Synthetic Monitoring - why & when to use it
Introduction
A bad digital experience can be very costly. On an average, the cost to acquire an online visitor is around $5 and while only a 3% of them actually convert, any performance issues can burn a big hole in your pocket. But how can you set a benchmark for a quality digital experience?
HubSpot claims, if the site brings in $100,000 / day in revenues, a 1 second page improvement will bring in extra $7,000. That’s how much a second is worth; it can mean a 7% increase or decrease in your revenues.
Though upholding performance at a benchmark is difficult, monitoring them to take proactive actions can be easy. But performance monitoring yields results only if you choose the right tool. This is because most monitoring tools offer some basic features like pinging URLs to check availability and load time. Moreover, it is different from how a real user would behave on your website. So, why would you use a monitoring tool that doesn’t tell you anything about the user journey?
The only way you can analyze whether a user journey meets the customer expectations is by using a Synthetic Monitoring tool. It is used to monitor critical user journeys on a website and warn developers in case something doesn’t work or perform as expected. If you’re not familiar with Synthetic Monitoring, this article will help you learn about it in-depth.
What is Synthetic Monitoring
Synthetic monitoring (also known as active monitoring or proactive monitoring) is web application monitoring that is done using a Web browser emulation or scripted recordings of web transactions regardless of whether or not users are currently visiting your site. Synthetic monitoring tools are used to identify and resolve web application performance and availability issues from different geographical locations worldwide.
When and why is Synthetic Monitoring needed
- Synthetic monitoring is useful to check your site’s availability and end-to-end performance 24/7 from the end-user perspective. Synthetic monitoring predicts, to a fair degree of accuracy, how your application will perform in which geographies and isolate the root cause of any detected bottlenecks. Synthetic monitoring prevented or pre-empted 95 to 99 percent of previous performance issues for most of our customers.
- To provide rich customer experiences, websites and applications increasingly depend on 3rd parties such as shopping carts, ads, customer reviews, web analytics, social networking, SEO, video, and more. In fact, retailers around the globe increased the number of 3rd parties by 21% in 2015, from an average of 25 to 30 3rd parties.These third parties help increase traffic, conversions, and improve customer satisfaction, but any one of them could weaken your site’s overall performance or even take it down. While SLAs may let you point the finger, from the end-user point of view, you’re the one who’s accountable. Using synthetic monitoring, you can evaluate components via pre-production testing for proof of concept and load testing based upon what you expect during high-traffic periods.
- To objectively measure your own service level agreements (SLAs) and also your third party service provider’s. When your site is not functional, loading so slow and timing out that customers go elsewhere, then your site becomes unreliable. Imagine if this happens at a business-critical moment such as cyber Monday or during a major news event, this will impact your entire year negatively. Continuous testing and synthetic monitoring are the best possible ways to ensure your web applications are reliable. No application is immune from performance issues. Complex infrastructure and integrations that support web application delivery in today’s world makes it even more vulnerable. You can get ahead of any problems and preserve your customer experience by taking a top down approach and having visibility into how your application is performing and all of the factors that affect that performance.
- To measure, baseline and analyze performance trends of your application across different geographies, browsers and resolutions. If you don’t track and baseline your performance during normal operations, you won’t know how your applications are performing for your end-users during peak periods. Synthetic monitoring enables a consistent, reliable approach to measure performance throughout the days, weeks or months. With synthetic monitoring, baseline tests can mirror the way your end users access your applications. These simulated user tests can monitor key transactions across geographies, browsers, and devices. Armed with this data, you can assess whether you are meeting user requirements, identify areas to improve, and use the data for capacity planning. Established baseline metrics from synthetic monitoring will give you concrete data to support negotiations and to monitor going forward before entering an SLA.
- To track and benchmark your application performance versus the competitors. Benchmarking using synthetic monitoring provides valuable insight into how market leaders and the competition are performing. It also provides a context for shared goal-setting between IT and business stakeholders to improve business results, and as a tool for measuring performance over time.
What Synthetic Monitoring will not do
- Resolve end user complaints. Synthetic Monitoring would tell nothing about what the end user was actually doing or experiencing when a real user raises a support ticket.
- Troubleshoot servers and network problems. Operation team may need visibility into servers and network related problems. Synthetic monitoring does not provide this visibility.
- Determine how server or network performance affect application performance. Under resourced network or servers impact application performance. Synthetic monitoring is not a good solution to identify root cause is such cases.
- Analyze the performance of every page in an application. Creating and maintaining scripts for synthetic testing is time-consuming, even for one application. It should be judiciously used for monitoring only critical flows of an application
- Validate the impact of change because of change in application or infrastructure. When IT team makes changes to applications or infrastructure, they must validate the impact of those changes on actual end user experience to determine the impact. Comparing metrics from synthetic testing won’t provide the full picture.
How to evaluate Synthetic Monitoring tools
- Creating monitoring scenarios:
- It should be as easy as navigating the website like real users. The best way to do that is to use a record & playback tool which captures user-like actions and starts monitoring them. Not all monitoring tools offer record & playback. A few website monitoring tools like CloudQA and Dynatrace have this capability while some of the popular tools like Pingdom requires you to code for synthetic monitoring.
- Due to ever increasing complexities of modern web applications, synthetic monitoring tools should be able work on websites with technologies like single page applications. If the monitoring tool does not support working with complex web functionalities, the scope for catching performance issues is severely narrowed.
- Once a scenarios is created for monitoring, a Synthetic Monitoring tools should provide an easy way to extend it. If such feature is not available then any incomplete scenarios can frustrate the tester working with the monitoring tool.
- Management of monitoring scenarios
- Agile methods bring frequent updates to the application. Hence, any synthetic monitoring tool should allow modification of scenarios that are as agile as as the application development process.
- Websites usually have many workflows; however, there are some common scenarios like Login, Logout and Payment. Synthetic monitoring tools should allow joining separate scenarios to create a new one; thereby, reducing a lot of effort in creating new scenarios. Hence, a monitoring tool should contain a library of scenarios which can be tested collectively and interchangeably.
- Data driven monitoring
- No monitoring is complete without verifying all possible combinations of data inputs. For example, a website may have calendar settings/filter. In such case synthetic monitoring tools should be able to verify performance with alternating ‘Date’ inputs. The same can be said about other variations like a ‘Filter and Sort’ option on the website.
- Minimize false positives
- False positives are one of the most annoying thing about website monitoring. They however cannot be escaped. Issues arise from server maintenance, glitches, poor coding and problems with the CDN. Synthetic monitoring tools should be capable of reducing false positives.
- A well designed synthetic monitoring tool reduces overheads like retesting for any false alerts.
- Only a handful of synthetic monitoring tools are resistant to unexpected behavior of an application. For example, web elements of apps developed using AngularJS are dynamic and load unpredictably. In such cases, an element loading slower than the speed of detection by a monitoring tool can misfire an alert.
- Location based monitoring
- Latency could be higher from certain location/s from where your customers access the website. Monitoring from different geographic locations is a way to see how the application performs with latency and load issues.
- Alerts & Notification
- Synthetic monitoring tools should offer an alerting mechanism for situations where the desired performance goes below an established threshold . You should check whether the alerting tool of your choice integrates easily with your technology stack. Bare minimum requirements for monitoring alerts are SMS, and email. Integrations with popular tools like Slack and Pagerduty are also sought-after.
- Reports
- Baselining is an important feature for comparing current performance to a historical metric, or a baseline. The performance figures can be used as a comparative baseline for configuration changes.
- Reports from a synthetic monitoring tool must include performance metrics for availability, uptime, full loaded time, network time, client time, server time, and resource timings.
- Data retention is variable for monitoring tools. The retention offered is generally 6 months or 1 year.
- Other criteria
- Synthetic monitoring is not readily available with most APM tools. An integration with APM tools is desirable in such cases.
- AI / ML can be an important feature to reduce false alerts, prioritise problems and for prediction of issues.
- Few synthetic monitoring tools offer mobile browser support. This is important for businesses having sizeable web traffic through smartphones and tablets.
- To see how the application behaves with different bandwidths, a bandwidth throttling support is essential.
Where does TruMonitor excel
If you are an ONLINE INSURANCE company
When someone is trying to buy an insurance online, the personal data like date of birth, gender, location, etc are deciding factors for premium calculation. To get a true sense of how your application behaves for a true user, you should have the ability to monitor & measure your application performance with multiple data sets. TruMonitor is the only tool that makes multi-dataset monitoring possible and extremely easy to setup. With other tools, you are forced to a use single data set to monitor an insurance purchase journey and you will never be close to truly measuring your end user’s experience. Having different monitors for each data set is not a very good option from maintenance perspective. Imagine if you have a synthetic monitoring solution that could solve both these problems by using data as a variable executing one single script. That actually will give a very good idea of how your application is perceived by different types of end users.
If you are an E-COMMERCE company
When someone is buying a phone from an eCommerce site, they would look for product description, price, color and other important attributes of a product. Most synthetic monitoring solutions will not allow you to verify different types of elements like images, links, buttons etc on a page which your end user would need. With TruMonitor you can verify all types of elements on page and also add custom scripts for complex logical verifications.
If you are an ONLINE BOOKING company
Usually flights and hotels are booked by providing a date of travel or date of check-in. You record a script in June 2018 picking July 2018 travel and check-in dates. If you use the same recorded script for synthetic monitoring continuously it will start breaking after the chosen dates. The reason being calendar will start showing up differently from August 2018. Unlike other monitoring solutions which either do not offer this capability or have a flaky approach for addressing dynamic dates, CloudQA provides you an option to work with dynamic dates f which would never break.
If you have an AGILE APPLICATION ENVIRONMENT
Online e-commerce can be very competitive and companies constantly have tweak their application to improve user experience. For example, in an eCommerce purchase, let’s say you had a dropdown to display cities based on a state for a user to select. To improve this process you change the dropdown to an auto suggest text box. With other monitoring tools all your synthetic monitoring critical workflows would have to be discarded and created from scratch. With CloudQA, you have the flexibility of deleting the dropdown stop and add step to type into an auto suggest text box by just click of a button.
If your web application has EXTERNAL DEPENDENCIES (API’s)
While purchasing a product on eCommerce website, one would have to go through a delivery process by providing zip code and in most cases delivery is done by a third party service provider. Generally, third party providers use APIs for verifying if they can deliver to the specified zip code. With CloudQA, you can combine web application flows with API invocation into one single script and monitor them together make it a smooth process.
Support for MODERN WEB APPLICATIONS
Modern web applications are complex and beyond the capability of a basic synthetic monitoring tools. For example, web application based on AngularJS are difficult to monitor because their of dynamic web element loading capabilities. TruMonitor has robust engine which can handle such complex interactions.
Support for NEWS / MEDIA APPLICATIONS
News and Media web pages are content heavy. Having a 5Mb web page is common among the leading media sites. And having third party scripts like ad for marketing can drag the performance further. TruMonitor helps you gather performance intelligence to help you optimize web page for any user behaviour.
TruMonitor Features:
| Category | Feature | CloudQA – TruMonitor |
|---|---|---|
| Creation | Recording Scenarios | Advanced recording of user actions with support for assertions and hovers |
| Monitoring | Browsers | Real browser support |
| Location Based Monitoring | Monitor multiple locations around the world | |
| Complexities | Monitor complex web applications like Single Page Application | |
| Maintenance | Editing Scenarios | Perform complex actions- Adding scripts, store variables, random variables, alerts, waits etc |
| 3rd Party Integrations | Out-of-the-box integrations with popular tools | |
| Scheduling Frequency | Choice of 5, 10, and 15, 30 and 60 minute frequencies | |
| Reliability | False Positive Resistance | Retrial of monitoring, automatic waits and robust CSS selectors |
| Reporting | Dashboard | Basic dashboard |
| Performance Metrics | Availability, uptime, full loaded time, network time, client time, server time, and resource timings. | |
| Performance Analysis | – Line graphs and histograms based date and time series analysis of Load times for URLs, and User actions – Interactive graphs with user action and URL filters – Run trends for status on application glitches while monitoring a scenario – Drill-down for glitches with screenshots | |
| Alerts and Notification | SMS, email, and custom integrations like Slack, PagerDuty and JIRA | |
| Competitive Benchmarking | On request | |
| Other | Support (Basic Subscription) | – Product onboarding – Dedicated Support Assistant – Live chat – 8 Hours / 5 Working Days available |
| Infrastructure | On premise and Cloud |
Summary
With growing performance expectations and web complexities, synthetic monitoring is indispensable for companies who want to stay ahead of the competition. Unlike other synthetic monitoring tools, TruMonitor offers a comprehensive solution for analysis of user journey even on complex web applications. Contact us to learn more on how you can leverage synthetic monitoring to become proactive in delivering quality digital experience.
LIKE THIS POST SHARE IT WITH YOUR FRIENDS
RECENT POSTS
Guides
How Test Automation Accelerates Time-to-Market and Boosts Software Quality
How Test Automation Accelerates Time-to-Market and Boosts Software Quality
How To Select a Regression Testing Automation Tool For Web Applications
Regression testing is an essential component in a web application development cycle. However, it’s often a time-consuming and tedious task in the QA process.
Switching from Manual to Automated QA Testing
Do you or your team currently test manually and trying to break into test automation? In this article, we outline how can small QA teams make transition from manual to codeless testing to full fledged automated testing.
Why you can’t ignore test planning in agile?
An agile development process seems too dynamic to have a test plan. Most organisations with agile, specially startups, don’t take the documented approach for testing. So, are they losing on something?
Challenges of testing Single Page Applications with Selenium
Single-page web applications are popular for their ability to improve the user experience. Except, test automation for Single-page apps can be difficult and time-consuming. We’ll discuss how you can have a steady quality control without burning time and effort.
Why is Codeless Test Automation better than Conventional Test Automation?
Testing is important for quality user experience. Being an integral part of Software Development Life Cycle (SDLC), it is necessary that testing has speed, efficiency and flexibility. But in agile development methodology, testing could be mechanical, routine and time-consuming.