How we scale our experience monitoring for the biggest websites

More and more online businesses today are understanding the value of their visitors’ site experiences and the direct impact they have on customer acquisition and retention. With the focus shifting towards visitor experience over purely technical performance statistics, the need for a unified digital experience monitoring solution has become mission-critical.

This is especially true for industries such as e-commerce or publishing sites, where large amounts of frequently changing content can have a direct effect on whether their visitors become converted customers or take their business to competitors.

If You Build It, They Will Come

While we have made strides in the realm of experience monitoring with our Real User Monitoring tools, we saw the need for a dedicated monitoring tool that combines site metrics that are valuable to both engineers and business managers.

That’s why we built Visitor Insights, which offers a single, holistic view of your website’s performance and how the changes you make to your site impact on your visitors’ experiences.

We understand that visitor experience is important to all businesses, from the smallest online store to the biggest news publishers. Small online operations value the experience of a few hundred site visitors equally as much as an international newspaper values the experience of the thousands of daily visitors they receive.

One of the key requirements we set ourselves when building Visitor Insights was the ability to scale our monitoring to accommodate all of our customers, regardless of their traffic volumes.

With this, we have restructured our pricing to offer digital experience monitoring to everyone without the need to upgrade to more expensive plans due to higher traffic volumes. After all, growing site traffic is something we encourage!

It’s also why we’ve built Visitor Insights from the ground up, with new technology that allows us to offer our customers to monitor millions of pageviews at very affordable prices.

Building Technology That Scales

When we put scalability as a key requirement to building our new technology, we took the biggest publishers, like NYTimes or The Washington Post as examples.

After all, if we could accommodate the needs of their sites and traffic, we could build a tool that gave anyone the confidence to grow with Pingdom.

Our original Real User Monitoring tool was built with good intentions but we quickly realised that our customers’ needs with regards to scalability made our tool both expensive and hard to maintain.

Moreover, as we aggregated our RUM data directly, we saw limitations when customers wanted to dig deeper into their historical data.

With Visitor Insights, we solved this issue by building our tool’s storage requirements on a partitioned, hierarchical solution.

The result is a high performing, yet affordable, system that allows us to retain raw data for longer, and allow customers to perform advanced queries on live and historical data quickly.

Data Is Only Useful If You Can Learn From It

Big data is not new, and understandably the more data you gather from as many of your site visitors, the better you can understand how they browse and interact with your site. While having access to large quantities of your visitors’ data is a good thing, it is only as useful as the insights you glean from it.

One of the main challenges we faced when building Visitor Insights was designing an interface that could support and present big data in an efficient and user-friendly way.

We’ve created several tools to allow our customers to easily understand their site visitor data, including:

Page Groups

Whether your site has 20 or 2,000 pages, we have built in the ability to quickly view data on specific page groups. This allows you to group any and all of your most vital pages and see an overview of how they perform as a whole.

Say you have an online shop and your developers recently deployed an update that affects your product pages.

You’d want to see the impact of these changes across all your product pages, without having data from irrelevant pages skewing your results.

We’ve made grouping easy: using a simplified RegEx function based on exact matches and wildcards, you can group together any pages you choose.

Easy to Understand Interface

Sometimes you just need to be reassured that your site visitors are happy. We built Visitor Insights to be accessible to all, from engineers to business owners.

Visitor Insights gives you a unified, holistic view of how your site performance reflects on visitor satisfaction.

We designed our graphical user interface (GUI) to present business-critical data, i.e. your site’s bounce rate and load times at a glance. This way, someone wanting a quick overview can have the data they need at a click of a button.

Of course, behind the simplified overview lies a wealth of raw data that your engineers can slice and dice when troubleshooting issues or pinpointing performance bottlenecks.

Leave a Reply

Comments are moderated and not published in real time. All comments that are not related to the post will be removed.