Understanding the real end-user experience

Richard Hepplestone, Application Performance Management Consultant at Compuware South Africa

Do you have a transactional website where customers can go online to purchase - and pay for - your organisation's products and service? Do you know how many of your online customers are happy with their online experience?

Today, few organisations can answer this question with any confidence or accuracy. Research commissioned by Compuware has found that despite 50% of European retailers and financial organisations claiming to monitor and manage the performance of their transactional web site, just 6% of organisations have specific information about the actual end user experience for each individual logging on.

With such insufficient methods in place, it is therefore perhaps unsurprising that poorly performing transactional web sites are costing a third of retailers and companies in the financial sector across Europe approximately R5 250 000 per year. An additional 21% estimated that poor web site performance was costing them around R15 750 000, with a further 23% estimating the cost to be in the region of R21 million.

It is only by monitoring and understanding each individual customer's online experience that an organisation can ensure a quality online service.

This is great in theory, but as soon as you look into end-user experience monitoring, you usually get bombarded with information. You are not sure whether or not the recommendations you are given are based on what is best for your organisation or whether they are recommendations to help a vendor sell its products.

So, what are your options?

Transactional- or Session-based monitoring

This refers to two general types of information that can be extracted. Transactional monitoring refers to the monitoring of response times and availability of business transactions. For example, the time it takes for the system to generate a purchase order or even the time it takes to log in to the system.

Session-based refers to monitoring on a more technical level where monitoring is typically performed on the network and response times are calculated based on looking at the TCP session response times.

Like anything else, each of these general techniques has its pros and cons. The main advantage of transactional monitoring is that you get a true reflection of the actual end user experience, making it easier to relate the monitored user experience to business drivers. On the other hand this can be relatively more complex to implement and maintain.
The main advantage of session-based monitoring is that it is typically quick to implement. Usually, all that is required is the deployment of some kind of network probe or probes in the right place and the majority of applications can be monitored.

The 'however' is that it does not give you a view of how long business transactions are taking, although it is good for technical troubleshooting, particularly if you start comparing response times at different places in the network. This is also a form of end user experience monitoring that has developed in recent times to overcome its disadvantages.

Transactional Server or Client side monitoring

Another decision to be made when it comes to end-user experience transactional monitoring is whether to adopt server or client side monitoring. Placing a network appliance in front of the application server typically performs server-side monitoring. From this point it captures user transactions as they go in and out of the application server. The advantage of this is quick easy deployment. The disadvantage is that only certain middleware protocols blend themselves to facilitate the accurate definition of end user business transactions. Also, some question the response time accuracy of measuring the user experience from the server perspective.

Measuring from the client side is the (supposedly) most accurate method of measuring, but also has considerations in terms of the implementation and maintenance. There are typically two ways to implement client side end user experience monitoring - agent-based and agentless. An agent-based client side solution involves distribution of agent software, which can monitor transactions either actively or passively.

An agentless client side solution involves writing instrumentation code into the application. The obvious downsides to this are the risk of introducing bugs into the application, additional maintenance steps needed when upgrading the application, the overhead and of course the risk of 'missing' the instrumentation of key areas. Also, in this context the term 'agentless' is a little misleading. All you are actually doing is building the agent in to the application.

Transactional Client Side Monitoring: Active or Passive?

Active monitoring typically refers to a type of client side end user experience monitoring where robot devices are set up at key locations on the corporate network and are configured to playback pre-recorded transactions at a specified interval. This is sometimes referred to as using 'synthetic' transactions. This is currently the most popular method of end user experience monitoring according to a Gartner poll published in April 2005.

The biggest advantages of this technique are that it is relatively unobtrusive - each robot simulates a single user on the network - and more importantly the predictable nature of the monitoring (you know when each transaction is going to happen) allows for:

• Proactive notification and diagnostic capture of problems (the monitoring discovers the problem before the first user experiences issues)
• Long term trending and analysis of application service levels
• Service levels are monitored and reported upon, even when no users are transacting

As with everything else, this comes at a price, and once again that price is the time required for installation and maintenance due to the need to script transactions. The other challenge that is often raised when it comes to synthetic transactions is that you need to consider the effects of simulating write transactions or in other words anything that makes changes to production data. If you generate a new purchase order every 15 minutes you are going to have one very happy supplier. This can however be countered by running cleanup scripts to delete the data created or using dummy accounts so that the purchase orders you create don't actually get processed.

Passive client monitoring refers to monitoring actual users' transactions, through placing a non-intrusive agent on the client machine, which then monitors for defined start and stop events associated to end-user transactions.

So what's right for you?

At this stage, all I have probably achieved is to highlight the fact that there are multiple techniques out there for end user experience monitoring with advantages and disadvantages to each one. Which one should you choose?

Unfortunately, like most things in this industry, there is no simple answer to that. The first thing to look at is your reasons for wanting to implement end user experience monitoring. If your aim is to create better alignment between business and IT, then monitoring that shows the performance of business transactions is needed. If the aim is only to provide a means to detect and troubleshoot a performance problem, then you can take advantage of the quick installation and low maintenance requirements of a session-based solution.

It is also important to look at internal capabilities. Is rolling out numerous agents at all feasible? Is application scripting possible? Who owns the network - do you have the access necessary to install monitoring probes? And then of course you need to consider the application architecture and whether a certain type of monitoring supports your environment.

Conclusion

Monitoring the end-user experience has numerous uses and benefits to an organisation, and there are definitely many ways to skin this particular cat. What is best for your particular organisation could vary. Currently the most popular method is active, client side monitoring, however passive, server-side monitoring is rapidly gaining popularity, particularly since new technologies, such as Compuware's Vantage Agentless Transaction Monitoring give the benefits associated with passive, server-side, session based monitoring (rapid deployment, low maintenance) and have evolved to allow for the definition of business transactions, making it useful for business/IT alignment