Feature Article (Pub:December 03) Measuring Value, not just hits by Paul Rudman & Andy Caddy
The explosive growth of Internet usage in the past few years has led to an ever-increasing investment from vendors trying to tap into online consumer spending which, according to Forrester Research, will exceed US$100 billion by the end of 2003.
In parallel with this increase in retail spending, companies are also investing heavily on internal, web based systems, adding portals and application aggregation to their intranets and interacting with their suppliers and partners through extranets.
But what drives this spending, and where will investment reap the greatest rewards? Without a way of measuring successes, and failures, there is no way of knowing if you money is well spent or poorly invested. Creating and understanding an holistic approach to metrics will help you prioritise developments and justify business investment, no matter what platform you are operating upon.
What are metrics ?
The term metrics is used in the web context often to mean just plain website log analysis, using such tools as Webtrends, Urchin or Analog. But this is taking a very narrow view of the subject that really should be defined as the understanding of website usage, activity and performance through the study of all available data. It is only when this wider and more complete analysis is carried out that your site can be truly understood.
Why are metrics important?
As economic slowdown has taken a grip since the heady days of the late 1990's, organisations are looking more and more at the value they are getting from their technological investment. Justifying ROI has now pushed the role of metrics to the forefront, as the ability to capture and analyse figures and datasets represents the ability to define current and future marketing initiatives. Without understanding current usage and performance, how can you justify creating any strategy for further spending?
So what type of metrics are available?
This is where Information Technology meets Marketing, where site traffic analyse joins CRM data to provide the complete profile of the user, their activity and their experience that cannot be achieved without this union.
User activity leaves a technological footprint that provides details of the location of the user, what they looked at, and how long they spent on your web site. Combine this with user contact details and opinions from surveys and feedback/ contact us forms and you've got everything required to tailor your web site to your target audience.
What are the options?
There are a plethora of different methods for gathering data. They vary from the complex to straightforward, from technology-based to paper-based, and from hugely expensive to virtually free. But before you dive in and start defining the tools, first you must understand the problem. What are you trying to achieve? Metrics provide the means of measuring the success of your site but what do we mean by success? Hopefully you will have some well defined success criteria distilled from your own business objectives, but these will vary across different business. Although based on the same technology, the objectives of intranet, internet and extranet are usually quite different: Intranet's improve employee productivity, internet is about retail and marketing and extranets reduce costs in the supply chain or differentiate services provided to customers and partners.
At a top level, some measurements that might help you judge the success of you sites might be:
Success Criteria, medium and Example Metric:
- Reach more customers - Demographics of users.
- Make more Sales - Ratio of visits to conversions.
- Improve Customer Satisfaction - Analyse results of customer feedback.
- Target all employees with comms - Use ERP system to target specific departments.
- Increase online learning usage, Encourage knowledge sharing - Traffic Analysis highlights areas of high usage, these can be used to promote internal messages.
- Differentiate your service - Customer feedback / questions will refine the services you offer.
- Reduce cost/ Increase revenue - Track usage to justify investment in Extranet functionality.
Internet sites, Intranets and Extranets all have unique requirements; Intranets focus on hard and soft savings, Internet sites drive revenue generation or brand awareness, and Extranets improve customer satisfaction and generate savings. Of course there are not black and white rules for what will work in a particular situation, but the following are the examples of relevant metrics: -
- Traffic Analysis
- Measurements from the log analysis may give most popular pages, top referrers, least visited pages, length of visit (site 'stickiness'), visitor conversion rate, location of user, and details about the visitors technology such as browser and OS to help give a picture of what activity is occurring on your site, where people are visiting, and what the relative strengths and weaknesses are of your web site.
- Calls to Action
- Interacting with your customers to capture surveys, questionnaires, interactive votes and forums. "Question of the day" and "Two minute questionnaires" are popular ways of achieving this.
- Call centre monitoring
- Understanding the amount of calls received for key deliverables of your site - product enquiries, store openings, service descriptions, product ordering etc. Reducing these calls by providing the same information or service online may be a key deliverable and an easily acquired metric.
- Customer specific data
- What geographies do your customers come from; this may indicate how successful your marketing campaigns have been or whether your site is too geared to a particular language or culture. How many orders are repeats from loyal customers? How long does an order take? What are the peak periods for ordering? Querying the customer database can give a number of important metrics that when cross referenced against other figures give a much full understanding of site usage
- Site usage, number of unique visitors (leading to what percentage of your company is being reached), most/least used areas and length of visit. Though similar to internet analysis the emphasis will be more on getting usage out of your investment rather than making the site sticky and attractive to customers.
- Employee interaction
- Feedback forms that are used for department enquiries, moderated employee forums, employee polls. All these tools help gather employee views and opinions but need to be co-ordinated to ensure that the data they capture is structured and suitable for analysis.
- Traffic Analysis
- Site usage, number of unique visitors and most/least used areas will be amongst the statistics you'll be analysing. Extranets may have to be justified and therefore usage needs proof, but more often than not they will be about improving a customer relationship or reducing costs therefore other metrics will reap better rewards.
- CRM interaction
- If your extranet links to your CRM (and really, it should do) then you should be able to measure the number of calls logged, or service requests made or whatever the particular nuance of the customer relationship is that you drive to improve. The CRM system should also be able to provide overviews of your customer geographies that can be cross-referenced with the traffic analysis to see how successful your site is at reaching all your target users.
- Call centre interaction
- Perhaps you look to reduce the number of calls your call centre receives. If this is the case then the baseline figures need to be captured to understand what you are trying to improve. Being able to differentiate the source of a call is also therefore critical; is it a phoned in call or was it generated over the Internet.
To illustrate how different approaches to metrics can assist in many ways, the following business scenarios should help demonstrate a variety of approaches.
Investing in an intranet project to reduce HR admin. The project must quantifiably reduce the amount of administration performed by the human resources department such that headcount reductions can be made
Implement a self-service intranet with complete employee penetration that automates administrative tasks such as hiring forms,performance review administration, agency interaction, employee details and password resets, etc along with online versions of policy manuals and FAQ/ Knowledge management base to improve productivity and reduce human interaction.
In order to assess the relevant success of this project, all user activity has to be tracked and compared with data about the processes that were in place before the new system was integrated.
Traffic analysis will provide information on who is using the new application and how frequently. Departments with low usage may require training and more exposure to the benefits of the system or highlight that its relevance is certain user group specific.
The tracking and analysis of different file types downloaded will justify the save in human administration that providing a corporate resource to access HR documentation was set-up to achieve.
Analysis of the usage of the FAQ/ Knowledge base highlights which sections and new processes users are finding difficulty understanding, so providing user feedback without having to contact the users. From this the new system can be fine-tuned for ease of use. After all, the whole purpose of introducing the knowledge base is to minimize the need for human support and administration, so the new experience has to be as painless as possible.
Employee Feedback through the system can be directed into organized databases and the information dissected to improve the user experience and gain a valuable insight into ways to improve the system, how to tailor the knowledge base, etc. This data is stored within an centralized environment and easily accessible for review, providing a far more efficient feedback loop than with more human involvement in the HR administration tasks.
The automated HR system should have a search function that logs all user activity including all queries made, most popular queries, number of zero returns, and number of documents viewed from the returned results.
Is you search helping users find what they want? What are users looking for? Queries such as "jobs and vacations", "legal policies" "travel and expenses" will give you the knowledge about what employees want to find, and also how simple it is for them to access the appropriate documentation.
A company selling products primarily through an online channel to a UK audience wants to expand into Europe but needs direction. It has been noticed that some sales are coming from non-UK customers and the company wishes to further expand on this opportunity.
Web logs can give an overview of geography though this is not necessarily reliable. By examining times of access, clues to timezones can be achieved. Matching the order time to the site usage could also give indications of where the traffic originates. Ultimately, completed orders on the sales order system will be able to show where orders are going, but not what potential audience there is; non-English speaking countries will not get very far in the system and will not complete the sale.
A start point might be to introduce a number of non-English language navigation pages that help users around the site - after all it is extremely expensive and time consuming to re-write the site in multiple languages. Alternatively a Multi-lingual questionnaire may also give the same result. Examining the usage of these pages after a pre-defined period may help to understand which cultures and languages need to be targeted first.
Web Metrics are essential for understanding current marketing investment and justifying future spend. Some sales have come from Europe, are these isolated instances or possibly the seed of a future target market that had not previously been considered? This question will need to be addressed as it raises the implication of a multi-lingual site, potential changes to an e-commerce solution (shipping costs & currencies), accessibility (review of all text, images etc. to make sure they are universally understand and not just colloquial) maybe even has legal implications (consideration of different laws etc.), and all demand cost.
The company needs to consider where did the users come from? Analysis of search engine and directory referrals, customer feedback / queries, and click-throughs / rich media exposures from current marketing campaigns will provide a complete picture about the non-UK leads and how they arose.
Conducting a pilot marketing campaign with various companies in different countries such as search engines and industry portal sites will allow the company to test the waters, without an initial substantial investment, and tracking the campaign to conversion ratio will determine whether further investment in the foreign user base is to be encouraged and developed.
Market research metrics will also provide an insight into whether your products and offerings are likely to appeal to a greater audience in Europe. Email campaigns evoking feedback and response are important, but more so are the opening and conversion rates of these campaigns to determine what that market wants, and whether you are positioned to take advantage of that. After all, you may think you understand your target market from your success in the UK, but understanding European audiences is essential to justify the ROI of targeting a demographically different set of users.
A service delivery company wants to reduce cost and improve service by giving customers the ability log their own fault calls, make service orders and view online bills.
Adding extranet capability to your company is not a task to be undertaken lightly - you are, after all, about to change the way you interact with your customer base and introduce a new model of engagement. Designing the perfect system first time is clearly nothing more than a pipe dream and therefore an iterative approach would seem a sensible way forward. The problem is how to decide which areas to target; there are many touch points for the customer supplier relationship and therefore how do you decide where to start?
The most important part in the planning phase of creating this new web accessible method of customer communication and method of interaction is to understand the rationale for doing this in the first place, and here is where Metrics is essential.
You'll have all the relevant data to hand from length of time taken to answer a call from a client on a subject to costs associated with the creation and distribution of the current billing process, because if you don't have this benchmark you will lack any fundamental criterion for success.
Holistic metrics on the processes you are going to replace will help create a framework or scorecard focusing on the goals that you expect to be achieved by implementing the new system. This approach will not only manage the expectations of the project shareholders and sponsors but also present a rational reason behind the move, thereby encouraging others to embrace the change rather than rally against it.
When defining a strategy for your website or when putting a project together, the term "metrics" is often mentioned but with little attention paid to the actual detail of what it may describe. It may be ticked off as complete once a log file analysis package has been procured and installed yet there is so much more that is required in this field to truly understand the usage, successes and failures of your web site. Once your web site is live, it is virtually guaranteed that you will be questioned for statistics of how the site performs and without your holistic metrics in place you answer is going to be very limited.