How Web 2.0 shifts the paradigm

Perhaps you’ve been experimenting with the free Google Analytics solution and are ready to apply more powerful technology, or simply apply the service in a more rigorous way.
It turns out that you have many choices for solutions. And each will come with its own implementation rhythms and complexity. Forewarned is forearmed.

Selecting a web analytics solution
If you have completed your research and requirements phase thoroughly, you have likely narrowed down potential vendor options. To build your request for proposal (RFP) or tender, use the functional specifications, and add additional system-related and general requirements. Avoid check-the-box requirements and instead ask substantive questions of the bidders.
Regarding data collection, ask:

What data collection methods are available and how they are implemented;
How page tag implementation will work with your content management system and what other alternatives there are for global tag implementation;
How your product filters out and reports on domains and search engine bots;
How your product captures and reports on non-HTML content – for example, PDFs, Excel, Word, streaming media, AJAX;
How reports can be obtained both on a main domain (www.com.com) and its sub-domains (careers.com.com) without doubling count users;
What the strategy is for deploying cookies and what the impact is on reporting if we cannot use cookies.

Regarding data storage, security, and accuracy, ask:
For software as a service (SaaS)

How long will the data product store and in what form,for example, with raw data or reports;
Whether data backed up frequently;
What type of failover infrastructure is kept in place;
How complete and accurate placement and syntax of page tags verified.

For software

What the disk requirements for storage are;
What back up and archive practices are recommended.

For data integration

What data types can be imported and exported;
How data integration can be accomplished;
How marketing partners can integrate data.

For support, professional services, and training

Find out the available levels of maintenance, support, and training and how much they cost;
Find out the availability and expected response time of web, phone, and e-mail based technical support options;
How the knowledgebase and/or other self-service support options are supported and if updates are added frequently;
What the professional services offerings are and costs;
Available courses, how much they cost, where the courses are given (can they be taken on-site or online?) and whether or not free educational seminars available.

For company stability

Number of years in business;
Number and size of clients;
Staff growth and turnover;
Credit rating through services such as Moody’s or Standard & Poors;
Recent acquisitions.

For product development

When the next release is and what it will include;
What new products and offerings are planned and when;
Strategic direction of offerings/products.

Adding these questions to the use cases produces an RFP tailored to your requirements, instead of boilerplate lists of functions that all vendors claim they can perform. While this process is labour intensive, the upfront planning will pay off in selecting the right tool for your organisation, and implementing it effectively.

Evaluating web analytics vendors
The vendor’s response to the RFP is of critical importance, but remember that words are just that. You’ll want other inputs to help you make your decision, including:

Vendor demo;
Product demo;
Proof of concept;
Customer references.

Vendor demo viewing tips
Vendors will want to show you reports during the demo. While this might be fine as an introduction to the tool, you are now interested in seeing how the tool really works. For this, you need to focus on functionality related to the specification that you’ve created and user administration functions that have been outlined in the use cases. Recall that your business case may be driven from some very specific reports. In addition you may also want to understand how to do the following:

For report development

How to create visitor segments and apply them to reports;
How to create campaign tracking reports;
How to create Excel-based reports;
How to determine whether online help adequately assists users;
How to use report filters.

For user administration

How to set up report users and groups;
How to publish reports on a schedule for distribution;
How to publish reports on an as-needed basis.

Remember, the idea here is to get an idea of ‘a day in the life’ with this tool. You and your team will be living with this choice for some time. Don’t let the vendor sidetrack you with a presentation that doesn’t address your requirements.

Product demo
Vendors often provide demo accounts so that you can use the web-reporting interfaces on a dummy data set. These demos can be quite helpful in assessing the tool’s usability, access to online help, and the ability to run reports. Understand from the vendor which version the demo uses, whether optional modules (which cost extra) are included, and if there are any limits or constraints on the demo’s functionality.

Proof of concept
Remember: it doesn’t work until it works with your data.
Vendors will typically engage in proof of concept trials, albeit typically for a cost. It makes sense to do a proof of concept when you have narrowed your choices to two vendors. This will give you an opportunity to thoroughly test the application on a section of your site, as well as to implement its data- collection methodology.

Customer references
Vendors will give you hand-picked customers to speak to, but you can get added value if you request to speak with customers who have businesses and environments like yours.
To help identify potential issues that might surface with the tool, ask them what challenges they’ve faced and what features they’d like to see going forward.

Making a decision
Determining the cost for a web analytics solution can be quite complex. Obtain a price quote that itemises all products and services and negotiate on price and services. The Web analytics space is very competitive.
Use this to your advantage for a negotiated price. To make your final decision, meet with your team to evaluate the vendor against your requirements. Make sure that there is buy-in from those who will be expected to support and use the tool.

Implementation tips, techniques, and best practices
Once you’ve selected the tool, the real work begins. Usually this means meeting expectations to begin cranking out data as soon as possible. You can get help if you need to jump start your program. There are three ways you can go about this.

Professional services from the web analytics vendor
A web analytics vendor will happily sell you professional services. In the vendor’s favour is their expertise in providing implementation and training services, as these are directly related to the product. Some vendors offer best practices and analytics services as well. However, they may sidetrack a bit in cross-selling additional product and service features.

A consultant who partners with the vendor
Hire a consultant who partners with the vendor. Like the vendor, they may be interested in selling additional product features, but could offer a wide range of support services, including implementation; best practices; analytics; and strategy.

The vendor-neutral consultant.
The third option is to work with a vendor-neutral consultant who represents your interests only and does not partner with any vendors. Independent consultants can offer assistance in developing requirements and metrics, as well as requests for implementation (RFIs) and RFPs, implementation, training, best practices, analytics, and strategic services. Some independent consultants focus their work on specific products based on their experience, not because of partnerships. In general, the downside to most neutral consultants is their lack of expertise with any single tool.

Let’s take a look at tips, techniques, and best practices that focus on creating early successes and actionable web analytics programs.
These include:

Using requirements and metrics development assistance:
Wanalytics buyers perennially fall to the allure of full-colour reports with beautiful trend lines. That lasts until they realise that they have more data than they need and that their user base can’t figure out what to make of the reports. If you follow the protocol described in ‘Error! Reference source not found’ on page ‘Error! Bookmark not defined’, this won’t happen to you. However, if you don’t have the resources or bandwidth to go through this phase diligently, call on some help.

Using implementation assistance:
A frequent lament among analytics customers is: “I wish I’d used the vendor’s implementation assistance.” Many enterprises spend their money on the product and think they’re saving lots of money taking the ‘do-it-yourself’ approach. This often results in mistakes in data collection in SaaS systems: or if you are hosting the software yourself, tool configuration and data collection errors. In page tag systems, vendors can typically show you how to enable global tagging through your content management system – a strategy that larger enterprises need to address for viable data collection. Address other issues during implementation such as how to best distribute cookies. Vendor experience with and exposure to a multitude of environments can be of benefit. Also, because organisations rarely have resources available to spend entire days on implementation, the DIY method sometimes proceeds in fits and starts, often with significant delays. In addition, if your staff spends time with implementation consultants, they’ll learn a lot about the product, helping them to maintain the tool going forward.

Phasing in metrics development and availability
If you followed the ‘Error! Reference source not found’ section recommendations and developed a functional specification and determined priorities for metrics availability, this should be fairly straightforward. You create a development schedule to enable your implementation of the metrics. If you didn’t follow the recommendations, you could find yourself in a situation where all potential users are bombarding you with report requests at the same time.

Developing Web analytics processes:
There has been – and continues to be – a sense of organisational confusion about the web analytics function. For example, where it belongs in the organisation, whether it should be in marketing, IT, finance or an independent function. There are examples of it being seated in all of these departments, both working smoothly and not. Success doesn’t depend so much on what department owns it as long as you put effective processes in place. You also need to think about how to integrate analytics into your web-application development lifecycle. Unfortunately web developers, designers, and marketers frequently do not include analytics folks in their development cycles. This causes web applications, content, and promotions to be developed without proper preparation for data collection. This results in data not getting collected and/or needing to be re-worked and re-tested, causing production delays. To ensure that web analytics is integrated into the web development process, you need to implement business processes and procedures to manage data collection, analysis, and presentation.

Training your user base
At a minimum the tool administrator and web analytics manager should receive vendor training on how to manage the product, create reports, and use any additional modules you purchased. Although some enterprises get by with simply reading the vendor documentation, training is almost always essential when the principals on the project have no prior web analytics tool experience. In fact, it can be a very good idea to expose a technical resource to the training if they are responsible for development work associated with data collection. This can also help mend the disconnection that sometimes exists between web analytics managers and IT concerning the complexity of data collection. Beyond vendor training, the web analytics manager will need to spend a good deal of time training the user base to employ the solution to create reports, interpret data, and to develop metrics and functional specifications for new content, applications, and campaigns.

Analysing and presenting actionable data:
Ensuring that all stakeholders receive data in a usable form, in spite of the reporting options available, still challenges many organisations. Web analytics practitioners struggle with the best way to break through the clutter to get their analysis and recommendations across. This can be mitigated if stakeholders are brought into the metrics requirements from the beginning so that they have ownership over them. In addition, having a report presentation process in place also helps the data to be seen, whether this is a daily e-mail with targeted information, or a weekly meeting. In any case, the data and metrics must be made usefully available, or the analytics initiative will be perceived as a failure.

Eight steps for success
If you do nothing else follow these eight steps:

1) Research the marketplace – understand key differences about the vendors and their history, tendencies, technologies and approaches;

2) Know your requirements – map analytics requirements and tool assessment to site objectives; functions and metrics;

3) Ensure your support – help senior management, technical team, and report users understand why analytics is important to their agendas;

4) Build process around tool administration and usage – all tools need to be maintained to work optimally;

5) Don’t scrimp on training and implementation support – make sure it meets your objectives;

6) Negotiate – don’t accept the list price;

7) Use the data for decision making – initiatives fail if no one uses the data;

8) Build analytics into web strategy and production process – analytics must lead web content, navigation, and marketing initiatives.

This article is adapted from the CMS Watch Web Analytics Report, http://www.cmswatch.com/Analytics/ ?

Phil Kemelor is CMS Watch Contributing Analyst and lead author of the Web Analytics Report, which evaluates 13 major Web Analytics solutions. http://www.cmswatch.com/Analytics/Report/