‘Your Say’ – Measuring ROI

Implementing a comprehensive knowledge management programme demands a considerable investment of resources in any firm. The rewards, however, are not always readily quantifiable. Simon Lelic talks to Marcus Blosch, Paul Flew, Nick Milton and Geoff Smith, and asks how businesses should approach the challenge of demonstrating a clear return on investment.

“Knowledge management is catalytic,” says Nick Milton, managing director of Knowledge Transformation International. “A little investment goes a long way. You are not asking your people to do extra things; you are asking them to do different things, or do things in a different way. If your KM programme does not depend on massive IT purchases, then investment can be relatively small.”

But ‘relatively’ is of course the crucial word. Knowledge management has gone beyond the status of being perceived as a passing fad; it is now a global, multimillion pound industry in itself. The amount of time, money and effort being invested in KM world-wide has surged dramatically over the last five years. It is therefore entirely understandable that companies should demand the ability to quantify a return on their investment.

“In the early stages of a knowledge management programme, demonstrating return on investment is crucial,” emphasises Milton. “The initial period is all about evangelism – firing up the company to rethink old paradigms and embrace a new way of working. And nothing supports evangelism like a miracle. If KM can work financial miracles, then the company will buy it. ROI answers the question ‘what’s in it for my business’, and answers it in dollar terms. In the later stages, once KM has become established as good business practice, you won’t need to calculate ROI any more.”

To reach this point, though, several obstacles have to be overcome. As Marcus Blosch, a business modeller at DHL Systems, points out: “The direct effects of knowledge management are extremely difficult to judge.” Milton agrees, citing his experience developing a knowledge management programme at BP as an example: “Often KM is a key element of a number of different approaches, an integral part of the recipe, but not the only ingredient. As one business unit leader at BP said: ‘You are in there with the spaghetti – we can’t isolate the KM component on its own.’ But without KM, the recipe would not have worked. So you use language like ‘KM contributed to an improvement of X’, rather than ‘KM delivered X’.”

Furthermore, knowledge – and hence knowledge management – has no obviously quantifiable value. “A major issue with any KM project is how do you measure the intangible? By its very definition, this question has no answer,” says Paul Flew, managing director of Future Edge. The trick, Flew argues, is to assign knowledge a tangible value. “Sales, marketing, or business process projects have specific goals defined by the globally accepted ‘dollar value’ measurement; knowledge management projects must either use the same measurement or produce a new asset valuation system, by which benefits can be measured.”

Flew suggests one possible method for determining a knowledge dollar value, relating to the job market: “Knowledge is traded at values based on supply and demand; what is yet to exist is a definitive index that gives a value for particular areas of knowledge. However, in the absence of this index, it is possible for a company to implement its own valuation system on a company-wide or project level basis. Project managers may then assign as much project time as deemed appropriate to developing an accurate assessment of project benefits.”

While assigning a monetary value to knowledge is the most obvious approach when calculating ROI, and the one to which company managers will probably relate best, there are limitations with this technique. As Blosch points out: “Commonly the return on investment measure has been purely financial, but this myopic view does not take account of critical aspects of an organisations strategy, such as customer satisfaction, employee motivation, and so on. If the organisation is to be innovative and successful over the longer term, then it must recognise that there are other key elements to this success, and manage them accordingly.”

By way of analogy, Geoff Smith, business development manager for Knowledge Transformation Services at Cap Gemini Ernst & Young, describes a simple scenario: “You’ve just had double-glazing installed, and are now sitting back, thinking about the ‘intangible’ benefits – how smart the new frames look, and how cosy it feels now those niggling draughts have disappeared. The prospect of being able to use rooms that were previously uncomfortable is rather nice, but how do you put a value on this?” Smith goes on to explain that there are several ‘intermediate’ benefits, such as the house being more secure, which are more tangible and easier to measure, though still not large enough to justify the investment on their own. The big payoffs, in terms of reduced energy bills and the increased value of the house, are analogous to major ‘tangible’ knowledge-based performance improvements, and increased shareholder value, benefits that make an investment case command attention.

“In the case of the ‘intangible’ benefits,” says Smith, “I could probably get some objective measure of reduced ‘draughtiness’ in terms of how fast the air changes, but the instrumentation would itself prove rather costly and intrusive. It is better simply to ask – does it feel better? In the same way, we know that a knowledge-sharing culture brings staff satisfaction improvements. To measure this, rather than try to assign a monetary value, why not simply include an appropriate question in the next staff survey? In other words, we need to be careful in our choice of measures, to ensure that the process of quantifying benefits does not in itself become a costly overhead.” Smith maintains that while some benefits are indeed harder to measure than others, a way of proving the ‘upside’ can always be found; “It’s just a matter of knowing where to look, and choosing sensible measures that form a framework that can fully reflect what has been achieved.”

Whatever approach in assigning a quantifiable value to knowledge is taken, each of our contributors highlights the importance of first identifying a baseline reference point from which to proceed. “The post-implementation KM return on investment equation needs to take into account not just the new level of performance, but also what performance levels existed beforehand, and what improvement we expected,” says Smith. “In other words, the key to successful post-implementation measurement is a set of planned objectives, and a baseline of ‘as-is’ performance levels, so we can look at the difference.”

Flew explains how knowledge extraction techniques can be used on a sample of the workforce within the project domain to establish a baseline. “There are a number of methodologies available to ascertain the breadth and depth of knowledge that an employee possesses, not the least of which being the simple structured interview. This allows a ‘knowledge map’ to be created, which, along with knowledge valuations, will be applicable as a baseline for the project, subject to the confidence limits set by the sampling process. This process gives clear indications as to where strengths and weaknesses lie within the organisation, and hence is applicable to projects that are aimed at sharing or creating knowledge within the organisation.”

According to Milton, demonstrating ROI using this approach is relatively straightforward if a knowledge management system is applied at a project level. “You establish the predicted cost/revenue of the project, apply KM, and see what the actual cost/revenue is at the end of the day,” says Milton. “You need some baseline figure so that you can demonstrate improvement. Maybe the baseline is the predicted revenue, or maybe it is how much the project cost last time, or maybe it is a benchmark against similar projects elsewhere.”

Milton describes how BP applied knowledge management practices and processes to the 1998 shutdown of the Nerefco refinery. “Compared to the previous shutdown, four years earlier, Nerefco cut costs by 20 per cent, reduced the time of shutdown by nine days, and increased time to next shutdown by half a year (with no safety incidents and no environmental breaches). Those improvements were worth nearly $10m,” he says.

Closely linked to this idea of identifying a baseline reference point by which to calculate ROI, is the use of a control group to measure the effects of knowledge management, and to assess the difference in performance of a group on a holistic basis. “This is something we have had direct experience of within Cap Gemini,” says Smith. “We monitored a group of salesmen who were given access to a portal designed very specifically to meet their needs, and saw some definite returns in terms of performance.”

And just as it is important to have a reference point for the position your company at the start of a KM programme, it is crucial you identify how you want your business to develop. Blosch emphasises the importance of establishing a clear link between the knowledge management initiatives undertaken and the strategic aims of the organisation. “To avoid making large-scale investment without significant benefit, knowledge management projects need to be carefully scoped and prioritised, and their cause and effect managed,” explains Blosch. “These measures should be in place before any major investment in KM initiatives takes place, which link investment with results in terms of the organisation’s strategy.”

In order to do this the organisation needs to understand the key components involved in the realisation of its corporate plan. Consequently, Blosch recommends that companies adopt a ‘balanced scorecard’ approach, as detailed in Kaplan & Norton (1996). “In essence, the scorecard developed should reflect a rounded view of the organisation’s strategy, and promote customer satisfaction, employee motivation, process performance and profitability,” says Blosch. “Investments in knowledge management initiatives may then be judged on a wider, more balanced framework to the organisation, thus ensuring they contribute to the fulfilment of the organisation’s strategy.”

There are, of course, less complex techniques companies can employ to measure ROI. Flew describes several different approaches, although he emphasises that each should really only be considered for short-term, well-defined projects. The first relies on instinct and perception, but Flew concedes that while this method is popular and very straightforward, it is also extremely unquantifiable. The second depends on time analysis. “This is one of the simplest methods for determining project ROI,” explains Flew. “By measuring the benefits of project deployment in terms of time-savings, the dollar value of the extra time available to perform other tasks can then be calculated.”

Finally, Flew describes a method based on structured referencing procedures, which is well-established in the academic community. “This approach enables you to track the development of papers back through the sources that have been useful. If a KM project’s aims involve the better sharing of information among employees, then a referencing system makes this task quantifiable. To build the idea of knowledge sharing into our valuation system, we need to include an ‘audience index’, which indicates the greater value of the knowledge when available to a much larger audience. Cases have been reported where cost savings running into millions have been directly attributed to making large bodies of knowledge available to just a small number of people.”

Indeed, it is not unreasonable to expect a return on investment in the hundreds of per cent after KM implementation. At BP, although admittedly no major IT investment or cultural change was needed, Milton quotes savings of $260m from a two-year investment of only $5m. “In my experience,” says Smith, “this level of return usually relates to bringing products or services to market faster, reducing time to peak sales, eliminating major areas of duplication, accelerating change programmes that themselves bring major savings, or increasing revenues as a result of objective improvements in product or service quality.”

The potential benefits of initiating a knowledge management programme are certainly attractive. As Milton says: “The behaviours and culture of knowledge sharing will have massive payoff across your entire organisation. Costs will fall, efficiencies will rise, innovations will emerge, new attitudes will be created. Once your KM programme achieves critical mass, the benefits will start to roll in. You need to have faith in this, but faith will not be enough to get your organisation started.” If your business is to persist with knowledge management, the ability to demonstrate ROI, particularly in the earlier stages of KM implementation, is vital. If the challenge is ignored, the danger is that a genuine opportunity to improve your company’s efficiency and, most importantly, profitability, will be wasted. KM

“An investment in knowledge always pays the best interest.” Benjamin Franklin

Marcus Blosch is a business modeller at DHL. He can be contacted at: mblosch@lhr-sys.dhl.com
Paul Flew is Managing Director of Future Edge Ltd. He can be contacted at: prf@futureedge.co.uk’
Nick Milton is the managing director of Knowledge Transformation(SM) International. He can be contacted at: nick_milton@ktransform.com
Geoff Smith is business development manager for Knowledge Transformation Services at Cap Gemini Ernst & Young. He can be contacted at: Geoff.Smith@capgemini.co.uk