Jerry Ash | Measured opinion

Metrics don’t kill knowledge management projects; people do.

Some KM advocates are becoming less timid and more testy when confronted with the perpetual challenge to provide metrics to prove the value of knowledge management. During a recent dialogue among KM thought leaders and practitioners hosted by the Association of Knowledgework, Hubert Saint-Onge, developer of the Knowledge Assets Framework, started an avalanche of passion when he recalled a key moment from his career:

“When I was senior vice president of strategic capability at [insurance company] Clarica, I had to present my business plan to the CEO on a quarterly basis. In addition to knowledge management and learning, my portfolio included strategic planning, internal and external communication, human resources and corporate branding – in other words, the full basket of intangibles.

“The chief financial officer attended these meetings and kept bringing up the measurement question. I was always able to side-step the issue. One day, he became more vociferous than usual on what he said was (his words) ‘the need to measure all this crap’. “Luckily, I had many opportunities to practice an answer. I said that I admired his passion for measuring and that I would like to take his lead.

I promised right there that if he would share with me the process he was using to measure how his organisation’s finance and actuarial work was adding value to the company, I would right away adopt and apply this framework to the activities for which I was responsible. “He looked at me dumb-founded. It had never occurred to him that he should measure what was considered a conventional activity in the company. In his mind it was that new so-called ‘crap’ that we needed to measure.

“Isn’t it interesting that we put the onus of measurement on what is new when we have pile upon pile [of activities] we don’t measure because it is already accepted practice? As all of us who have worked on this for decades know, measuring the impact of growing intangible assets on the bottom line is not easy. It certainly cannot be trivialised because sometimes having wrong answers is worse than having no answer at all.

“I fully subscribe to the need to measure all that can be measured, but I don’t believe that you can only manage what you can measure. As a matter of fact, I find this one of the most mindless dictums ever to be uttered by people who appear to be otherwise reasonably coherent.”

Peter Marshall of Helix Commerce joined in. “One can only manage what one cannot measure,” he said, reasoning that if something can be measured, then it can be reduced to a control procedure and, “does not require management or intelligence.”

In his opinion that is the domain of IT – to capture and codify the routine, which can be either automated or outsourced. Knowledge, on the other hand, cannot be measured, automated or outsourced. He blamed the measurement problem on the illusion of the metrics-minded – a dashboard displaying every key indicator a manager needs to navigate the business.

Saint-Onge concurred, saying that business leaders err when they use measurement as they would use a machine to tell them what to do. In Saint-Onge’s view, managers often like to find a formula that lets them off the hook when it comes to the hard work and vulnerability of having to exercise judgment.

Also, Saint-Onge noted that measurement does not always convince. “Senior managers who don’t believe in KM ask for measurement because it is a safer political approach than questioning the initiatives directly,” he said. “When you do the numbers, they find fault with them and do not accept them at face value.”

One painful experience behind Saint-Onge’s point of view was the occasion when he had to cancel a $2.5 million programme in which he could point to a 175 per cent return on investment. In the end, the programme was nixed, quite simply, because “they did not like it”.

Marshall, likewise, criticised the dashboard concept. An automobile dashboard hardly navigates the car, he said. “The dashboard provides some useful metrics that the car’s operator occasionally consults – engine too hot, rev’s per minute approaching the red line, speed over the limit and so on,” Marshall said. “But only the driver knows where, why or when to drive the car. And the driver uses all kinds of tools and skills to actually do the driving that are not captured on the dashboard.”

Scott Shaffar, Northrop Grumman’s director of knowledge management, took the middle ground. He pointed out that working out metrics has a price. He engages in measurement only when he feels it is necessary to gain and maintain support. “After ten years I am constantly in the role of salesman and if a key leader tells a peer that there is value in KM, I get more work. The best measure I have experienced is personal testimony from those involved in valueadded work.”

That prompted an equally positive story from Saint-Onge. “One day when I presented to the CEO, I included in my budget a measurement initiative. He asked why I wanted so badly to spend such an amount on measuring instead of focusing on moving the work ahead?

My answer was that I wanted ammunition to counter the critics in the organisation. He then told me that it was not necessary to make this kind of investment, that I should proceed with the work at full speed and that he would provide the ‘air cover’. In time, my colleagues on the senior management team became very supportive and the CEO provided consistent support for the following four years.”

The group went on to discuss the positive reasons for measurement, even in KM, and they concluded it wasn’t measurement per se that was the problem. Rather, it was the motives, purposes and viability at issue. It made me think of a bumper sticker common in the US: Guns don’t kill, people do.

Hubert Saint-Onge is principal of the SaintOnge Alliance, based in Waterloo, Ontario in Canada. He can be contacted via the organisation’s website: www.saintongealliance.com.

Peter Marshall is a partner at Helix Commerce, based in Irvine, California. Likewise, he can be contacted via his web site: www.helixcommerce.com.

Scott Shaffar is director of knowledge management at defence contractor Northrop Grummann, based in El Segundo, California – www.northropgrumman.com.

Jerry Ash is KM coach, founder of AOK, www.kwork.org, and special correspondent to Inside Knowledge. He is also the author of the Ark Group’s latest major report, Next Generation Knowledge Management. To order, please contact Adam Scrimshire at publishing@marketing.ark-group.com. Jerry Ash can be reached at jash@kwork.org.

To engage in e-mail and online conversations with many of the world’s leading KM thought leaders and practitioners, join AOK at: www.kwork.org/explain_join.html.