denotes premium content | Oct 15 2008 Regular
posted 6 Dec 2005 in Volume 9 Issue 4
Conducting a knowledge audit: Part II
In the first part of this masterclass we looked at some of the issues involved in starting a knowledge audit; the need to establish a project, the advantages of mapping out the tangible and intangible benefits, the value of thinking about what outcomes the organisation wants before the project starts; establishing some common definitions of key terms and analysing what the purpose of the knowledge audit is.
The purpose of the audit is to uncover what knowledge the organisation has and what knowledge the organisation needs to pursue its aims and objectives. It can then be used to identify what gaps there might be and what improvements are needed. This link between the audit and recommendations and improvements is important and we will return to this in the third and final part of this series next month. This article, however, will consider what questions to ask and how to produce some answers.
Defining the knowledge audit
If in doubt, ask questions. Producing a list of questions at the outset is a good way to define what the audit is actually going to cover. Consideration can then be given as to how answers will be produced. The questions need to reflect the culture and the orientation of the organisation, but they could include some or all of the following:
· What knowledge does the organisation need to support its business?
· Where is that knowledge in the organisation?
· How does that knowledge flow around the organisation?
· How is that knowledge captured, stored and exchanged?
· How is that knowledge made visible?
· How do people keep their knowledge up to date?
· How is knowledge defined in the organisation?
· How is knowledge created in the organisation?
If the knowledge audit is going to inquire into the relationships between staff within the organisation and between the organisation and external agencies, then phrase questions to cover this, too. The audit can examine either the explicit or tacit knowledge of the organisation, or both. The explicit knowledge is contained in documents, websites, e-mails, databases, notebooks and so on. Coverage of this could include volumes, storage, types, formats and so on.
The tacit knowledge is held in people’s heads and extracting this – if that’s the right phrase – and making sense of it is about the interactions between people, processes, existing knowledge, information systems, environment and context. Introducing ‘live’ people into this equation, rather than inert objects like computers, introduces an interplay of many unusual factors. Tacit knowledge is often seen as a personal resource by people, which means their emotions will be involved. This dimension must be respected.
The tacit knowledge that people have only becomes visible to others if they write it down or articulate it. Sometimes it can be hard to clearly identify a process, even though it is done on a highly frequent basis. Care and patience is therefore needed for this work. A great deal of this tacit knowledge may never be shared; it is what someone uses to do their job. But what if they move on? Or what happens if the organisation wants to assess the quality of that knowledge? And how is that knowledge updated? Are people using organisational knowledge to resolve issues and respond to questions or their personal opinions? The latter may carry risk and wider implications.
Who are the knowledge auditors?
Carrying out a knowledge audit is not rocket science (and rocket science itself is not that difficult). It is certainly within the remit of able staff. Ideally, knowledge audits should take place on a regular basis so it is definitely worth considering the development of in-house expertise.
Let’s presume that the audit will be done in-house, by a small group of people who are already working in some field of information or knowledge management. As well as their professional status and qualifications, they should ideally have an excellent set of social skills. After all, real people are going to be involved. To be really effective, the auditors should be regarded as independent, if possible, and certainly drawn from senior management. The auditors need to work like detectives, not spies. Smart organisations are the ones that empower staff, not stifle them. Do not stereotype or categorise people or ignore anyone. Often the most unlikely people will step forward and offer the most trenchant observations about good or bad practice or about a continuing problem which needs to be solved.
Most organisations are class-based hierarchies and there are many subtle and not so subtle differences between those who work on the front line for low wages and those who work at the top for high wages. This does not mean the front-line staff lack knowledge. They have lots of it. But if the audit is going to determine the nature of this, how it is used and what can be done to make improvements, then the audit needs to establish a lot of trust and good will. If that can be done, then there are potentially big benefits for the organisation as whole.
There will be no trust if the audit is perceived as the basis of a cost-cutting exercise or as a disguise to push through unpopular reforms. Fight to keep these type of organisational politics away from the purpose and the implementation of the audit.
The audit team will set the tone of the project. It needs to think about short, sharp productive interactions with people. It needs to consider the medium-term goals and the long-term implications. It needs to create a project name that captures people’s imagination, not the ‘annual knowledge audit to ask you what you do or don’t know’. A project with a slogan such as, ‘Know where? Or Nowhere?’ ought to generate interest because people will want to know more about what this could possibly mean. Create a buzz round the project and generate curiosity. Most people cannot resist the latter.
Methods, questionnaires and surveys
Once the key questions have been established, a methodology is required in order to provide some answers. The methodology is a set of actions that will be followed to collect the evidence, facts, opinions and ideas. The opinions and ideas are useful because they will help shape the recommendations. There are different ways to collect the evidence, including questionnaires, surveys and interviews, of course. These can be both formal and informal, individual or in groups. Workshops can be useful, too, as can informal ‘knowledge cafes’ where people are encouraged to come and talk in a more relaxed, and perhaps non-work environment. Let’s look at some of these techniques in more detail.
We’ve all completed questionnaires and surveys; the tantalising short ones that offer exotic holidays; the incomprehensible ones from the tax people; the infuriatingly complex ones when all we want is a basic service; the ones that feel like they will never end; the ones we ditch because they become too personal. Yes, the design of questionnaires matters.
There are two main reasons for using questionnaires. First, to answer questions about the use of explicit knowledge and the use of tacit knowledge in the organisation and, second, to analyse how this supports the core business. Mapping out the use of explicit knowledge, where it is stored, the volumes, the retrieval tools and so on can be complicated and can unearth all sorts of parallel systems within the organisation, alternative communications channels, deep silos, multiple databases with the same data (or rather, with slight variations of the same data).
Making enquiries about tacit knowledge needs to be done sensitively and with skill. To re-iterate a point made above, asking people ‘what tacit knowledge do you have?’ is not going to work in most cases. Asking people about their job, the good parts, the bad parts, the frustrations, the triumphs, the things they like, the things they do not like, what grievances they have and so on is going to produce richer veins of knowledge about their jobs, which should be vigorously mined.
Many people will not be familiar with the terms ‘knowledge audit’ or ‘knowledge management’ despite the fact that they might have been successfully doing the latter for a long time. Approach the respondents in the natural language of work social relations, not the jargon foisted on the world of work by multi-million software companies.
There are different types of questionnaires, which can be structured, semi-structured and unstructured. In this modern age we live in, questionnaires can be as easily constructed electronically and added to the corporate intranet.
A structured questionnaire has a clear set of questions, usually with yes or no answers or multiple-choice options. These are the type of questionnaires you get in market research. They are useful in producing quantitative data but they could have a place in the audit, particularly in looking at sources of explicit knowledge such as the number of databases, the frequency with which people use key systems like the internet, the number of times per day, on average, that people cannot find what they are looking for. They can include scoring systems where, for example, one is very bad and ten is very good.
Semi-structured questionnaires contain answers and options, but also free text boxes where the respondent is invited to add some comments of their own. For example, you might want to ask people things such as, ‘What are the three best things about working here?’, ‘What are the three worst things about working here?’, and one of my personal favourites, ‘How do you think things could be improved?’.
Unstructured questionnaires can be harder to carry out, but they are potentially more rewarding as they can be used to explore issues and may raise issues that had not been considered by the organisation. However, if an internal team is being used with no experience of interviewing and questionnaire design this may add an unnecessary layer of complexity.
A semi-structured questionnaire that enables inquiries about the use of both tacit and explicit knowledge may be the best. This could be used to determine activities such as the quantitative use of the corporate intranet – how often do staff use it, what sections do they use? It can also be used for the more qualitative analysis – what do staff think about it? How does it help, or not help them in their work?
Questionnaires should always be piloted to ensure they work in an interview setting. Even doing this with just two or three people will make a huge difference in spotting any questions which simply do not make sense or that are too cumbersome to answer. Avoid double questions as they are harder to deal with.
They should being with easy questions that people can answer – name, rank and number is a good start. Then progress to the harder and more complex questions.
If questionnaires are going to be filled in by staff on their own, think about the numbers that will need to be distributed, how this will be done, whether it will be paper or electronic and what resources will be available for collation and analysis. Is a blanket coverage wanted, a random sample or a weighted sample?
Interview time
Interviews can be very valuable in capturing experience and in giving insights as to how processes actually happen in the organisation, rather than how they should happen. Interviews can be carried out using any questionnaires that have already been created (see above), or they can use different questions because they may seek to find out more detail about activities and process.
Interviews can be conducted at both a formal and an informal level. Before marching off with a clipboard and pen, mull over a few considerations: How many people are going to be interviewed? At what level in the organisation should they be? What are they going to be asked? How long are the interviews going to last? Interviews can generate a lot of content so make sure there are enough resources to collate and analyse all of the outputs.
The audit leader may want to take a ‘wide and shallow’ approach in which a large number of people go through a short interview to determine some general points and issues. Or it could be a narrow and deep approach where detailed interviews are carried out with a specific team. This could include everyone from the most senior person to the most junior. Indeed, it can produce fascinating results as a multi-faceted view emerges based on how different players in the same process view both the different component parts and the totality of their collective actions.
Interviews should take place somewhere without interruptions, without builders drilling through the floor above – it is not unknown – and where people can feel reasonably relaxed.
Explain to the interviewee what the interview is going to be about, how long it will take, what follow-ups will occur, what the purpose of the knowledge audit and the interview is and what the outcomes will be – for example, a written report with recommendations to develop a knowledge management strategy. The interviewer may offer to write up the notes of the interview in draft format and send to the interviewee for comment and possible correction.
This is a useful device to ensure that final reports are accurate and often, people will send back additional comments that they didn’t think about during the interview.
Do not be shy about asking questions. I have worked with at least three technical geniuses at different times. It has always been of great interest to me that they would never hesitate to ask the most basic question if they were unsure about something. They were never ever afraid to show their ignorance where they had any, despite their obvious, impressive knowledge and deserved reputations.
There is also much to be learned at the end of a formal interview when the respondent relaxes. It can be extraordinary how two minutes of conversation at the end can be more illuminating than an hour of formal questioning. If this happens, start making notes again as soon as possible so any points are not forgotten.
However, there is also a great deal of validity in carrying out informal interviews and you may find this works better with some people and in some contexts. This is not a scientific research study; the audit is being carried out to improve knowledge management across an organisation. I personally think are some things which people say which instinctively feel right; these points should be recorded.
Workshops
Workshops should be just that. An event in which to do some work. Tastes differ. I generally dislike role playing and I do not like workshops that are nothing but lots of power points and a wise guy presenter.
People who are invited to a workshop should be given a clear agenda, times, an outline of what the organisers will do, a list of what the participants are expected to do beforehand. If the workshop is part of an knowledge audit, it is being used to answer some of the original questions that the audit seeks to answer, using group dynamics.
Workshops can be equally successful by lasting a day, half a day or an hour or two, but whatever the length they need a clear focus, even if that clear focus is to explore fuzzy ideas and hazy activities. A good facilitator who understands the importance of their role will help. They have a duty to the rest of the group to ensure everything runs to time, that the loud voices do not dominate everything and that more shy people are encouraged to speak.
The workshop may want to explore the relation of knowledge to processes in the sense of ‘how do you do a particular activity’ and examine this in detail. It can be extremely illuminating for people to map out a process and then declare, ‘we don’t know why we do that bit, or this bit or that other bit over there’. That is a key component of knowledge auditing; finding out how the knowledge, or lack of knowledge, actually impacts on the day-to-day processes in the organisation.
Desktop research
Much can be learned about knowledge management within an organisation by trying to use those workhorses of the electronic office, the internet and intranet. Is retrieval easy? Are they supported by glossaries, indexes, taxonomies? Are there frequently asked questions?
Can the intranet be used to discover whether there has been any previous knowledge-auditing work? Does it have references to knowledge management? These questions in themselves highlight the maturity or otherwise of knowledge management in the organisation. Knowledge can be usefully captured in case studies, articles staff have written and briefing notes. Do these appear and are they easy to find? Is there any record of how failure is recorded?
Take this real scenario: we had to try to identify all of the projects within an organisation over a three-year period. About 300 were discovered. Of those, 160 had been completed and the documentation was there for all to behold. There were 80 ongoing. And the rest? No one had a clue. They had started, staggered and fizzled out. But why? There must something useful to be learned from this?
Desktop research can be used to find out what skills people have. It is a good question to ask and if the auditors can not find it out from their desktops, then the odds are that no-one else in the organisation can either. That fact alone should already be formulating itself into a recommendation for the final report.
Knowledge café
On a Friday afternoon, people are starting to wind down at the end of another busy week. Let’s be honest, many are gently pushing difficult and complex tasks into Monday morning; there is a more relaxed atmosphere in many offices, but people are still working and expected to be working. That is a good time to have an informal knowledge cafe. Find a comfortable room (a training room, for example), fill up on some light refreshments and snacks and issue a general invite to people to come and spend ten minutes with a coffee and cake to talk about how they find things out at work.
Be amazed, be very amazed. The last time I did this, all sorts of people came calling. People who did not know you could add attachments to e-mails, who didn’t know you could add quotation marks to search engine queries, who didn’t know about a key library of resources – I am not making this up. In this case, (partly because of the way we structured it) we found out a huge amount about what people did not know – but surely that’s as important as what they do know? It provided some succinct examples for the final presentations.
Observation
Waiting in the entrance of a busy office during an assignment, I sat and watched the two reception staff deal with an endless stream of people. Delivery men with boxes, VIPs coming for meetings with senior staff, an angry customer, some polite customers, 30 people from another office who had arrived for a training day, someone who had wandered in off the street who wanted to know where to pay their electricity bill.
In a lull in this avalanche of people with their endless demands I went over for a chat, during which I asked if they used the intranet (remember, a potential key knowledge management system) to help them. They both laughed, replying, ‘we don’t have time to use that, and anyway it’s too slow and no one can find anything’. They were both using the tacit knowledge they had accumulated. It was clear from observation that they constantly made rapid decisions about who could be dealt with there and then to get them on their way, who would have to be referred to someone else, who was a priority. In this small observation and seemingly casual question a lot is revealed about how knowledge is being used in the process of delivering services – in this case a very high profile part of the organisation.
Now here is a real nugget of knowledge use (in this case two reception staff). This could be the basis of much more exploration for the audit. If they do not use the intranet (part of the official knowledge-management system), what exactly do they use? How do they keep up-to-date with the changing people, meetings and activities within the building? What do they do if they do not know the answer to something? It is a minor illustration, but it is important; doing a knowledge audit can be about using small examples to get much bigger insights into what is exactly happening in the organisation.
Who likes dealing with organisations where the reception staff are polite, knowledgeable and on the ball? And who likes getting somewhere on time, only to be made late for a meeting because no-one can be bothered to sort you out?
Lessons learnt
The knowledge audit should start by identifying what questions it wants to address and answer. A set of techniques such as questionnaires, interviews, workshops and so on can then be identified. These can be combined and used in different ways to gain different insights, to look at the same questions from different angles and to help create buy in and awareness of the general project.
In the third and final part of this master-class, we will look at some of the important issues surrounding the audit, such as how to measure its success and what can be learned from the experience of the project itself. The outcomes and presentation will be considered, along with the implications of the audit and how the findings can be turned into recommendations and implemented.
Danny Budzak is director of The Information Design Company, an information and knowledge-management consultancy.
