Quality control

Daragh O Brien describes a programme of information quality change at US telecoms provider TeleTech Services

Tele-Tech Services is a small data provider serving the US telecommunications industry. It provides a niche but highly valuable service in telecom billing and rate master data to a wide range of larger telecommunications service providers. That data is used to calculate customer bills for calls made between states in the US and between operators.

As Kimberly Russo, its co-president says in the case study the organisation has published on its website:1 “While we are a micro company by telecom industry standards, the biggest players have come to rely on us. Our data is used to determine billing on billions of landline and wireless calls each month. A small error becomes enormous in the robust environment in which our data resides.”

The journey begins
The organisation’s information quality journey began in 2002 when Russo began tracking error rates in the data that Tele-Tech Services provides to its customers to see if they could actually measure accuracy rather than just claiming it. The initial findings were good, with accuracy rates (calculated from the number of errors detected by the quality control process divided by the number of files) running at 99.75 per cent.

However, Russo soon discovered that not all the data needed to get a full picture of their true levels of quality was being captured.

For example, there was no metric to track errors which had been missed by the quality control process. This happened so infrequently it was not tracked.

So, a tracking process was put in place to record these reworks originating from customer reporting, with the error rates being adjusted in the month the error was reported.

By the end of 2003, they had established a defect rate that Russo was proud of and wanted to make part of the sales proposition to customers by way of a SLA.

Driver for the information quality project
This is where their real information quality journey began as address a number of key issues had to be addressed:

Verifying the accuracy and validity of quality statistics (after all, if they were to be the basis of a legally enforceable SLA they would have to be able to stand over their statistics);
Identifying the data that would be covered by the guarantee; and
Finding out whether customers really wanted an SLA.

Ultimately, the key driver was that if the organisation was going to offer an SLA (and potentially give money back to customers for errors) they would need to be approaching the challenge in the right way from the very beginning. While the metrics and data they had collated were impressive they were likely not adequate for the new purpose that was being proposed.

In addition to the key resulting outcomes identified in Figure 1, Tele-Tech also identified that the freedom from errors in data would help clients save money through reduced customer complaints, billing disputes or refunds. Also, once it had engaged the services of Dr. Tom Redman it became clear that the SLA that customers wanted was not a refund of fees paid if there was an error, but rather “customers just want the data to be right the first time”. At that point the SLA focus for Tele-Tech shifted from penalising themselves when errors arose to rewarding customers for finding errors that they had missed.

Support and ownership
The information quality initiative was driven from the top in Tele-Tech Services. Russo had a passion and interest in making quality a verifiable and validated key sales proposition for Tele-Tech that started the process. Given the size of Tele-Tech Services, it had a reasonably flat organisational structure to deal with. This allowed ownership and governance to be driven from the centre in a top-down fashion.

However, the approach taken to rolling out the culture change and philosophy of the big question was one that ensured that the entire organisation had a sense of ownership and responsibility for the quality of the information that was produced. While it is a small family-owned company, the ability to link the information quality mission to measurable commercial metrics (for example, increased sales and improved customer retention) helped keep the senior management team bought into the process and ensured that everyone was positively enforcing and driving the change from the top down, as opposed to a lone champion.

In addition, it took a company-wide approach to defining and maintaining a culture of quality. This was not an initiative that was confined to the IT department or a quality department. Through the judicious use of goals and incentives and a general sense of fun around the challenge, Tele-Tech has embedded a strong quality culture in the organisation.

That quality culture translates into ownership by staff of customer issues and of staff responsibility and accountability for errors, their detection, their analysis; and their remediation and prevention. All of this appears to be done in a way that does not spook people with the seriousness of the task, but allows strong messages to be communicated to instil a strong sense of ownership.

Finally, Tele-Tech uses the support of its customers to reinforce the message both internally and externally by publishing the quality statistics (and clearly defining what they mean); and by making extensive use of customer feedback in its marketing and internal messaging.

Methodology and tools
Tele-Tech Services enlisted the help and guidance of Dr. Thomas C. Redman to help it with its quality journey. In doing so, it opted to implement a formal and rigorous data quality methodology that is effective and flexible.

The first recommendation from Dr. Redman was that to achieve the objectives it needed to apply quality standards that exceeded Six Sigma. To that end, Tele-Tech began reporting the number of files with an error as a percentage of the total number of files produced. If there was any error in a file, the entire file was considered defective.

Redman’s second recommendation was to cease the practice of reporting errors identified by customers in the month they were discovered. It was more valid to report the errors in the month that the error was actually created. This meant that the reporting of errors was linked to the analysis of root cause and also created a moving target for quality that was a more valid statistical model to represent their accuracy.

It would not be correct to attribute defective records found in January 2008 to that month if the underlying error had entered the system in April 2006. No amount of analysis of January 2008 would help you find the root cause of the errors and, as a result, the organisation would not be able to eliminate the special causes of bad months through increased controls, root cause analysis, process changes, staff training, and so on.

People versus technology
As a small company, Tele-Tech Services was not in a position to invest heavily in the information quality tools which were available in the market during the early days of its data quality journey. As a result, it focussed on what I refer to as the human factor.

According to Stephanie Fetchen, another co-president of Tele-Tech, in an interview with DataQualityPro.com in 2008:2 “We focused on implementing personnel training programmes but rejected the use of additional quality tools. It doesn’t have to be that complicated, our focus is now on prevention at the point of entry as opposed to downstream cleansing and improvement.”

This is a strong counterpoint to the approach of investing in the data quality toolset first before addressing the strategic drivers and cultural issues, which runs the risk of the scope and approach to addressing data quality being driven in part, by the capabilities and limitations of a software tool.

The focus on improving the ability and capability of knowledge workers to do their job better is in line with the established best practices in quality management. Both W. Edwards Deming and Joseph Juran stressed the need to invest in appropriate training for staff. Juran, in particular, stressed the need to change management behaviour through quality awareness and training. Deming, for his part, mentions the importance of knowledge and training in both point six and point 13 of his famous 14 Points for Transformation.3 The team in Tele-Tech have also implemented measurement and reward systems for both their staff and for their customers to help track and prevent errors.

For staff, the incentive system is simple, but is tied directly to the top-line strategic quality goals. According to Fetchen: “With the scheme we award a bonus if one of our staff reaches a data quality goal. We try and tie their goals to the overall company goal. So if our corporate goal was a 50 per cent reduction in defects we would make the individuals goal to be cutting their own recorded defects in half. This motivates the team as no-one wants to see a month where defects occur, it definitely helps the entire team to keep the figures up.”

Given what we have learned about the long-term sustainability of extrinsic motivators, it is important to step back and look at the broader methodology that is used by Tele-Tech to sustain the quality culture. The target and reward structure is used in the context of a broader cultural message which establishes strong intrinsic motivators for staff to avoid letting errors through the quality processes. While the serenity prayer might appear twee and light-hearted, it communicates a strong value message which is then supported by the extrinsic motivators of rewards for hitting headline goals.

Russo advises: “Have fun with what you’ve done! It’s amazing how much a little laughter can do! Working with such a focus on accuracy can be a bit demanding – but being able to find the levity in a situation – or to enjoy your teammates and your work environment makes a huge difference. We recognise that and we try to encourage a sense of fun in what we accomplish together.”4 The key message that is communicated from the top, is that the business lives or dies on the quality of the data that they deliver, but that they will make it fun to achieve that quality.

With regard to its customers, Tele-Tech has an equally irreverent but effective approach to getting customers to report errors to it. A simple email address and a reward to the customer if the data does turn out to be flawed (a box of cookies from a local bakery) is the mechanism. This is the extrinsic motivator for the customer, with additional motivations being that by alerting Tele-Tech, the error will get investigated and will be fixed if it is an error or the underlying root cause for any misunderstanding will be documented and explained. As Dr. Redman pointed out, the customer doesn’t want a refund or a rebate if the product is defective, they want a product that works first time.

Prioritising higher value/higher impact data sets
Tele-Tech Services deals with one basic set of data – telecommunications tariff data. Therefore, the priority was essentially set by the nature of the business. It prioritised the importance of accuracy of the data.

More importantly, it has been focused on ensuring that it clearly define what accuracy means as a dimension of quality in its business and, for that matter, what quality is. Accuracy is defined as no defects in any files sent, changed, or investigated for change in a given month, with the defects being either detected internally or by customer reports.

By keeping it simple, Tele-Tech has been able to consistently publish information about its quality scores that are easily understood by customers. In its own quality story, Russo compares how other companies “come up with secret formulas to quantify quality” that “actually compromises quality”, with the simple approach taken by Tele-Tech.

Its prioritisation of accuracy as the key dimension for measuring its quality comes from almost two decades of market research with clients which emphasises the critical importance of accuracy to them. Accuracy drives the ability of Tele-Tech’s customers to generate revenue, avoid billing disputes and so on.

Process improvement to prevent problems recurring
To an extent, not having a traditional data quality tool to fall back on has meant that Tele-Tech Services is like a trapeze artist flying without a net – the execution of its processes has to be flawless or a less than desirable outcome will arise. To that end, there is a fanatical focus on process improvement to prevent errors. This in keeping with the structured methodology that Redman defines.

Examples of the process and continuous improvement approaches applied by Tele-Tech include conducting a post-mortem of errors to understand cause, effect and how to prevent them, seeking ways to automate control processes to further eliminate human error, and the elimination of tariff interpretation errors through training. This last process involved:

Weekly training sessions for local researchers to explore exceptions, new tariffs, unique circumstances and business rules, as well as tips and techniques discovered during on-going data maintenance;
Biannual review of the procedures manual to keep procedures accurate and up to date across 100 separate processes; and
Mock calling exercises, where researchers are presented with data files that have had errors inserted into them, which need to be identified (a data quality fire-drill).

Embedding and sustaining the information quality change
Tele-Tech is an excellent example of how a data quality change can (and perhaps should) be implemented. Regular internal and external reporting on the quality levels being achieved combined with an almost obsessive attention to the value of accuracy means it is impossible for anyone in the team in Tele-Tech not to know that this thing is of critical importance to the organisation and its clients.

Directly linking personal goals to the goals of the organisation and putting the accuracy concept into clearly understandable terms means that it is easy for staff to internalise the values and objectives of high-quality information and data.

The existence of a clear governance framework (albeit a by-product of the flat organisation structure and family-business model) makes it easy for everyone to understand their role in the quality equation. There are clear linkages between the quality objectives and the strategic priorities of Tele-Tech. Tele-Tech takes this one step further and articulates its quality objectives in the context of their impact and the value to the end customer, the billing telecommunications company.

Whether by accident or design, Tele-Tech has defined and executed a fairly rigorous value delivery system to underpin its information quality strategy. Not only has it identified how it can affect and influence the accuracy of the data it produces in a manner that is statistically valid, but it has linked that to both a clear set of resulting outcomes for it and its customers and a well-defined plan for communicating that value proposition to target customers.

In her interview with DataQualityPro.com, Stephanie Fetchen sums up the results of that value delivery system as follows: “We know that we have won companies from our competitors because of our data quality. Companies have also measured our data against that of our competitors and chosen us. Reducing customer churn and attracting new clients was one of our main goals and we have certainly been successful in achieving this.”5

While the non-adoption of formal data quality tools might appear to be counter-intuitive, it has served to put the focus on the human factor in information quality. My view is that were Tele-Tech to invest in a tool today, it would have a significantly greater chance of long-term sustainable success and return on that investment than if it had opted to invest in such software back in 2002. By focussing on the human factor, Tele-Tech has had to deal directly with issues of process design, human error, training and culture. As a result this strategic change has never run the risk of being seen as an IT Project. This has certainly helped to ensure buy-in and sustainable success.

The approach taken by Fetchen and Russo in documenting and communicating this story makes for exceptional sales and marketing collateral, but it also acts as a strong anchor point to drive springboard stories and other narratives in the organisation as part of sustaining the culture. This, combined with the use of social media and transparent publication of all quality statistics, even if they do not meet their targets, serves to further enhance the narrative power of the culture change.

Conclusion
To a great extent, Tele-Tech Services is a benchmark-worthy implementation of an effective information quality strategy.

It clearly defined the value proposition and the delivery system. It bolstered the best efforts of its internal team with external expertise and a structured methodology, which resulted in a radically different and more rigorous approach being adopted. It focussed on the human factors as part of a framework of continuous improvement and process development.

Ultimately, it proved that significant improvements can be achieved without the need for expensive tools. In my view, this is an extremely important lesson as it highlights the need for organisations to have their information quality vision and strategy defined from the perspective of the human factors necessary to drive the change before investing in tools. This will only serve to enhance the return on investment from such tools in the long-term and will lead to a much more sustainable and successful information quality change in your organisation.

It may be tempting to say that the Tele-Tech story is only relevant to small to medium enterprises. However, as a template for how teams within larger organisations can start to build their information quality strategy and practice, there are valuable roadmap lessons that can be learned from this story:

Define the resulting experiences that your customers (internal or external) will get from an information quality initiative;
Obsessively focus on linking your goals and delivery to high-level strategic goals;
Clearly define what you are measuring, how and why;
Adopt a clear m ethodology early on to avoid pitfalls. If you look like you are making it up as you go along you won’t be taken seriously;
Be ready to challenge your own assumptions and change your focus to better meet the needs of your customers;
Focus on human factors of knowledge, capability, understanding and suitability of processes before investing in tools; and
Don’t take it too seriously.

By modelling the success factors from the Tele-Tech story, fledgling information quality teams should ideally be able to achieve a point where they can justify investment in tools to support increased demand for their services or increased volume of data to be processed. But those services and the methodology for processing that data will already have been proven and battle tested.

This article is an extract of The Data Strategy and Governance Toolkit, written by Daragh O Brien and published by Ark. For more information or to obtain a copy, please contact Robyn Macé at rmace@ark-group.com

References

Russo, K., The Tele-Tech Services Quality Story, Part 1: A journey in the discovery of the Good, the Bad, and the Big Q, Tele-tech Services, for further information on Part 1 visit: http://bit.ly/dSWKgV, for further information on Part 2 visit: http://bit.ly/i3j9yS, last accessed 15 April 2011.
Jones, D., ‘Going beyond Six Sigma: How KFR Services Inc., took their data to a whole new level of data quality’, DataQualityPro.com, 17 September 2008, for further information visit: http://bit.ly/mpmrAS, last accessed 20 April 2011.
Beckford, J., Quality, 2nd edition, Routledge, 2002.
Russo, K., The Tele-Tech Services Quality Story, Part 2: Focus on the Big Q Results, and No More Bad Months, Tele-tech Services, for further information on Part 1 visit: http://bit.ly/dSWKgV, for further information on Part 2 visit: http://bit.ly/i3j9yS, last accessed 15 April 2011.
Ibid.