Data quality moves centrestage

The example set by telecoms giant BT reflects how data quality is being taken more seriously by major organisations – and they are reaping the benefits in terms of better customer communications and big efficiency savings.

By Graeme Burton

Just ten years ago, telecoms giant BT was grappling with a problem that will be familiar to all organisations of a similar size. It didn’t have just one customer database, it had hundreds – some 400 running in a total of about 700 systems – and scores of different IT teams to look after them.

The reason for such apparent profligacy was not inefficiency, but merely reflected the way that the company – and its IT systems – had grown over the years. Different units in BT had different needs, of course, and one system on its own would have been damagingly restrictive.

By 1997, the shortcomings were becoming apparent, especially in marketing where the fragmented nature of customer information was starting to cause embarrassment. With customer information held on 400 different systems, it was impossible for marketing to have a complete view of every product and service that a customer might be purchasing from BT.

But it wasn’t just the frequency with which customers were marketed products that they already had, nor the fact that 15 per cent of all mailings did not even reach their intended recipients because many of the customer databases were also out of date.

What, perhaps, was most embarrassing was the number of marketing mailings that the telecoms giant was sending to telephone boxes – even telephone kiosks on trains – because it could not easily distinguish between the various different ‘nodes’ on its network. “We couldn’t differentiate between a telephone box, a traffic light and a customer at that time,” concedes Nigel Turner, manager of the information and knowledge-management consultancy at BT.

Turner was brought in at a time when the organisation was starting to get to grips with the hundreds of different applications it was running, many of them in-house developed, with a view to shifting to off-the-shelf packages. They would not just be cheaper to run, but would, where necessary, provide enterprise-wide coverage.

The data-quality problems, meanwhile, were not simply down to a lackadaisical attitude. Far from it. “We seemed to be doing a lot of data cleansing all over the place, rather than taking a proactive approach,” says Turner. “The problem with data cleansing is that it’s like painting the Forth Bridge. You start at one end and by the time you have finished at the other end you have to start again.”

The data-cleansing activities that were taking place were being done in silos and they were not using common methods or toolsets either. This would have to change, with data quality becoming a central activity with the power to influence the way in which business processes were implemented and run.

Strategy

Turner believed that data quality was a business challenge and one that had to be addressed in business terms. “Information is first and foremost a business asset, so we had to do two things: to organise the business and the IT department accordingly,” he says.

The data-quality problem was also recognised by BT’s then chief information officer (CIO) Lizzie Beesley, who provided strong support. Beesley supported the argument that data quality ought to be a centralised, strategic activity. “We laid down some very simple rules, such as we would not do data-cleansing activities unless we were confident that the data we would be cleansing could be kept clean. Otherwise, we would have to do it over and over again,” says Turner.

Business discipline also demanded that Turner’s activities could be supported from a financial perspective. “When we are asked, ‘how did you measure data-quality improvement?’ The answer is that we measured it in money and how much had been saved for the company.”

That is to say, when different data-quality initiatives were being weighed up, Turner considered first and foremost how much money each could either save or generate BT.

Turner therefore started with the construction of a customer-data warehouse for the marketing department. Cutting the number of erroneous mailings in BT’s marketing activities would provide a double bonus to the bottom line, cutting the number of mailings sent to the wrong people or the wrong address, while improved accuracy could encourage a higher sales conversion.

This involved the migration of data from multiple sources, a process that would involve a data-cleansing stage before the information could be certified as fit-for-purpose. In line with the company’s policy of standardising on particular packages, Turner selected Ab Initio for the extraction, transformation and loading (ETL) process, while adopting Trillium Software for data quality. This combination remains in use today.

“We built Trillium functionality into Ab Initio as a series of callable routines, so that as we migrated the data, we could check it using Trillium, enrich it where possible, match it where we could and, where it couldn’t, the system would throw off the data to a separate file to be dealt with manually,” says Turner.

The data-quality software performed a variety of analyses on each item. These might include matching data from different sources – for example, to connect a record from one system under the name of ‘John Smith’ with the ‘Smith, J’ of another – while checking the accuracy of other data elements, such as postcodes.

Indeed, simply ensuring that every entry had a postcode meant a saving on the cost of the mailings because the UK Post Office offers a discount to bulk mailers for supplying them. “Within the first year, we identified about £5m of savings as a result,” says Turner. “Then, we started to look at the front-end systems and what we could do to improve them.”

The next phase involved what Turner calls introducing online data validation at the front-end, the ‘touch-points’ with customers. Customer-service staff would be given access to a name and address database and would have to confirm with customers these details as a first step when they called in.

This would prevent staff from generating new, duplicate records while providing an opportunity to for the company to make sure that its customer records stayed more up-to-date.

That led to an even grander project: the name and address (NAD) database. “We asked, ‘why are we holding name and address information in 400 different systems, when we could hold it in one system and stream those updates though to all the other systems that require access to customers’ names and address’,” says Turner.

This initiative dovetailed with a wider movement in enterprise-data management towards master-data management. The NAD now provides the master data to the other systems in BT that need access to customer data. The source systems have not been stripped of customer data, but instead, the NAD links to them and changes customer data accordingly.

“We make changes once within the NAD and then, because it’s all linked by keys, that change is automatically reflected in the 400 source systems,” says Turner.

The NAD did not just involve linking residential customers, but also the many different entities of business customers, too. For example, one High Street bank will have many branches. Under NAD, these were linked together, an initiative that also enabled the organisation to gain a more rounded view of the products and services that its business customers bought from it and, therefore, their true value to BT.

Big savings

Yet the biggest data-quality initiative run by Turner’s team was a series of inventory-management projects. “We basically looked at particular areas of inventory, be they private circuits or copper cables, to make sure that the information in our systems matched reality,” says Turner.

His team ran more than 50 projects targeting different areas, matching and comparing data in different systems that ought to dovetail with each other – a mismatch would indicate a problem. While inventory data is vastly different from customer data, the principles and practices are broadly the same.

“For example, a customer orders a new private circuit from BT. Halfway through processing of that order, the customer decides to cancel. We were finding that the cancellation wasn’t always reflected in our systems – our systems showed a private circuit occupied and awaiting a customer. Sometimes it would just sit there,” says Turner.

The discovery process was similar to the drawing of a Venn diagram: if the orders database indicated a circuit and that circuit was also reflected in the billing system, then it could be reasonably assumed that all was well. However, if a circuit appeared in one system and not another, a team of engineers would be sent to investigate. “If we found a circuit that was not being used [or paid for] by a customer, we would return it to stock,” says Turner.

While this may sound like an unexciting project, it yielded three-quarters of the total return on investment of the ten-year data-quality improvement programme – around £450m of the £600m of savings made combined with revenue attributable. Such figures demonstrate just how much money can be raised or generated by a robust approach to data quality – and one that prioritises business need and money saved or revenue generated above all other factors.

Processes

But technology can only do so much to improve an organisation’s data quality, says Turner. “It’s a complex problem because it’s not just about IT and data in systems, it’s about culture and processes. If someone in the front office inputs an incorrect customer name and address, then all the systems in the world are not going to help you. Garbage in, garbage out.”

Data-quality improvement is, therefore, as much about corporate culture and business-process improvement as it is about IT. It means, for example, making sure that front-office staff understand why complete addresses are required, as well as doing online checks every time a customer calls in – verifying their complete address with every contact makes sure that the record is always up-to-date.

Indeed, it is only by improving the processes and culture of an organisation that lasting improvements to data quality can be made.