The wheel of life: Information-lifecycle management

Information storage can be both costly and complicated, particularly given that the amount of data most organisations need to access has never been higher. Information-lifecycle management can help to bridge the gap between technological functionality and business requirements, and is unsurprisingly becoming a critical component of the knowledge-management toolkit. By Jessica Twentyman

Maintaining peace and public order in and around the urban conurbations of Sheffield, Doncaster, Barnsley and Rotherham in the UK is the daily challenge faced by the men and women of the South Yorkshire Police Force. Managing the data that they create in their daily working lives, meanwhile, creates another daily challenge – this time, for the force’s IT team.

According to Roy France, IT manager for South Yorkshire Police, ensuring that that data is stored securely and can be retrieved quickly when required is an imperative. “Everything the force does is concentrated on the fast, continual retrieval of information,” he says. “We need to be certain that information is always available year after year.” That data, he continues, comes from a wide range of different systems: the force’s crime-management system, operational intelligence, command and control systems, e-mail, and day-to-day finance and administration applications. But the data each of those disparate systems holds has one thing in common: from its creation to its disposal, it is stored in a way that recognises its value over time to the force.

Typically, that involves storing it on a high-cost, high-performance disk when it is likely to be accessed on a daily (or more frequent) basis, and then migrating it to lower-cost, lower-performance data-storage mediums as the likelihood it will need to be accessed (and therefore its value to the force) decreases. This practice is popularly known as information-lifecycle management (ILM) – a term used to describe the application of technologies and processes to the management of information across its complete lifecycle, according to its business value and how that changes over time.

Not every organisation is so savvy when it comes to managing data storage. But most are facing great pressure to be more so, for three key reasons.

First, the volume of data under management has reached such levels that storing, tracking and retrieving information has become daunting. Recent research from IT analyst company the Meta Group indicates net annual storage growth will average between 20 per cent and 25 per cent for enterprise (monolithic) storage, 50 per cent to 55 per cent for midrange (modular) storage, and between 80 per cent and 85 per cent for low-cost, capacity-based (SATA/ATA) storage, yielding aggregate storage growth of 45 per cent per year. In five years, such robust growth rates will yield 34 times more storage capacity on the floor to manage. “Without enhanced storage processes, management and automation, effective utilisation of storage assets will remain a major data centre issue,” says Carl Gruener, a Meta Group analyst.

Second is the demand for high availability – users want rapid access to data of varying ages, with no threat of it disappearing forever.

Third is the increased pressure on organisations to meet regulatory requirements and to ensure better corporate governance. This means they are holding more data for longer periods – and in the process, compounding the storage headache.

The typical response to these pressures, says Randy Chalfant, chief technology officer of storage manufacturer StorageTek, is “a knee-jerk reaction based on maintaining the status quo”: storage managers buy more disk space, continue to back up all data in the same manner, and manage swelling data volumes through established, labour-intensive processes. This is, perhaps, understandable: the cost per gigabyte of physically storing data is lower than it has ever been and continues to fall. As a result, storage costs have never been lower at exactly the time when data volumes have never been higher.

For many hard-pressed IT managers, that makes simply buying more hardware to accommodate burgeoning data seem like a relatively pain-free option. In a recent survey conducted by market-research company Vanson Bourne, 80 per cent of respondents admitted that they simply purchase more storage capacity rather than investigate ways of getting more out of existing resources.

However, it is not a sustainable approach, says Chalfant: “Typical IT organisations can’t simply throw money at the problem by buying more primary disk, nor can they throw more staff at the resulting storage-management problem.” This is certainly the case at South Yorkshire Police, where IT must be carefully budgeted. “We are continually tasked to deliver best-value information technology,” says France. “It therefore makes sense to move older information to different online storage media that still provides the right levels of protection, replication and recovery at the lowest possible cost. That’s why we adopted an information-lifecycle-management strategy,” he explains.

ILM technology

The ILM approach recognises the varying costs of storage media. For example, monthly financial reports may combine sales orders, shipments, inventory expense and other data for the month. During the processing cycle, the finance department needs access to verify and analyse this data frequently and rapidly. For that reason, it might sensibly be stored on high-performance disk. After the reports are generated, however, the previous month’s data is referenced less frequently as the focus changes to data for the current month. Previous reports can then be migrated to lower availability, lower-cost storage, such as tape, thus freeing up primary, high-cost storage.

The numerous sales opportunities that this represents to the IT industry has sparked a flurry of activity as suppliers scramble to repackage existing products and release new ones that claim to enable customers to implement ILM in their own organisations. Among them are some of the giants of the data-storage industry, including IBM, Hewlett-Packard, EMC and StorageTek.

Implementing ILM is far more involved, however, than simply buying such products. First, the company needs to establish a thorough understanding of its own data environment: what categories of data are stored, how often they are retrieved, how long they need to be stored for and where in the environment, and at what point (if ever) might they safely be destroyed. “It’s vital to analyse current storage usage before embarking on any major storage implementation or upgrade. You need to know what you’re holding, where you hold it and what data has specific compliance requirements. That is a huge job in itself,” says Nigel Ghent, UK marketing director at storage supplier EMC.

That requires an assessment of the tiers of storage that exist below primary, high-cost, high-performance disk, says Tim Mortimer, business manager at storage-integration company InTechnology. “We generally advocate four levels of storage: first, fast access, high-performance primary disk; second, low-cost disk such as SATA (serial advanced technology architecture) disk; third, tape technology where data must be retained but is unlikely to be referenced again; fourth, offline tape in a secure facility, possibly offsite, which can be manually reintroduced into a tape library in the very unlikely event that it needs to be recalled,” he says.

The emergence of SATA disks has done much to boost ILM efforts, Mortimer adds. SATA arrays can store data at a fraction of the cost of high-performance disk. When shifting point-in-time copies of data, for example, SATA is often the obvious choice.

That is not to say, however, that tape technology is becoming redundant, argues Derek Lewis at Morse: “Despite advances in disk, there are still huge advantages to tape technology. I’ve recently seen tape technology that can hold around 1.5 terabytes on an £80 tape. The costs involved in storing large volumes of data that will probably not be accessed again on tape are now staggeringly low, and disk – even low-cost disk – still cannot match them.”

Software, too, plays an integral role in ILM. According to Guy Bunker, chief scientist at storage-management-software company Veritas (acquired by Symantec in December 2004), there are now a huge range of tools available to help data-storage managers monitor the storage environment and allocate data to different storage tiers according to pre-set policies. “When you look at the problems faced in data storage – deciding where you keep data and how you retrieve it again – users quickly realise that intelligent software tools are necessary not only to underpin ILM efforts but to future-proof the ILM environment,” says Bunker.

Many of these tools are already available from companies such as Veritas, StorageTek, IBM, Commvault and EMC, says Lewis. “Storage-resource-management tools, for example, are enabling companies to monitor and analyse large, complex and dispersed storage environments so that they know, for example, when a particular tape drive is full or how much bandwidth is taken up by a particular back-up task,” he says. Other ILM areas covered by storage-management software include: back-up and recovery; e-mail archiving; and, data-migration tools.

ILM promises

By matching data to storage systems according to its value, IT managers are able to tackle two of their major areas of concern: disaster recovery and e-mail management.

Protecting data from loss, theft or destruction is an expensive and time-consuming struggle for most IT departments. In fact, says Phil Goodwin, an analyst at IT market research company the Meta Group, back up and recovery processes remain “a primary pain point for most storage support organisations”, taking up around 60 to 70 per cent of the time allotted to general storage-management tasks. Information-lifecycle management, however, can help organisations take away some of that pain.

Once data has been classified according to its value, a vital part of the ILM process is to apply policies that ensure that critical data is not lost forever if a server fails, for example, or if fire or flood sweep through the building. Typically, many IT departments back up every piece of data they hold. But this involves huge storage costs and expands back-up windows to unmanageably long periods of time. Not only that, but organisations that take this approach commonly find that they not only back-up vast quantities of irrelevant or redundant data as a matter of routine, but that redundant data is duplicated many times. Using ILM principles, by contrast, a level of protection is defined for a piece of information as soon as it is received or created. That dictates whether it is backed up to a mirrored or off-site system, and when it can safely be moved to cheaper storage that offers lower protection. “What disaster recovery and business continuity does is ensure information is available to you when you need to retrieve it, whether it is on a server or a back-up tape or off site. ILM encompasses all that information,” says Ghent of EMC.

E-mail management, meanwhile, is another important driver of many ILM projects, because companies face increasing regulatory and legislative pressure to keep e-mail for long periods of time and are looking to maximise the use of lower-cost storage mediums for that purpose. Not only that, but e-mail archiving is an important facet of many ILM projects where companies are looking to maximise the use of lower-cost storage media. After all, there is little point in keeping a three-year-old e-mail on high-cost, high-availability disk if the chances are that it will never be accessed again – better to keep it on tape in a library, and eventually, on an offline tape in a secure storage facility.

As a result, ILM not only benefits hard-pressed IT budgets but also provides better alignment between IT and the business as a whole, says Gruener of Meta Group. “[Better] storage management is only a portion of the benefit of ILM. ILM pervades the organisation by combining content management with storage infrastructure to bridge the gap between technological functionality and business requirements with respect to data flow, use, regulatory compliance, availability, recovery, security, retention policy and management.”

Until organisations get that message, however, they will continue to flag under the burden of labour-intensive and costly storage practices. It is time, says Chalfant of StorageTek, for such companies to “get smart” about storage management. “Everywhere I go, I hear CIOs complaining about high storage costs and troublesome storage management tasks,” he says. “The simple fact is, they’re not doing it right.”