Avoid the five biggest email migration pain points

As European businesses face mounting pressure to upgrade outdated legacy email archives, we are seeing growing numbers of them moving to the cloud. Research by Cloud Industry Forum found that in 2013, 69 per cent of organisations across the UK market formally adopted at least one cloud service within their business, an eight per cent increase from the previous year.1 By the start of the next decade, we expect tens of thousands of enterprises worldwide to move their email to the cloud.

It’s not uncommon for organisations to have hundreds of terabytes of unstructured data stored in legacy archive systems. These archives are often teeming with redundant or obsolete information that does nothing but add to storage costs. Overloaded archives are difficult and time-consuming to search for eDiscovery, data minimisation or risk management. As a result, organisations are looking to migrate this data – to deliver operational efficiencies, make storage management someone else’s problem and change capital costs to operating expenses.

But these legacy archives are a major obstacle to adopting cloud email services. The process of moving data from legacy archives to the cloud can be painfully slow. Migration timeframes often extend to months or even years.

Here are five of the most common migration pain points organisations are likely to run up against:

1. Slow APIs
Most migration technologies available today use the legacy archive’s in-built application programming interface (API) to extract and migrate data. Unfortunately these APIs typically weren’t designed to deal with large volumes of data. Many can only process one item at a time.

At this rate, even small volumes of data can take months, or in extreme cases years to migrate. As the volume of data we create and store annually doubles in size every two years2, migrations will only become even slower.

2. Delayed search and discovery
Gaining access to more responsive or functional search tools is a common driver for migrating data. However, the new platform’s search won’t return accurate results until it has ingested all the legacy data. Working with the slow APIs mentioned above, this means it can take months or years for an organisation to access the search capabilities that were the point of migrating in the first place. This delay effectively voids the anticipated return on investment (ROI) that justified the migration in the first place.

3. Inefficient or corrupt indexes
API-based migration relies on the legacy archive’s internal index containing accurate records of which messages and attachments are stored where. It’s common for these indexes to become from years of use. Extracting data with a corrupt index can lead to compromised data that is damaged or incomplete.

4. Risky data
Data that contains sensitive private or financial information, such as credit card numbers and personal customer details, can often be buried deep within terabytes of old data. This risky information should be identified before moving data to the cloud, however most legacy archives and migration tools lack advanced search capabilities to find it.

5. No way to leave behind the ROT
Traditional migration approaches have no efficient way of distinguishing redundant, outdated and trivial data (ROT) from important files you need to keep. For example, they can’t selectively leave behind data by date or other criteria that determine a file’s value. This leaves organisations no choice but to move the entire archive to the new platform. While cloud platforms and new hardware may be more efficient, migrating an entire legacy archive onto these new platforms doesn’t solve the original issues. Organisations will continue to struggle with issues such as search, structure, irrelevant data, and unnecessary storage costs.

A more intelligent approach

When looking to migrate email to the cloud, most organisations place their attention on finding a best-fit new platform. Yet few pay any attention to another factor which has the potential to impact on the success or failure of their transition just as much: the choice of migration technology.

Recent advances in technology have made it possible to index and extract data directly from archive storage rather than through the API. This means organisations can quickly examine data directly within the archive storage. In turn, they can avoid common pain points of migration by indexing and making previously unsearchable data searchable before migrating it to the cloud. Organisations can then make informed judgments about risk and value.

Before migrating, they can pinpoint business risks of private and financial data, and filter out data they want to leave behind in the legacy archive. This may include data past its retention date, very large and infrequently accessed files, duplicated email messages such as company-wide email memos and trivial content containing keywords such as ‘lunch’ or ‘kitten’. Organisations can also prioritise information to be migrated first, such as data on legal hold or belonging to executives.

The ability to streamline years’ worth of data makes migration drastically faster. It also brings major cost savings. A quick migration means spending less money maintaining the legacy archive while transferring data to the new platform. Similarly, it lowers reduces storage costs, as the new archive contains less unnecessary data.

Move faster, move smarter

We recently completed a project with a global financial institution that had 330 terabytes of emails in multiple archive systems. The company simply could not search messages within a reasonable timeframe in response to discovery requests for ongoing litigation matters. Its legal and technology advisers estimated that migrating the data to a new archive platform would fix the problem but would take years, literally.

Using the intelligent migration approach we have been discussing, the company made all 3.1 billion emails in its archives searchable within six weeks. It eventually left behind more than 75 per cent of the duplicated, outdated and irrelevant data in its legacy archives. The company completed the entire migration in eight months and saved millions of dollars in storage costs.

Searchable data opens up endless possibilities

In addition to achieving time and cost savings, organisations can start afresh and take charge of their data by introducing and enforcing retention policies. Staff productivity will also likely increase in line with the new ease of searching and using only data that has current business value. Offering businesses the ability to easily search their data means they are able to analyse it, which may give them a competitive edge over many other companies who will continue to struggle through masses of unstructured data.

Rocco Seyboth, Senior Vice President, Intelligent Migration at Nuix, a technology company that enables people to make fact-based decisions from unstructured data.

1 Cloud Industry Forum, Cloud UK Paper twelve, UK Cloud trends and the rise of Hybrid IT http://www.softwerx.com/wp-content/uploads/2014/03/UK-Cloud-trends-and-the-rise-of-Hybrid-IT.pdf

2 EMC, The Digital Universe of Opportunities, www.emc.com/leadership/digital-universe/2014iview/executive-summary.htm