Imperial College London uses new Half PB storage to support research and e-learning

A new highly-scalable storage system is supporting staff, students and researchers across multiple departments at Imperial College London. Designed and implemented by HPC, big data management and analytics provider OCF, the centrally-funded half PetaByte system enables secure, cost effective storage of vast amounts of research data produced at the College, whilst underpinning a Panopto video capture system that records lectures from up to 100 separate theatres on a daily basis.

The new Linux-based, IBM GPFS storage system sits across two separate data centres. It replaces a legacy Microsoft solution that had grown on an ad-hoc basis and become increasingly difficult to grow and administer.

As well as long-term, cost-effective, secure storage requirements of research data, e-learning is a major area of focus, so the College needed a solution that was capable of storing captured and encoded video recordings of lectures, making them available for editing and viewing by potentially 20,000 students. Multiple lectures can be recorded in parallel on the system, then saved and encoded with a turnaround time of less than one day from raw video capture to encoded version. Uptime was also a key factor - the College wanted a solution that would remain online 24/7 to support students.

The new system, which comprises of 8x IBM x3650 M4 GPFS clustered storage and NAS servers, along with 4x IBM Storwize v3700 storage arrays provides increased uptime, greater resilience, and is more responsive, incorporating a number of disaster recovery features to protect against data loss. In the event of data loss or corruption, users can also recover previous data copies from up to 14 days themselves; saving the user time and reducing the burden on IT administrators significantly.


“We wanted a system that could grow and scale more easily and protect large data sets more effectively and efficiently,” said Steven Lawlor, ICT Datacentre Services Manager at Imperial College London. “Our old system no longer scaled and was becoming a bit of a beast. The new system will be easier to expand and more cost effective, in part because we can now move data to tape for long-term storage, a key requirement for certain types of research data. Uptime is one of the most important factors, we wanted a system that could also be maintained and expanded whilst still running.”

Julian Fielden, Managing Director of OCF comments: “We’re seeing a massive explosion of data growth. Academia is feeling the pressure of keeping up with storage demands, especially around research data. Imperial College London’s real challenge was around storing the lecture capture side. Research data can be broken up and separated, whereas its video capture system was one large system across the College. By developing a scalable storage solution based on GPFS they are now able to grow and expand storage in a simple and cost-effective manner.”

Imperial College London has already migrated all of its video content to GPFS. It is currently migrating research datasets from the Microsoft environment to the new Linux-based storage system.