SGI to provide massive data storage capability for the Australian Square Kilometre Array Pathfinder

The Pawsey Centre will process huge volumes of data. The two largest generators of data are expected to be the Australian Square Kilometre Array Pathfinder (ASKAP), Australia’s largest and most capable radio telescope ever constructed, and the Murchison Widefield Array (MWA) project, which studies the signals from the dynamic radio sky as well as measurements of the Sun and heliospheric plasma. These initiatives will expand understanding of the universe and drive technological development worldwide. It is anticipated that these two projects combined will generate eight petabytes of data each year, all of which will flow through the Pawsey Centre. When adding in data from other research areas, such as geothermal modelling and rock characterisation, iVEC forecasts supporting data volumes at approximately ten petabytes annually for the foreseeable future. To manage such volumes, CSIRO selected an SGI® InfiniteStorage™ and SGI® UV™ 2000 based solution to address the scale and cost-efficiency requirements for a project of this magnitude.

The SGI InfiniteStorage solution comprises disk storage systems and licences to support up to 100 petabytes of online storage that is virtualised across multiple performance tiers by SGI® DMF™ software, with data ingest and workflow managed by SGI® LiveArc™. The system’s six petabytes of primary storage is virtualised to provide further flexibility and savings with an additional cache of SGI MAID, the company’s energy-efficient zero-watt disk technology. This environment integrates with 40 petabytes of data tape libraries and provides expansion capabilities to support a seamless 100-petabyte hierarchical storage management (HSM) online environment.

Beyond big data management and storage, the SGI UV 2000 solution delivers both big data analysis and visualisation capabilities. Working as a set of data-analysis engines to move and process huge amounts of data very quickly, the SGI UV 2000 can be incorporated into an array of different workflows to provide pre- and post- processing for a range of scientific applications. The versatility of the SGI UV 2000 allows the Pawsey Centre to deliver extremely powerful big data visualisation capabilities, enabling its researchers to view and manipulate vast amounts of data in new ways. This new technology will allow the Pawsey scientists to visualise images in the order of four terabytes, an order of magnitude greater than the size of images previously available, which in turn provides a quicker path to results and interactions.

“iVEC is committed to ensuring Australia maintains its place as a world leader in research and scientific computing, and the Pawsey Centre is a critical pillar in this strategy,” says Neil Stringfellow, iVEC’s executive director. “SGI's storage and data analysis infrastructure is a vital component of the Pawsey infrastructure. In particular the SGI UV 2000 visualisation system with its very large shared memory capability will enable our researchers to manipulate their data in a completely new way, leading to the potential for new insight and ambitious analysis.”

The Pawsey Centre is expected to be fully operational and in production by October 2013, with users gaining early access before June. It is anticipated that the first component to be heavily used will be the SGI InfiniteStorage based HSM as the radio-astronomy community moves into the final testing phase for their apparatus and begins to stream significant volumes of data. The SGI UV 2000 based data-analysis engines and visualisation systems are expected to come online shortly thereafter.

“For decades, SGI has been solving Big Data challenges for researchers across science and industry in an effort to find answers to the world’s toughest challenges,” said Jorge Titinger, president and CEO, SGI. “We are very pleased to support the data management needs of the Pawsey Supercomputing Centre. They are conducting impressive research, and with our InfiniteStorage and UV 2000 technology, will be able to reach results and interactions more quickly. We look forward to continuing this partnership and seeing the Pawsey Centre’s revolutionary solutions to challenges in science.”