Managing a data mountain

The world’s largest telescope, the Square Kilometre Array (SKA), is expected to generate more data in a single day than the world does in a year at present. And even its prototype, CSIRO’s ASKAP, is expected to accumulate more information within six hours of being switched on than all previous radio telescopes combined.

Such gargantuan streams of data require serious management, and that will be one of the jobs of the $80 million iVEC Pawsey Centre in Perth, which is due to be completed in 2013.

The planned Pawsey High-Performance Computing Centre for SKA Science in Perth (photo credit: Woodhead/CSIRO)

“There are two issues here,” says Andrew Rohl, Executive Director of iVEC, a joint venture between the CSIRO and the four public West Australian universities, which will manage and operate the facility. “The raw data is being generated by telescopes out in the desert, and we have to get it to a computer to reduce that data and to generate useful products out of it. So the first problem is to get it to Perth, and the second is to process that amazing quantity of information.”

Solving that first problem means mastering high-speed data transfer. “The only way to transmit quickly the huge amount of data that would be generated by the SKA telescope is by high speed networks,” says Chris Phillips, who works on the problem at CSIRO Astronomy and Space Science. “We’re talking terabits per second.” Transferring one terabit a second is the equivalent to transferring 50 full DVDs every second. “Over the last few years we have had to develop custom software to efficiently utilise high-speed networks because standard software just couldn’t cope.”

Black holes in real time

In a test of their technology, two Australian radio telescopes worked with others in China and Japan to observe a distant black hole. Connecting the telescopes electronically using high-speed data transfer allowed astronomers to collaborate in real-time, rather than waiting months for the data to be stored on disks and then shipped around the world. “That demonstration showed the world that Australia can be the data processing centre for these international experiments,” Chris says. In the next two years they plan to increase data transfer rates tenfold.

Once the data has been collected, it has to be processed. Powering the high-performance computers needed to process that data avalanche will take a lot of energy. The Pawsey Centre will need the power supply of a large shopping centre. “A lot of that energy will be turned into heat,” says Andrew.

However, at least part of the energy needed to keep the computers cool will come from a renewable source—geothermal energy. In June 2010, the Government announced a $47.3 million green energy investment in the SKA project, to build a geothermal cooling plant for the Pawsey Centre and to build a solar array at MRO to power the observatory itself.

The Centre, which will be equipped with one of the 20 most powerful supercomputers in the world, will cut its teeth on data processing for ASKAP and MWA telescopes.


iVEC Professor Andrew Rohl, Tel: +61 (8) 6436 8831,,

CSIRO Astronomy and Space Science, Dr Chris Phillips, Tel: +61 (2) 9372 4608,