Huge Radio Telescope Gets New Database
The world’s largest radio telescope, the Square Kilometer Array (SKA), will be getting a data storage and analytics upgrade thanks to a grant from the organization that operates the telescope and partner (AWS).
The Dutch database technology company MonetDB Solutions said Monday (Aug. 10) it would use the grant to develop new storage and analytics techniques for SKA along with partner Centrum Wiskunde & Informatica (CWI), the Dutch research institute for mathematics and computer science. MonetDB Solutions is a spin-off from CWI.
The SKA Project is an international effort to establish the world’s largest radio telescope with more than 1 million meters of collecting area. It would consist of thousands of dishes and nearly 1 million antennas stretching from South Africa to Australia.
Column-store specialist MonetDB called the database management system for the SKA telescope “a critical component in the processing of astronomical data.” Along with MonetDB’s open-source columnar database management system geared to analytical workloads, SKA operators will use the vendor’s embedded R and Python capabilities for statistical data analysis along with the MonetDB Data Vaults extension for handling external data files.
The SKA project is an international effort to build a radio telescope that would eventually have over a square kilometer (one million square meters) of collecting area. The telescope is expected to generate hundreds of gigabits of celestial data per second, which must be processed in real-time.
The partners said they would develop database technology that “ensures that the system can sustainably handle growth in terms of both query complexity and data volume.” The database also will be designed to allow for the addition of new features along with improved software and algorithms needed to boost data processing performance.
The grant to develop the SKA telescope database is part of the “AstroCompute in the Cloud” program initiated by AWS in April 2015 and the organization operating the SKA telescope “to accelerate tools and technique development for storage and analysis of the vast data volumes produced by modern telescopes.”
The grant program aimed to “ensure that mature, high-quality data management and processing solutions are in place by the time the SKA starts to pump out data in 2020 or so,” Jeff Barr, chief evangelist at AWS, noted in a blog post.
Amazon will provide 1 petabyte of storage capacity for all grant recipients. The MonetDB database project is scheduled for completion by the end of 2016.
Members of the SKA organization include Australia, Canada, China, India, Italy, New Zealand, the Netherlands, South Africa, Sweden and the U.K.
In July, the Australian SKA radio telescope revealed a galaxy 5 billion light years from Earth. The Australian instrument is one of three “precursor telescopes” that will serve as the basis for the Square Kilometer Array.
The radio telescope detected a radio emission from the galaxy with the “imprint” of hydrogen gas as it passed the Earth. The gas absorbed some of the emission, causing a tiny dip in the signal that was detected by the Australian telescope. Normally, the dip would have been hidden in background radio noise, but it stood out at the “radio quiet” site in western Australia.