News

Software to manage increased research data

 

14/06/2018

New technology continues to offer better solutions for researchers, as QCIF happily witnessed at a national meeting earlier this month over software for high-performance computing.

QCIF staff attended the national, CSIRO-organised Data Management Framework (DMF) Users Group meeting at UQ, 6–7 June.
 
Currently, QCIF and UQ’s Research Computing Centre (RCC)— the event’s local host — both use DMF servers as part of HPC infrastructure to service the data storage and data analysis needs of researchers.
 
Hewlett Packard Enterprise (HPE), DMF’s vendor, flew two people from its US software development team to talk about the big changes to the new version of DMF (DMF 7), which is a major rewrite, and provide an opportunity for current users (including QCIF) to ask questions.
 
Martin Nicholls, a specialist systems programmer with RCC, said the most significant feature of DMF 7 is its ability to scale to larger numbers of files. “Together, both RCC and QCIF have more files than would fit into one single server, so this scaling is important. It [DMF 7] enables RCC and QCIF to efficiently manage the ever-increasing amount of research data,” he said.
 
A projected feature of DMF 7 (due mid-2019) will provide closer integration with MeDiCI, the high-performance data storage fabric that underpins QCIF data storage and provides seamless access to data regardless of where they are created, manipulated and archived.
 
Some technical questions asked at the DMF Users Group meeting provided the HPE development team with feedback to ensure that features essential to QCIF and RCC were incorporated into the product as early as possible.
 
Almost 40 people attended the meeting, including 15 vendors and 24 DMF customers.