Like many government agencies, you probably are faced with solving complex problems on a daily basis. While typical IT infrastructure may offer some solutions, you have likely found yourself searching for a mechanism that solves these complex problems in a more efficient way. High-performance computing offers a solution outside of typical IT infrastructures to tackle these complicated problems.
GovLoop and Red Hat’s The Future of Tech: Understanding High-Performance Computing (HPC) roundtable offered an interactive discussion on what HPC is and how you can use it in your agency. Ted Hwang, Customer and Technical Solutions Architect at Cisco Public Sector and Adam Clater, Office of the Chief Technologist, North America Public Sector at Red Hat led the discussion on what HPC is and why it matters to government.
Essentially, HPC is enabling research in federal labs that are conducting experiments that require processing large and complex collections of data. “The idea is to bring everyone who is working on a problem together into the same infrastructure and leverage all available resources to solve the problem,” Clater explained. HPC is a single organism that breaks down the typical IT silos by combining computing power, network, and storage into a single computational cluster.
Why It Matters
Utilizing HPC allows agencies to seamlessly solve problems that a siloed IT infrastructure would find more challenging. “Leveraging HPC gives you processing capabilities and applies to it to many parallel processing situations, touching almost every aspect of IT,” Hwang explained. The ability to harness that much computing power and focus it on a single problem allows agencies to solve bigger problems more efficiently.
For example, the Massachusetts Open Cloud is working with private sector partners to enhance the readability of fetal MRIs. Historically, fetal MRIs are very challenging to read because each image has a lot of noise amongst the relevant information. “HPC has been able to rationalize the noise out of fetal MRI images and get readability of the image down from months to a few days. By using HPC, Massachusetts Open Cloud was able to tackle a very big problem and break it down piece by piece to offer a consolidated solution,” Clater explained.
Getting Started
While HPC works to break down silos in IT, it is important that the workload of HPC researchers stays separate from typical IT. “We see success in HPC when agencies are able to manage the hardware and HPC research separately,” Hwang emphasized. By carving out separate infrastructures, agencies can allow their HPC researchers to work on problems without getting bogged down by other aspects of IT.
Additionally, agencies should be sharing their information and data as much as they can. Clater underscored that, “we are beginning to see centers of excellence for HPC in the government where agencies are sharing information and data to encourage efficiencies across government.” Information sharing ultimately allows public sector organizations to overcome some of the challenges associated with HPC.
Biggest Challenges
- Administration: Utilizing an HPC infrastructure means that systems administrators have to switch from managing a few computers to up to a couple thousand. To overcome this challenge, Clater recommended practices that foster intersection between HPC and other infrastructures. For example, he emphasized that utilizing cloud tools in conjunction with HPC could be very useful at an HPC administration level.
- Automation: Automation is key to enabling the administration of large data sets and managing multiple HPC facilities. This is where private vendors come in. “Agencies need to figure out the automation tools they need and work with private vendors to optimize infrastructure capacity and fully utilize HPC,” Hwang explained.
- Scaling: HPC researchers often have a difficult time predicting the scale of infrastructure they will need to meet research requirements. In order to counter this, Hwang recommended starting small and building up your HPC infrastructure.
“We’ve all got problems that we are trying to solve and everyone is trying to figure out which IT infrastructure will best solve these problems,” said Clater. “HPC can do this but you have to take your problem to vendors and collaborate to find the right tools you need to solve your agency’s problem.”
For more information on how your agency can optimize HPC, check out GovLoop’s recent report, “Empowering Agencies with High-Performance Computing.”
