OpenFDA: Ushering in a New Era of Government

Below is an excerpt from GovLoop’s latest guide, Capitalizing on the Open Data Revolution. You can access the full guide here. 

MedWatcher is a free mobile and web app that allows patients to understand and report side effects of drugs, medical devices and vaccines. The service was created in 2010 and is an early example of the power of using FDA data to build new services for patients. Its use of FDA’s adverse drug events data and information collected from the community informs more people about side effects from medications and improve FDA’s ability to collect and disseminate such data. MedWatcher is an example of how open data can support new services, which government may not have the resources to create.

For years, FDA has been collecting and sharing data related to adverse drug events. But until recently, the agency did not have an efficient way to share data, make it easily accessible or enable users to contribute to it. Previously, companies and the public had to submit Freedom of Information Act requests to receive data. In some cases, the data was already public, but even then, it was often hard for developers to find accurate and timely data on which to build applications. FDA found that for companies to truly make the most of its data, it needed to make the data accessible.

That’s why FDA created openFDA. With openFDA, data is now publicly available in structured, computer-readable formats. This makes it easy for developers to build mobile applications and web services and create data visualizations. Researchers can also use openFDA to quickly access FDA data through APIs. Now developers can easily integrate the data into other sources. “OpenFDA really shows the power of not only releasing data, but having a very focused effort to make sure that data is usable and accessible to citizens and showing what the potential could be when the community takes up the data and does things that we could never imagine,” said Sean Herron, former Presidential Innovation Fellow and product and software engineer at 18F. He served on the team that created the openFDA platform.

Data Discovery: The Process of Delivering Value

To be sure they were delivering value to their data community, FDA officials first needed to know what kinds of data would be most useful for their users. They started by conducting a thorough inventory of their data, exploring which datasets were most actively used and already released.

They found that although they had released a considerable amount of data, such as food recall information, it was rarely being used and no one outside the FDA community was accessing it. Officials assumed this was because those who could find the data valuable — startups, developers or citizens — did not know the data existed, or because there was little documentation and structure around the information. Herron hoped to change that.

After taking inventory of the data, the FDA open data team identified three of the most valuable datasets and decided to build openFDA around them. The three core datasets were:

• Adverse drug event report: An adverse event is submitted to FDA whenever an undesirable experience is associated with use of a drug. This includes side effects, product use errors and quality or therapeutic failures. Since 2004, FDA has been collecting this data and has more than 4 million records. It is important to note that the reporting of adverse drug events are completely voluntary in the United States, but sharing information can help combat unintended consequences of medications.

• Product and food recalls: This openFDA API is for all product recalls FDA monitors. At times, the agency may remove a product from the market to protect the public. Companies almost always do recalls voluntarily. FDA’s role during recalls is to oversee strategy and classify the recalled product according to the level of hazard. FDA then posts all recall information in its Enforcement Reports.

• Product labels: FDA also collects data around product labels, such as those on the back of pain medication bottles listing active ingredients and dosage. This data is useful for researchers, citizens and developers to help bring more clarity to what is in medications.

FDA officials were determined to maximize the utility of these three datasets. “We really wanted to dive super deep into those three datasets and produce an elegant interface on them,” Herron said. For a year, FDA conducted user testing, did small beta releases and iteratively built openFDA as the team gained more insights from their community. But doing so did not come without challenges.

Addressing Data Challenges with Collaboration

“Although cloud computing and technology resources are cheaper, faster and more effective than they have ever been in the past, there are still many challenges to make any kind of dataset available on the Internet,” Herron said. “If a dataset potentially contains something like a Social Security number or a patient health identifier, scanning data is still a very labor-intensive process to actually make sure that data is safe to be released.”

To work through the final challenges on preparing data to be public, FDA brought together a team of software engineers, user interface professionals, and policy and legal experts. The openFDA initiative’s beta version was publicly released in June 2014. Since then, the agency has continued to improve and build on it. After the first week of the beta release, it had more than 1 billion queries.

“We saw a number of applications and websites that allow FDA information to be mashed up with other datasets out there, making information more easily understandable for citizens,” Herron said. OpenFDA removed the previous barriers to FDA data. Rather than having to download gigabytes of data with very confusing XML files, openFDA now allows easily accessible data for developers. Now, software developers can use FDA data to create their own applications and websites, outside the scope of the agency.

Open Data Lesson: The User’s Needs Come First

The major lesson learned from FDA is that in order to excel with open data, organizations must have a firm grasp on user needs. By working so closely with its users, FDA was able to figure out what the three most high-value datasets were and deliver the APIs and resources that were in demand. “For me, my users, my customers in all the work that I do, are the citizens of the United States of America, so I need to be responsive to their needs,” Herron said. Herron is not alone in his transformative work. In our next section, we explore the open data work at Commerce, which is helping “America’s Data Agency” leverage information in new ways.

To learn more about open data and case studies from data.gov and the Department of Commerce, be sure to check out GovLoop’s latest guide, Capitalizing on the Open Data Revolution.