A useful Open Gov Commitment? Focus and Harness the Winds of Public Comment

The White House Recently asked citizens to post suggestions on how to improve regulations.gov, data.gov, and the Federal web strategy. This is part two of a multi-part series on how to and more importantly, why we should consider changing the way these, (potentially) game-changing efforts could be improved. Here, we present some comments for regulations.gov and some suggestions on how changes could help improve the federal rulemaking process.

Regulations.gov is an award-winning attempt to better engage the public in the Federal rulemaking process, by making documentation about rules available in one place using an easy-to-use user interface. As well-intentioned as the Regulations.gov effort has been, there are many specific steps that can be taken to improve this important tool. The shortcomings of Regulations.gov are well-documented[1] and, while the Regulations.gov team is frequently responding to input on how to improve, the changes have often left much to be desired. Recently, the Regulations.gov team responded to the White House’s blog post on GovLoop but the plans the team discusses won’t address the more fundamental issues this tool faces.

Unstructured data collection = an unstructured mess

In my PhD class work/research we were encouraged (forced?) to put method behind our madness. My work involved doing social science research on a government agency involving interviews with stakeholders involved in a government process. If I had proposed to approach these interviews by saying “Please give me your comments on X process” my advisor would have laughed me out of the building and, at the end of the day I would have had a collection of information that, when aggregated, would have amounted to the equivalent of a 3-year-old’s finger painting; a sloppy, meaningless mess. Unfortunately, what I’ve just described isn’t too far off from what the government receives from Regulations.gov.

Public input is an important part of the democratic process. The Administrative Procedure Act codified the public’s right to participate in the regulatory process. Public input helps agencies deliver services in a more focused manner to the public. Additionally, offering the public an opportunity to comment serves an important role of building citizenship by allowing people to feel that they are being heard and to hold the government accountable for its actions. On regulations.gov, the government presents rules and the public can give their “comments”, provided their comments are 2000 characters or less, or they can upload an attachment. In my work, I’ve had the privilege of seeing, first hand, what similar input from the public looks like: about (and these are my approximations) 80% of the public submissions about the rule/action/document are opinions with a for or against stance on the issue, 10% of the submissions are substantive information about the matter at hand and/or include some kind of actionable suggestion, and the other 10% are either blank or rants against government in general. There are several problems with comments received through regulations.gov and the method by which these are collected. My top three are:

1) The mountain of unstructured data that regulations.gov creates requires qualitative analysis and coding organize and categorize it to be of any use and to be available for future reference.

2) The public, who has little or no visibility into the decision making process, has to make their own determination about what the government wants or needs to know from them in order to improve the process. If 100 people comment, you can bet that there will be almost 100 different guesses as to what would be helpful or useful.

3) Opinions on a rule or project are not actionable by the government. The intent of seeking public input is not a referendum on the rule or project, but that’s often all that comes out of public comment mechanisms like regulations.gov.

These issues with the collection of information through regulations.gov create a couple down-stream problems:

1) Qualitative data analysis is time-consuming, expensive, and, to be good at it, analysts require years of experience. Most agencies don’t have the necessary expertise to handle this analysis and, and as a result, will contract this type of work out.

2) The only interaction that the general public has with the federal government where they can be assured that their input is heard is through elections. Without direction, the public seems to be inclined to treat commenting like an election. If they feel that their input is not heard, there is a real risk of alienating these people from providing future input on government issues.

A simple solution

The main, overall problem with regulations.gov is “garbage in, garbage out,” “you get what you pay for,” or some other cliché. So here’s my idea, White House: Regulations.gov needs focus. Requests for information/comment from the public should be direct and tailored to the specific needs of the agency and effort. Here are six simple things that can go a long way toward improving the quality of data received:

1) Allow agencies to structure the comment form to pose each of their questions about the rule directly to the public and also include a section for “other input.” This will focus the public input to topics that are relevant to the rule.

2) Support surveys. A simple, well-written survey, about a rule making effort can provide very rich data about public opinion on an issue and areas to improve upon/change and the data received from free text fields can then be used to support the findings of the survey. [2]

3) Make it clear, this is not a vote. A good start would be to reiterate the points from the “how does effective commenting work” from Cornell’s regulationroom.org. Positions are fine, but be clear that the public’s rationale for their position is much more important/useful than the position itself.

4) Accept anonymous submissions and allow the form to accept more than 2000 characters. To this point, I’ve stayed away from criticizing Regulations.gov’s user interface and functionality, but this needs to be addressed. Requiring name and contact information and limiting the text field to 2000 characters are arbitrary limits and barriers to participation. There is no rule to say that comments have to be associated with a person in order to be considered. If the public can anonymously submit a comment by mail, then they should be able to do so by the web.

5) Incorporate social science research methodology expertise. Preparing a document for comment, creating an instrument to gather input and data analysis should be done by those with the knowledge to do it properly. This knowledge may or may not currently exist at agencies. If expertise does exist, those with the expertise should be consulted prior to making regulations available for comment; if it does not, agencies should consider a plan to gain such expertise in the future.

6) Incorporate (or encourage the use of) tools with regulations.gov. The government has spent millions on text mining software and analytical tools for the DoD; why not repurpose those for civilian use? Or encourage agencies to share tools that they have developed for their own purposes, such as the Forest Service’s Comment Analysis and Response Application which is used to analyze comments received through regulations.gov or from any other source.

It’s time to move regulations.gov from just a presentation agent for the Federal Register with companion comment portal. We’ve seen how the innovations in Federal Register 2.0 have improved readability and the potential for comprehension. We’ve also seen how efforts such as Regulation Room have radically restructured rulemaking information into useful chunks of information. It’s time to start thinking strategically about how the information received from the public should be used and collected rather than just opening the window and seeing what happens to blow in.

(Cross-posted at www.phaseonecg.com/blog)