Using Evidence and Evaluation to Govern

OMB released new guidance supporting the use of evidence and evaluation in making budget trade-off decisions. The guidance reflects a high degree of sophistication on how to do this in a budget-constrained environment.

The Office of Management and Budget issued new guidance to agencies encouraging them to use program evaluation and evidence-based decisions when developing their budgets for FY 2014. This commitment continues a trend begun in 2009 when President Obama took office. But in similar memos in the past, the commitment was demonstrated by offering agencies more money if they undertook evaluations. For example, in 2010, OMB encouraged agencies to submit proposals for evaluations, offering up a pot of $100 million for such studies.

This year, a commitment to new monies isn’t as prominent, but the guidance reflects a genuine commitment to evaluation and evidence-based decision-making in a number of specific ways. The OMB memo offers details and examples of existing approaches agencies could pursue. For example:

• Use administrative data or new technology to lower the costs of program evaluations. It points to a recent brief by the Coalition for Evidence- Based Policy for promising approaches.
• Use existing waiver authority and performance partnerships as ways of testing variations in approach, and then subject them to an evaluation. For example, OMB supported cross-agency performance partnerships in the FY 2013 budget, allowing agencies to blend funding streams to “test better ways to align services and improve outcomes.”
• Expand the use of evaluation in existing programs. Agencies don’t have to rely solely on dedicated funding to support evaluations. They can add evaluation requirements into existing grants or contracts (which may require legislation).
• Include measures of costs and costs-per-outcome as a part of routine reporting. This can allow comparisons of cost effectiveness across programs.

In this memo, and in an accompanying memo providing agencies FY 2014 budget guidance, OMB encourages agencies to use cost-effectiveness data to allocate resources, pointing to a model used by a dozen states that rank programs based on the evidence of their return on investment. Several new approaches it encourages agencies to consider include:

• Using evidence as a factor in distributing grant funds. For example as a condition of participation in a formula program, applicants would have to adopt evidence-based practices, or in competitive grants, programs can assign points to applicants if they agree to integrate such practices.
• Use evidence as a factor to be eligible for more funding. For example, the Department of Education’s Investing in Innovation program and the Department of Health and Human Services’ Teen Pregnancy Prevention program use tiered evidence as a component of their grant determination processes.
• Use a “pay for success” approach. The Departments of Justice and Labor plan to pilot programs where foundations provide the up front monies and they are repaid with government grant funds only if certain outcomes are met. OMB also encourages agencies to use the relatively new statutory authority to conduct challenges and award prizes.

In addition to encouraging agencies to propose initiatives in their upcoming budgets, OMB asks agencies to designate champions for program evaluation and participate in interagency forums. It directs agencies to designate “a high-level champion who is responsible for program evaluation” and outlines the kinds of duties that person would have, such as creating a research agenda, attracting and retaining high quality staff, and providing independent input into agency decisions. It says this position could be held by someone in an existing role.

OMB says it will “reinvigorate the interagency evaluation working group established in 2010” and encourages agencies to participate in a series of interagency forums “to improve use of evidence.” These include forums OMB will organize with the Council for Economic Advisors, with the Performance Improvement Council, and via “communities of practices” being sponsored by the Office of Science and Technology Policy.

The memo lays out a remarkably specific agenda for agency-level program evaluation and champions the use of evidence in decision-making. It will be interesting to see how agencies respond!