New Jersey’s 100 Data Fellows

If evidence-based decision-making is all the rage in government these days, how do agencies develop the skills to conduct such analyses? And how do agencies create a culture to support this management approach so it isn’t just another flavor-of-the-month fad?

New Jersey’s child welfare system may have cracked the code. A new report for the IBM Center for The Business of Government, by David Lambert and Julie Atkins at the University of Southern Maine, tells the story.

Background. Faced with years of spotty performance in ensuring the safety and welfare of at-risk children, New Jersey’s Department of Children and Families (DCF) felt good about its progress after a three-year upgrade in data and technology. By 2009, the authors wrote: “Managers could now turn on their computers and see in real time the proportion of investigations completed within the target of 60 days in each of New Jersey’s 10 areas and 47 local offices.” Nevertheless, even with all of this new investment in computers and data, departmental leadership felt performance had not measurably improved. Managers were not using the data to manage.

At about the same time, the US Department of Health and Human Services (HHS) had funded a program to help state child welfare agencies design and implement programs to improve system outcomes. New Jersey became part of HHS’s “Manage by Data” initiative and developed an 18-month program to train 100 Data Fellows from different levels within the agency. “This initiative succeeded in changing the technical skills, attitudes, and practice of a diverse group of workers,” note the authors.

The success of the initial program has encouraged New Jersey’s Department of Children and Families to extend the program to two additional cohorts of departmental employees, using their own funds, since the leadership realized this could not be a one-time investment.

Getting Started. New Jersey launched the development of its program in mid-2009, based in part on experiences piloted in five other states. Initial questions included: How many should be trained? Which parts of the agency should they be from? What training format would work best? What is the best approach to ensure leadership support for the initiative?

While the department initially considered training all frontline employees, there was concern by the program organizers that this might result in superficial skill development. It ultimately chose to train a smaller number in more depth, from different areas of the state, and levels of supervision and management. The training, they concluded, needed to cover technical data analysis; communication and presentation skills; and creating confidence among participants that they could use their new skills in day-to-day practice, with their own staff and colleagues.

DCF leadership targeted training to mid-level staff with at least 10 years with the department, and being competitively selected was seen as an honor that would enhance their potential for promotion. Participants – called Data Fellows – had to have the support of their supervisors in order to take part.

In addition, departmental leadership identified agency priorities which were built into curriculum design and became the focus of projects to be completed by participants, so they also became invested in the success of the program.

Launching the Program. The DCF Fellows Program launched in mid-2010. The curriculum was based on adult learning principles, particularly applied learning. It involved classroom work, coaching, group projects, and special events. According to the authors: “The 100 Fellows were divided into five groups of 20 each; classes were held at different locations across the state.” There were three phases of six day-long seminars covering: (1) how to be knowledgeable consumers of data, (2) using data to understand and manage change, and (3) how to put the diagnostic pieces together to improve outcomes.

Fellows’ projects were based on challenges that bubbled up from area and local offices, and were in the context of the departmental strategic priorities. For example, one group of Data Fellows “focused on improving parent/child visitation practice and examined the frequency of visits,” etc. They found children placed with kin were more likely to have parent visits (as expected), but fewer visits were occurring than expected because the visits typically took place in forbidding-looking welfare offices. So the group recommended better training around how to make the visits more welcoming for the families, so they would initiate more of them.

Impact of Initiative. The Managing by Data initiative has led to a number of changes in the way DCF conducts its operations. For example, Fellows found that offices that took the time to hold more frequent supervisor conference investigations with staff tended to save time and produced better results. By adopting this practice in a struggling field office, it was able to double the timeliness of investigations. In another case, Fellows learned that hotline call screeners had highly variable rates of anonymous calls (ranging from 3 percent to 50 percent). By observing the best screeners, they were able to develop a script for use by all screeners that resulted in far fewer anonymous referrals – dropping by more than 2,000 in an 18-month period.

More strategically, the Managing by Data effort gave Data Fellows the opportunity to exercise leadership and to serve as “data ambassadors” with their colleagues, regardless of their role within the department. It also contributed to the use of monthly ChildStat meetings at the departmental level as learning sessions to create culture around data — not a punishment or accountability session (which had been a prior experience). Area directors began to hold quarterly sessions on their own, comparing their performance with other area offices in order to improve their own.

Lessons for Others. The performance movement over the past few decades has been successful in creating a supply of performance information. It has had trouble, however, in creating a demand for it by decision makers. There has been a missing step – analyzing and presenting the data in useful forms. This has given rise to several technical efforts such as PerformanceStat meetings, GIS systems, and executive dashboards. But the New Jersey Managing by Data program demonstrates the importance of engaging managers and frontline staff in using the data they collect. The New Jersey program successfully developed staff capacity, helped develop a candidate pool for leadership promotions, and established a model for data-driven management. And that’s good management!