Expertnet Prototyping Using Quora: Participation Metrics at Week Two

Early on during the Expertnet consultation last year, I shared the following (sadly, the Expertnet wiki is locked due to its paid subscription having expired, but Google still has most of it cached):

re: Notifying Experts
tbonnema Dec 18, 2010 12:43 am

It looks like ExpertNet is trying to solve at least two distinct problems:

1) The matchmaking piece between experts (however defined) and opportunities to contribute: For some time now, I have suggested the creation of a national participation calendar (http://bit.ly/h7mwmv, http://bit.ly/6h18sc), so maybe this is something that ExpertNet could help accomplish.

2) The consultation piece once a group of participants has been assembled: There is already a broad range of tools available today to support these kinds of processes (see http://ParticipateDB.com). Depending on the circumstances, some are more appropriate than others, yet success will to a large extent depend on “soft factors” unrelated to the technology. My question is whether ExpertNet should build or buy a solution or whether it should rely on existing tools on an as-needed basis.

(Full disclosure: My company created ParticipateDB, and we’re developing a tool in this general e-consultation category.)

Just as a reminder, the range of processes Expertnet would potentially aim to support basically came down to this key question:

2. Fact-based advice or deliberative policy creation? The original draft focused on seeking verifiable, fact-based advice from citizen experts. However, later statements have hinted at a significantly broader scope that would allow officials “to pose questions to the public about any topic we’re working on.” Given the nature of many of the topics mentioned (job creation, preventing homelessness among veterans etc.), ExpertNet would have to support policy consultations that are much more deliberative by nature. This has huge design implications.

This general idea of a more agile approach whereby citizen expertise is solicited following a variety of processes and using a variety of existing tools was supported by several other participants. Here is Tim Huegerich (Google cache):

The benefits of using already established, external platforms for responses
TimHuegerich Dec 30, 2010 9:23 am

1. Easy and cheap – less work and expense for setting up ExpertNet
2. Allow the external organizations to perform moderation, etc., sidestepping concerns about how to moderate without the appearance of “Big Brother” trampling free speech.
3. Furthers the original principles:
   • Participation must be easy. (by allowing participation through platforms than citizens are already familiar with)
   • We are not building a new Facebook. (the original proposal does not really come to terms with this principle because it assumed that a community would form around the new platform to moderate responses, etc. – this alternative is much more consistent with the principle)
   • Innovation requires experimentation. (By allowing various partner platforms to compete in generating useful responses, this proposal encourages experimentation and innovation.)

David Karger also made the same point (on Januar 23, 2011), saying that “… I will argue as I have elsewhere that instead of building a [government-sponsored social networking site] we should look to leverage an existing system.” (Google cache).

They really did get a lot of smart answers during the Expertnet consultation, didn’t they?

To this date, there has never been an official follow-up or debriefing, so we can’t know for sure whether the current experiment is intentional or even related. But last week, the White House announced on their blog that they would be using social knowledge site Quora to solicit input on a number of questions: We Want to Hear from You on Quora: Announcing the Startup America Policy Challenge

From the post:

Yesterday, the White House announced $2 billion in public and private resources to help entrepreneurs start and grow their businesses. In the spirit of open and participatory government, we also announced the Startup America Policy Challenge. We’re calling on entrepreneurs and the broader public to share their ideas on how to accelerate entrepreneurial innovation in the areas of healthcare, energy and education. Aneesh Chopra, US Chief Technology Officer, kicked off the challenge in a post on Quora and asked a few questions to get the dialogue going.

[…]

All Americans are invited to reply. But I’m especially interested in hearing from entrepreneurs in these areas – and so is President Obama’s Cabinet, particularly Energy Secretary Steven Chu, Education Secretary Arne Duncan, and Health and Human Services Secretary Kathleen Sebelius.

You don’t have to be a policy wonk to respond, and you don’t need to describe a detailed policy solution—there are others in government and in academia who will help us with that, as part of the Startup America Policy Challenge. Just tell us what’s on your mind.

So have a good idea? Let us know…

Let’s look at participation metrics to date for the three questions posted:

Education: In the U.S. education system, what can the government do to best enable the use of new learning technologies?

• 18 answers
• 12 comments
• ca. 70 votes
• ca. 6,000 word count (not including comments)
• ca. 325+ average word count per answer
• 516 views
• 53 followers

Energy: In the U.S. energy system, what can the government do to best enable the use of new clean energy technologies?

• 22 answers
• 29 comments
• ca. 50 votes
• ca. 6,000 word count (not including comments)
• ca. 250+ average word count per answer
• 609 views
• 65 followers

Healthcare: In the U.S. healthcare system, what can the government do to best enable the use of new health information technologies?

• 8 answers
• ca. 40 votes
• ca. 3,500 word count (not including comments)
• ca. 425+ average word count per answer
• 297 views
• 37 followers

At a glance, it looks like overall participation rates are still fairly modest so far. However, individual responses are much more detailed as compared to the roughly 125 average word count per comment we saw during previous online dialogues on Change.gov and the Open Government Dialogue (see our metrics update from June 2009).

Quora provides a few distinct features that allow participants to collaboratively improve their answers over time. Unfortunately, there is no easy way to measure the level of activity various answers have seen or whether they have been modified, but it is something that occurs regularly on the site. This an other design choices, such as the limitation to allow only one answer per participant, can greatly help improve quality and reduce overall word count.

As far as I’m aware, no end date has been defined by which input has to be submitted in order to be considered, nor has it been made clear what will happen with the input. Also, I’m not seeing the conveners (Aneesh Chopra and team) actively engage.

Having said that, this is a promising approach that will further advancing the idea of harnessing citizen experts. Assuming that the policy questions at hand are actually ripe for public input, using an existing tool is one way to get there relatively quickly. Given the intended audiences (technology entrepreneurs) and the type of input the White House is after (policy ideas), Quora appears to be an appropriate choice of tool.

If Expertnet were ever to be built as its own tool (which may or may not be necessary), it is experiments like this that will inform its design. Hopefully, we’ll see many more of these.

Original post