Agile Document Development: Part 2 of 2: Applying it to documents

The point of these posts: Agile methodologies used by software (and web) developers can be adapted to provide similar value to the rest of us. Not just for techies any more!

Waterfall document development.

I put it to you that document development in the OPS also tends to follow a waterfall methodology.

(If “waterfall methodology”doesn’t mean anything to you and you want an explanation of what waterfall and agile methodologies are, and how I came to write this post, read the previous post.)

In our document development, as I’ve experienced it in going on twenty years in the OPS:

1. A person or group is tasked with preparing a document (briefing note, strategy, policy, what have you). The first phase is research. They do an environmental scan. They look at the literature. They consult stakeholders and executives. This is the phase when they are really open to external input. (It’s a bit like requirements gathering in the IT waterfall.)
2. Taking the results of that research, the person or group will do analysis. They will come up with one or more options and a recommendation. They will share the analysis, options and recommendation with management to gauge temperature and get direction for writing the document. (It’s a bit like solution design in the IT waterfall.)
3. With the direction from management, the person or team goes off and writes the draft document. Drafts go up, revisions come down until the document is at a stage that the approvers are comfortable with. (It’s a bit like development and testing in the IT waterfall.)
4. Once all the revisions are complete, the document is sent up for final approval. (It’s a bit like the move to production in the IT waterfall.) Then people answer questions about the document and eventually look to update.

Sound familiar? As the IT waterfall involves a lot of contact between IT and the business at the beginning and end, but has IT developers beavering away without business contact in the main development stage, so this also involves our document developers drafting behind closed doors. How often have you asked for something only to be told it wasn’t ready for sharing yet? I’m not talking about sharing with the public, I’m talking about sharing with other public servants.

It doesn’t have to be that way. But the culture (and methodology) encourage it. Just as a good waterfall IT project manager wants to protect his developers from the business during development, to avoid distractions and changes of direction (scope creep) after the requirements have been locked down, so a manager wants to keep his team shielded from external influence. And just like the IT team doesn’t want to show their development for user acceptance testing until they’ve completed their own set of tests, to reduce the risks of embarrassment from buggy software, so the manager wants to hide the document until it has cleared approvals.

The similarities go further. Just as waterfall is a “big bang” methodology, working to solve the who problem at once, our document development strategy does the same. We don’t deliver our social media guidelines one guideline at a time. We work on the document until we have the perfect guidelines for all social media. If we are working on a new procurement directive, we don’t release it one procurement type at a time. It is built and approved as a complete package.

Once you see the parallels with the IT waterfall methodology, wondering what our document development process would look like if it were more closely aligned to an agile methodology is natural. When challenged with talking about leveraging the “Culture of innovation and collaboration to work smarter”, I was driven down that path.

Agile document development

What it is

Let’s see what an agile document development methodology might look like. Here’s one possible flavour that I’m more or less making up as I’m typing. Consider it a first sprint prototype.

1. The person or team responsible for developing (or coordinating the development) of the document meets with the executive9s) who are asking for it. They get a list of the types of problems it is meant to solve, the situations it is meant to address, the questions it is meant to answer. Broad problems or questions are broken down to a group of smaller, more specific ones. These are prioritized. What are the most urgent/important ones? Which should be tackled first? These get tackled in the first sprint. The rest go into the backlog.
2. Research is quickly undertaken and a draft solution is documented. To keep things quick, a couple of strategies are used:
   • the problem is narrow so the research can be focused
   • the research and analysis is crowd-sourced. Input from others with similar interest is actively sought. The research and analysis are done out in the open where others can see and contribute.
3. The draft solution is shared with the sponsor. Minor changes are noted for edits. More significant changes are added to the backlog. If significant changes are suggested, the sponsor decides whether the document is okay for provisional release (i.e. despite the need for change, is better than the status quo). If not, the changes become the focus of the next sprint. If so, the document is released as plan/strategy/guidance as of DATE. The changes go into the backlog. The backlog is prioritized. The highest priority changes or other problems/questions are selected for research/analysis/writing in the next sprint to refine or extend the document.
4. Repeat forever. After any sprint the paper can be released as “best available plan/strategy/guidance”.

Opportunities/challenges

As you can see, the process parallels agile software development pretty faithfully.

It mitigates many of the same risks. It enables constant updating to ensure that the document can keep pace with a rapidly changing environment. It enables quick delivery to meet the most important problems/challenges as well as providing for continual growth and refinement while providing plenty of off-ramps.

There are, however, some challenges we’d have to sort through. Many of the larger documents we work on, the ones that might benefit most from an agile approach, go up pretty high in the hierarchy for approvals. It’s quite possible that the senior executives responsible for providing the approvals won’t be ready to review and approve changes on a regular, rapid schedule (every few weeks, if it parallels software sprints).

One way to approach this challenge might be to look at major and minor releases. Release the various versions of the document as “unofficial, beta” until it is substantive enough to warrant going up for formal approvals. In the meantime, it can still provide “best advice of experts”. After approval, further sprints will extend with “unofficial, beta” advice until they, too, get rolled up in a new, approved version. If a sprint includes an important change to the approved document that is required by changing circumstances, then either (a) a new major revision could be approved and released or (b) that single change can be brought up for approval and released as a memo/edit to the official document in advance of the next major release.

Another challenge will be information overload. With all of these fluid, always changing documents being released, it may be challenging to keep up with the guidance – or rules – relating to our activities. Of course, we have this problem now – how many of us are really familiar with all of the directives, policies, strategies, etc. affecting our work? How can we know that we are? But it will get worse if guidance is being released frequently, piecemeal and constantly changing.

I think some sort of central repository bringing together this knowledge would be pretty key to addressing this (even for our current state of waterfall documents). I’m starting to think of some functionality we might want to add: tagging to indicate what the document relates to, the ability to subscribe to documents to be notified of changes or to tags to be notified of new/changed documents. If we think of documents as solutions to problems, they could be tagged with the problems they are solving. Then I can subscribe to “my problems” and ignore “not my problems”.

I think this is a solvable problem and the solutions would be useful with or without agile document development.

In conclusion

I think there is a lot of potential here. This is only a couple of days thought. I’m sure with others approaching the problem of agile document development, we can rapidly refine/extend these thoughts. 🙂

Of course, agile is only one approach we may want to consider learning from. Anyone want to do a blog post on open source document development?