Notes on Data Retention (and other stuff)

This is a revision of an email I sent to a colleague. Some names will be redacted because I’m not sure if they would like to be posted here.

K,

I’m sharing this with Mark Patrick at Joint Staff because he’s a smart guy and may have some good stuff to say. He lives ECM at JS and has boots on the ground experience with the attitudes that we are facing. Mark also has an good explanation of the kinds of explicit and implicit knowledge. I think there is one additional type of knowledge and that is intuition or instinct. It’s the knowledge that is just “there” when we need it. It’s not the product of active synthisis or analysis. It may be an epiphany or something more subtle.

Here’s kind of a synopsis of enterprise information from a records management perspective. There are two kinds of information/data…records and “non-record material.” Records are temporary or permanent. They may also be “vital,” meaning they are the records an organization absolutely MUST have to continue operations.
Non record material is broken into a couple of categories “reference” and “personal materials.” There is also a gray category of “de facto” record, which is that product under development that has not been officially published(stamped) yet. The rule is that if there *is* an official version of the record, that is *the* record and the replacements that are in work are not records yet….the lawyers want those anyhow in discovery…but they probably don’t apply to PA/FOIA requests. If there is not an official version, then the de facto record is considered official for all purposes. In the real world there is one more category of information/knowledge and that is what I call “water cooler” information. It’s the result of brainstorming, mind mapping, chatting, and other creative activities. This information generally has a very short lived usefulness and very focused in purpose. It’s the “you had to be there” kind of information that doesn’t work well out of context. (It actually may work adversely out of context).

Retention:

1) Records. There is a legal definition and we are required by law to manage this data/information under an approved records schedule. Records are media and format agnostic. Records are TRUSTWORTHY and AUTHORITATIVE. D___ has a records schedule and J___ has added a couple of categories to help with our project generated records. I think P J may have some insight into this. Records can include sponsored blogs, wikis, and twitter like things, including social networks for posting complaints/resolutions/ideas. This can include IM and chat room/dco logs if they are designated as the “official” minutes of a business encounter.

2) Reference. References are things like uncompleted forms, dictionaries, books, algorithms, wikis, encyclopedieas, etc. They are kept around as long as useful. They may be authoritative.

3) Personal materials. This information does NOT belong to the organization and is managed solely by the individual. But, people need to be aware that “dibsing” information does not make it personal to avoid scheduling it. Personal materials would be copies of travel vouchers/receipts, notes made in a local personal file, etc. Personal materials stored in corporate space may be “discoverable.” These materials are managed by the person who owns them. This can include blogs, wikis, twitter and such done on non-corporately provided networks, even if it’s on corporate time. Not necessarily trustworthy or authoritative.

4) Water Cooler / Digital Detritus. This is information that may or may not “belong” to the organization, but certainly is useful to getting work done. It is also the kind of information lawyers love to find because it is so easily taken out of context. Given the short term usefulness and narrow focus of such materials, as a rule (IMHO) they should not be retained for longer than six months (I prefer 90 days, because it fits with the rule for turning “working papers” in to formally classified documents. If it’s not done in 90 days, it’s either destroyed, or an exception must be sought.) This can include IM and Chat rooms and most email. Blogs, wikis, twitter may also belong here if the space is provided for social networking specific to morale, getting to know you, etc. People should not expect this information to be trustworthy, authoritative or kept around for a long time. It should actually churn to keep the creativity levels up.

So, that’s the how long.

I tell people there are five key things you have to consider when managing records (and pretty much any other info)

1. Identifying and capturing – What is this and why does it exist? When was it created and by whom? Where was it developed? So things need to be looked at with an eye towards characterizing stuff and deciding what to do with it. If we choose to keep it for the organization’s knowledge sink, then we need to tag it, so we can find it again, and so we have some contextual reference/meaning for it when we retrieve it.

2. Storing and Protecting – Storage is affected by the medium of the thing stored, but it should always be easily accessible to people with appropriate access. The appropriate access thing can be a long discussion, but primarily is making contextually relevant information easy to retrieve in an exploitable format. There is no legal reason preventing record and non record material from sharing the same storage space. Protecting includes environmental issues such as mold and technology refreshes as well as keeping the hackers at bay.

3. Scheduling – Scheduling is setting up the environment so that information falls into the appropriate bucket off the “assembly line.” This requires some malice of forethought in considering what kinds of information the organization generates before said information is created. Scheduling should not be a burden on the day to day workforce and they don’t really need to be “Sit, Spot, Sit” trained to do records management. The law says that all federal employees are responsible to know their obligations, but that is generally limited to “Don’t take home documents in your underwear.” kinds of responsibility. Disposing of materials follows the record schedule and corporate policy, but tools must be provided to help the people responsible (not each AO, thank you very much) to identify what needs to happen and when. This includes packaging up permanent records to send to the National Archives to be included in the National Experience. It also includes tools to manage vital records, ensuring that information is routinely reviewed and backed up such that it can be quickly restored into an exploitable format should something ungood happen.

4. Finding and retrieving – Those tags and contextual information are critical to finding the appropriate information. But on top of that the information must be appropriately exposed to varying degrees of access permissions. And once the information is found, it must be easily retrievable in an exploitable format for a purpose that we probably didn’t know when we captured and tagged ten years prior.

5. Interoperability and portability. The cell walls don’t have to be completely porous/transparent, but there needs to be a method for exposing information to different perspectives and access permissions. We also need to remember that this is OUR data and not some vendors’ It is our responsibility to develop and maintain processes and procedures for getting OUR data out of one format into another and verifying the TRUSTWORTHINESS, AUTHENTICITY, and AUTHORITY levels of the transformed data.

Combined with an open corporate attitude towards information sharing and ad hoc cross CofC collaboration these things are probably going to generate a nice return for your Knowledge Management buck. If you don’t have the right corporate culture though…find yet another software toy, run it through the usual gauntlet straight to the shelf, and get yourself a beer or two when it’s finally over. On the other hand, if there is softness in the culture to change, and you are willing to be in it for the long haul, get a six pack, cause it is a long haul. Think 100s of years after you retire. I’m pushing the Records Management wet string up a frozen mountain for the next 10 years or so, cause I think it’s important. (I retire in 10-11 years. Hopefully when that happens I will have made enough headway that my great grandchildren 50 years hence are not facing the exact same intractable problems and losing history like we are. They can have new and improved problems to deal with.)

Does that make your brain bleed? It does mine. BTW, this email is a defacto record, because we don’t have a policy that we know of and we are starting to work one, but when the real policy gets signed or something better gets drafted, this can be deep sixed…because for right now, it’s got a fair amount of “jana’s perspective” embedded in it.

Hope that is somewhat useful,
jana