One of my roles is to manage the Great Library. Another is to manage our corporate records and archives team. As you might imagine, there is some substantial overlap in skill sets between the two teams. But there are substantial differences in service delivery that mean that they have very different needs. We talked about one of those the other day: how to acquire a preservation system that will enable us to fulfill the 70-year plus retentions of our regulatory documents.
Libraries keep a lot of things but we tend to weed down to a core set of older materials. The type of law library you are in may determine whether you bother to keep anything ancient. Our collection goes back a couple hundred years.
There are a couple of luxuries a law library has that an archive doesn’t. One is that our older formats tend to be books and there is a tried and true way and relatively inexpensive way to preserve them. Another is that most current information is produced digitally and often managed through external providers like legal publishers.
I am not an archivist. In the current lingo, I am archives adjacent. I value our team’s subject matter expertise and know enough not to second guess them. This post reflects thoughts on decision-making for an archives team, not an archivist’s perspective.
We have a corporate records retention policy and schedule, as well as some awareness that those things exist. Like many organizations, we are making that transition where a highly paper dependent environment is pushing up against the digital possibilities.
On second thought, let me rephrase that. We have corporate teams that create documents that may or may not be records, and which may or may not be part of a greater workflow, and those teams want to stop storing (and pay to store) paper.
This means we are lining up some initiatives as our highly decentralized organization decides how to deal with digital records. There are lots of process issues (documents v. records, official v. unofficial) but one big one is: where will they live forever?
A Lifetime is … well, a Lifetime
I read a lot of technology media and rarely a week goes by when I don’t see a great price for a lifetime service or license to a product. That sounds great. Given North American mortality rates for a doughy white male, I’m probably going to be around for another 25 years or so. I can make a cost assessment based on price v. my own longevity.
Of course, that’s not what they mean. Or they may mean it but it’s not really a reliable world view. The web has been around for 25 or so years. Some of the biggest entities on it have been around for 15 or fewer years. Many, many technology companies have lasted only as long as one season of your favorite Netflix program
The reality is that you’re buying a product or license for the lifetime of the company. If they outlive you, great. If not, you may have some tough decisions to make.
It is common when acquiring a technology to understand what the obsolesence is. If I license a content management system, the content in it is usually in a format that is pretty portable. I recognize that in 3-5 years, I may want to move that web content into another CMS that provides [new | better | different] functionality or is cheaper, fits into our technology skillset better, etc.
The challenge seems a bit greater when you realize that the content needs to be stored securely and preserved for a very long time. I’m not sure if it is or not, but it definitely shifts the focus a bit when thinking about licensing a product.
Preservation Management Selection
Our archivist walked through the products he felt would best meet our requirements. Unsurprisingly, they had a lot of similarities and some features that might be a challenge for our organization.
Cloud or On Premises
Our organization has not embraced cloud computing (even though I wrote a book on it about 6 years ago). This is not to say our organization doesn’t use the cloud. It just means that there is still a lot of FUD, and no clear strategy, surrounding it.
All of the potential candidates offer cloud-based service, though, and it’s very appealing. Now that Microsoft Azure and Amazon AWS clouds have Canadian regions, this is a viable option without having to have the discussion about storing records outside Canada. That’s a huge obstacle to remove from our path.
The preference for “on prem” is a challenge for another reason, though. There is a lack of trust in our corporate technology team’s ability to maintain the records. If you adopt the three-copy approach to digital preservation, this may matter less so long as the corporate IT responsibility is only the dissemination information package (DIP).
It’s what the end user sees but it’s not necessarily what you are storing on a glacier server or whatever. If IT is responsible for more than one of your three copies, then you will need to have trust that they can manage it. In our case, we’ve already experienced DIP loss when a backup wasn’t running. It means that we – and other teams – are using external backup drives to ensure recovery if corporate resources fail. It’s not ideal.
Fake it until you make it. Until our archivist took me through it, the only dip I knew was for chips. Now I know AIP and SIP as well. More importantly, our archivist made me understand in a way that will make it easy to explain to decision-makers outside our information teams.
Since we’ll be looking at keeping two copies off-premises anyway, it may be simpler to keep all three off site. Otherwise, we’ll need a good method to move content from one system and network to another.
Open Source or Proprietary
This is another area were our organization is in flux. Traditionally, our IT teams have avoided open source technologies as a principle. This is somewhat undermined by the number of open source-based tools we have implemented, from Red Hat Linux to Firefox’s web browser.
I have a huge preference for open source. But the reality is that one decision-maker’s preference shouldn’t drive acquisition decisions. I heard recently of a CEO who likes a particular two-factor authentication app, so that’s what the entire company is going to use. It is not, in fact, superior to other products and the decision should have been left to the subject matter experts the CEO has hired.
It’s a bit of a red herring, I think, because what we really are concerned about is what happens if the company folds or we want to walk away. Will we be forced into a move to another provider during a wind-down, or can we keep the product code and run our own instance (in the cloud or wherever).
Open source provides some benefits in this area. None of our options are 100% open source but, to the extent they rely on something that is, it gives us some flexibility. Our final selection will need to be explicit in the contract about what software we can continue to use after our relationship with the vendor ends.
Our organization was caught out by this a few years back, when a vendor just closed and a key system was proprietary and now without support. Our IT team thought they’d licensed ongoing access to the software but they hadn’t. They ended up having to buy that access and are essentially running their own fork of the product now.
It’s not realistic to expect any of these vendors to be around for the life of our digital files. But as we start to shift our corporate files to official, permanent digital copies, our organization is going to have to be thinking about change on a much longer timeline. The selection of a preservation system will mean planning how to mitigate those gaps as we need to move from vendor to vendor and maintain preservation of files well beyond the career span of the people managing the records.