Considerations for long-term key storage (paper backup, media for vault storage)?

Question

I'm reviewing our company's security procedures regarding cryptographic keys and master passwords, and also my private system at home. The key generation process is pretty much the same as that outlined in Recommended operational security for generating one's primary key pair?. Current procedures for paper backup and media for vault storage are more than sufficient for the operational lifetime of the material, which is at most three years for the certificates used for data exchange in the German health care system.

What I'm having trouble with is long-term storage, though, both for the paper-based part and the 'digital media for the bank vault' part. For several reasons in can be necessary to use certain keys a long time after their operational life has ended, even decades later. For example when digging in old archives, or when trying to read a really old PGP message.

And it is the 'decades later' part wherein lies the rub. I couldn't find good data for the durability of printed matter, beyond various recommendations/requirements to use dot-impact printers or black-and-white laser printers. Laser printers offer higher quality and hence higher possible data densities but from the info I found I couldn't say whether their printouts really last longer than those of dot-impact printers or not...

As regards digital media, those in current use all have an expected durability of only five years or less, except for flash devices and hard disks to which a somewhat longer - though unspecified - durability is attributed. Our only long-term durable media - MO disks - had to be scrapped and shredded when we retired the last available MO reading device.

Based on my unscientific Googling I'm currently favouring USB sticks for digital storage, as they seem to combine long-term data retention (internally) with a good prospect of long-term accessibility via the USB plug/bus interface (externally), but I'm not quite satisfied with the scarcity/quality of corroborating information that I've found so far.

Another consideration is that printed matter can be inspected visually but digital media can not. There's no indicator that the data might be hanging on only by the tips of its fingers (figuratively speaking) and the next stray cosmic ray might blast the last 'good' electron from its perch and thus cause data loss in a cell. Digital media might be close to a catastrophic collapse without anyone being able to detect that fact.

I couldn't find recommendations as to a possible refresh write every n years or so, or regarding suitable storage formats with high redundancy (ECC/Reed Solomon). Given the size discrepancy between key material and storage capacities, even a hundred-fold size increase would be perfectly acceptable. It might be possible to put the redundancy features in tools like RAR to good use here. However, I have no experience in such matters, and procedures/schemes designed by amateurs pondering first principles tend to be cr*p in actual practice...

I'd appreciate pointers to studies, papers and articles regarding the secure long-term (vault) storage of key material, both on paper and on digital media. I'd also welcome pointers regarding practice and experience in this context. The focus would be mainstream tools and technology of the sort accessible to a small company or a private citizen like me, not specialised tech with price tags of four digits or higher.

Note: I'm also interested in the practicalities of paper backup, in particular tools that allow adding error-detection and error-correction capabilities similar to what RAR does for archives (i.e. dial-your-desired-security), and perhaps encodings that are more efficient and/or robust in the face of OCR than, say, base64.

Update: durability of print is the subject of ISO 11798 which helps with googling related information. See Erstellung von alterungsbeständigen Personenstandsunterlagen at the Landesarchiv Baden-Württemberg (in German), for example, or HP Supplies Print Permanence and Durability. It seems that laser printers have no problems passing the tests (whereas ink jets have to work at it, hard).

Answer 1

What about something like Verbatim's Archival Grade Gold DVD-R>? It gives 4 GB storage. They claim a life of "up to 100 years" but I find no independent verification. Frankly I'm skeptical. But maybe you can boost your chances by just burning the same data on to multiple disks and hope any failures are uncorrelated? I'd buy disks from different lots or different manufacturers too. The hard part might be ensuring that the drives needed to read these disks are available and functional when the time comes (perhaps beyond your lifetime)

Also look at this new disc technology from Millentia called M-Disk They offer an incredible 1000 year life estimate but I'm an eternal skeptic. Caveat Emptor.

If you really want to splurge & the keys you want to store are not too large in size (think MegaBytes and not Terabytes) you could look at Norsam Technology's HD-Rosetta product. They actual use electron beam etching to etch your data on to a metal platter. I suppose that should be archivally very safe. Though I cannot imagine anything more inconvenient to read back. Norsam provides an addressable microscope for automated readbacks. An independent evaluation of the durability done by Los Alamos Labs years ago exists.

Another option is to outsource it to someone like Amazon. Have you looked at their Glacier project? They claim an average annual durability of 99.999999999%. It is targeted specifically for long-term data archival storage.

A fourth option that comes to mind is to not treat the archive as a cold storage, dump and forget but actively manage it? This is speculative but I'd be happy deploying a highly redundant RAID array and then spinning it up on a cycle often enough. Sure, disks will die but you can replace them. Dealing with technology obsolescence would be tricky but could be managed. At least the fact that you can actually read the data, say, at 6 month intervals gives you confidence that the data will actually be accessible when you do want to read it.

A highly non-conventional way of doing this would be rolling out something like Hadoop Distributed File System (HDFS) on existing servers your company already owns & just keep a policy to maintain the system & port the nodes to newer hardware as and when the company grows.

My experience has been that if data isn't being regularly read it gets hard to be sure that it can actually be read when the time comes.

In general I would try to not put all your eggs in one basket. If you have the budget incorporate redundant strategies. Hedge your risks.

A cynical question to ask would be, what's the expected life of your company anyways and is the data expected to be able to outlive the company? If so, you really need either a third party solution (where you trust the third party's ability to survive more than you trust your own company. PS. Who pays for storage once your company is dead?) or a true cold storage solution where the data just continues to reliably live on and on (where would you physically keep the media if your company no longer exists?) irrespective of if the company exists.

Answer 2

@curious_cat has a great answer.

I'm going to add a few other areas, though.

First, what's your budget for this? You need the budget in three areas:

• Capital and operational expenditure for inital start-up
  • buying HD-Rosetta capable addressable microscopes, equipment to etch stone or clay tablets, OCR scanners that can handle stone or clay tablets, etc.
  • multiple times, for geographically redundant sites
  • additional expense for secrecy - if you're storing encryption keys, they should also be protected.
    • this makes outsourcing either reading or writing very difficult over a decades long timeframe - the new owners of the vendor you used may well have your data, and may not care about your company's privacy, and you may have limited legal leverage, particularly if they're in a different national jurisdiction than the original owners.
• Ongoing operational expenditure
  • Keeping training materials up to date
  • training new employees over decades on procedures they'll only use once
  • regular (no less than annual) random, verifiable test reads
  • adding additional data to the long-term store
  • equipment replacement as time goes on
  • geographical redundancy - as the company changes, adjust your storage sites, as the company may move operations to or from countries or even continents
• End of life expenditure
  • Unless you choose stone or clay tablets, you're going to have to continually transition to new technologies over the years.
  • End of retention for whatever data reaches final end of life probably requires its destruction, which is interesting and fun for stone or clay tablets, but difficult for outsourced storage
    • i.e. how do you get them to purge the data from their own backups?

Answer 3

I'm also interested in the practicalities of paper backup, in particular tools that allow adding error-detection and error-correction capabilities similar to what RAR does for archives (i.e. dial-your-desired-security), and perhaps encodings that are more efficient and/or robust in the face of OCR than, say, base64.

Have you considered combining paper storage with machine readable data? I am thinking encode the key data on the paper not only as human readable printed text (as a last resort) but also in a "machine readable" format so it can be easily re-imported without having to either trust OCR or type each and every character and hope you get it right. Something like (depending on your key size of course) QR codes (they can take up to 2,953 bytes of data in each QR code).

Answer 4

Backups and Archives Oh My!

Based on my unscientific Googling

First issues first, let's turn this into a scientific exercise (library science!), the decisions you make today will affect people that you haven't (or might never) meet.

You've mentioned two of your three options for key management; analogue and digital. Have you considered your third option? Decryption of all data that uses personal ephemeral keys and then perusing storage and bulk encryption of the actual data only. Key escrow is a tricky business, one that your organization may not fully understand the ongoing cost it presents to the organization (both in time and money).

You need to stop thinking in terms of "backups" and start thinking in terms of "archives". Backups are short lived compared to archives, and they do have their own definitions and expectations on the phrase "long-term".

IMO flash thumbdrives (built cheap and not protected from memory decay or the stray cosmic ray) are not appropriate for archival systems.

Bet let's approach this one topic at a time.

Paper Archives

There are many resources for archival storage of printed materials.

Our good friends at ISO have a series of standards for paper document preservation:

ISO/TC 46/SC 10 - Requirements for document storage and conditions for preservation

Here's an example of what ISO 16245 describes. (Direct PDF Link, ISO standardized boxes)

Align your paper documentation with these practices and you probably won't go wrong (barring environmental affects, even the library of Alexandria was burned down after all).

Digital Preservation

While paper is pretty easy (relatively speaking, chemicals and temperature work as expected over the course of decades), digital preservation is much more complex. You need to deal with not only the expected decay of information (as mentioned above, decay of the storage medium as well as soft-errors introduced from the environment). You must also deal with the pace of technology. How hard is it to currently find a 5 1/4 floppy drive and software that will support the drive on a modern OS? How hard will it be decades from now?

The great thing about Digital Preservation, is that there are so many standards to choose from (sic), wikipedia actually has quite a nice run down of the current digital preservation standards being worked on around the world.

Your Data Preservation System should follow a widely adopted standard, if your intent is to provide a long-term and inter-operable data archive. Otherwise you are just creating backups.

The Third Option

In general cryptography is applied to data that can have either major impacts on short term operations of an organization (e.g. tactical co-ordination) or impact major long term operations (e.g. strategic positioning). So the question for the third option is this:

What value, in practical terms, does keeping data encrypted with ephemeral keys provide to your organization?

Archives need to be searched (why else would you be keeping the data?), either by an auditory agency or internal organizational resources. Retention and Search for ephemeral keys adds needless complexity to the already complex issue of Digital Preservation.

Keeping your data encrypted with personal keys (that you are holding in escrow, at great expense of money, time, and attention) just increases complexity needlessly.

We can consider that personal keys should be used for "tactical" purposes (short-term), and your data archives should be encrypted for "strategic" (long-term) purposes. This would mean a symmetrical key would be used on your actual data archives of your de-crypted public and private messages.

At this point you might be thinking, "Well we just traded one problem for another." Yes and no, we have reduced the problem space considerably by removing ephemeral keys (generated over the short-term) and replaced them with long standing (well-known) keys to be used on your backups.

Closing Thoughts

While you have only talked about key management. Who manages the data itself? What archive system do you currently have in place that will actually keep data for decades? If you can't, then why worry about the key management?

If you already have a multi-decade archive system, use the third option, remove the ephemeral keys from your data and then encrypt your backups in bulk. Protect those bulk keys in a manner that is appropriate (two person integrity, ACLs, or a few of those Verbatim Gold disks under Gates, Guards, Locks, and Guns).

Answer 5

First of all, please take @Joseph Kern's answer seriously!

Think how hard it is to play an audio tape or access the data on a floppy disk nowadays—both storage formats were prevalent and widespread in the 1990's, only two decades ago. You mentioned USB sticks: the standard USB-plug will be gone within a decade, replaced by the new USB-C plug and that will probably be gone a decade after as well. Whatever physical media you choose, you should store a compatible "playback device" along with the physical media, e.g. a computer that has a USB port and will still boot in a few decades to display the data on the USB stick. And most consumer electronics is not built to last that long.

Two additions I'd like to point out:

• (acid-free) paper
• (digital) microfilm

For small amounts of data (encryption keys etc.), paper has the significant advantage that you can read it without any tools: no microscope or compatible hardware and software needs to be preserved along with it. In particular, acid-free paper is a good idea and should be fine for a few decades, as long as you don't use ink that decomposes the paper or fades (that's why dot-impact printers are preferable). If you're looking into centuries, you'll have to take into account proper storage and other preservation practices as well (acid-free boxes, humidity, temperature, etc. plus organizational practices: what person will still know where the stuff is and how to retrieve it in a few decades and how will people interested in the matter find that person).

For storage of large quantities of digital data, printing to microfilm may be a very good option. (Don't use any media that have been around for only a decade like DVDs. There's really no way to be sure how long something lasts except to wait and see.) Traditionally, microfilm has been used to save miniature versions of images to film (similar to an analog slide or analog film). This has the advantage that the only tool a human needs to read the documents is a microscope. However, to get back to digital you would need OCR for text and for images invariably lose some data (think noise, scratches). The one advantage of digital data concerning long-term storage (along with its many disadvantages mentioned above) is that checksums etc. can be used for error detection and correction, so even if some bits are unreadable after a few decades, you might not lose data. So instead of printing a miniature image on microfilm, you can save digital information (something like printing a black dot for 1 and a white dot for 0). However, this process again requires special software to get your text or images back into human readable format. Needless to say, anything you should only ever use well-documented open source file formats for longtime storage (like plain text or PDF/A).

But again, you really should be talking about this to your organization's archivist. This is not something taught in 101 computer science courses.