ASCII by Jason Scott

Jason Scott's Weblog

From the Mailbag: Archiving Yourself —

Date: Wed, 7 Feb 2007 00:40:07 +1100
From: David Dean
To: Jason Scott 
Subject: article request

Do you get many (any?) of these?

Anyway, I figure as an obsessive hoarder, you've got some great system
for storing your old files - notes about projects ideas, half
completed code-bits, word files that you might just want in six months
but have no use for currently .. etc etc.

I'm in the process of copying everything off my desktop right now, and
I figure someone who cares so much about storage must have a better
system for this than me - I tend to just lose everything every few
years when I upgrade computers.

I think the fact that I've not once rummaged through burnt discs
looking for a file means that my system works for me .. but maybe
you're using tricks us mere mortals can learn from?

Thanks!
David

Thanks, David. Since I’m now up to a six-times-a-week blogging schedule, I have the space to take specific requests from folks who have questions and then ruin their opinions of me by answering them. In your case, I’m reading the question this way: how does one keep track of their own old shit? Or, to be even more specific, how do I keep track of my own old shit?

There’s entire courses in this, but here’s what I do, basically.

  • Decide if something is worth saving, and dispose of it immediately if not.
  • If something is worth saving, encapsulate it.
  • Encapsulate the encapsulations.
  • If possible, keep many copies of the encapsulated encapsulations.

This sounds a little weird and general because I use the same methodology for computer files, papers, books and magazines, and artifacts. At this level, it works for all of them, and it’s generally what I do.

Honestly, a lot of it comes down to realizing what you’re going to save and what you’re not going to save. If you’re storing it, you’re saving it. If you’re saving it, you might as well take the effort to save it well, or throw it away. I am reminded of a story by an old associate of mine. He’d moved from one apartment to another, and then to a house, and then to a house he was buying. In the basement, he found about a dozen boxes he’d packed while moving from the first apartment. He’d not opened them in six years. His decision? He threw them away, sight unseen.

I would never do this, but I understand his thinking. If he hadn’t needed the stuff in six years, he would likely never need it. But this pre-supposes that he would have a need for everything he owned at least once every six years. But there’s stuff that doesn’t fall under that: old medical records, family heirlooms, baby photos… basically, nostalgia and historical items. He’d decided he wasn’t a historical person, and moved accordingly. I have decided I am and therefore I save a lot more and continue to.

Assuming you’re a historical person, then, the goal is to take a few steps now that are easy to do and which will give you the most leeway down the road.

Physical artifacts are probably the easiest thing for me right now; I have a pile of comic book sleeves and backing and tape, and if some old pamphlet, magazine or catalog interests me, but I know I won’t be looking at it again any time soon, I bag it, tape it, and throw it into a box. After a while, I’ve got tons of these things in boxes. The boxes are in my attic. Maybe I’ll end up regarding them later and throwing them away (unlikely) or sorting all the catalogs into one pile and the magazines into another (very likely) and then sub-sorting the magazines by a specific issue set like Spy or Mad Magazine (also very likely). What happens then all depends, but I’m leaving my options open here. I don’t have an impossible-to-go-through-without-ripping-something pile of old magazines in the corner, and I don’t have the task of re-sorting things again and again because I keep leaving the piles untouched. This method also works for old video games, business cards (I have a special case for these) as well as CDs (I have cases and cases of old CDs, labelled “Shareware” and “Personal).

So we’ll leak over here into “files”. As I mention in a talk I gave at HOPE one year called “Saving Digital History” (which, I may add is itself, saved digital history), I think there’s a little too much stress put on getting things “perfect” than just saving things as best you can. If you can take the time to demarcate and describe everything going in, great. But if you can just drop things into general stuff like “Photographs”, “Audio”, “Movies”, and so on, you end up with piles that you can later go through quickly. For example, it used to take many days to browse through JPEG files; it would take up to 30 seconds to “render” one to the screen. Now you’re faced with a default thumbnail selection rendered instantly. Who’s to say it won’t get even faster, which recognition software that you can say “return every photograph with this car in it”? But the best thing to do is just drop them into massive directories and then subsort from there.

There is the potential to lose ethereal data, like “The reason I have this file is because my buddy steve told me to hold it” or “this was a girl I slept with and then never saw again”, where anyone but you seeing it will not know “the story”. There’s ways to attach these meta-descriptions to files, and if you can, good, but if you can’t, you can’t. I’d rather err on the side of data than not.

In the case of hard drives, we’re often very lucky that hard drives have increased so much that you can actually take an entire old hard drive and drop it in the new one as a folder. I have this situation, with a “Old Work Hard Drive” I have from my game industry days, as well as later computers I found. I have zipped up directories, collected images, and documents ranging from love letters to games I never finished. Some of it still has meaning and some has lost all meaning to me, but because disk space is so cheap, I just keep shifting it around.

And there we get to the final bit of our little sketch: backing up. I use a number of programs to back these things up, notably Synchronize It! and rsync to keep multiple copies across multiple machines. I also, when I have time, burn DVD-ROMs of everything I can, although who knows how well that’ll hold up.

There is data I consider “vital”. These are financial records, writings, and of course my documentary data. In those cases, I have a rule of “three hard drives, two DVD-ROMs”. The data has to be in those five places or it’s unsafe.

Could I still lose data? You bet. That’s the risk of being alive: of dying. You have stuff, you might lose all your stuff. Not doing your best with what you have because it can’t be perfect is no way to go. Sometime you’ll find an extra weekend or a bored evening and you’ll sift through your old crap and make it a little better. The key is to make it so the jump from “wow, a lot of crap” to sorting it is very short. This is why I have the magazines in bags and the files in folders. I just go in and make it “a little better”. That’s all we can ask.

I’m not perfect; I just discovered a day or so ago that some of my old macintosh disks are actually getting moldy. I’m going to go get them transferred off where I can and save the data from them… then encapsulate them… then encapsulate the encapsulations….


Categorised as: Uncategorized

Comments are disabled on this post


4 Comments

  1. fuzz says:

    Games Industry days? I don’t think we’ve heard any ramblings from those days, more please.

    {btw, for syncing files, I use Pathsync
    http://www.cockos.com/pathsync/
    (by Justin Frankel), just in case anyone wants more choice}

  2. If someone wants to get rich, I came up with a great idea that would revolutionize the search engine industry; create a program that allows people to attach metadata to any file. I have a much longer diatribe that I’ve been rattling off to anyone who will listen, but that’s the general idea.

    I’ve been using a program called Advanced Disk Catalog to create a software list. CDs and DVDs full of programs, music, files or whatever can be sucked into the program’s database in a matter of seconds. The program allows for exporting to several friendly formats, and has a nice built in search feature. The program does allow for user added notes and information, and if you’re archiving zip archives, it’ll suck in the file_id.diz files automatically as well.

    One problem with saving all those tiny text files is that it can take a long time to build up to 650 megs (a CD’s worth) of data, and if you lose a hard drive before it’s been archives, it’s bye bye bits. Make sure you keep two copies using on of the previously mentioned sync programs. If you want to be really secure, find one that supports FTP and back those directories up nightly to an online FTP site somewhere else. Having two copies of your irreplacable files on the same computer won’t help you in a house fire.

  3. Chris says:

    I’d also like to say taking one copy of your backups offsite is mandatory. Buy another external drive and once a month take your backup to your safety deposit box or lacking that — a good friend’s house. Natural disasters can and do happen. I’d also recommend scanning all family photos for this same reason.

  4. Michael Kohne says:

    fuzz, thank you. I’ve been looking for a tool like PathSync for some time now.