ASCII by Jason Scott

On the post I did about Floppy Disks, that’s gotten a lot of attention in its own right, there was a comment the weblog software marked as suspicious, and possible spam.  The content itself was rather innocuous and in theme:

I will always keep a few floppies and a 3.5 inch drive. Can’t install older versions of XP or 2003 without them to load custom drivers at setup.

I also have the 5.25 inch drive and disks (just in case).

In 25 years some government secret on a floppy will need to be read and I’ll have the last drive and hardware to read it. Then I’ll be rich. Mark my words.

But yes, it was definitely spam – the website attached to the user, their e-mail address, and the name of the user were all references to a specific fruit that appears to be the new thing garnering the need for attention and weblink juice to get its message of “please fucking buy me” out to as maximum an audience as possible. It is, without question, spam.

There’s a range of commentary I make where I am probably not saying something new to a lot of people, and I get a lot of response of “well duh”, but I still feel the need to mention it, for historical context later.

Spam, as you know, is the idea of unsolicited commercial speech being thrust upon audiences without their consent, assent, knowledge or awareness. I’m going to leave it at that general definition for the purposes of what I’m talking about.  Originally a reference to endless messages on Usenet (and emulating the Monty Python Spam Sketch), it expanded out to e-mail as e-mail became more prevalent. From e-mail, where it got the most hand-wringing and programmatic work to divest it away, it has followed, at various paces, to every single platform in which any amount of people congregate. Spam has cropped up in twitter, facebook, online games, IRC chat, Flickr, wikis, and places infinite.

Someone solved the spam problem a while ago: go after the tiny handful of banks that the spammers take payment from.  But, knowing how to solve something and solving it are two different plateaus on a very large landscape with dangers and silliness fraught between.

So instead, let me focus on what I’m observing here, in this specific situation, and why I think it needs to be made into a historical note.

A person made this. For me.

It was weird to have machines scripting spam in the 1990s beyond the standard “and take this pre-written pitch and shove it in as many newsgroups/e-mail addresses as possible”. You wouldn’t have it, say, look at the writing of a newsgroup, re-arrange the last major topics talked about, plagarize items from a year or two previous, and compose a “new” message that turned out to be links to spam. If Usenet had survived into the later 2000s, it would have. And that’s what I’ve been seeing – scripts that knock on my doors with composed text taken from the weblog itself. So there’s been advancement there.

But this… I checked. This is a valid comment. They’re saying something reasonable and clear, something that would be a perfectly fine comment about Floppy disks, their perspective to them, and what they might have said if they were some pleasing retro-nerd happy I was talking about 5.25″ floppies. It was composed for real, for this weblog posting.

This is an enormous human cost, compared to just scripting once and blowing it out across a thousand million weblogs.  It means, and I knew it was coming to this, that hiring humans to do things so bone dull has become enough of a commodity that spammers are employing this to get their messages injected into the conversation. If my software didn’t show me the spammy e-mail and website attached to the posting, I’d have approved it.

I don’t like it at all.  I don’t know where this is going to lead. But it’s worth noting, for the future.

Update: It turns out this was a spam script, which took one of the comments on a Reddit link to my entry and then refashioned it into my commenting structure. Brilliance. So, we’re still in machine land. I am actually rather relieved. There are some things human beings just shouldn’t be doing.

4 Comments

Someone has to break it to you, and that person is me.

It’s over. You waited too long. You procrastinated or made excuses or otherwise didn’t think about it or care. You didn’t do anything and it’s too late now.

I’m talking about Floppy Disks. And I mean the five-and-a-quarter (5 1/4″) floppy disks that actually are somewhat floppy and which are long and flat and which were the mainstay of home computing for well up and over a decade, back then. A decade, I hope I’ve made clear, that means quite a bit to me. And the history, the thoughts and dreams and knowledge and information that people put onto those floppy disks with a grinding noise and a large LED lighting up and flickering? It’s gone. Three-and-a-half-inch floppy disks, which are not really floppy at all and which got a real hayday in the waning years of the 1980s and the 1990s? Not as gone as the fives but definitely in bed in the ward that things go into but don’ t really come out of, but which you can still visit, if you remember to.

If you still have boxes of floppies sitting in your attic or basement or grandparents’ place or wherever else, I’m telling you the days of it being a semi-dependable storehouse are over. It’s been too long, too much, and you’ve asked too much of what the floppies were ever designed to do. If you or someone helping you gets data off of it, then it’s luck and chance, not engineering and proper expectation. A lot of promises were made back then, very big promises about the dependability, and by most standards, those promises came out pretty darn good – it has often been the case of extracting data from floppies long after the company that wrote the software, that made the computer, that manufactured the disk drive parts, and manufactured the disk have gone into the Great Not Here.  You could be a totally different person, with people who you helped create running around your feet and many years younger than these floppies, and you could pull data off them to show the little people what their parent was up to so long ago. Maybe even get them excited about their turn at the screen and keyboard when the time came. It was like getting two sodas for one buck out of the soda machine. Cool!

No longer. Edge cases exist, and will always exist, but the ship is sinking; it’s not seaworthy. With some perseverance and faced against all the odds stacked against you, something might get out of these poor black squares, but I would not count on it.

Why am I telling you this?

I am telling you this because I am grabbing you by the fucking collar and shaking very hard because it is obvious you need to be shaken very hard and told that this is it. This is the endgame for floppies. We went over the hump, and the chances of rescue are slim to none now, but there are still chances. It’s a chance that needs to be taken now.

If you have an archive or cache or hoard of floppies, you need to get in touch with me. I will help get the data off of them for you, whatever piecemeal amount is still thriving on there. We’ll get errors up the wazoo, and some of them will be simply unreasonable, but it has to be done, I have to try.

Archiving history is now my full time job. Let me tell you how much I love that. I love it THIS MUCH.

So I’m throwing myself into the fire. I have people who have said they’ll step forward and help this happen. We can transfer the data off the floppies, get a hold of history before it goes into the zero device.  Get in touch with me.

And please, one other group.

There are libraries, archives and collections out there with floppies. They probably never got funding or time to take the data off – there’s a great chance the floppies are considered plain old acquisition items and objects, like books or a brooch or a duvet cover. They’re not. They’re temporary storage spaces for precious data that has faded beyond retrieval. If nobody got around to pulling that information off, then a fundamental goal of many of these places dissolved under their noses and they’ve failed. I’m willing to forgive and forget, myself, if we can just ferret out these caches and help the items get into a more stable state. (As an aside, the conclusions of this study are wrong, although I appreciate the effort.)

Help me with this, before it’s too late. Because it is too late.

Help me now.

Update: There’s now a page on the Archive Team Wiki that I have created to give people options and information about the transfer of floppy disks into a more modern storage location.  Please read or contribute.

79 Comments

OK, now that I’ve gone pro with the archiving and I’ve been at it pretty hardcore for a few months, my programmer brain has kicked in and I’m trying to find inefficiencies and kill them so we end up with a ton of cool stuff online.

I already know what I am and what I’m about with all this. I’ve known for sometime: I’m the weird little widget, the fucked-up little strange item that is going to link a lot of people who haven’t linked up before. That said, there’s always the danger of being called, by each set of people, someone who is a poor replacement for them. And I am! I am an enjoyable archivist but not a full replacement for a “real” archivist and I’m a home computer enthusiast but there’s some people who truly pants me over compared to what I know.  But my mission and forseeable future will be spent linking all these sets together, and that’s how it’s going to be.

So now that we understand each other, let’s get to work. Let’s get to work, in fact, on the biggest single problem I’ve found.

It’s metadata. Metadata is the slowdown.

If you don’t really know what metadata is, that’s cool, it’s a kind of weird concept. In simple terms, metadata is information about other data. If you have a pile of Apple II Floppy Disk images, then the fact they’re Apple II floppy disk images is the metadata, as well as what’s on them, when you transferred them, who owned them… anything that’s not the data itself is the metadata.

See, it’s not that hard to add lots of data somewhere – see cd.textfiles.com where I have three million files, and yes I am going to port cd.textfiles.com to archive.org and yes I am going to be adding scans of the discs themselves and the ISOs and oh yeah, that’s right baby we’re going big with that. But the three million files, left alone, wouldn’t be interesting or more accurately would be very interesting but be flooded by all the other files not in your immediate interest path for whatever you’re looking up.

Metadata, you see, is really a love note – it might be to yourself, but in fact it’s a love note to the person after you, or the machine after you, where you’ve saved someone that amount of time to find something by telling them what this thing is. Lives have been absorbed getting metadata, and so there’s an entire field of computer study about this idea, and making your machines do the hard work for you. Google’s got some interest in this, I heard. If you could completely generate Metadata, life would be pretty awesome. But you can’t. Not really. Not completely.

So let me announce another collection I’m working on. The Bitsavers Archive. Yeah, that’s right, that one. The one that people have been scanning, donating, and working on for over 15 years. I’m going to import it into archive.org. I got permission from them and we’re going in.

But again, Metadata becomes the immediate issue. I’ve written a script that lets me point at a bitsavers asset, say An Apple I User Manual, and type in a title and then the description and date for the item, and then the script does the rest – upload it, generate a metadata .xml file the archive.org system uses, and check in the item so it can going the collection. Truly, fire and forget. Everything automated is now automated. But the fact remains, I had to title the item, and then write a description. That’s the big holdup.

See, bitsavers.org has 19,000 items in its collection. They are not described in the manner they really need to be to be useful. Someone, me or other people, will need to describe them, before they should be checked in to archive.org. Sure, I could slam all 19,000 in WITHOUT descriptions, but then very little has been accomplished – imagine stacks of a library where all the back of the books have been covered with white-out. Not useful at all.

Similarly, the arcade manual archive I created to test out my scripts and acquisition approaches is a wild success, except in numbers. Of the 362 manuals in there, though, they’re all described nicely, thanks to a team of volunteers who stepped forward to do the reading and typing necessary. They’re credited in the front page for the work they did. Notably, though, I have 4,000 more manuals to add. I’ll be adding them as I go for a little while, but it’s just going to take a huge amount of time, even with every step but “describe” automated.

So, it basically comes down to me asking you, if the idea interests you, to sign up to be a metadata warrior, someone who will work with me to describe these items. I’ll help you find something personally interesting – some people really dig reading old computer manuals, others care about arcade stuff, and I’m sure there’s even more in what I’m adding to keep your attention up. I’m not asking you to do everything – even if you add a handful of stuff, that’s more than was there a week previous – and that’s helping.

And in case the thought occurs to you, Archive Team is not really the best thing for this – this is about long-term presentation, not saving burning data from assholes. When you help me with metadata, we’re helping make available really cool stuff that has been saved but which needs a nice tag on the outside so later people and generations can know what’s in there.

The e-mail is metadata@textfiles.com and I suspect I’ll have a mailing list to discuss this in the future.

1 Comment