ASCII by Jason Scott

Jason Scott's Weblog

Archivebot and Automatic Betterment —

One of the most successful (and ongoing) projects that Archive Team has done is Archivebot. It’s a crowdsourced archiving operation that goes after anything that needs archiving, be it webpages, tweets, videos and other online scrapings, and allows them all to be captured in a competent, useful manner, 24/7. It has grown over the years and is ridiculously flexible now, with a command language so variant that it has this unbelievably high quality manual for all it can do.

Archivebot is at its best with on-the-ground shifting-sands events, like tragedies/celebrations, they-said/others-said controversies, and capturing things the way they “are” just before the new news flattens the pages before they’re taken down. If you watch the Archivebot Twitter account, you can see the bot in action, in real time.

As it goes along, generating these WARC’d pages, it stores them in 100 gigabyte chunks. (Yes, Gigabytes.) Usually between 100-300gb of this data comes in a day. The chunks are then uploaded into a collection, where they are then absorbed into the Wayback machine within a day or two, meaning the world benefits from the data almost immediately.

ArchiveBot has two mascot images. They are both accurate:

VY5wNAvslogo_header

Looking at the Archive’s statistics, some of these pages are called up through the wayback machine hundreds of thousands of times. (One of them has gone past 600,000 recalls.) Many, many more are called “merely” tens of thousands of times.

Occasionally, Brewster gets an idea in his head, and I hear of it, and fly with it. (It’s nice to have a boss that inspires.) One of them was that the new Internet Archive interface is rather visual in nature (by default), and so it would be nice if some of these more data-heavy items in the collection had some sort of visual component to them, if possible.

I set off on that work last year. The idea was to use a WARC playback mechanism (WebArchivePlayer) to bring back pages out of the WARC files, take screenshots, and then upload those screenshots into the items as “previews” of what’s inside.

It’s taken a while, because these have to be downloaded, then played back, then screenshot, then checked if the screenshots are any good, and then removing the ones that don’t have any data in them, and so on.

But it’s coming along really well.

news.ycombinator.com-shallow-20150322-124509-4t3kz-00000.warc.gzwww.cubcentral.org-inf-20150322-141813-3c758-00000.warc.gz

The played-back WARC files work remarkably well. Naturally, if something is a Javascript or embedded-object nightmare, it doesn’t look good at all. But many do.

I’ve got the script working chronologically right now, so it’s doing the oldest items first and then moving on from there. It takes it anywhere from a half-hour to a couple hours to make the preview images. Most of that is because I’m giving every single grab a chance to produce images, and some of those grabs go into the gigabytes.

archivebot

The result, which is now 99% automated, are pages after pages of beautifully rendered, verified-as-historically-relevant or at least gawkishly-fascinating web pages and sites. The thumbnails look good, and going to the individual chunks will give you an interesting (and potentially disturbing) slideshow of events and strangeness.

I mention this all for two reasons. The first is just to put a line in the sand at a point in Archivebot’s journey to reflect on how far along that amazing project has come. It continues to innovate, thanks to the efforts of an all-volunteer force, and addresses the ever-changing aspects and requirements of being a chronicle of the archiving of the web.

But the other is the template that this thumbnail and slide-generating aspect represents at the Internet Archive, which is heavily machine-augmented human work. I go through the items that come out of the contraption, pulling out the sideways-broken ones and the weirdly-off rejects, leaving thousands of screenshots with no human intervention whatsoever. It’s grinding away 24/7, doing something both useful and not worth throwing a person at. It’s how I think a lot of the work will continue to be handled with an ever-increasing workload.

And it’s fun.

00_coverimage00000_Header00000_Header (1)


Even Just the CD-ROMs —

CD-MOM_The_Mother_of_all_Windows_Books_CD_Version_Adn-Wesley

The joke, which I used in a lot of introductory speeches about the Internet Archive Software Collection, was that we are the largest collection of downloadable software on the planet, period. Find us a bigger one, and we’ll download it and add it.

The reason I can make that declaration is because the first few years I got involved, it was a binge of acquisition both online and off, including downloading of FTP site collections, CD-ROM sets, floppies, and all sorts of online collections. It’s very big. It’s somewhat problematic to step through it all, but it’s all there, in one form or another.

Circuit_City_Games_CD_Bonus_from_Advantage_Protection_Plan_Circuit_City_2004 Delphi_Programming_Unleashed_SAMS_Publishing_1995 developerWorks_Speed-Start_your_Linux_App_-_3Q_2002

It was important to me to gather as much software as possible as quickly as possible because I was very worried we were going to fall into a “too late” situation, be it one of original media dying, or previous (excellent) attempts to gather software fading away into obscurity. There have been some pretty amazing large collections in the past; but then again, most of them ended up on CD-ROMs, so it’s been a case of just gathering CD-ROMs and getting the data off them. Which I did, by the thousand.

The Archive now has something on the order of 8,000 CD-ROMs at least, contributed by myself and dozens of other people. They range from installation disks for modems to entire collections of software from various historical sites. They are companion CD-ROMs to magazines, driver compilations, and promotional one-offs. In some rare cases, they’re CD-Rs acquired from collectors that have very rare material indeed.

They are rather difficult to search.

Dr Duke_Nukem_3D_Complete_Version_3D_Realms_1996 Estate_Planning_Adr_Research_Institute_of_America_June_1996

In the Archives “Biz”, there’s various schools of thought about willy-nilly acquiring “stuff”. All acquisition comes at a cost, be it large or small, and it comes with ongoing issues of maintenance, accessibility, and resource draw. That’s all to be expected, but the approaches have variance and there are some hard-core beliefs and policies out there that, when encountering someone not going in that direction or following those policies, gets a little bit of shade thrown.

I get a lot of shade thrown.

Games_Machine_CD-ROM_Gangsters_1998 Games_Machine_CD-ROM_Hollywood_Monsters_2000 Games_Machine_CD-ROM_Rayman_1995

2015 was the year that, getting a few articles about me in various rags, and URLs being available towards those articles, that I stumbled on the most scintillating subtweeting that had been going on (for some time!) about the work I do and how I approach it.

Now, I assure you, not a tear was shed on my part, mostly because I’ve been involved in tight-knit communities, and their little paper dramas and kabuki theater of outrage and nose-raising. It was mostly a surprise because my whole thing of “Archive Guy” had been, generally, a positive one – folks either liked what I’m doing (I thought), or had no particular opinion and a kind of “well, it’s your shitpile” approach. Not so! The anger is palpable out there, in a small and delightful crowd of archivists and librarians who think I’m doing actual damage.

I listened thoughtfully to the arguments, engaged a few folks, got the lay of the land with regards to criticisms.

And then? Well, full steam ahead.

Games_Machine_CD-ROM_Total_Annihilation_1997 Graphics_Blaster_RivaTNT_Creative_1998 HP_Application_Server_8

Related to my 2016 goals, I’ve gone ahead and have started shoring up a small choice I made some time ago, and which had some interesting outcomes.

In the rush to get CD-ROMs online, I chose to rip them as fast as I could and get the ISO images up into the Archive, while not scanning any of the CD-ROM covers or discs and certainly not going into any excessive metadata work.

This is a mortal sin in some circles. I did it willfully and gleefully, knowing that materials would be harder to find, but they’d be up somewhere, ready for people who really needed to know it was there, and where to send their own collections. It was a gamble, and it paid off.

Now, I’m in the process of scanning in those CD faces, as the images in this entry can attest. They’re weird as art, and very helpful as reference points. I’ve been intrigued that for many people, the art alone is giving them visual alerts to the materials inside. That means that it’s one thing to have a CD-ROM of old computer art, another entirely to have the hand-drawn label on the CD-ROM that people remember clearly from the past.

The collection, in other words, did the important thing first, and the next-important thing is happening. I am digitizing these sleeves very quickly, at about 500dpi, and taking these TIFF files, uploading them, then running a script that makes a nice JPG image that you can look over quickly.

As I finish a pile of CD-ROMs, these discs will be going to the Internet Archive’s physical storage, where they are available for reference and access in the future, or even rescanning.

Of course, they don’t have an indexing system there, yet.

That’ll happen later.

Onward!


What 2016 Is —

tumblr_mzz6cfLpLU1rzik3go1_500

As is usually the case, my need to get things done means this weblog does not get anything updated on it. I see the last time I posted was in November, and here we are at the strike-out of the new 2016 year. 18 years of running TEXTFILES.COM! Five years at the Internet Archive! And I am now in the back half of my forties. What an excellent time to hatch plans, in advance of them all being dashed because of something I haven’t forseen yet.

Speaking of forseen, one thing I’ve noticed is that a lot of people in my age/lifestyle, that is “born in the 1960s or 1970s, and then sitting down a whole lot for 40 years while staring at glass”, usually have something crazy bad happen to them health wise in their 50s. Super incredibly bad: strokes, diabetes onset, full-on heart attack… you know, something. And then depending on where and how this happens, they keel the fuck over.

As someone who gave a rather life-affirming speech some time ago, I am most interested in sticking around a bit. So changes are in place.

First, I have a general plan that, before I am 50, everything under my direct physical control will fit in a 15-foot moving van. This means that I will have books that either friends wrote or who I know the author of or they are particularly inspiring. It means that I’ll have my computer tech that represents what I need to edit video or to do some amount of scanning but I will not have 14 Apple IIs in a massive storage container 50 feet from my bedroom. And it will certainly mean that I won’t have, like I currently do, thousands of floppies and CD-ROMs piled around me like some testimony to lost digital landscapes.

The reasoning behind this reduced-space goal is this: It’s obvious that this monk’s lifestyle of working on things and travelling is something I like, and there is no indication that I will ever want to move into some big house again, or have a mortgage, or any of that. The chances of me being one of those people with a RV outfitted as a working base is non-zero. So I don’t want to be the place for storage of masses of computer history that should really be in a different entity. Additionally, it is not very fair to the material I have that has some very good utility and use, that it languish under my direct control in an area with absolutely no policy and facility for others to interact with them. In other words, do not think this is me saying I’m going to throw out all this stuff I have here; it means that I’m going to find homes for them that are open, easily accessible, and with them doing the maximum good. I always thought of the Information Cube as an Ark, not a final resting place – this is just the end result of that.

Another major change is the boring health stuff. I’ve stopped drinking anything but water (carbonated and spring) and I’m on one of those ridiculous diets that makes you hellishly thin by freaking your body out. We’ll see how that goes, but before I move into an exercise regimen, I want to get the ol’ carcass down to something workable. One side effect of a thinner frame will be the exotic and interesting outfits I’ll be able to fit into, but another will be that I can kill off a range of health problems I’m already exhibiting and maybe hit a more respectable lifespan. The days where I could just reach for anything at the supermarket and grab something wrapped in plastic that’s been sitting there for weeks and eat whatever the hell is in that thing are pretty much over. I’ll enjoy a good restaurant here or there but not as a staple. It’ll be good, trust me. Note that this is not an invitation for you to send me a multi-paragraph, multi-link treatise on how to do my health.

12274470_466074340231059_1603672885059975092_n

Most importantly, 2016 is now officially scoped as the year I really hope to fulfill my to-do list and promises.

Living the way I do and getting the attention I do, I’ve now built up a backlog of requests, goals, intentions and wishes that is yawning back for years. And then what do I do? Agree to travel to talks, agree to get involved in time-consuming quick projects, and always taking on more and more things. In one context, this is fun, but in another I am now beset with what feels like an endless pile of promises that are not fulfilled. Some people are rather pissed (or melancholy, or zen) about this. Others are under the impression I’m waiting around for the chance to do something, so they send more to me. It’s now officially out of hand.

As such, I will be making very few trips and appearances this year. I’ve already committed to Shmoocon and Magfest, two events I try to attend for various reasons, and I doubt strongly I will miss Kansasfest (although HOPE happens on weekend of that event). But I’ve been intentionally cutting back on saying “yes” to any appearances in 2016. Podcasts I’ll do. Stuff in New York City (which I live very close to) I will also do. But 2016 is the year I am going to be spending a LOT more time working on things and a LOT less time talking about how I do things.

Subsequently, this weblog is going to cover things, but they’ll be projects that I need assistance for, or which I want folks to be aware of. It’ll (hopefully) be a cacophony of completion, at least for this coming year.

pabc2015_a1920_04

Let’s see what happens!

 

 


JSMESS Achieves a Hero’s Death —

logo

With the release of MAME/MESS 0.168 today, JSMESS achieved something special and something final: Irrelevancy.

Through the work of JSMESS team member DopefishJustin and MAME/MESS developer Micko, assisted by a number of other contributed factors by both teams, per-driver compilation of MAME/MESS into Emscripten-converted Javascript just “happens” now. It’s one of the features built into MAME/MESS, and further work can be refined there as needed.

Volunteer teamwork for this project therefore shifts over to The Emularity, which is the loading structure for Javascript-based emulators, including MAME/MESS, EM-DOSBOX and others that will be added. The Emularity allows and will continue to allow ease of loading for this breed of emulator in a variety of ways, making the embedding of software history available everywhere, ubiquitously, for a very, very long time.

Pushing it through Emscripten also makes way for a future in which a replacement candidate like WebAssembly will be the eventual final target. Emscripten’s continued dedication to cross-platform compatibility and refinement of the compilation process means that now there’s a dedicated team for compiling (Emscripten), and a dedicated team for emulation (MAME). It’s as sweet as it gets.

swagger

This has been a very long road for me. I announced this project idea in this very weblog a mere four years ago. 4 years! (And note that DopefishJ is the first to jump in with assistance. Four years ago! And he’s never wavered.)

4 years is a very long time to bring something like this together. Granted, we had something sort of working within the first year, but to continually refine, improve, find the bugs, re-engineer the whole thing and attempt to make it functionally easier to keep on top of… that took a core set of people a lot of time.

They’re owed a lot of gratitude and thanks, and I need to assemble the canonical list of everyone who helped, but the efforts of DFJustin, Vito, bai, devesine, dreamlayers, clb, jvilk, yipdw, antumbralbalrog, MooglyGuy, haze lord_nightmare, and many others are what brought us to this point.

So, what’s next?

Well, the emscripten support in MAME/MESS is not perfect – it definitely needs eyes looking at it to improve the accuracy and the implementation. But it just got added this month, and I’m quite patient about these sorts of improvements.

The Emularity will need more refinement in terms of making it easier for “just folks” to start embedding software wherever they want it. The code works nicely, it’s just a matter of sitting down and going over how a person who had not had to program javascript would make something run.

And of course MAME/MESS can always use the addition of more people helping it with support, refinement and improvements. The Emscripten/website use case is a strong one – it’s going to be very easy for museums, university teachers, and everyone else to be interacting with this emulator going forward, and so the more focus on getting it comprehensive and quick as well as accurate, the better. It’s instant reward.

jssmurf3

As I’ve indicated earlier this month, my focus is not on making sure emulation in the browser is a fact – that’s been established. My focus is entirely on transferring as much lost or in-danger digital information into modern-computer-readable-form as absolutely fast as possible. The emulators are here, and they’re waiting. Now we have to focus on these poor, solid magnetic souls keeping their precious contents, day by day, until they’re rescued.

I am not sad in the least. It was so fun to work with this team to get things where there are, and it’ll be so great to refocus them on parts that need more attention and love (like automatic new-driver building when new versions of MESS/MAME come out).

It is, all told, a great day.

Thanks to everyone.

 

 

 

 


The Rest of the Infocom Cabinet —

Here’s an update on the Infocom Cabinet, with a side order of ethical debate.

Michael Berlyn

I’ve now dumped the balance of materials I have around into the Infocom Cabinet collection on the Internet Archive. There’s some scatterings left on my hard drives, but they are either 100% personal (think: pay stubs and employee evaluations) or they’re duplicated in many ways in what did go up.

So with this little update comes:

All told, we’re somewhere in the range north of 5,000 individual scanned pages in this uploaded collection. It’s worth it to note that this wasn’t even the full extent of Steve Meretzky’s file cabinet – this was just as much as I grabbed with a sense of “this needs to be saved” cross-purposed with “this will look good scrolling by in the final film”. There’s likely piles of interesting material in the collection, including all of Steve’s work with Boffo and Legend Entertainment, two companies he worked at after Infocom – I just had to call it at some point, or I’d probably be scanning to this day. I again note that Stanford University was donated the entire Meretzky collection, where it sits safely to this day.

For the “what about” crowd… yes, there’s a few other items in the collection, and I may put them up if it makes sense to, but this should really be enough for anyone to produce a reasonably informed opinion on the goings-on, from nearly day one through to the office closing and all the remaining items shipped away, of Infocom, Inc., 1983-1989. They’re readable in the browser, and the original scans that are up are all 600dpi, meaning they can be zoomed in for artistic meaningfulness, which as a documentary film guy I’m pretty big on. It’s a triumph! People are talking about it! It’s making waves!

Now what?

Scan-091007-0001

So, there’s an important factor in this. The vast, vast majority of Infocom employees are very much alive, some still working in the games industry, and others who have possibly not thought about the words “Zork”, “Infocom”, or “55 Wheeler Street” for a very long time. And now, out of nowhere, related to no particular anniversary or event, the sum total of the company’s materials are now online somewhere, browsable, and thousands of people are poring over them, studying and commenting.

Some will be delighted. Some will be confused why anyone cares so much, and maybe one or two will be in some ways horrified or nervous, especially if they haven’t gone over what’s been posted themselves.

For the film, I interacted with a variety of Infocom staff, some of them for just one day (interview), some just over a single phone call (saying they wouldn’t be in the film), and others on and off for years. I can’t pretend to call myself their friend beyond the Meretzky family and especially Steve, who I spent a large amount of time with during production and who I occasionally see when I’m in California.

There’s a situation in making a documentary I call “Stop-Motion Interaction” where you interview someone, spend 3 years working on the movie, and then either have the person at the premiere or run into them, and you have been spending months inside your head getting to know the person from what they talked about, and then you see them and to them, you’re just this old dim memory and to you, you’re seeing an old friend again. It can be jarring for both parties.

But there were people I didn’t get the opportunity to talk with at all. They literally have nothing to know about me or my methods or what I’m about, beyond I made some sort of film and that film had an Infocom aspect to it. (Some Infocom alumni just called GET LAMP “The Infocom Movie”, presuming that’s what it’d be about.) For some of them, they will likely see what just happened with all this documentation and have a “reaction”.

So.

downloaddownload (1)

This gets enormously complicated. And painful. But if I’m going to talk about where I got to with releasing all this historical information, and to stand as some sort of example of the issues involved, I gotta go here.

It’s explained in excruciating detail in this podcast, so I’ll go with the Cliffs Notes version, like someone explaining why one shoulder blade is 2 inches higher than the other, and why there’s a scar going from one ear to the forehead.

Besides this treasure trove of infocom documentation in Steve’s basement, I had someone contact me saying, basically, “So, you’re working on this movie. Would you like The Infocom Drive?” Like everything else, I said YES without needing any details because that’s how I roll. When the Infocom Drive arrived (a roughly 150mb .zip), it was essentially a snapshot of Infocom at the end of days, Knowing that this was a goldmine that needed to be in some way preserved, I gave three copies away to trusted sources, and one of them wrote an article about a particular narrative thread in the drive’s contents, got a ton of attention, some extremely angry ex-Infocom folks (both privately and public, to me), my movie almost died in the cradle and I didn’t talk to the author for about six years.

Again, the podcast goes into this whole thing for the sake of the looky-loos, but I’m trying to get to the core of the discussion/debate here – that to tell this narrative thread, this article used e-mails, entirely private, pulled from the hard drive and which were never, ever published anywhere and I’m sure the employees on both sides of each letter had no idea their writings survived and just imagine waking up to that nightmare scenario.

Reconciliation did happen, and I did have conversations with a lot of people about it, and I definitely still harbor both the sadness at the initial event and the lost opportunities of six years of potential collaboration.

So then, what exactly am I doing here?

Scan-090904-0059

First, I tried to take lessons from the debacle of a half-decade-plus ago and implement them in a way that would protect people:

  • Removal/blacking out of personal information in the realm of addresses and phone numbers, that are surprisingly still intact to the present day;
  • Employee evaluations and specific medical information
  • Anything that might be construed as a personal attack, especially on a person not along the chain (name-calling against specific managers, or a parody gossip article naming two employees, the “parody” aspect possibly misconstrued in modern times)

Steve has been rather open with how he does his work, so there are things in there that I wouldn’t do if I hadn’t worked things out with Steve and gotten his opinion on what’s acceptable. For example, I left in a salary listing for Steve just because it’s both historically interesting and because I think if he had it on his computer, he’d make it part of a presentation at GDC. But only Steve gets that treatment in any way.

I worry about someone defending decisions made decades ago, with 20/20 hindsight applied by groupthink hive-mind perfection-oriented knowledge. I hope that doesn’t happen. People in this group range from early 20s to early 30s (with a few noted exceptions) and Infocom was often either their first job, or a completely crazy 90 degree change in career. They did what they did, and it came from competence and doing the right thing as they saw it. I don’t know if any of us could stand up to such scrutiny and get top marks across the board.

Beyond this, though, there’s the situation itself.

This is probably going to be the only time, outside of maybe Sierra and Broderbund, that this level of depth of the life cycle of a game company will ever end up online. And while I know there are archives of some game companies, I don’t believe any had the meticulousness that Steve showed in gathering up company work and management product and placing them into perfectly boxed-up folders indicating what aspect of the firm they were. We literally have the memos introducing the start of the sales team, company library and health insurance… and then the “we’re not doing so well”, the resignation letters, the calls to sell furniture and office supplies. It’s all in there.

Scan-090927-0003

It is my strongest belief that this collection will instruct, inform and change things in games, if only to show what situations have persisted for years, and what aspects are evolved from how things were. It’s hard, cold source material, unprettified and unsummarized, and showing something else: Just how fucking amazing Infocom was.

These were good people. Hardworking employees, creative geniuses, and driven towards the goal of being the best of the game companies. A place that people dreamed of being part of from the outside. A company that stood as doing all the right things, until it wasn’t doing the right things. A chance for people to figure out where the cracks showed, where the triumphs were, and where dreams were actually and truly formed and hewed on a daily basis. That’s pretty amazing.

Infocom alumni can e-mail me (jason@textfiles.com) any time if they have concerns about something, or which I overlooked the nature of (I tried to be very careful about this and all the thousands of pages have been vetted by me personally – the buck stops here.) Naturally, the world at large can e-mail me too.

I should rush to say that the reaction on the part of everyone I’ve found has been 100% positive. I’m writing this not because someone complained, but because I saw in a potential scenario that angry and betrayed researcher I was so long ago with my friend (who is still, again, very much my friend) Andy. I know that the result is often not shouting but seething. That solves nothing. I wanted to get ahead of it.

For everyone else, please enjoy this rare and possibly unique peering into what is, ultimately, one of the high holy grails of gaming history.

zork_feat-2


A Cabinet of Infocom Curiousities —

This is big news, in the realm of game design studies.

During the production of GET LAMP, I spent a lot of time digitizing or photographing all sorts of artifacts and documents related to Interactive Fiction and text adventures. This included books, advertisements, printouts, and various ephemera that various players or programmers had lying around from that era. This would usually involve one or two ads, maybe a map or two that someone had drawn, and one or two photos snapped at a convention.

But not in the case of Steve Meretzky.

If you’re coming into this relatively new, or even if you need a little brush-up, let me state: Steve Meretzky has earned the title of “Game God” several times over, having been at the center of the early zenith of computer games in the 1980s and persisting, even thriving, in the years since. He continues to work in the industry, still doing game design, 35 years since he started out as a tester at what would become Infocom.

But more than that – besides writing a large amount of game classics in the Interactive Fiction realm, he also was an incredibly good historian and archivist, saving everything.

EVERYTHING.

When we finally connected during production (as it turned out, we lived within 10 miles of each other), Steve showed me his collection of items he had from the days of Infocom (which spanned from roughly 1981 through to the company’s eventual closing and absorption by Activision in the early 1990s). And it was a hell of a collection:

img_7272jpg_18330726335_oimg_7230jpg_18142881450_oimg_7229jpg_18326662102_o

Recognizing the value here, not just for my documentary but for the world at large, I gained permission from Steve to start scanning these items. First, in his basement, and then, when the job extended past a few weekends and it got annoying to have this guy in Steve’s basement, from my home, in a setup that I would work from with a set of pliers (for staples) and just scanning, constantly, as I could:

img_7228jpg_18330610795_oimg_1822jpg_18313469866_oimg_1789jpg_17717243854_o

This took a long time. I scanned as much as I could, and after working on Steve’s “design binders”, which are very large combinations of every scrap of paper related to a game, I took a run at the file cabinet, which had pretty much every major communicated aspect of the Infocom company, from memorandums and business process through to interoffice softball game preparations and crab race outcomes. I definitely didn’t get everything, but I got a whole lot. Something on the order of roughly 9,000 scanned items, in fact.

Ultimately, Steve moved out of his lovely home and went to the west coast. His binders, artifacts and other items went to Stanford University, where they are housed today. I sent them copies of my hard drives, and they are using them (to my delight) to house their own digital form of the archives, and intend to bring in the remainder of the materials over time.

I ended up using a lot of material in GET LAMP, with loving pans across these 600dpi images of puzzles, writing and advertisements while people talked about text games and the craft of creating them. And after the movie was done, I put the scans away and moved onto other projects.

Until now.

Today, I’m dropping the first set of what I hope will be the vast majority of the stuff I scanned during that production year, onto the Internet Archive. The collection is called The Infocom Cabinet, and right now it has every design notebook/binder that Steve Meretzky kept during the period of what most people consider “Classic” Infocom. This includes binders for:

Right there are nearly 4,000 pages of material to go through related to the production of these games.

img_1764jpg_18313412216_o

Bear in mind: Steve did not mess around when it came to assembling these folders. He includes the light, drizzly roots of a given game, whether it be some cut-out newspaper articles or an exchange between employees of “what should Steve work on next”. (In some cases, heavy descriptions of the games Steve never got a chance to make, including a Titanic game and Minute Mysteries.) It then follows through many iterations of the maps, puzzles, references of any given work. Often, there are draft versions of the artwork and text for the manual and hint books, including all correspondence with outside vendors (like G/R, the copywrite/design group Infocom used heavily and which Steve has the occasional huge disagreement with). Then, once the game is functional, we have letters and feedback from playtesters.

(PLEASE NOTE: I HAVE REDACTED THE NAMES AND PERSONAL INFORMATION OF THE PLAYTESTERS INVOLVED – ORIGINAL UNREDACTED COPIES ARE NOT ONLINE BUT EXISTENT.) 

For someone involved in game design, this is priceless work. Unfettered by the crushing schedules and indie limits of the current industry, the designers at Infocom (including Steve, but not limited to him by any means) were able to really explore what made games so much fun, where the medium could go, and what choices could be made. It’s all here.

img_7239jpg_18332087951_o img_7237jpg_18144421759_o

But more than that, and I mean much more – Steve kept all the memos, business process, and related papers that were generated through Infocom Inc.’s life. Like, pretty much all of it.

This gets slightly harder for me to put up – I am going to have to work with Steve and some of the other people involved as to what can go up now and what should stay in Stanford’s stacks for researchers to work with. But for now, a healthy set of materials have gone up:

This is a relatively tiny amount of the total internal company scans I have made, but these are the ones that I can put up without worrying about it crashing into anyone’s life. Again, personal information has been removed, and the focus has been on company process and interesting historical documents.

img_7253jpg_17708048524_o

There’s so much more not up right now, but this 4,000 page cache should give you something pretty extensive to chew on. I also can’t promise when the ‘next wave’ will come, as it really will be time consuming to go through compared to the relatively light (personal-information-wise) design binders. But it will!

I can’t thank Steve enough for what he did during the timespan of Infocom – he just absolutely captured a very special company during a very special time and kept it, well-sorted and updated, for years and years. That we have this at all is a tribute to his staying firm to this approach, even with the side-effort of, you know, completely revolutionizing computer games.

Enjoy this holiday treat.

img_7255jpg_18142931390_o


Close the Air Gap —

Close the air gap. Close the air gap. Close the air gap, the air gap, the air gap.

11657342_10153396452613834_634704471_n

Close the air gap. Close it. The air gap. Close it.

IMG_3235

Close the air gap. Close it right now. This weekend. This month. Close the air gap.

sharewarezcds5123851623_e313aa7341_o

There’s a layer of atmosphere, of air, in between us and the data, and that needs to go. It needs to be pulled in. It really needed to be pulled in years ago. It needs to go in now.

In the nascent period of “will Javascript be up to the task” that came out of the beginning of the JSMESS project, there was a worthwhile point to focusing on the “will it work” aspect. We got it to work. It’s not perfect, but it’s very, very good. If you have a very fast machine, and as time goes on, we will all have fast machines, then the experience can feel great. What’s left is polish and documentation. The beast lives, the machine hums. It needs help, yes, but the amount of people who can help are relatively tiny in the world.

What’s needed is to close the air gap.

I’m not worried about entire classes of “stuff”. Console games made before 2000. The full run of “Friends” on DVD. Most anything released audio-wise on CD that had a UPC symbol. I’m not worried in the same way you shouldn’t be worried about falling rocks when driving through a road in cornfields. It might be a problem, but it shouldn’t be at the forefront of your mind. It’s not in my concern circle that there are probably some releases by Tommy Boy Records or an occasional post-2010 feature film that is difficult to find in just the right video format.

When people talk about ‘saving’ materials into digital form, some of the “big things” get immediate attention and love. For example, Nintendo and especially Mario. We’re set on Mario. But there’s entire swaths of material I call “Advocate-less Items” that just need someone to digitize them regardless. Old user group floppies. Placemats. Rave flyers. Handmade cookbooks. Shop manuals. Beverage distribution one-sheets. Matchbooks. You know, stuff.

I will be resuming work on DIGITIZE THE PLANET because I think that will help bring an army of digitizers to the fore. People who will drop a little money, go get scanners, and help us add materials that are sitting around, give them useful metadata, and then upload them to places like the Internet Archive.

But as I sit here in my room, ripping hundreds of CD-ROMs, floppies, and tapes, and as I start to scan in newsletters, books and pamphlets, I feel like we’re running towards the Tsunami with buckets, trying to catch it. We need more people, and we need more scanning, and we have a lot of work ahead of us.

Close the air gap.

IMG_20150209_004756

Close the air gap.

Close the air gap.

Close the air gap.


Archiving 3,500 Hours of Stupid —

The title has little context, which I think is appropriate.

One of my friends in high school insisted, nay, demanded that I listen to an album called A BIG 10-8 PLACEwhich was either about or by something called Negativland, which I didn’t have a clue as to what that could mean, and before I could really absorb that, I had to hear the album, and oh man.

61LgxDno63L._SL500_SX355_

I can’t even easily describe this album. It’s primarily a set of audio collage, some of the finest that can blow through headphones, with ridiculous self-reference and unusual disturbing moments. It was just a game and life-changer for me, some crazy kid in 1986, a mere 16 years old and up to that point living with Art of Noise as my most wild band.

I bought a number of Negativland albums over the years, and in college, by a very, very lucky set of circumstances, I got to attend an actual, live, Negativland concert at The Western Front in Cambridge, MA – one of their rare East Coast appearances. I showed up way too early, and met the band.

I’m compressing time insanely – sorry about that, but my personal life with this band is not important to the main points other than to say that it’s progressed on to what will now be a life with Negativland for 30 years next year: 1986-2016.

Along the way, I saw them live a handful of times, and in one case, even put the entire band up in my house in Waltham when they had some Boston appearances. The whole band! In my house! What fan wouldn’t love that?

negativland_2

Maybe I’ve just hit that age where I approach liking different bands to “take it or leave it” – you either listen and are grabbed, or you listen and say “not for me”. But if you like clever or really brilliant sound collage, cultural ridiculing combined with satirical commentary, and the occasional burst of disturbing, intense music, then you may enjoy Negativland.

Along the way, I’ve become really close with one of the members, Mark Hosler, who I’ve had the pleasure of hanging out with in various locations, and who has never failed to delight me with his presence. Kevin Smith says “Don’t meet your heroes” but what I counter with is “Get better heroes.”

In the middle of 2015, one of the members of Negativland, Don Joyce, passed away, at the age of 71. He was amazing in his own right. There are some really excellent obituaries and memories out there, and again, I’m not going to add to them here.

Don-Joyce-e1437688180848

What I’m going to do is tell you about how this nearly-30-years-long fan got the chance at the dream of his lifetime, when he was able to leverage his position at a very large, very open network-connected archive to bring forward a collection of Negativland and Don’s work that nobody ever thought would be in one place.

It’s called the Over the Edge Collection.

See, during the time he was in Negativland and doing a lot of sound work with the band, Don was also the host of a weekly radio show in KPFA called “Over the Edge” – an hours-long sound collage of themes, commentary and thrown-into-the-mix listeners that he called Over the Edge. That’s three to five hours a week, every week, for decades. Unique enough as a show, it’s definitely unique in the consistency and execution of his vision. He did his last show, died a few days later, and various people from the band and long-time collaborators showed up and did the very next show entitled THERE IS NO DON, and the show was announced as continuing in some form, but the era of Don Joyce was over.

There were scads and scads of recorded shows from the many years, some of which had been remixed into sold items for the Negativland Seeland Records catalog, but which the vast, vast majority had not seen the light of day since broadcast.

Until now.

Working with the Band’s Archivist, Tim Maloney, who has painstakingly digitized tapes for years, the Internet Archive now has up over 3,000 hours (THREE THOUSAND HOURS) of this radio show. It’d take you a good part of a year to listen to it all.

There are now lots of articles about this hosting of all this material (I particularly like this one), but there’s two things I wanted to address before I set you off to it.

negativland_4

First, I work for such a great place that adding 941 shows of hundreds of megabytes of audio each wasn’t even a rounding error – I didn’t have to bring it up with a single person before I helped do it. This dream of Don’s and others in the band, that all the Over the Edge material would be up somewhere, instantly listenable, came true in hours as the hard drive uploaded the items.

The Archive’s derivers convert all incoming audio to various forms, including .ogg and related formats, as well as generating waveforms and just generally trying to give listeners the most options. It is now possible to listen to all 941 shows (with more coming) and not have to hit a paywall, a banner ad, a rate limiter, or any other barrier. It might be worth considering to donate to such a generous organization.

negativlandcontent

Second, you need to realize that a band this meticulous, this directed in their focus and approach and art, was making a huge and ridiculous leap when they just threw up these thousands of hours of shows. Up to this point, they’ve crafted nearly every second the public heard of their work, and so presentation was done at great pains and effort. And then, a short time after Don’s passing, we are suddenly given all of these items with, in some cases, almost no descriptions at all.

The choice was pretty clear – only release what had gone through rigorous metadata refinement, fully describing and cataloging this collection until it was 100% ready to go as a perfect crystalline museum piece… or just dump everything up there and figure it out later. It was very difficult to do a 180 and put it all up as is.

As a result, there’s a call for help in describing these shows. Don only described a percentage, and efforts by fans new and old to describe these shows would be appreciated. It might take years, or a few insane weeks, but the literal life’s work of Don and the members of Negativland (and many many others, I rush to say) deserves the community it should get.

Let’s get Stupid.

b002


The Internet Archive Telethon —

I’ve been given permission to go ahead and schedule an Internet Archive Telethon to allow us to have a gosh-darn-it actual showcase at the amazing space that is the San Francisco headquarters. My goal, humble as always, is to try and raise a million dollars for the Archive in a weekend.

To do this, there will be a live telethon from noon on Saturday, December 19th to noon on Sunday, December 20th. I will be co-hosting with others, and it will be quite a hoot.

I’m now working on stuff regarding logistics for it, but I’m letting people know about it because of two reasons:

FIRST. If you know or are a group who makes use of the Internet Archive, either hosting with the Archive, or are a prominent name (I’ll leave the definition of that to you), then please write me at telethon@textfiles.com and let me know your ideas for an act, appearance, pre-recorded message, or other contribution that will be good to have during the live streaming.

Second, if you donate to the Internet Archive in advance of the Telethon, and include the phrase LOCKBOX in your donation, it will be added to the Telethon’s pre-telethon “Lockbox” and that will give us the initial boost towards the goal.

I’ll return to my regularly scheduled weblog shortly.


In Realtime: Some Initial Sorting and The Power of Two —

I drove down to Baltimore yesterday, sorted manuals for the day, and then drove back.

First, I stopped by the Manuals Plus warehouse to see the current status of the manuals that we left behind. Answer: Gone. Thrown out. Recycled.

20916883838_6eb8937192_k 20917977539_937dba2e48_k 20917978649_b96502b73b_k 21104777875_c7e768cd21_kSo there you go – that’s what would have happened if we’d not organized and arrived within 5 days of being told it was being thrown out. Except, of course, we got quite a bit of things before having to leave the rest behind.

I’ve seen some share of criticism of this process, but people need to remember that whole “It was Friday and I was told it would be thrown out on Wednesday” thing. Spoiler: It’s kind of vital to why we had to work we did at the speed we did. And the outcome was really nice.

The manuals we saved are in three storage units at a storage place down the road from the warehouse. I planned to go to the units, and work with whoever showed up. As it turned out, one person showed up.

The reasons for this are obvious – little relative warning, middle of the day, etc. But there we were – me and Eric and a storage unit full of stuff:

IMG_7153We did just this unit for the day, because of two main reasons: There were only two of us, and my back is still in very rough shape from the initial load-out.

There were a notable amount of people horrified that the boxes were sitting straight on the concrete – apparently outside of flood risk, concrete has a habit of transmitting moisture through the ground to whatever’s sitting on it. Something had to be done, and we went with wooden pallets.

20916681120_64091c287f_kOnly $3 apiece! We bought twenty.

Then it was a “simple matter” of us going through all the boxes in the storage unit and making a very quick determination if it was “Hewlett-Packard”, “Tektronix”, or “Other”. You see… there’s an awful lot of HP materials – like thousands of manuals. Same for Tektronix. Those should be noted and set up to be mostly near each other, while I figure out the next move.

The result of a full day’s work was this:

IMG_7156 IMG_7159 IMG_7160 IMG_7161 IMG_7162Everything’s on wood now. Stuff is still piled a little high, but we sorted it out, so there’s a clear set of “HP/Tektronix” piles and then piles and other stuff. (Any situation where these HP/Tektronix items go to another home will require a last QA check to make sure other manuals aren’t in there too.) We labelled as best we can, so that eventually, the HP might have its own unit, while the brand names on the box will make scanning choices slightly better.

It was a hot box. I sweat. A lot. This was sweat:

21112477581_3b45ce6fac_kBut ultimately, we hit a good break point – the boxes are better than the initial load-in. Now I just need to come back down and do the rest. I also have a lot of calls to make to engineering museums, libraries, archives, and other groups I know. These things need a permanent home that doesn’t have the word “EZ-Storage” at the front of it.

I drove 570 miles in 24 hours just now to do this shoring up.

Totally worth it.

IMG_7154