ASCII by Jason Scott

Jason Scott's Weblog

Love and Noir in the Time of Ebooks —

I first met Chris Orcutt at the age of 11, and we both remember it exactly the same way – the wall at the condominium village we both lived in, who was hanging around, even who was standing where. 2015 is 34 years later. 34! That is a very long friendship.

We’re now in our forties. I’m a filmmaker, archivist, annoyance. Chris is a writer.

Here’s a good one of Chris these days, at his writing desk:

orcutt8

It’s a still from from a film I did about his insistence on writing his first drafts in pencil. (The film is here.)

Chris photographs well.

IMG_4764

However, we usually look like this when we’re together:

Image4 Image6

We’ve both moved around a lot, but for the last few years, we’ve lived within a few dozen miles of each other, and getting together on a semi-regular basis is the easiest it’s been in over a decade, so we’re making the most of it. Good lunches, good walks, good conversation.

In his early 30s, Chris became a full-time writer. He’d been a writer for years before that, including for a newspaper, as a speechwriter, and as a copywriter, but after a certain point, he and his wife made the decision that he would dedicate himself, full-time, to writing. Writing novels and short stories and plays. Nothing else.

They’ve kept at that choice, with various amounts of hardship attendant, since then. Chris writes. He gets up in the morning and he writes the entire morning, then either does some reading or other research. Pretty much every day. For years.

Back in his twenties, when we were both going to colleges near each other in Boston, my friend started taking a poor turn. Drinking got amusing, then prodigious, then alarming. It got to a point I simply assumed my very good friend, my best friend, was going to die, so I tried to enjoy what time I had with him.

As he has discussed in the open more recently, Chris had an un-diagnosed bipolar disorder. With medication and medical supervision, it’s under control, the drinking is rare, and my friend is alive to this day. With this control came the ability to focus on tasks and quality of work that shows in his efforts since then.

And that effort, I will say one more time, is writing. Pretty much every day. For years.

The result of this writing has been a collection of novels. They include a three-novel detective series, the Dakota Stevens Mysteries, and two short-story collections.

They’re good. They’re unquestionably good.


He has not wasted his time, his years, poured into these projects. I’ve not had to look away and make excuses for them, or call a glue-ready horse a racing champion. He’s done fantastic work here, and will continue to do so.

Here’s why I’m talking about this.

The third novel in the detective series just came out. Of his going-full-time novels, this is the fifth overall. He and I both do these long-term projects (he the books, I the movies) and we don’t look at each other’s stuff, in-process – we wait until the end and enjoy it with everyone else. So on the day it came out, I bought and downloaded the third novel, A Truth Stranger Than Fiction, and damn if I didn’t devour that thing in a day. I read very fast and vociferously, but I take the book in along the way. And the book’s good. Damn good.

While I didn’t read the in-process novel, I went to a lot of lunches with Chris during the year he’d worked on this book (and he’d written drafts before the intense year) and I know how many times he read the thing, how many drafts he pounded out to get it where it was, and how much editing and polish he put on it, across months, to get the book where it is. This thing’s a craftwork wordsmithing of the highest order for Chris. You may diverge from opinions stated by characters or you may be surprised at which way things go, but none of his books, none, have the sour feeling of a good idea burdened by crap writing. He wrote these for the ages.

This turns out to be a problem.

IMG_4826

If you haven’t observed this already, I’m a whirlwind of input and output. I splash through a hundred hangouts, online and off, a week. Tracking down the most minute of factoids just to prove a point in a web-based conversation I wasn’t even in two hours ago is my idea of a good time. Shit howdy do I like splashing through life’s intellectual wading pools and swinging a bazooka of manic lyrical fireworks. Pump me full of the hottest images from imgur, the latest insanity out of hackernews, the twitter tornado of the moment. My tolerance for the always-on glut of intellectually stimulating-and-stultifying river of human experience is legion. Bring it.

Not so much, with Chris.

The rough years of his twenties and the writing years of his thirties and forties have diversified the two of us. In our teens, we were both pretty hyper and bring-it kids. I’ve turned that up to dangerous, slap-a-warning-label-on-it levels. Chris has instead focused on his craft, his skills, in the art of writing. It’s made him different. Not less of a person, mind you… far from that. It’s his years of focus that’s made him really great at the writing he’s aimed his life around. He is the monk, waking to the cold water of the stream and practicing, and I am the insane brother living deep in the city, buried under twelve crazy deals and all kinds of trouble. Different choices.

All said, we still get along.

But it does mean that each time Chris emerges from this thing that means so much to him, this novel that he’s poured hundreds, thousands of hours into… he just hates the way things are. It’s like he’s brought the best pie he could bake to a fair where folks are gleefully handing out Hostess Fruit Pies and crowing at how the flaky crust and the cherry-flavored-filling is a meal fit for a king.

hostess pie

 

He gets really sad about it, sometimes. Books were, and continue to be, special. But the modern world is working very hard to make the kind of effort Chris puts into his part of the writing landscape a waste of time.

What he wants is for the books to be available in 1975, making a splash with the effort he’s put into them, meeting up with others doing equivalently hard work, and taking his residuals while taking his wife to exotic locations to do research for further writing.

What he gets books are available on Amazon. Amazon takes away the problems of a warehouse of rotting copies that your publisher isn’t promoting, and enables to-the-moment direct control by the author of the sales and pricing of their book. For that advantage, Amazon has progressively, and I do mean progressively, destroyed writing books as a career for any writers who are not in some way self-promotion machines or blowing three quarters of their time that should be spent on their books, on blog entries meant to drag in readers.

Amazon’s structure favors people shooting out a bunch of crap, in serial form, to constantly stay in the promotional frame. Amazon’s routines disfavor older books, with “older” being a month or two ago. And ever-present “sales”, “prime” and “deluxe” situations are existing where Amazon is heading towards making books into an “all you can read” Netflix-like situation, where people pay Amazon $10 a month and they get all the books they can slam down their gullet.

This is not what he signed up for.

RJcVTtH

 

I’ve done this rodeo before.

Here, I’ve told you my friend’s craft and pain, and the more passionate reactions will tend to be along two lines of thought, with a bonus track.

The two lines are

  • Fuck your buddy. Sorry to hear he came down with the born-too-lates. It’s a world of self-promotion and staying-with-the-times and he better learn those skills as well as he’s apparently learned to write. Or get a second job.
  • Writing has never been a self-paying job – congratulations on learning what we all knew.

(The bonus track is, “oh, a book series to read, I’ll check it out.”)

 

So, both of these are wrong for what I’m talking about here.

First, I think the world is seriously suffering when everyone is writing garbage teenage supernatural romance with a chapter’s worth a month. I realize there has always been crap and Sturgeon’s Law reigns supreme, but that doesn’t mean Sturgeon’s is an end goal. I make the argument that there’s got to be some middle ground – that as we’ve been working so hard to make it easy to transport electronic books in seconds and share them among ourselves in seconds more, that we lost some of that imprint feeling, of knowing a house of publishing was going to give us winners and it was worth the effort to seek out the newest titles. I think it’s a problem, and a problem that needs solutions.

Second, there have been writers who have made it a career, after they put in their dues. Chris has, trust me, put in his dues. After saving for years, and with a boost from sales from a book, husband and wife took a trip to Paris (which was then used for research for a later novel), and this was the first time they’d done any vacation in a decade of marriage. Dues? Paid.

I can’t wrap this entry in a bow. I can’t even bring you the place the plot is going to go. I don’t know what’s going to happen. My friend has skills that outstrip mine in many ways, and I get lots of goofy press for my efforts, while he works in relative shadow.

I thought, maybe, I’d shine a little light on a friend’s lifetime effort, since it deserves a beam many times brighter than it has gotten.

And maybe, just maybe, send them to Paris again, one day.

orcutt2

 

 


That Whole Thing With Sound in In-Browser Emulation —

We’re past 4 million verified MS-DOS players on the Archive this week.

So that’s something.

It’s time to really lay out the floating elephanty-thing related to this whole in-browser emulation kick – sound.

Sound on the in-browser emulators (be they JSMESS, EM-DOSBOX, or others) suffer from sound strangeness. In JSMESS, it works fine except when it doesn’t. In EM-DOSBOX, it works, but it does this kind of strange fluttery-ness that, if you’ve got experience with the originals, you notice right away.

Both are the results of a specific situation with browser audio, and the work that the Archive has been doing to bring this all to a mass audience is bringing the situation to the forefront. I just needed somewhere to send people who go ‘sound seems weird’, and so here we are.

tumblr_mdg63iqTL81qkmqyho1_400

One of the side effects of this project is that I had to spend years following the week-by-week advances and activities of the current lead Web Browsers in how they handled things like Javascript engine improvements and other features.

As might be expected, I’m not a fan.

The rest of this is insider baseball, but if you want to know why audio has an issue, here it is.

Currently, there’s four active large-scale browsers with any traction and attention. They are Chrome, Firefox, Internet Explorer, and Safari. There’s others, just like there are other land-line telephone companies and there’s other network hardware companies and there’s other search engines, but come on. As part of the testing procedures for JSMESS I took a sample of the other browsers that were hitting the Internet Archive and installed all those things. Many of them are Chrome and Firefox, recompiled with features added in or features ripped out as the tastes/anger of the maintainers dictate. This just adds more lint to the Big Four.

(Also, as regards Safari, I’m being really nice including them.)

With four browsers to consider, it then comes into play about how they will handle certain standards so that something that appears on one browser will appear some level of the same on the others. Examples include rendering of fonts, tables, adding buttons, allowing control of keyboard, and so on. On the whole, generally, the world has gotten by.

Here, at the beginning of 2015, the big drag is Internet Explorer. Forget the future and the present; the past has been rolling updates for browsers, so that Mozilla/Firefox and Google/Chrome are basically making major, wholesale changes to the operations of their browsers a few times a month, as it suits them. Safari sort of follows suit with much less frequent work, but the system is in place to allow them to do it when it pleases them. Meanwhile, Internet Explorer drags along, basically doing a major scary X.0 release every year… except 2014, when they didn’t do a single release.

Here’s where we start to run into the issues with stretched supply lines, a la warfare, except here the war is to bring runtime-quality items on webpages.

Firefox and Chrome both have in-house speed-up flav-o-paks for making their stuff run quicker. Firefox has ASM.JS, and Chrome has NaCL. ASM.JS will work in all the major browsers, and NaCL is kind of a sleight of hand trick. They both want the same thing – to cheat out certain programming actions and runtimes so that they run very, very fast in the browser. ASM.JS, since it’s basically Javascript, will run in the other browsers, but Firefox has gone ahead and written in-program routines for noticing ASM.JS is in place and then optimizes it further. Therefore, programs like JSMESS, that use Mozilla’s Emscripten compiler to make their run-times, run significantly faster in Firefox then Chrome. In an ideal world, the end-user doesn’t notice because once you get to a certain speed with, say, emulating DOS or a video game, you just have it. It’s done. But it’s not an ideal world.

So, essentially, JSMESS and EM-DOSBOX run pretty fast in Firefox, run a little slower but fast enough in Chrome, run notably slower in IE, and run OK in Safari. Tiny-share browsers using Firefox’s engine therefore get the speed boost, and the Chrome-based browsers get that amount of speed.

IE could release another version soon, and blow everyone away, and then another year will happen. For better or worse, they really should be on the same ridiculous twice-a-month schedule the other browsers are rocking.

Let’s step back.

00_coverscreenshot (11)

All of this back-room crapola is part and parcel of the entire software industry as a whole -there’s nothing special about it related to browsers. The only reason it even gets attention is because the nature of the web has, for the moment (a moment lasting 20 years) forced natural adversaries into cooperation, because initial attempts to be “THE” browser for the web by both Netscape and Microsoft fell on their face. In other words, because nobody could really “own” the way webpages were being made, all the browsers had to do work to make themselves at least superficially coherent when it came to standards.

Along the last 15 years, there’s been weird little deals and tricks employed to make one browser seem “better” than others. For example, IIS webservers (made by Microsoft), if they detected they were dealing with Internet Explorer, went into this kind-of-not-a-standard mode that basically wrecks TCP/IP but ensures really fast data transport. (In high-level terms, it stops waiting for acknowledgement and just blasts the data.) Google added some really persistent programs running in some operating systems (Google Crash Handler, among others) that were “helpers” that gave their browser some resilience and options. And now we have ASM.JS and Firefox’s both providing the new idea and the others not really wanting to endorse it in any meaningful way.

Most of this is very boring.

But let’s proceed, in this fogged landscape of standards and trickery, to sound.

smurf

So, you start up a game of Smurfs: Rescue in Gargamel’s Castle, and it either works for you, or it doesn’t. Generally, it will run at full speed in all browsers. Sound, however, is another situation entirely. When the game starts and the music begins, it either sounds clear, or it does not.

The reason for how it sounds is multi-fold.

First, when you listen to an audio track (or a video track with audio) or you play a javascript-native videogame, or anything that’s using audio in a webpage, in general these items have a set of pre-made, rendered, finished audio files in them. An audio track is obviously that – a multi-megabyte .wav or .mp3 file, and a video has an audio track that is same. A videogame will have a little pile of samples. All the audio routines in browsers assume this is how the world is, and what the world will be. And generally, that has been the case and all has been good.

JSMESS breaks that, as does EM-DOSBOX.

Instead, these items are generating chunks of data, live, and mixing them as they go. They’re not preformed samples being pushed out – every millisecond is brand new, and won’t repeat.

The current standards don’t like this situation at all.

Mozilla had cooked up an audio standard that sort of dealt with mixed audio, but that standard has been removed in favor of a shared standard called the Web Audio API. Here’s the current version of that standard. As you can see, Google and Mozilla have people working on it. Microsoft isn’t evident, but who knows. Apple is MIA as well.

I’ve been told that if this standard gets formalized and gets implemented, it will just “fix” the audio problems. It accounts for programs like JSMESS/EM-DOSBOX doing dynamically created audio. Emscripten will then observe this standard, and then compiling JSMESS will push it through and we’re in happy audio land.

I want nothing more than this. I have nothing I can contribute directly to the standards bodies other than to say it’s obvious millions will benefit.

So that, in a nutshell, is why audio currently has problems in the in-browser emulators.

But let’s go one step deeper.

geocities-1996

At the Chaos Communications Congress this past December, there was a talk by Olia Lialina called The Only Thing We Know About Cyberspace is that it’s 640×480. it’s a wonderful walk back through the Geocities pages of yore, which Archive Team was one of the contributing groups to get the data.

It’s a great talk. You should watch it.

Fundamentally, there’s a theme in Olia’s speech (and the speech of others in that space, like Dragan Espenschied, Ben Fino-Radin, and so on) bemoaning the move away from a space on a website being the province of the users, and being turned into a homogenized, commodified breeder farm of similar-looking websites with only surface implementations, like WordPress, Facebook Pages, and so on.

Well, if you’re wondering how the good goddamn that happened, look no further than what I’ve just been talking about with the Web Audio standards and the endlessly shifting goalposts of the browsers.

There was a time when a person who was not particularly technical, or whose technical acumen was sufficient to get applications running on a machine and not much more, could code a webpage. The tags were pretty straightforward, the uses of them clear, and the behavior pretty dependable. Much how one could, in a weekend, learn sufficiently how to pilot a sailboat… such was that a few weekends of study could allow a person to craft a fun little webpage, with their voice, their stamp, and the idiosyncrasies of their personality shining through.

Those days are gone. Long gone.

Instead, we have this (as my buddy Ted Nelson calls it) nightmare honky-tonk of interloping, shifting standards soirees that ensure, step by step, bylaw by beta, that anybody who isn’t willing to go full native will be shut out forever.  The Web’s underpinnings, at least on the basic HTML level, have been given over to the wonks and the engineers, making it an impenetrable layer of abstraction, not worth your time to learn unless you were looking to buff up your resume, or if some programmer pride resided in this whole mess being in your job description.

As I’ve said before, the idea behind JSMESS is meant to be greater than Browsers, Javascript and the MESS emulator. I use all three of these components because they reach the widest and have the greatest chance of survival on all sides. If one fails the project, it’s out.

In the world of Browsers, right now, there’s a group (Emscripten) which is making the porting of C++ programs (like MESS) pretty simple to compile. They’ve gotten so good at it in the last few years, but we should never lose track of the original fact, which is that the entire project is botard insane and the result of an insane and terrible situation. I appreciate everyone’s efforts, but that fact remains.

IMG_0218What we’re left with, and this is the god-awful truth, is a world where people have either gone full immersion into this bizarro world and can talk, with a straight face, about coding Ruby or Python for The Web, and people who have long ago given up understanding it and who just slam the side of their computers like a TV that isn’t getting the local news station properly. And both sides have contempt for each other, and it didn’t have to be this way, but here we are.

My hope is that the Web Audio API will get the overhaul from the new standards that it needs, Emscripten will reflect these sooner rather than later, we will tell the shut-out masses to “upgrade your browsers to the newest version”, and we’ll all just have nice little games and programs in our web pages and we can forget how much of the landscape of Brazil is lurking underneath our browsers’ clean, crisp buttons.


The MS-DOS Flood (And the High Flight of V2) —

A couple days later from my entry above.

This has already significantly dwarfed the Internet Arcade. Thousands of people are hitting the site a minute, playing MS-DOS programs, writing reviews or keymappings or just freaking out about seeing this whole collection available and playable.

The choice to send everyone to the Version 2 beta interface has also been nice – lots of useful, interesting feedback, good comments, and people really working out the rough spots. The system got a fix-up during this and runs notably faster. The whole Internet Archive infrastructure is getting the workout it deserves.

A long time ago, I wrote about Opinion Spectrum Collapse Disorder, where if enough online attention is aimed at a thing, whatever that thing may be, the resulting flood of commentary will quickly expand out, like a gas, into every conceivable tangent, subject or opinion, until the whole thing is kind of meaningless for “this is what is being talked about”. Celebrity gossip stories seem to attract this particularly: talk about a celebrity buying coffee at a supermarket when they are anti-coffee, and the comments will quickly blow out into the celebrity, the notion of celebrity, something the celebrity said or did 10 years ago, coffee, supermarkets, Ayn Rand, and cat ownership. It’s just the way things are, now. I don’t know if there’s a vaccine for that. OH NO I SAID VACCINE

14223550930_ff50c301dd_oSo yes, people are coming to the Archive in droves to get their MS-DOS on. I think the number is either over a million already or will be soon, and it may be more. The Archive (very admirably) does not log users in any meaningful way, so it’s not like I’m going to be able to run Webalizer on the stats directory. I suspect that, like the arcade, we’ll be into multiple millions before the week is over. Enjoy.

And I think the feedback from the Beta/Version 2 interface has been a spectacular boon – it’s a direction the archive is going in, and the massive stacks of information it maintains will be that easier to go through. The journey to the upgrade isn’t over – but it’s bringing a lot of people into contact with the Archive’s information that might have missed out on it before. The bounty was always there, and folks inside and out are happy to enjoy it.

The rest of this entry is me just answering a bunch of questions for people and reporters, because both camps just kind of make stuff up when they’re not confronted with actual facts. The answering will be in the form of an essay, not a FAQ.

Actually, speaking of a FAQ, here is the Internet Archive FAQ for the MS-DOS emulator. That answers a lot of the technical questions, or what seems to slow people up. So if you’re just here because SPACE BLORTS isn’t working for you, I think that’ll help. Otherwise, let’s just throw out some thoughts I have and some information people seem to want.

Internet Archive Employees, Weekly Engineering Meeting

 

It is adorable that press and some bloggers have turned the narrative into “Jason Scott did this”. I am a top-hatted loudmouth for a very, very large amount of people who have poured decades of human years into what you see when you start the emulator and it goes “boop”. This includes the dozens of employees of the Internet Archive who keep things running, the developers at the Archive who have made the emulator work in the infrastructure, the developers and volunteers working on the JSMESS and EM-DOSBOX emulators and their porting to Javascript, the Emscripten developers (many) who have made the system work incredibly in the last couple of years, and the emulator authors (now we’re up into the hundreds) who have slaved, tirelessly and with scant recognition, for coming up on two decades to save the software by saving the running of it in simulated/emulated hardware.

I have a strong presence and personality, and a very strong will, so keeping my eyes on the focused goal over the past three years has been something of a contribution, but I’ve written a minuscule amount of code, mostly in the form of scripts. I’ve watched people max their brains out trying to make a system do something it was never designed to do, and watched others figure out how to wend deep into code to make completely indifferent and goliath codebases interact.

I am fearful to write out a list of names right now and miss people, but maybe sometime soon I’ll do an entry where I just talk about all the people and the worlds that have collided to do all this. It’ll be quite long.

IMG_2419

 

Some quick technical things. This weblog is absolutely packed with entries describing the JSMESS system that the vast majority of systems on the Internet Archive use to emulate consoles, computers and arcade games. Just use the search box and look for “JSMESS”; you’ll be rewarded with the years of entries on the subject, including pleas, triumphs and tribulations. I don’t feel the need to repeat that here – if you want it, it’s there.

What I do want to say is a few words about the differences between JSMESS and EM-DOSBOX, which is what the MS-DOS systems work on.

I was not happy about adding an exception-based emulation system into the mix, but JSMESS isn’t quite ready for MS-DOS yet, and I think it’s a ways away from it. When it works, it’ll work really nice, and maybe we’ll even switch over to it, but for now, it started to become a real problem in the timeline of the Archive’s software collection that we were emulating the ZX Spectrum and the Atari 800 and Apple II, but not the massive juggernaut that has been decades of MS-DOS compatible programs. So I made that decision and a bunch of us put our faces to the grindstone and that’s what got announced – EM-DOSBOX.

A random slashdot anonymous poster claims they did critical work before the implementation of EM-DOSBOX, but the EM-DOSBOX Repository appears to be a straight fork from DOSBOX, so jury’s out, but I didn’t have any direct involvement in DOSBOX or EM-DOSBOX, but I’m fairly sure the lineage is direct (although there have been other javascript ports of DOSBOX in the past). If so, step forward and take your glory, soldier.

EM-DOSBOX is a very fast, very slick porting of DOSBOX, using the Emscripten compiler to bring the code into the browser. It implements some tricks which even JSMESS can’t do quite yet, and which might lead to speedups in the future along that line. I can run some EM-DOSBOX emulations at full-speed on my phone, which is sufficient for me to throw up my hands and say the future’s here.

The biggest difference between EM-DOSBOX and JSMESS is that JSMESS wants disk images, gathered magnetic copies of the original booting medium. (An example on the Apple II would be SPACESHIPS.DSK, where this is a “Spaceships” Apple II floppy portrayed as sectors that JSMESS reads like a disk). EM-DOSBOX is quite happy with a .ZIP file of all the files needed to run. It’s rather convenient because the vast majority of MS-DOS materials are sitting in .ZIP files (or .ISO files if they’re CD-ROMs, but we’re not doing those at the moment). The only downside to this is that you have to tell EM-DOSBOX what the “starter file” is in a collection. That’s something that can’t easily be automated, so I have to go into each .zip and do a best guess of what probably makes it go. AUTOEXEC.BAT’s a good guess, as well as PROGRAM.EXE where PROGRAM is the name of the program being run. But I get caught out all the time, and with thousands of programs, how the hell did I deal with that?

15958989827_95976b1841_o

 

The answer was the Screen Shotgun, the automatic player-and-screenshotter script pile I’ve spent a while putting together over the last year. I’ve written lots of entries about THAT process as well, stalwart inquisitors – just search for “shotgun” in the weblog search box.

These days, I have a version of the script that can start up on an Amazon EC2 instance and I can run any amount of systems at 14 cents an hour to generate screenshots as fast as possible. These emulator-playing robots serve multiple functions, but the big ones are that they can take a bunch of screenshots of the programs and remove the doubles, and they can serve as a basic Quality Assurance team, because if the game doesn’t work, the screenshots are going to show it.

Hey, look, 2000 MS-DOS Screenshots taken by robots. That’ll hold you for a while.

The most annoying difference this time was that the MS-DOS programs switch up their resolution, with VGA vs CGA and so on, and there’s not a locked specific “size” of the canvas. A few kind folks on twitter took me under their wing, with various solutions, and I chose one that was very, very flexible, removing the need to hand-type-in resolutions going forward for future machines. The power of the crowd, they say.

1389149289357A number of people to pop in among both this and the other major software release events have said something along the line of “what’s the point”, or “this is a terrible thing”, and like anyone working on a project, it’s kind of a waste to engage people who are stopping by with that position. Obviously, if I spent years on it, I thought it was something worth doing. I’ve tried to articulate why on multiple occasions. It either sticks or it doesn’t.

But the fundamentals are pretty straightforward:

  • Software is history. It is culture, it is process, it is a signpost.
  • Software also has a ridiculous half-life. A year or two, with few exceptions.
  • Software, unlike many other media, has a massive barrier to using it.
  • This barrier does nothing but grow over time. Over enough time, insurmountable.
  • Coming up with solutions like in-browser emulators promises to lower barriers.
  • If people can’t see or interact with software, it disappears. Utterly.
  • This project is an attempt to lower that barrier, ideally to nothing.

After these statements comes a lot of endless debate and discussion, and I’d rather be sitting in some room with a microphone arguing with someone than dealing with e-mails and weblog entries, but I think, much like my work with Archive Team, that once the core subject being debated is deleted or gone (or effectively gone), the conversation essentially ends.

What this army of volunteers and collaborators have done with javascript emulators is we’ve stopped the conversation from ending. We’ve essentially jammed the door open. It’s here now – we’re in a world where computer and software history runs in browsers, and that’s that. It’s refinement and iteration here on out. If my time or priority takes me away, the work will continue. That’s done, that’s a fact. Nothing will take that away, now.

Going forward, I expect debate in places where this effort collides with commercial interests, but I can tell you one thing I’ve learned, maybe something intuitive to people who work in the space of preservation or curation, but still worth mentioning:

In the realm of historical objects that have started their cultural/contextual journey as products, there are a specific subset that have gotten the attention, effort, quality preservation and presentation befitting a worthy project to do so. But that’s it. The vast, vast, vast majority of commercial products, including hardware and software, sink without a trace, or end up in attics or collections with only glancing references made. Maybe a completist providing a text listing of “what there was”, or a wrap-in of a few more-obscure B-list examples, but then it’s this giant, yawning abyss.

I figured, in the example of Arcade Games, that The Nerds would have long ago gutted, garrotted and filleted the history of all arcade machines enough to tell us who worked on it, what it consists of in terms of gameplay, or the usual love one sees showered on a Pac-Man or a Mario. No. No, not at all – there were (and are) games for which we have the ability to emulate a ROM dump someone did as long as 20 years ago, and a photo of a machine sitting neglected in a warehouse. Nothing else. And this is arcade games, where one could argue there was a perfect storm of subject interest, intensity of fanbase, and overlap with the most curatorial personalities in terms of maintaining information. Nope. Sometimes the games exist as a single found circuit board and everything else was imparted from that for the purpose of an emulator.

Now extend that to software, and it’s much, much worse. Now extend it to non-gaming software, especially custom software, and it’s even worse than that.

While people wring their hands over the preservation of Doom, a hundred thousand programs are at risk of being locked away only to people willing to burn hours to spellcast them into function in private geek caves.

This is a real and functional problem, and it has real consequences. This set of projects is, at some level, a strike back against that. It’s worth the doubt and the derision.

IMG_3916

Finally, the little prayer I always say.

I’m very lucky – I’m working for a wonderful non-profit library, a dream of a lifetime. It allows me to work on projects like this, to maintain the effort to keep the project focused, and to have a platform to make the project available. That’s due to the forward thinking of Brewster Kahle and my various managers at the Archive, that they let this maniac onboard to work on things. I never forget to be grateful.

 

 

 


The MS-DOS Showcase (And The Ascension of Version 2) —

2,400 MS-DOS games and programs have gone live on the Internet Archive. The full collection is here and the games-specific section (basically all of them, for now) is hereBut wait before you click on those two links.

That’s the headline, and based on the thousands of plays that this collection has already gotten, a pretty well-known thing if you were looking for it. Considering the Internet Arcade has dragged in over 5 million people, this new collection will probably bring in a flood of its own. Welcome.

But.

Let’s talk about what makes this different than the other playable software libraries I’ve been putting up.

First, I really worked hard to have only fully-functioning programs up, or at least, programs that gave viable, useful feedback. Some of them will still fall over and die, and many of them might be weird to play in a browser window, and of course you can’t really save things off for later, and that will limit things too. But on the whole, you will experience some analogue of the MS-DOS program, in your browser, instantly.

mobygames-logo-sm

Next, this is the first time I’ve successfully bridged metadata with Mobygames, the game information database which has been running for many years and recently returned to private management. My scripts yank in metadata from the entries, and place them in a relatively sane way into the descriptions. It’s not perfect, but it’s pretty good. Best of all, if someone overhauls the entry on the Mobygames site, I can see about pulling in the improved data into the Internet Archive item over again. So it’s still a one-way pipe, but it’s a fast one-way pipe, and the games make much more contextual and informational sense now.

Anyway, here’s the next big deal – I want you to start really getting into the Version 2 interface of the Internet Archive. So, go here:

v2

https://archive.org/details/softwarelibrary_msdos_games/v2

This is a way to go directly to the Version 2 of the Internet Archive’s interface. In the upper right, there’s an exit beta button, so you can always pull the rip cord.

The Archive introduced v2, or “the Beta Interface” late last year. It was slow, stocky, and freaked people out. But folks got the idea, mostly – it was taking a site that had only incremental changes for 13 years, shaking the whole story up, and re-imagining the whole thing as a visual and browsing collection, as well as a way to dig deep into the materials.

Since last year, it’s gotten faster, slimmer, has added a bunch of features, and continues to become better to use. But it needs feedback, which is why I’m pushing you at it.

Enjoy the games, and check out all those beautiful screenshots! Play a few programs, note how you get around to things, and talk about what works and what doesn’t work for you. There’s a feedback button – use it. The goal is that you will be able to do everything you can do with the old interface with the new, but that you’ll have so much more happening on the new one. And remember, v2 works across all of the Archive… all the collections are out there, be they movies, books or audio, and the new interface has cool ways to interact with them as well.

100327_2b

As for the software library itself… many titles will stick around – some will go, or change, or be replaced. This is still brand new territory, and with the addition of the MS-DOS programs so far, this no-plugin in-browser experience is one of a kind in the world. I’m working just as hard as the V2 devs are to make everything work, and it wouldn’t be worth my time if I didn’t have people trying it out, commenting on it, and making use of it.

So please, let’s start 2015 right – in CGA and with a brand new way to use the Internet Archive.

 


Each New Boot a Miracle —

This week, we got the emulation of MS-DOS programs (mostly) working on the Internet Archive.

Up to now, every single emulated system has been via the JSMESS project, which supports well over 600 systems and which does a hell of a bang-up job on it. The fact that one system (MESS) emulates everything from an Atari 2600 to an Apple II to a PDP-1 and so on… that’s pretty amazing stuff. And when we had well over a dozen people intensely involved in porting MESS to Javascript, it made sense to use this one emulator for everything, since once we got the work done, it just worked, for many many systems. It was good, economical effort.

There are, of course, many other emulators out there, some of them little hacks meant for one machine and some which handled multiple systems. Some of them, over the years, simply got absorbed into MAME/MESS, and others died out.

The MESS developers have biases and preferences, like anyone else, so there are systems that have gotten a ton of attention, and systems which have gotten some rough scaffolding, some proof of concept, and then little else. This isn’t to say they’re slacking off, of course – it’s just that there’s a lot of work to do in a lot of directions and they move along at the speed they’re comfortable with. Also, they’re more focused on getting it accurate and hardware compliant than fast and easy. This is, absolutely, the best long-term solution – just hacking things to “sort of work” is the province of forgotten half-functional messes that don’t function with an OS upgrade or an app change.

Also, there are cases where a system gets a burst of love from a developer and it just sings. Recently, the Apple II driver within MESS (and therefore JSMESS) got a massive boost in cleanup, rewrite, and expansion – and the result was a fast-moving (nearly 10 times increase in speed) wonderbomb that can handle graphic modes and settings that previously dropped everything. MESS continues to be the best choice in emulation, long term.

However, projects that focus, intensely, on one platform for emulation will do amazing work in support and accuracy. They’re not 100% flexible and they certainly take shortcuts here and there, but they work really well for what they are.

DOSBox_screenshot

One such project is DOSBOX, the DOS emulation system that has been around for over 10 years, and which does one thing, the x86 architecture and specifically running of DOS-compatible programs, pretty well. It has different goals and approaches than MAME/MESS, but it a very respectable and well-run project, and we’ve all benefited from their work. Except for the fact it only does one platform instead of 600, it would behoove the JSMESS team to attempt a port of it.

Luckily, someone already did that.

Dreamlayers ported DOSBOX via Emscripten into a browser-functional emulator. He did it all by himself, and he did it very well, all things considered. His name for it is em-dosbox.

I’m just going to lay it out and say that Dreamlayers is a software engineering genius, one of those people with a gift for coding and making things work not just better, but understanding what things have to be left tied down and waiting for later improvements. Most of his em-dosbox notes are where Emscripten falls down as a compiling and conversion platform, with indications of how they can be improved. And buried in the code of his is an alien artifact that makes the generated javascript from the process run extremely fast. It’s called extractfun.py and let’s not go too deep into it, other than to say that if we ever figure out how to plug it into JSMESS, fireworks will result.

Now, em-dosbox is not perfect – far from it. It can freeze a browser, and it runs into problems of all sorts when running some of the oddball machinery that the DOS world produced between 1980 and (roughly) 2003.

But when it works, man, it just works. Clear as a bell, right as rain, a DOS window boots into a browser. If you go fullscreen, the mouse even works. (It works in not-fullscreen, but then you see both the desktop and the emulator cursors and your brain is sad.)

It’ll be a little while before the collection is really public, as there’s now the process of throwing thousands of DOS programs into the system, verifying they boot, making screenshots, and all the rest of that delightful curation the world hopes is waiting for them.

But it’s happening. It’s really happening.

PC_DOS_1.10_screenshot

And now more fundamentally, the personal take, and why this three year journey has been so worth it.

MS-DOS was always my main thing; I coveted the Apple II world, really got into the Amiga when it came out, danced among Ataris as I could, but for the vast majority of my “work” or whatever a teenager’s activities count as, I did it on MS-DOS.

MS-DOS was what my primary computing machine was from 1983 to 1988 – I hacked on the exact same machine, with minor improvements, all that time. I grew used to the clack of the keyboard, the arrangement of the fonts, and I never really used a windowing system on it for any time in high school. I called BBSes and online services with the internal 2400 baud modem – all the files I’d downloaded for my textfile collection, and which ended up on TEXTFILES.COM, passed through that machine.

Work under any system for long enough and you buy into its processes, whether you want to or not. And I bought in big. The grey on black text, the directory structure, the FILENAME.EXT system, the all-around aesthetic. I feel DOS systems glorify text in a way that other computers didn’t, due to IBM’s long, long history at providing the cleanest, crispest fonts for typewriters and correspondence. I was an IBM kid, after all, and my father’s acquisitions of company-made computers had me fully involved in it for years and years.

So, MS-DOS is my thing, and getting this working was my thing.

A lot of people poured themselves into the original DOSBOX and EM-DOSBOX. But two people, Daniel Brooks (freelance) and Hank Bromley (Internet Archive Dev) worked very hard for a significant amount of time to jam the EM-DOSBOX system into the Internet Archive environment, which is much more difficult than one might expect. Imagine playing a chess game by keeping the board in your head and dropping coins with move descriptions written on them down a grate and you start to get it. A lot can jam up in the pipes, and the whole thing wasn’t designed for what I’ve been making it do. They worked for days on end, bouncing iterative changes, seeing the results, and at the end, it worked.

The functionality of the MS-DOS/EM-DOSBOX emulator within the Internet Archive environment has more caveats than a gym membership, to be sure – dropping to the prompt currently freezes it. You have to tell it what exact file is the “boot” file, the one that will be executed on startup. Too big a file, and it blows out your stack. It doesn’t write out to the emscripten filesystem, causing some to blow up because they can’t write to “disk”. And the sound… well, sound continues to be an issue until the nerds-that-be fix them in the browser world.

But…

amnesia

When it works, it’s glorious. The speed is really clippy, the prompts blink like I remember, and as it plays CGA, EGA and VGA graphics out and hits the notes, I can sit back and enjoy that clean, crisp little world I spent so much of my childhood in.

And more than that…

The Arcade, while fun, is not a technologically advanced project, and the systems are beyond simple – they work or they don’t. They boot up, they play music, they take your quarters, they play a game. I knew people would love them, and they certainly provide entertainment, and they’re historically important and culturally relevant. (And the effort to accurately reflect them in the emulator is a Herculean effort, I rush to say.)

But they’re also very simple affairs, and designed to impart 3 to 10 minute experiences. You get Pac-Man in a few minutes – you understand nearly all to know about Defender in a day or two. Granted, the skills to keep past that 3-10 minute period are legion and deep, but that’s not what most people would experience anyway.

But with a computer program – especially one designed to be sold for a good amount of money, or which represents a lot of work by someone, there’s a depth in there, and an intensity of the product, that can’t be ignored. A strategy game that doesn’t care how long you take to enter the next move, or a simulator that carefully gives you dozens of knobs to twiddle or points to tweak… that’s the harbinger of the computer experience. Not to mention the applications, those tools of productivity that have permeated computers since the beginning. It’s rich and lovely, and it’s my favorite thing.

Getting the Apple II and ZX Spectrum and Atari 800 collections running at the Archive have been a pleasure, and the resulting screenshots are beauty to behold. But MS-DOS… that’s my neighborhood.

The neighborhood looks good.

ezana

The system uses a .ZIP file that contains all the files that will be used inside the emulator as the “drive”. So if you have 5 files, including some batch file and an executable, accompanied by some data files, those will all be loaded in at the emulator’s boot. It will then require a designated “start” file, one of the collection, that it will attempt to run and execute against.

Sometimes this doesn’t work. There’s a lot going on there and my sources are variant.

But sometimes, it just works.

ciscoheat

And when it works, I see smooth as glass graphics, beautiful pixel art, and those beautiful words, and the whole thing is alive again, in the emulator’s box, infinitely reproducible, instantly referenced, ultimately preserved in a pretty useful fashion.

ather

border

gotowar

Folks who are the sort of folks who professionally quibble will professionally quibble, but the now-years of work I’ve been involved in this (and others have been there right with me, by the way) are utterly justified every time I run the ingestor and push a .zip file of a gathered-away program online. Within seconds, it’s a single click away from beeping, booping and playing music, or standing stoically waiting for my input, unaware of the science-fiction universe it now lives in, destined to never really die, and to stand and be counted by history.

It’s like watching a miracle with every boot.

It’s a great feeling. It’s the best I get.


The Tourist, The Researcher, The Maniac —

This year was primarily about software, specifically collecting and emulating it.

That went well.

Asked what my goals or projects are for work for the next year (2015), I have resolved that it will be cleaning up and making it easier to go through the mountain of material in the Internet Archive’s stacks.

Naturally, I’ll be continuing to help with Archive Team, improving JSMESS, and acquiring tons of crap to throw into the hard drives as well, but I really want to take advantage of the tools and time to really dig deep into the Internet Archive’s wonder as a collection. It is a huge, huge set of materials and I’m positive people would have minds blown if they really came into contact with the true size and breadth of the material.

So that’s how it’s going to go in 2015.

But, as usual, I can’t wait and I’m starting now.

archive-1200

First, understanding the mass of material already in the Internet Archive that has come flooding in in the past few years is a task in itself. We don’t have a few monks chanting – we have 4,700 chants by a group of monks in France. Or maybe you want your sermons in english. Well, that’s fine, here’s a set of 21,000 of them. Or religion bums you out – that’s OK, take your mind outside and look at 8,000 books from the library of the New York Botanical Garden. Or go back in time to see a hundred episodes of Starcade, the game show for video gaming.

I can do this all day. There are thousands of groupings of thousands of items down there. It goes in many directions, and it goes deep as your tolerance can go. Books? The Internet Archive’s contributors added 1,000 books yesterday, and 1,000 the day before that as well. (It’s been a good couple of months.) Audio, don’t get me started. Newspapers, tens of thousands of issues of newspapers!

Stuff. We have it.

IMG_5922

Before I talk further, let me also say I’m not the only one thinking about this, especially within the organization. There is a very intense effort underway to completely redo the Internet Archive’s interface. If you visit http://archive.org/v2 you will go into the Beta Interface, which you can see is coming along swimmingly. (There’s a button in the upper right to turn it off and return to the current interface.) A team of people spent a year dealing with both the backend and the frontend aspects of this new design. It has a way to go, but it actually brings this whole issue to the forefront, since it’s going to become a lot easier to browse the collection.

But for my little bit, I want to find ways to get a better hold on the millions of items, move aside spam and broken items, and group up really cool and similar things without me spending every waking hour doing it.

But before I even do that, it’s time to talk about who I think the audience for the Internet Archive’s media riches are in the first place.

Hence, the Tourist, the Researcher, the Maniac.

10658828_10152506233184527_5746589902833348620_o

The Tourist washes up on land, pushed in by the general tools of contemporary discovery – popular links and search result. The Tourist can be very compelling, because whatever dragged a Tourist to your site probably dragged a lot of other Tourists. They can almost seem like a coherent movement, or a congregation. But they’re not, any more than one or two methods out of a building results in an actual orderly line. They’re a mob, a gawking fog of incoherent attendance, a group who has come in because there was a blinking sign somewhere, and having found something similar to what they want, have smiled and gone away, or they’ve stuck around just enough to pass judgement on somewhere they’ve never been and will likely never return again.

You can tell a Tourist because they go back to comment at the place they heard about the link, and the commentary they make tends to be along the lines of “I have a lot to say about the way about how nothing impresses me.”

Tourists can definitely become recurring users, of course. But their introduction to the place will almost always be “here was a thing you might like”, and they run in and out, because these are busy people, and there’s a lot of things to like.

All is fine and good with Tourists, unless your site or the site attracting Tourists thinks Tourists are the best people to acquire, what with their bloating numbers and all. They’re not – they’re basically sugar. Quick high, bad aftereffects, no long-term benefit.

Img_1223

The Researcher has come into your life and your site because they’re Looking For Something. Hopefully you have it.

Their requests are usually somewhat strange, but they’re also really flexible about what they find. They browse, they look up similar items, and they try to do some of your sorting job for you. In a good day, they send some of the results of their work over – “I have a list of items that need to go into a different collection” is a common one Internet Archive gets in the inbox.

Researchers often want something that is not quite what search engines can provide. A query can be something like “I want all the magazines that reviewed the Apple III”, or “I want to look at movies that feature skateboards or skateboard parks.” You can feel like everything is an incomplete patchwork of metadata and fairy dust after a Researcher walks around. But they’re not trying to be mean or disruptive – they just really appreciate the place and want to get the most out of it.

Researchers hunt in solitary or in small packs.

IMG_3916

The Maniac sounds like who you’re doing this all for.

You are not.

The Maniac comes in, and never leaves.

The Maniac has a dream inside their head of how the world should be and you are failing at it, right now, even while you’re not doing anything.

The Maniac has lots of ideas for the site, how things should be done, but they’re really a bundle of unfunded mandates and issues that affect a very, very small number of people. As a bonus, they become anti-ambassadors, posting frequently and viciously in other realms about how your place wouldn’t suck if you only decided to use the sdfjkhsfsdjfh standard or you would just allow unicode in usernames. I’d say The Maniac hasn’t done anything, but they probably have done something and by Ramthar’s Crowbar you will hear about it, a lot, as explanation of why they get to come in and take a dump in the potbelly stove.

DPI matters a lot to the Maniac, as does impeccable presentation and completeness. These, in themselves, are excellent things to have, if you can get them. But Maniacs are very, very quick to flip out if any of an internal checklist isn’t met. Maniacs flip out a lot. It’s what they do.

Maniacs are almost always solitary, although they can give the impression they travel in packs – but to be honest, every Maniac thinks all the rest of the Maniacs are tourists.

IMG_3398-Scaled

So, there’s a fourth group, called The Future – but going too insanely over the top to appease people that don’t exist with goals and ideals you can’t possibly anticipate is not today’s exercise. So you keep out there in the hazy realms of Tomorrow, you crazy Futures.

I think the sweet spot are the Researchers. You want to push Tourists to consider becoming Researchers if the mood takes them, and you want Maniacs to either tamp down into becoming Researchers, or go spill their hot oil of hate out from another castle.

Make the site make sense, start culling out items that are spam into places that won’t discourage browsing, and help build collections that sing. That’s 2015 for me – researchers welcome, Tourists a delight but sorry about the sdfjkhsfsdjfh being sub-par – I’m sure we’ll get to it soon.

Now, let’s get some sorting done.

 

 


Pain —

10660609_10152695388558834_1918406404_n

Somewhere around the age of 26, I woke up and couldn’t really get out of bed.

The reason, in the immediate, was that any attempt to put weight on my legs sent shooting pains through them. Real, intense, shooting pain like there were rods lodged inside the legs and they sent out shocks when they detected any movement or effort. I have forgotten what I thought at the time, but I do remember how much it hurt.

The mattress was on the floor, so I was able to roll and crawl to a phone and call for help. After getting clothes on (somehow) and getting down three flights of stairs (somehow), I waited outside and was taken to the hospital. During this hour or two before I was picked up, I learned which positions hurt the most, which the least, and tricks for getting around, somewhat. I also stopped being shocked at how much it hurt and just winced or yelled an awful lot as I discovered a new angle, a new approach that would make it all hurt anew.

At the hospital, after some battery of tests, I was given a bunch of over the counter pain relief, some crutches, and no real idea of what was wrong. I do remember one young doctor saying “You know, this really acts like gout. But you wouldn’t have gout – you’re 26.”

I hobbled on crutches for a couple days, the pain went away, I lived on.

A couple years later I was arguing with my girlfriend while she was driving her car. I remember this because in the middle of yelling, I felt something very unpleasant and I told her to stop arguing because something felt very unpleasant.

The thing about pain is that you both remember and don’t remember it. I can, these many years later, describe it as a tightness, a strain deep inside my gut, like shoelaces of nerves had been pulled very intensely, and were now knotted in a bow, and my movement in most any direction was pulling at them. That’s what I remember made me go from thinking I’d had a bruise or a swallowed knick-knack in there – its that there was no logical sense to my ratio of pain to the motion. Moving and trying to get into a good painless angle did nothing, and then it would stop, and then it’d come back harder, and I was rapidly losing brain function. I asked and then really begged my girlfriend to take us back to her house.

On her couch, it just got worse, we didn’t know if I had a burst appendix, a knife I’d somehow fallen on, or if something inside me was just going necrotic and taking out my organs. But boy, did it hurt. It hurt until I ran out of ways to think it was hurt. I wanted desperately to take something to knock myself out but then I simply assumed that I’d die in my sleep and that seemed needlessly unpleasant and surreal. So I lay there, looking up at rafters, overcome with pain, waiting for either healing or death. Some measure of healing happened, and ultimately I fell asleep, as if I’d run a marathon.

Let’s move ahead to the diagnosis, that came in my mid-30s. Kidney stones.

There’s two kinds of kidney stones – the calcium ones and the acid ones. I have the acid ones, which are notably rarer and as a bonus, the treatments, which I had many, many, many meddling Dr. Google-Readers give me over the years, are entirely opposite. Drinking cranberry juice doesn’t work, for example – a lot of people told me to drink cranberry juice. No.

Due to a set of circumstances where I didn’t seek out the best doctor but just fell into doctors, I didn’t really have a doctor look at me, properly, with competence and awareness, until somewhat recently. This doctor has done wonders. So there’s your happy ending.

Now, let’s go back.

By my early 30s, I was passing 100 kidney stones a year. That is not an exotic allegorical statement –  I was passing some sort of stone every few days. For months, and then years. On the whole, over time, I didn’t even notice them. In fact, I almost got worried when one didn’t show up in a given week. It was just another fact of life, like waking up feeling tired or occasionally having a joint feel sore. Stuff happened, and some of it involved the fact that I ceaselessly created stones and passed them frequently, with small pinpricks of pain involved in the process.

By the time this was all happening, pain was rapidly not becoming a problem for me. The kind of pain one might have from hitting your foot or shin against something, or from grabbing something sharp, I could notice, but it was more like hearing a distant yell, like someone cupping hands and shouting that you should turn back, the woods are not safe. It wasn’t immediate, and it certainly wan’t concerning. By this time, most pain was being given a level of interest far beyond what it probably should have been.

The reason for this is because of the times when it got my attention.

Imagine, if you can, a very large meatball inside a bowl that is only a little larger than said meatball. Imagine that meatball is kind of sloshing back and forth, kind of randomly, in the bowl. That’s how it felt, and how it started.

I’d feel some overarching dullness inside myself, and I started to get really good at recognizing it. The feeling, you see, was a largish stone, and one that had gotten in some way stuck. I got really, really good at noticing this, because I knew what was coming next.

If I was lucky, I could run in some direction and get my hands on water. Lots and lots of water, like quarts of it. I’m really fast at drinking water, and I’d stop mid-conversation and drink everyone’s water at the table, or run towards a store and buy a gallon of water and start drinking it at the cash register. Anything, you see, to get as much water flushing through my system as absolutely possible.

If I was lucky, and a few times I was lucky, the meatball would evaporate. It’d kind of roll into a cloud version of itself, and waves of discomfort would subside, like a pool minutes after someone jumps in it.

I wasn’t often that lucky.

I am a weaver of metaphors. I am a person who tries to find the middle ground between the listener and the moon, and concoct a pointing finger that will lead you to the moon. But where do you even begin with describing pain like a pain of a stone that’s lodged in your system?

First, it’s often in your gut, deep inside, so it doesn’t feel like the sharp sting of a skin burn or a radiating line of a strained muscle. It feels like something deep inside you is twisting apart, and has no recourse to stop, and probably will never stop. After a few minutes, you forget when the pain started and you definitely stop assuming it will ever end. Oh, you definitely hope it will end, but it never seems to.

Sometimes I was just two eyes staring at something in the room and pain. One time I was a flattened person on the floor of an airport, screaming, everyone looking at me, while a stranger held my hand. Another moment, I was in the softest bed ever, in a visit to my father’s home, with my siblings watching me appear to be dying for no good reason. They scrambled through medicine cabinets looking for something, anything, but ultimately all they could do was sit elsewhere in the house until the screaming stopped and I fell asleep.

This happened a lot. I’m not telling anyone who has lived with or spent a lot of time with me something new – they probably have at least one memory of me either having to leave very quickly to go be possessed elsewhere, or they’ve sat nearby, wondering what you do with a body when that happens, when your friend is gone. They know. Things people shouldn’t have to consider with guests, my occasional attacks gave them time to consider.

I only got operated on once, by a doctor I’ve called Doctor Coldfront since the time he operated on me. He had the bedside manner of a mailman to a mailbox. I’m sure he was competent, but ironically, he made me hurt the most, although with that pain came some interesting revelation.

He inserted a stent into my left Ureter, which is the tube from a kidney to the bladder, while I was under. I didn’t mind that – I was asleep.

Some time later, and I’ll spare you too many details, he removed the stent under local anaesthesia, as an outpatient procedure in his office. I had requested a full knockout but he wasn’t interested in doing that – I assume out of 10% concern about putting someone under and 90% of the inconvenience for what he perceived as a minor operation.

I had nightmares about that day for a decade.

But as I said, some revelation came from that – which was that there are doctors and professionals all throughout the world, some even here, who use some level of medical procedure to torture out information. Some of them are paid by governments, some by private organizations, to do so. Back in the hazy glance of youth’s cobbled knowledge and self-assuredness, I thought that a reasonably strong-willed person could resist giving whatever information was being extracted, under torture.

Considering how I was treated by a doctor who was merely inattentive to my wishes, I gained the knowledge and the understanding that no – information is probably not worth keeping from someone who is intentionally, unendingly, aggressively intent on you staying alert and bathed in pain for as long as is necessary for you to give up whatever pearl of knowledge they wish to extract.

That’s an interesting lesson to have gotten.

I never spoke to that doctor again. I hope someone broke his heart.

So, why go into all this? Well, two reasons.

First, my weblog has been suffering a little from a case of the Nerd, and has way too much technical information without much self-revelation and consideration. So there’s a little self-revelation.

Second, I wanted to make clear to a certain subset of fans and followers as to what my secret is, since I’ve had people literally say that they are trying to “Jason Scott” something, i.e. throw a lot of energy at it, or never stop.

My secret is that I am all too cognizant of how quickly life changes, and how one can go from a moment of relaxing in the sun to feeling pain so great, so intense and miserable, that you will cry tears of anger at your previous self for wasting precious minutes and hours in the paradise of existence you had been granted. It is why I don’t like lying down for too long unless I am very tired, and why I often (but not always) get up with a burst of intensity, worried about what I missed and what needs to be done.

It’s also why I tend not to wear coats, and underdress for the weather. It’s why I can hold my hand over a candle for a stupid amount of time. And it’s why I hand people things and am confused at the look on their faces, only to realize it’s because I handed them something blistering hot or bitterly cold.

It’s because pain’s kind of an abstraction now. When I go to the hospital these days, I have to exaggerate what pain I’m feeling, because I realized after a few times that they were asking if it hurt, and my definition of hurt and pain is not theirs anymore. I can see clearly, and I can think about things, and I call that not being hurt, even with a hole in my arm or a knife cut that needs stitches.

I’d call it a superpower, if I didn’t make an effort to remember how I got it, and how truly useless it is, and how dangerous. But I have it.

In the meantime, here I am, working very hard, doing a lot of things, and counting the missed opportunities and the upcoming chances to do even more things. Because of the pills I got from the right doctor, three years ago, I have not had a single kidney stone, or a single day in true screaming pain that defined my life for so long.

And if that’s what I have now, a day without that pain, then that is a day it would be a crime to waste, to not pull the most out of, to not work on things until I barely can stand up and then fall asleep until the next round of things to do.

I do not recommend the journey to this situation to anyone.

Have a great day.

 


It’s Time To Keep Solving the File Format Problem —

Here’s how a planned large-scale set of roving ideas and goals ends up being a case of just doing one thing right.

Theproblem

The original plan when I cooked up Just Solve The Problem Month was that there was a set of problems out there that just needed a few hundred people to contribute time and effort, and some otherwise seemingly insurmountable problems could be solved or really, really beaten down into a usable form.

Aaaaand what instead happened was:

  • We announced and set up a Just Solve The Problem Wiki for the first problem.
  • A lot of people worked on the Wiki.
  • I got very busy.
  • People kept working on the Wiki.
  • It’s been two years.

So, basically, we have one single output from this idea.

And that idea is the File Formats Wiki.

The File Formats Wiki has been progressively, constantly updated to for two years and has now entered its third year of existence. If you check the statistics page, you can see there’s been thousands of pages added.

I had previously said that November could be Just Solve the Problem Month. Well, the machine that this Wiki is on had to get some maintenance, so I moved it to December. And now, it’s just File Formats Month, I think…. that’s a good enough project itself.

That is, now.

So let’s go over it again, shall we?


 

The world has a problem. Well, the world has many problems, but this one is a big one.

File Formats.

Oh, we have been so good at file formats. We’ve happily come up with file formats whenever and however we want to, whenever information has been captured up into a bundle. We’re so good at it, we often don’t even look at what formats are out there – we just make a new one. And then we later change the file format we created to accommodate some problem we didn’t account for, and so the file format becomes “file format v2.0″ or “file format NG” or even worse “entirely new name for file format”.

And then we forget about them.

Really new file formats, that is, ones created in the last five to ten years, are pretty lucky, relatively – there are webpages and standards posted into all sorts of locations, and utilities available besides, to deal with them. And really, as time has gone on, one person or another will document a file format to the best of their ability, stick it on a webpage, and there it will set for some time before being deleted or forgotten.

All of this is a problem. This is a problem that can be solved.

We’ve set up a Wiki, called the File Formats Wiki.

Everything on it is Creative Commons 0 (Public Domain-ish) because it turns out you can’t just say something is “Public Domain” and have it stick worldwide. But the contract is basically that – everything on the FF Wiki can be used anywhere else – the knowledge can be spread to other similar projects (and there are some) and every boat can be lifted.

Every file format or container for information is eligible. We even have Piano Rolls, Tree Rings, DNA, and Spoken Languages. Obviously, our big focus is on computer file formats, though.

We’re looking for descriptions, links, clarifications, and tracking down of documents. We want to make this stuff get assembled in one place as best as one can.

We want it that when you find something is in ‘a format’, that this Wiki could be your first stop on the research, and possibly your last one as well.

You register, you can edit it, and you can get started basically immediately.

I’ll be working on it all month. I hope you will too.

Let’s solve this problem!


Gamepads! —

A week into the explosion of the Internet Arcade, with what has now been millions of visitors to the Archive as a result, and we’re getting pretty far down the “what still confounds people” realm. As someone who worked in tech support as his first “real” job at Psygnosis, (shout out to Chris Caprio and Jim Drewry for the hire), I know both the frustration of not having things work right when you expect them to, and the propensity on the technical support side to go ‘oh come on, why don’t you have this already”, like the mass of humans out there is just one person calling you back over and over.

In the last week, the JSMESS team made the software load in Internet Explorer and Safari, fixed aspect ratios on full-screening of the video, made keys work in fullscreen mode on all browsers, and shored up a bunch of little details not worth going into here.

This leaves the #1 thing we’re still getting mail about: Gamepads.

SO YOU HOOKED UP YOUR GAMEPAD AND IT WAS A LITTLE WONKY OR DOESN’T WORK RIGHT

  • Please bear in mind that JSMESS has basically nothing to do with Gamepad Support. We used Emscripten to compile it in. Whatever Emscripten does and whatever the generic Gamepad API loader does, we do – we do no development on it. So, first….
  • In all circumstances, definitely visit the Gamepad API Tester. This is a nice little page that will show what the current version of the Gamepad API thinks your controller is. If you can’t see it, or it acts weird, it’s the API.
  • Some gamepads actually press keys instead of doing “gamepad signals”. For example, we used the X-Gaming Tankstick during testing and it worked out of the box, since it’s basically another USB keyboard, and it was mapped to MAME controls. (Disclaimer – after X-Gaming found out I was using their stick at the Archive/Arcade, they sent a couple more a day or two ago.
  • There are software packages out there for various platforms that will convert joypad/gamepad movements to keyboards. For example, there’s Glovepie (Windows), Xpadder (Windows), Joystickmapper (OSX), and many, many more. This is a known thing of interest and people have made a lot of programs that do the same thing.

Ultimately, the message that we have at the bottom that says you should hook up your gamepad and then will go “Gamepad detected, you may need to refresh…” – well, this is because the way the API works, it’s not exactly guaranteed the signals from the gamepad will “Catch”. I tried to put in a message to go “keep messing with it if you don’t get it”.

Some people also got confused that this meant you could ONLY play these games with a gamepad. That’s not true – it’s just better in a lot of cases, once you have it working. The keyboard controls of MAME were always a weird compromise, and you can really feel it with some of the classics.

Remember, there’s a TAB command in all JSMAME emulations, letting you see the key mappings, and testing them with your joypad. It’s not perfect, but it at least informs.

The Joypad API specification is very new. It was barely up and running when we got it going in September of 2013. A year later, it’s there, and it works, except when it really doesn’t. That’s the nature of all of this – cobbled together pretty-good things presenting you a programming miracle. Like the fullscreen and the browser compatibility, I expect it to get incrementally, notably better.

Good luck.


Before It All Arrives —

The numbers are a little hard to calculate, because the Internet Archive, heroes to the core, do not keep logs for pretty much any time at all. But using a few methods, comparing some general graphs, and doing math, the admins handed me an estimate of how many people have played video games at the Internet Archive in the last 3 days.

Two million.

Two million.

While I’ve had bursts of somewhat public facing events and wonders over time, none of them have ever have crashed into the lives of millions in just a couple days. And certainly not in the continually ongoing way I am finding people are returning, over and over, to look at the games. Additionally, it hasn’t taken people long to find other software collections, and other places where the emulator has been utilized for consoles and general computers. In other words, they are cascading through all of the services that the archive provides, and poking at them all with brand-new eyes. Many thousands will have never heard of the Internet archive before Sunday.

I am hesitant to use a term like “turning point” in regards to the archive in general, since you could go on any subway platform and somebody who was waiting for the train would probably know that there was something called the Wayback machine. It’s been around for 15 years, and even if people don’t know all the details, they get with the Archive is about.

Not so with emulation, or the state of software preservation. The last three days have been, unquestionably, a turning point in emulation.

Emulation made the local TV news. The ability to play old software in your modern browser and on your current machine was a decently packaged story that found its way into magazines, newspapers, blogs, podcasts, and an unknown amount of water cooler conversations.

People who know me let me know about overhearing conversations about the Internet arcade, and therefore Emulation, in classrooms, workplaces, and in the street.

For the contingency of people who were hoping that one day society would understand what Emulation was, and would be able to articulate, even in the lightest fashion, its underlying theories, well, that day has arrived. That day is here.

Certainly, there were and always will be factions within the regard of historical software. Some people believe that actions like this are a dreadful mistake, and could only lead to horror and sadness. Others are flying high today, getting exactly what they had always hoped for.

A rough travel schedule last week has left me rather sick, and I am still recovering. Regardless, I can’t let these moments slip away. My work in historical circles only highlights for me how fleeting moments of happiness can be. We were watching as many as 1000 people a minute running into the Internet archive, overrunning resources within, having an incredible time. The sky is the limit for them. For this time, a bounty lay before them, utilizing the strength of their own computers to provide them a nostalgia roller coaster, a recovered treasure trove of memories, a new insight into a world nearly 30 years dead.

It can’t last. Nothing lasts.

What’s coming forward are questions, and ideas, and arguments, and discussions. They’re on the way, I can feel them formulating now. Letters and think pieces and essays, and debates.

I’m sure I’ll play a part in them at that future time, but I just want to go a little bit into the past.

A year and a half ago, I participated in what was probably the most white-hot conflagration of software preservation experts in the country. It was called preserving.exe. Here was the agenda and information on it:

PRESERVING.EXE (May 2013)

I gave a talk there. It was called “I did it 35 minutes ago” and in it I laid out the importance of software preservation, and the efforts of the Internet archive to move forward in that space. I am more than happy to admit that the speech was equal parts passion and plans.

I predicted the inevitability of what’s happening now, this week. I asked peers and contemporaries what they were going to do about it, what moves needed to be done, and if we are going to sit by and let software be buried, forgotten, or if we would work to give it the due it deserves and the access it needs to have.

I asked, fundamentally, what software is, in a historical context. Is it an eternal product, service, and item that is always to be sold, treated as such, perpetually unreachable, and distributable until the very media it is on rots beneath it? Will software have to be something described, as a lore, instead of presented for people to experience themselves and understand?

In other words, would software’s very specific nature of easy duplication and transfer ironically doom it to disappearance? Would the software preservation experts refuse to make their hard work available, out of fear, concern, lack of guideposts? Would the newness of the medium paradoxically lead it to be lagging decades behind in advances made with almost every other medium? Is that the world we want to leave behind?

I pounded my little podium, and I sat down. And when I went home. I went back to work.

Like blasting a firehose into a kitchen, the pure sheer numbers of people now running through the Internet archive are a fascinating spectrum of regard. Some people are disappointed when they run into rough patches, others are delighted that the whole thing comes together at all. Shouts of disbelief clash against demands for decade-old playing tips. The project has been praised and cursed, lauded, and questioned.

Through it all, one thing is obvious: software is more than code. Programs are more than executables. This medium we have formed forms us, and returning to it brings back joy and the agony, no different than a book, a poem, or a face.

Soon, a stunned populace, an alerted establishment, is going to come forward and have a lot of questions and want to debate.

But before they arrive, before the buzzing in the distance becomes a roar in the valley and a scream in our yard, I have one last question, to colleagues and bystanders:

Where will you be?

It is going to be most interesting for me to find out.