ASCII by Jason Scott

Jason Scott's Weblog

The MS-DOS Showcase (And The Ascension of Version 2) —

2,400 MS-DOS games and programs have gone live on the Internet Archive. The full collection is here and the games-specific section (basically all of them, for now) is hereBut wait before you click on those two links.

That’s the headline, and based on the thousands of plays that this collection has already gotten, a pretty well-known thing if you were looking for it. Considering the Internet Arcade has dragged in over 5 million people, this new collection will probably bring in a flood of its own. Welcome.

But.

Let’s talk about what makes this different than the other playable software libraries I’ve been putting up.

First, I really worked hard to have only fully-functioning programs up, or at least, programs that gave viable, useful feedback. Some of them will still fall over and die, and many of them might be weird to play in a browser window, and of course you can’t really save things off for later, and that will limit things too. But on the whole, you will experience some analogue of the MS-DOS program, in your browser, instantly.

mobygames-logo-sm

Next, this is the first time I’ve successfully bridged metadata with Mobygames, the game information database which has been running for many years and recently returned to private management. My scripts yank in metadata from the entries, and place them in a relatively sane way into the descriptions. It’s not perfect, but it’s pretty good. Best of all, if someone overhauls the entry on the Mobygames site, I can see about pulling in the improved data into the Internet Archive item over again. So it’s still a one-way pipe, but it’s a fast one-way pipe, and the games make much more contextual and informational sense now.

Anyway, here’s the next big deal – I want you to start really getting into the Version 2 interface of the Internet Archive. So, go here:

v2

https://archive.org/details/softwarelibrary_msdos_games/v2

This is a way to go directly to the Version 2 of the Internet Archive’s interface. In the upper right, there’s an exit beta button, so you can always pull the rip cord.

The Archive introduced v2, or “the Beta Interface” late last year. It was slow, stocky, and freaked people out. But folks got the idea, mostly – it was taking a site that had only incremental changes for 13 years, shaking the whole story up, and re-imagining the whole thing as a visual and browsing collection, as well as a way to dig deep into the materials.

Since last year, it’s gotten faster, slimmer, has added a bunch of features, and continues to become better to use. But it needs feedback, which is why I’m pushing you at it.

Enjoy the games, and check out all those beautiful screenshots! Play a few programs, note how you get around to things, and talk about what works and what doesn’t work for you. There’s a feedback button – use it. The goal is that you will be able to do everything you can do with the old interface with the new, but that you’ll have so much more happening on the new one. And remember, v2 works across all of the Archive… all the collections are out there, be they movies, books or audio, and the new interface has cool ways to interact with them as well.

100327_2b

As for the software library itself… many titles will stick around – some will go, or change, or be replaced. This is still brand new territory, and with the addition of the MS-DOS programs so far, this no-plugin in-browser experience is one of a kind in the world. I’m working just as hard as the V2 devs are to make everything work, and it wouldn’t be worth my time if I didn’t have people trying it out, commenting on it, and making use of it.

So please, let’s start 2015 right – in CGA and with a brand new way to use the Internet Archive.

 


Each New Boot a Miracle —

This week, we got the emulation of MS-DOS programs (mostly) working on the Internet Archive.

Up to now, every single emulated system has been via the JSMESS project, which supports well over 600 systems and which does a hell of a bang-up job on it. The fact that one system (MESS) emulates everything from an Atari 2600 to an Apple II to a PDP-1 and so on… that’s pretty amazing stuff. And when we had well over a dozen people intensely involved in porting MESS to Javascript, it made sense to use this one emulator for everything, since once we got the work done, it just worked, for many many systems. It was good, economical effort.

There are, of course, many other emulators out there, some of them little hacks meant for one machine and some which handled multiple systems. Some of them, over the years, simply got absorbed into MAME/MESS, and others died out.

The MESS developers have biases and preferences, like anyone else, so there are systems that have gotten a ton of attention, and systems which have gotten some rough scaffolding, some proof of concept, and then little else. This isn’t to say they’re slacking off, of course – it’s just that there’s a lot of work to do in a lot of directions and they move along at the speed they’re comfortable with. Also, they’re more focused on getting it accurate and hardware compliant than fast and easy. This is, absolutely, the best long-term solution – just hacking things to “sort of work” is the province of forgotten half-functional messes that don’t function with an OS upgrade or an app change.

Also, there are cases where a system gets a burst of love from a developer and it just sings. Recently, the Apple II driver within MESS (and therefore JSMESS) got a massive boost in cleanup, rewrite, and expansion – and the result was a fast-moving (nearly 10 times increase in speed) wonderbomb that can handle graphic modes and settings that previously dropped everything. MESS continues to be the best choice in emulation, long term.

However, projects that focus, intensely, on one platform for emulation will do amazing work in support and accuracy. They’re not 100% flexible and they certainly take shortcuts here and there, but they work really well for what they are.

DOSBox_screenshot

One such project is DOSBOX, the DOS emulation system that has been around for over 10 years, and which does one thing, the x86 architecture and specifically running of DOS-compatible programs, pretty well. It has different goals and approaches than MAME/MESS, but it a very respectable and well-run project, and we’ve all benefited from their work. Except for the fact it only does one platform instead of 600, it would behoove the JSMESS team to attempt a port of it.

Luckily, someone already did that.

Dreamlayers ported DOSBOX via Emscripten into a browser-functional emulator. He did it all by himself, and he did it very well, all things considered. His name for it is em-dosbox.

I’m just going to lay it out and say that Dreamlayers is a software engineering genius, one of those people with a gift for coding and making things work not just better, but understanding what things have to be left tied down and waiting for later improvements. Most of his em-dosbox notes are where Emscripten falls down as a compiling and conversion platform, with indications of how they can be improved. And buried in the code of his is an alien artifact that makes the generated javascript from the process run extremely fast. It’s called extractfun.py and let’s not go too deep into it, other than to say that if we ever figure out how to plug it into JSMESS, fireworks will result.

Now, em-dosbox is not perfect – far from it. It can freeze a browser, and it runs into problems of all sorts when running some of the oddball machinery that the DOS world produced between 1980 and (roughly) 2003.

But when it works, man, it just works. Clear as a bell, right as rain, a DOS window boots into a browser. If you go fullscreen, the mouse even works. (It works in not-fullscreen, but then you see both the desktop and the emulator cursors and your brain is sad.)

It’ll be a little while before the collection is really public, as there’s now the process of throwing thousands of DOS programs into the system, verifying they boot, making screenshots, and all the rest of that delightful curation the world hopes is waiting for them.

But it’s happening. It’s really happening.

PC_DOS_1.10_screenshot

And now more fundamentally, the personal take, and why this three year journey has been so worth it.

MS-DOS was always my main thing; I coveted the Apple II world, really got into the Amiga when it came out, danced among Ataris as I could, but for the vast majority of my “work” or whatever a teenager’s activities count as, I did it on MS-DOS.

MS-DOS was what my primary computing machine was from 1983 to 1988 – I hacked on the exact same machine, with minor improvements, all that time. I grew used to the clack of the keyboard, the arrangement of the fonts, and I never really used a windowing system on it for any time in high school. I called BBSes and online services with the internal 2400 baud modem – all the files I’d downloaded for my textfile collection, and which ended up on TEXTFILES.COM, passed through that machine.

Work under any system for long enough and you buy into its processes, whether you want to or not. And I bought in big. The grey on black text, the directory structure, the FILENAME.EXT system, the all-around aesthetic. I feel DOS systems glorify text in a way that other computers didn’t, due to IBM’s long, long history at providing the cleanest, crispest fonts for typewriters and correspondence. I was an IBM kid, after all, and my father’s acquisitions of company-made computers had me fully involved in it for years and years.

So, MS-DOS is my thing, and getting this working was my thing.

A lot of people poured themselves into the original DOSBOX and EM-DOSBOX. But two people, Daniel Brooks (freelance) and Hank Bromley (Internet Archive Dev) worked very hard for a significant amount of time to jam the EM-DOSBOX system into the Internet Archive environment, which is much more difficult than one might expect. Imagine playing a chess game by keeping the board in your head and dropping coins with move descriptions written on them down a grate and you start to get it. A lot can jam up in the pipes, and the whole thing wasn’t designed for what I’ve been making it do. They worked for days on end, bouncing iterative changes, seeing the results, and at the end, it worked.

The functionality of the MS-DOS/EM-DOSBOX emulator within the Internet Archive environment has more caveats than a gym membership, to be sure – dropping to the prompt currently freezes it. You have to tell it what exact file is the “boot” file, the one that will be executed on startup. Too big a file, and it blows out your stack. It doesn’t write out to the emscripten filesystem, causing some to blow up because they can’t write to “disk”. And the sound… well, sound continues to be an issue until the nerds-that-be fix them in the browser world.

But…

amnesia

When it works, it’s glorious. The speed is really clippy, the prompts blink like I remember, and as it plays CGA, EGA and VGA graphics out and hits the notes, I can sit back and enjoy that clean, crisp little world I spent so much of my childhood in.

And more than that…

The Arcade, while fun, is not a technologically advanced project, and the systems are beyond simple – they work or they don’t. They boot up, they play music, they take your quarters, they play a game. I knew people would love them, and they certainly provide entertainment, and they’re historically important and culturally relevant. (And the effort to accurately reflect them in the emulator is a Herculean effort, I rush to say.)

But they’re also very simple affairs, and designed to impart 3 to 10 minute experiences. You get Pac-Man in a few minutes – you understand nearly all to know about Defender in a day or two. Granted, the skills to keep past that 3-10 minute period are legion and deep, but that’s not what most people would experience anyway.

But with a computer program – especially one designed to be sold for a good amount of money, or which represents a lot of work by someone, there’s a depth in there, and an intensity of the product, that can’t be ignored. A strategy game that doesn’t care how long you take to enter the next move, or a simulator that carefully gives you dozens of knobs to twiddle or points to tweak… that’s the harbinger of the computer experience. Not to mention the applications, those tools of productivity that have permeated computers since the beginning. It’s rich and lovely, and it’s my favorite thing.

Getting the Apple II and ZX Spectrum and Atari 800 collections running at the Archive have been a pleasure, and the resulting screenshots are beauty to behold. But MS-DOS… that’s my neighborhood.

The neighborhood looks good.

ezana

The system uses a .ZIP file that contains all the files that will be used inside the emulator as the “drive”. So if you have 5 files, including some batch file and an executable, accompanied by some data files, those will all be loaded in at the emulator’s boot. It will then require a designated “start” file, one of the collection, that it will attempt to run and execute against.

Sometimes this doesn’t work. There’s a lot going on there and my sources are variant.

But sometimes, it just works.

ciscoheat

And when it works, I see smooth as glass graphics, beautiful pixel art, and those beautiful words, and the whole thing is alive again, in the emulator’s box, infinitely reproducible, instantly referenced, ultimately preserved in a pretty useful fashion.

ather

border

gotowar

Folks who are the sort of folks who professionally quibble will professionally quibble, but the now-years of work I’ve been involved in this (and others have been there right with me, by the way) are utterly justified every time I run the ingestor and push a .zip file of a gathered-away program online. Within seconds, it’s a single click away from beeping, booping and playing music, or standing stoically waiting for my input, unaware of the science-fiction universe it now lives in, destined to never really die, and to stand and be counted by history.

It’s like watching a miracle with every boot.

It’s a great feeling. It’s the best I get.


The Tourist, The Researcher, The Maniac —

This year was primarily about software, specifically collecting and emulating it.

That went well.

Asked what my goals or projects are for work for the next year (2015), I have resolved that it will be cleaning up and making it easier to go through the mountain of material in the Internet Archive’s stacks.

Naturally, I’ll be continuing to help with Archive Team, improving JSMESS, and acquiring tons of crap to throw into the hard drives as well, but I really want to take advantage of the tools and time to really dig deep into the Internet Archive’s wonder as a collection. It is a huge, huge set of materials and I’m positive people would have minds blown if they really came into contact with the true size and breadth of the material.

So that’s how it’s going to go in 2015.

But, as usual, I can’t wait and I’m starting now.

archive-1200

First, understanding the mass of material already in the Internet Archive that has come flooding in in the past few years is a task in itself. We don’t have a few monks chanting – we have 4,700 chants by a group of monks in France. Or maybe you want your sermons in english. Well, that’s fine, here’s a set of 21,000 of them. Or religion bums you out – that’s OK, take your mind outside and look at 8,000 books from the library of the New York Botanical Garden. Or go back in time to see a hundred episodes of Starcade, the game show for video gaming.

I can do this all day. There are thousands of groupings of thousands of items down there. It goes in many directions, and it goes deep as your tolerance can go. Books? The Internet Archive’s contributors added 1,000 books yesterday, and 1,000 the day before that as well. (It’s been a good couple of months.) Audio, don’t get me started. Newspapers, tens of thousands of issues of newspapers!

Stuff. We have it.

IMG_5922

Before I talk further, let me also say I’m not the only one thinking about this, especially within the organization. There is a very intense effort underway to completely redo the Internet Archive’s interface. If you visit http://archive.org/v2 you will go into the Beta Interface, which you can see is coming along swimmingly. (There’s a button in the upper right to turn it off and return to the current interface.) A team of people spent a year dealing with both the backend and the frontend aspects of this new design. It has a way to go, but it actually brings this whole issue to the forefront, since it’s going to become a lot easier to browse the collection.

But for my little bit, I want to find ways to get a better hold on the millions of items, move aside spam and broken items, and group up really cool and similar things without me spending every waking hour doing it.

But before I even do that, it’s time to talk about who I think the audience for the Internet Archive’s media riches are in the first place.

Hence, the Tourist, the Researcher, the Maniac.

10658828_10152506233184527_5746589902833348620_o

The Tourist washes up on land, pushed in by the general tools of contemporary discovery – popular links and search result. The Tourist can be very compelling, because whatever dragged a Tourist to your site probably dragged a lot of other Tourists. They can almost seem like a coherent movement, or a congregation. But they’re not, any more than one or two methods out of a building results in an actual orderly line. They’re a mob, a gawking fog of incoherent attendance, a group who has come in because there was a blinking sign somewhere, and having found something similar to what they want, have smiled and gone away, or they’ve stuck around just enough to pass judgement on somewhere they’ve never been and will likely never return again.

You can tell a Tourist because they go back to comment at the place they heard about the link, and the commentary they make tends to be along the lines of “I have a lot to say about the way about how nothing impresses me.”

Tourists can definitely become recurring users, of course. But their introduction to the place will almost always be “here was a thing you might like”, and they run in and out, because these are busy people, and there’s a lot of things to like.

All is fine and good with Tourists, unless your site or the site attracting Tourists thinks Tourists are the best people to acquire, what with their bloating numbers and all. They’re not – they’re basically sugar. Quick high, bad aftereffects, no long-term benefit.

Img_1223

The Researcher has come into your life and your site because they’re Looking For Something. Hopefully you have it.

Their requests are usually somewhat strange, but they’re also really flexible about what they find. They browse, they look up similar items, and they try to do some of your sorting job for you. In a good day, they send some of the results of their work over – “I have a list of items that need to go into a different collection” is a common one Internet Archive gets in the inbox.

Researchers often want something that is not quite what search engines can provide. A query can be something like “I want all the magazines that reviewed the Apple III”, or “I want to look at movies that feature skateboards or skateboard parks.” You can feel like everything is an incomplete patchwork of metadata and fairy dust after a Researcher walks around. But they’re not trying to be mean or disruptive – they just really appreciate the place and want to get the most out of it.

Researchers hunt in solitary or in small packs.

IMG_3916

The Maniac sounds like who you’re doing this all for.

You are not.

The Maniac comes in, and never leaves.

The Maniac has a dream inside their head of how the world should be and you are failing at it, right now, even while you’re not doing anything.

The Maniac has lots of ideas for the site, how things should be done, but they’re really a bundle of unfunded mandates and issues that affect a very, very small number of people. As a bonus, they become anti-ambassadors, posting frequently and viciously in other realms about how your place wouldn’t suck if you only decided to use the sdfjkhsfsdjfh standard or you would just allow unicode in usernames. I’d say The Maniac hasn’t done anything, but they probably have done something and by Ramthar’s Crowbar you will hear about it, a lot, as explanation of why they get to come in and take a dump in the potbelly stove.

DPI matters a lot to the Maniac, as does impeccable presentation and completeness. These, in themselves, are excellent things to have, if you can get them. But Maniacs are very, very quick to flip out if any of an internal checklist isn’t met. Maniacs flip out a lot. It’s what they do.

Maniacs are almost always solitary, although they can give the impression they travel in packs – but to be honest, every Maniac thinks all the rest of the Maniacs are tourists.

IMG_3398-Scaled

So, there’s a fourth group, called The Future – but going too insanely over the top to appease people that don’t exist with goals and ideals you can’t possibly anticipate is not today’s exercise. So you keep out there in the hazy realms of Tomorrow, you crazy Futures.

I think the sweet spot are the Researchers. You want to push Tourists to consider becoming Researchers if the mood takes them, and you want Maniacs to either tamp down into becoming Researchers, or go spill their hot oil of hate out from another castle.

Make the site make sense, start culling out items that are spam into places that won’t discourage browsing, and help build collections that sing. That’s 2015 for me – researchers welcome, Tourists a delight but sorry about the sdfjkhsfsdjfh being sub-par – I’m sure we’ll get to it soon.

Now, let’s get some sorting done.

 

 


Pain —

10660609_10152695388558834_1918406404_n

Somewhere around the age of 26, I woke up and couldn’t really get out of bed.

The reason, in the immediate, was that any attempt to put weight on my legs sent shooting pains through them. Real, intense, shooting pain like there were rods lodged inside the legs and they sent out shocks when they detected any movement or effort. I have forgotten what I thought at the time, but I do remember how much it hurt.

The mattress was on the floor, so I was able to roll and crawl to a phone and call for help. After getting clothes on (somehow) and getting down three flights of stairs (somehow), I waited outside and was taken to the hospital. During this hour or two before I was picked up, I learned which positions hurt the most, which the least, and tricks for getting around, somewhat. I also stopped being shocked at how much it hurt and just winced or yelled an awful lot as I discovered a new angle, a new approach that would make it all hurt anew.

At the hospital, after some battery of tests, I was given a bunch of over the counter pain relief, some crutches, and no real idea of what was wrong. I do remember one young doctor saying “You know, this really acts like gout. But you wouldn’t have gout – you’re 26.”

I hobbled on crutches for a couple days, the pain went away, I lived on.

A couple years later I was arguing with my girlfriend while she was driving her car. I remember this because in the middle of yelling, I felt something very unpleasant and I told her to stop arguing because something felt very unpleasant.

The thing about pain is that you both remember and don’t remember it. I can, these many years later, describe it as a tightness, a strain deep inside my gut, like shoelaces of nerves had been pulled very intensely, and were now knotted in a bow, and my movement in most any direction was pulling at them. That’s what I remember made me go from thinking I’d had a bruise or a swallowed knick-knack in there – its that there was no logical sense to my ratio of pain to the motion. Moving and trying to get into a good painless angle did nothing, and then it would stop, and then it’d come back harder, and I was rapidly losing brain function. I asked and then really begged my girlfriend to take us back to her house.

On her couch, it just got worse, we didn’t know if I had a burst appendix, a knife I’d somehow fallen on, or if something inside me was just going necrotic and taking out my organs. But boy, did it hurt. It hurt until I ran out of ways to think it was hurt. I wanted desperately to take something to knock myself out but then I simply assumed that I’d die in my sleep and that seemed needlessly unpleasant and surreal. So I lay there, looking up at rafters, overcome with pain, waiting for either healing or death. Some measure of healing happened, and ultimately I fell asleep, as if I’d run a marathon.

Let’s move ahead to the diagnosis, that came in my mid-30s. Kidney stones.

There’s two kinds of kidney stones – the calcium ones and the acid ones. I have the acid ones, which are notably rarer and as a bonus, the treatments, which I had many, many, many meddling Dr. Google-Readers give me over the years, are entirely opposite. Drinking cranberry juice doesn’t work, for example – a lot of people told me to drink cranberry juice. No.

Due to a set of circumstances where I didn’t seek out the best doctor but just fell into doctors, I didn’t really have a doctor look at me, properly, with competence and awareness, until somewhat recently. This doctor has done wonders. So there’s your happy ending.

Now, let’s go back.

By my early 30s, I was passing 100 kidney stones a year. That is not an exotic allegorical statement –  I was passing some sort of stone every few days. For months, and then years. On the whole, over time, I didn’t even notice them. In fact, I almost got worried when one didn’t show up in a given week. It was just another fact of life, like waking up feeling tired or occasionally having a joint feel sore. Stuff happened, and some of it involved the fact that I ceaselessly created stones and passed them frequently, with small pinpricks of pain involved in the process.

By the time this was all happening, pain was rapidly not becoming a problem for me. The kind of pain one might have from hitting your foot or shin against something, or from grabbing something sharp, I could notice, but it was more like hearing a distant yell, like someone cupping hands and shouting that you should turn back, the woods are not safe. It wasn’t immediate, and it certainly wan’t concerning. By this time, most pain was being given a level of interest far beyond what it probably should have been.

The reason for this is because of the times when it got my attention.

Imagine, if you can, a very large meatball inside a bowl that is only a little larger than said meatball. Imagine that meatball is kind of sloshing back and forth, kind of randomly, in the bowl. That’s how it felt, and how it started.

I’d feel some overarching dullness inside myself, and I started to get really good at recognizing it. The feeling, you see, was a largish stone, and one that had gotten in some way stuck. I got really, really good at noticing this, because I knew what was coming next.

If I was lucky, I could run in some direction and get my hands on water. Lots and lots of water, like quarts of it. I’m really fast at drinking water, and I’d stop mid-conversation and drink everyone’s water at the table, or run towards a store and buy a gallon of water and start drinking it at the cash register. Anything, you see, to get as much water flushing through my system as absolutely possible.

If I was lucky, and a few times I was lucky, the meatball would evaporate. It’d kind of roll into a cloud version of itself, and waves of discomfort would subside, like a pool minutes after someone jumps in it.

I wasn’t often that lucky.

I am a weaver of metaphors. I am a person who tries to find the middle ground between the listener and the moon, and concoct a pointing finger that will lead you to the moon. But where do you even begin with describing pain like a pain of a stone that’s lodged in your system?

First, it’s often in your gut, deep inside, so it doesn’t feel like the sharp sting of a skin burn or a radiating line of a strained muscle. It feels like something deep inside you is twisting apart, and has no recourse to stop, and probably will never stop. After a few minutes, you forget when the pain started and you definitely stop assuming it will ever end. Oh, you definitely hope it will end, but it never seems to.

Sometimes I was just two eyes staring at something in the room and pain. One time I was a flattened person on the floor of an airport, screaming, everyone looking at me, while a stranger held my hand. Another moment, I was in the softest bed ever, in a visit to my father’s home, with my siblings watching me appear to be dying for no good reason. They scrambled through medicine cabinets looking for something, anything, but ultimately all they could do was sit elsewhere in the house until the screaming stopped and I fell asleep.

This happened a lot. I’m not telling anyone who has lived with or spent a lot of time with me something new – they probably have at least one memory of me either having to leave very quickly to go be possessed elsewhere, or they’ve sat nearby, wondering what you do with a body when that happens, when your friend is gone. They know. Things people shouldn’t have to consider with guests, my occasional attacks gave them time to consider.

I only got operated on once, by a doctor I’ve called Doctor Coldfront since the time he operated on me. He had the bedside manner of a mailman to a mailbox. I’m sure he was competent, but ironically, he made me hurt the most, although with that pain came some interesting revelation.

He inserted a stent into my left Ureter, which is the tube from a kidney to the bladder, while I was under. I didn’t mind that – I was asleep.

Some time later, and I’ll spare you too many details, he removed the stent under local anaesthesia, as an outpatient procedure in his office. I had requested a full knockout but he wasn’t interested in doing that – I assume out of 10% concern about putting someone under and 90% of the inconvenience for what he perceived as a minor operation.

I had nightmares about that day for a decade.

But as I said, some revelation came from that – which was that there are doctors and professionals all throughout the world, some even here, who use some level of medical procedure to torture out information. Some of them are paid by governments, some by private organizations, to do so. Back in the hazy glance of youth’s cobbled knowledge and self-assuredness, I thought that a reasonably strong-willed person could resist giving whatever information was being extracted, under torture.

Considering how I was treated by a doctor who was merely inattentive to my wishes, I gained the knowledge and the understanding that no – information is probably not worth keeping from someone who is intentionally, unendingly, aggressively intent on you staying alert and bathed in pain for as long as is necessary for you to give up whatever pearl of knowledge they wish to extract.

That’s an interesting lesson to have gotten.

I never spoke to that doctor again. I hope someone broke his heart.

So, why go into all this? Well, two reasons.

First, my weblog has been suffering a little from a case of the Nerd, and has way too much technical information without much self-revelation and consideration. So there’s a little self-revelation.

Second, I wanted to make clear to a certain subset of fans and followers as to what my secret is, since I’ve had people literally say that they are trying to “Jason Scott” something, i.e. throw a lot of energy at it, or never stop.

My secret is that I am all too cognizant of how quickly life changes, and how one can go from a moment of relaxing in the sun to feeling pain so great, so intense and miserable, that you will cry tears of anger at your previous self for wasting precious minutes and hours in the paradise of existence you had been granted. It is why I don’t like lying down for too long unless I am very tired, and why I often (but not always) get up with a burst of intensity, worried about what I missed and what needs to be done.

It’s also why I tend not to wear coats, and underdress for the weather. It’s why I can hold my hand over a candle for a stupid amount of time. And it’s why I hand people things and am confused at the look on their faces, only to realize it’s because I handed them something blistering hot or bitterly cold.

It’s because pain’s kind of an abstraction now. When I go to the hospital these days, I have to exaggerate what pain I’m feeling, because I realized after a few times that they were asking if it hurt, and my definition of hurt and pain is not theirs anymore. I can see clearly, and I can think about things, and I call that not being hurt, even with a hole in my arm or a knife cut that needs stitches.

I’d call it a superpower, if I didn’t make an effort to remember how I got it, and how truly useless it is, and how dangerous. But I have it.

In the meantime, here I am, working very hard, doing a lot of things, and counting the missed opportunities and the upcoming chances to do even more things. Because of the pills I got from the right doctor, three years ago, I have not had a single kidney stone, or a single day in true screaming pain that defined my life for so long.

And if that’s what I have now, a day without that pain, then that is a day it would be a crime to waste, to not pull the most out of, to not work on things until I barely can stand up and then fall asleep until the next round of things to do.

I do not recommend the journey to this situation to anyone.

Have a great day.

 


It’s Time To Keep Solving the File Format Problem —

Here’s how a planned large-scale set of roving ideas and goals ends up being a case of just doing one thing right.

Theproblem

The original plan when I cooked up Just Solve The Problem Month was that there was a set of problems out there that just needed a few hundred people to contribute time and effort, and some otherwise seemingly insurmountable problems could be solved or really, really beaten down into a usable form.

Aaaaand what instead happened was:

  • We announced and set up a Just Solve The Problem Wiki for the first problem.
  • A lot of people worked on the Wiki.
  • I got very busy.
  • People kept working on the Wiki.
  • It’s been two years.

So, basically, we have one single output from this idea.

And that idea is the File Formats Wiki.

The File Formats Wiki has been progressively, constantly updated to for two years and has now entered its third year of existence. If you check the statistics page, you can see there’s been thousands of pages added.

I had previously said that November could be Just Solve the Problem Month. Well, the machine that this Wiki is on had to get some maintenance, so I moved it to December. And now, it’s just File Formats Month, I think…. that’s a good enough project itself.

That is, now.

So let’s go over it again, shall we?


 

The world has a problem. Well, the world has many problems, but this one is a big one.

File Formats.

Oh, we have been so good at file formats. We’ve happily come up with file formats whenever and however we want to, whenever information has been captured up into a bundle. We’re so good at it, we often don’t even look at what formats are out there – we just make a new one. And then we later change the file format we created to accommodate some problem we didn’t account for, and so the file format becomes “file format v2.0″ or “file format NG” or even worse “entirely new name for file format”.

And then we forget about them.

Really new file formats, that is, ones created in the last five to ten years, are pretty lucky, relatively – there are webpages and standards posted into all sorts of locations, and utilities available besides, to deal with them. And really, as time has gone on, one person or another will document a file format to the best of their ability, stick it on a webpage, and there it will set for some time before being deleted or forgotten.

All of this is a problem. This is a problem that can be solved.

We’ve set up a Wiki, called the File Formats Wiki.

Everything on it is Creative Commons 0 (Public Domain-ish) because it turns out you can’t just say something is “Public Domain” and have it stick worldwide. But the contract is basically that – everything on the FF Wiki can be used anywhere else – the knowledge can be spread to other similar projects (and there are some) and every boat can be lifted.

Every file format or container for information is eligible. We even have Piano Rolls, Tree Rings, DNA, and Spoken Languages. Obviously, our big focus is on computer file formats, though.

We’re looking for descriptions, links, clarifications, and tracking down of documents. We want to make this stuff get assembled in one place as best as one can.

We want it that when you find something is in ‘a format’, that this Wiki could be your first stop on the research, and possibly your last one as well.

You register, you can edit it, and you can get started basically immediately.

I’ll be working on it all month. I hope you will too.

Let’s solve this problem!


Gamepads! —

A week into the explosion of the Internet Arcade, with what has now been millions of visitors to the Archive as a result, and we’re getting pretty far down the “what still confounds people” realm. As someone who worked in tech support as his first “real” job at Psygnosis, (shout out to Chris Caprio and Jim Drewry for the hire), I know both the frustration of not having things work right when you expect them to, and the propensity on the technical support side to go ‘oh come on, why don’t you have this already”, like the mass of humans out there is just one person calling you back over and over.

In the last week, the JSMESS team made the software load in Internet Explorer and Safari, fixed aspect ratios on full-screening of the video, made keys work in fullscreen mode on all browsers, and shored up a bunch of little details not worth going into here.

This leaves the #1 thing we’re still getting mail about: Gamepads.

SO YOU HOOKED UP YOUR GAMEPAD AND IT WAS A LITTLE WONKY OR DOESN’T WORK RIGHT

  • Please bear in mind that JSMESS has basically nothing to do with Gamepad Support. We used Emscripten to compile it in. Whatever Emscripten does and whatever the generic Gamepad API loader does, we do – we do no development on it. So, first….
  • In all circumstances, definitely visit the Gamepad API Tester. This is a nice little page that will show what the current version of the Gamepad API thinks your controller is. If you can’t see it, or it acts weird, it’s the API.
  • Some gamepads actually press keys instead of doing “gamepad signals”. For example, we used the X-Gaming Tankstick during testing and it worked out of the box, since it’s basically another USB keyboard, and it was mapped to MAME controls. (Disclaimer – after X-Gaming found out I was using their stick at the Archive/Arcade, they sent a couple more a day or two ago.
  • There are software packages out there for various platforms that will convert joypad/gamepad movements to keyboards. For example, there’s Glovepie (Windows), Xpadder (Windows), Joystickmapper (OSX), and many, many more. This is a known thing of interest and people have made a lot of programs that do the same thing.

Ultimately, the message that we have at the bottom that says you should hook up your gamepad and then will go “Gamepad detected, you may need to refresh…” – well, this is because the way the API works, it’s not exactly guaranteed the signals from the gamepad will “Catch”. I tried to put in a message to go “keep messing with it if you don’t get it”.

Some people also got confused that this meant you could ONLY play these games with a gamepad. That’s not true – it’s just better in a lot of cases, once you have it working. The keyboard controls of MAME were always a weird compromise, and you can really feel it with some of the classics.

Remember, there’s a TAB command in all JSMAME emulations, letting you see the key mappings, and testing them with your joypad. It’s not perfect, but it at least informs.

The Joypad API specification is very new. It was barely up and running when we got it going in September of 2013. A year later, it’s there, and it works, except when it really doesn’t. That’s the nature of all of this – cobbled together pretty-good things presenting you a programming miracle. Like the fullscreen and the browser compatibility, I expect it to get incrementally, notably better.

Good luck.


Before It All Arrives —

The numbers are a little hard to calculate, because the Internet Archive, heroes to the core, do not keep logs for pretty much any time at all. But using a few methods, comparing some general graphs, and doing math, the admins handed me an estimate of how many people have played video games at the Internet Archive in the last 3 days.

Two million.

Two million.

While I’ve had bursts of somewhat public facing events and wonders over time, none of them have ever have crashed into the lives of millions in just a couple days. And certainly not in the continually ongoing way I am finding people are returning, over and over, to look at the games. Additionally, it hasn’t taken people long to find other software collections, and other places where the emulator has been utilized for consoles and general computers. In other words, they are cascading through all of the services that the archive provides, and poking at them all with brand-new eyes. Many thousands will have never heard of the Internet archive before Sunday.

I am hesitant to use a term like “turning point” in regards to the archive in general, since you could go on any subway platform and somebody who was waiting for the train would probably know that there was something called the Wayback machine. It’s been around for 15 years, and even if people don’t know all the details, they get with the Archive is about.

Not so with emulation, or the state of software preservation. The last three days have been, unquestionably, a turning point in emulation.

Emulation made the local TV news. The ability to play old software in your modern browser and on your current machine was a decently packaged story that found its way into magazines, newspapers, blogs, podcasts, and an unknown amount of water cooler conversations.

People who know me let me know about overhearing conversations about the Internet arcade, and therefore Emulation, in classrooms, workplaces, and in the street.

For the contingency of people who were hoping that one day society would understand what Emulation was, and would be able to articulate, even in the lightest fashion, its underlying theories, well, that day has arrived. That day is here.

Certainly, there were and always will be factions within the regard of historical software. Some people believe that actions like this are a dreadful mistake, and could only lead to horror and sadness. Others are flying high today, getting exactly what they had always hoped for.

A rough travel schedule last week has left me rather sick, and I am still recovering. Regardless, I can’t let these moments slip away. My work in historical circles only highlights for me how fleeting moments of happiness can be. We were watching as many as 1000 people a minute running into the Internet archive, overrunning resources within, having an incredible time. The sky is the limit for them. For this time, a bounty lay before them, utilizing the strength of their own computers to provide them a nostalgia roller coaster, a recovered treasure trove of memories, a new insight into a world nearly 30 years dead.

It can’t last. Nothing lasts.

What’s coming forward are questions, and ideas, and arguments, and discussions. They’re on the way, I can feel them formulating now. Letters and think pieces and essays, and debates.

I’m sure I’ll play a part in them at that future time, but I just want to go a little bit into the past.

A year and a half ago, I participated in what was probably the most white-hot conflagration of software preservation experts in the country. It was called preserving.exe. Here was the agenda and information on it:

PRESERVING.EXE (May 2013)

I gave a talk there. It was called “I did it 35 minutes ago” and in it I laid out the importance of software preservation, and the efforts of the Internet archive to move forward in that space. I am more than happy to admit that the speech was equal parts passion and plans.

I predicted the inevitability of what’s happening now, this week. I asked peers and contemporaries what they were going to do about it, what moves needed to be done, and if we are going to sit by and let software be buried, forgotten, or if we would work to give it the due it deserves and the access it needs to have.

I asked, fundamentally, what software is, in a historical context. Is it an eternal product, service, and item that is always to be sold, treated as such, perpetually unreachable, and distributable until the very media it is on rots beneath it? Will software have to be something described, as a lore, instead of presented for people to experience themselves and understand?

In other words, would software’s very specific nature of easy duplication and transfer ironically doom it to disappearance? Would the software preservation experts refuse to make their hard work available, out of fear, concern, lack of guideposts? Would the newness of the medium paradoxically lead it to be lagging decades behind in advances made with almost every other medium? Is that the world we want to leave behind?

I pounded my little podium, and I sat down. And when I went home. I went back to work.

Like blasting a firehose into a kitchen, the pure sheer numbers of people now running through the Internet archive are a fascinating spectrum of regard. Some people are disappointed when they run into rough patches, others are delighted that the whole thing comes together at all. Shouts of disbelief clash against demands for decade-old playing tips. The project has been praised and cursed, lauded, and questioned.

Through it all, one thing is obvious: software is more than code. Programs are more than executables. This medium we have formed forms us, and returning to it brings back joy and the agony, no different than a book, a poem, or a face.

Soon, a stunned populace, an alerted establishment, is going to come forward and have a lot of questions and want to debate.

But before they arrive, before the buzzing in the distance becomes a roar in the valley and a scream in our yard, I have one last question, to colleagues and bystanders:

Where will you be?

It is going to be most interesting for me to find out.


The Internet Arcade: And Where Are You? —

Yes, that’s right, three Internet Arcade posts in a row.

I usually like to mix things up, but the world is very quickly going nuts over the introduction of the Arcade. Hundreds of thousands of people have played the games in it, it’s been featured in dozens of sites, and it’s even hit TV news in a few markets.

B1hYwH-CQAAb5_f

It also (temporarily) crushed the Internet Archive servers, mostly to my putting up a relatively inefficient page and causing a few components within the system to work too hard. After an excellent and inspiring set of actions with the administration team at the Internet Archive, the hits haven’t stopped by a long shot.

render

So, this is great – hundreds of thousands of people are playing the games, the word is out that Emulation is Just Another Thing, and joy has come throughout the land.

However.

Putting up 900 arcade games, 2,300 console games, and 25,000 software titles means I didn’t get a chance to, individually, check every single one. And while I and the many people helping with JSMESS did our best to make stuff work right and fail gracefully, there’s a lot of browsers out there, and a lot of system configurations, and a lot can go wrong when you’re running, say, an entire arcade game inside a window of a browser.

This whole thing’s a proof of concept – a very public, very popular one now.

I am now making a call for you – yes, you – to help with the next round of improvements.

I need:

  • People willing to enter descriptions of games. Just the describing of games turns out to be a long-term thing – with the hundreds of Arcade machines, there’s entries that can be grabbed from (and should be cited), but I am actually kind of shocked that there are plenty of games that are not in any way online and described – there are games that MAME (and therefore JSMESS) emulate for which there appears to be really no canonical paragraphs about. We need those badly.
  • People willing to help smooth out the code. I mean, sure, I’d love if people helped reprogramming MAME/MESS itself so the improved emulation would be transferred over to JSMESS – but in this case, I just mean things like loader code (javascript) which could be more resilient, as well as better error messages for when things go wrong. Do you think you could help with that?
  • People willing to help describe idiosyncratic aspects of these games. Some of these games do not go gently into that good browser – games like Defender and Street Fighter have crazy amounts of keys and could really stand someone to at least document them where possible.

I’m sure we need even more people, but will you be one of the people who comes forward to help with this, to make it better? We’re doing fine on the people playing the games department – that thing’s sewed up. But now we’re building the largest virtual and playable collection of software. Can I get your support?

If you’re interested, either come to #JSMESS on EFNet (IRC), write me at arcade@textfiles.com, or hit me up on twitter, or … well, just scream out a window.


So, You Found the Internet Arcade —

Woo hoo.

Soon after my initial announcement on this weblog, a few tweets of the posting, followed by some press attention, means tens of thousands of people have now visited the Internet Arcade. Many are happy. A goodly amount are stunned. A few are annoyed and wondering why it “doesn’t work”.

This quick post from a plane is for the last group, who are having issues. Hopefully this can help clear things up for you.

Browser Choice

First, the Internet Arcade works best in Firefox, and a very recent flavor of Firefox at that. If you needed an excuse to upgrade, this is it. Chrome also works well, although due to disagreements between Mozilla and Google that bore me, Chrome runs about 20% slower on average. This can mean the difference between good and clunky emulation. Safari, mostly recent Safari, works good but sound does not currently work on it. Same for Internet Explorer. Believe it or not, I’ve had great response from a 64-bit Windows version of Firefox called Waterfox. As they say, your mileage may vary. We continue to improve this experience and we’re constantly doing fixups to the code to work with other browsers.

People occasionally complain about the irony of how you need the latest and greatest to run such old stuff. Well, you’re really running an emulation system that itself is running an old system, and that emulation system is in a browser. Let’s call it even on the “why is it so intense” issue. Work continues to make it run as best as it can.

Controllers/Gamepads

This is entirely un-intuitive. I’m sorry about that – we’re going to make it clearer soon. Basically, you need to start a game, i.e. after the MAME logo goes, and you see the game booting, and then you hit keys on your joypad until it says “joypad detected”, and then you need to refresh the browser. The Gamepad API we use is somewhat janky, and it takes a while for the browser to “get it”. Also, only Chrome and Firefox currently support the Gamepad API meaningfully. So again… start game, wait for booting, twirl buttons until it notices, then it will generally work for all the games you click on after that.

Another issue, if you live with the keyboard controller, is that MAME wants to use CTRL, ALT/OPTION, and a few other wacky keys. This comes into conflict with MacOS especially, as hot keys and accessibility come from combinations. There are two solutions, both are annoying. One is to Press TAB after the game starts, and you can go into a menu and change the keys… problem is, you lose these settings currently after you refresh the browser. Another solution is to go into your OS, make a guest account, and make that account not have hotkeys – that worked fine in OSX and in Windows for us. As this expands out into other setups, I’m sure other solutions will present themselves.

Sound

Sound, like Gamepads, is a little strange. It starts out muted by default. This is because the way the Webaudio API works is really poor, and you can thank Mozilla AND Google for that one. They claim they’re going to make it better. But here, you have to start the game, wait for the booting of the game, and then hit the UNMUTE button. Then you (you guessed it) refresh the browser to make sound work.

The good news is, once you set this cookie, it’ll have sound for everything. The other slightly annoying news is that that Webaudio API thing means that sound can be really fuzzy and crackly on a browser, and doing anything with a lower-powered machine (including, with some of them, even moving the mouse or swapping tabs) will get that fuzziness. I get it too, especially when you’re running a post-1983 machine in this emulator. As I’ve said to people testing it the past few months, when it works, it works great. When it doesn’t, hoo boy.

Credit Where Credit is Due

As this thing busted wise, it gets way too easy for people to report that “Jason Scott” did all this. Jason Scott by no means did even a notable percentage of this. JSMESS is a port of the MAME/MESS emulator. JSMESS, (github repo is here) was the work of a huge amount of people, listed there and on the JSMESS main page. And MAME/MESS has its own massive, massive collection of people who are working on the actual emulation, coding, driver refinement, and related coordination to making the most comprehensive emulator system on the planet. Every single time that this project has broken wide, some places don’t do a good job of reporting on the amazing nearly-20-years project of MAME/MESS. So that we’re clear.

I’m sure there’s other questions, or considerations. That’s some of the stuff that comes up.

Oh, and if you get a blank screen in the window…. did you turn off Javascript? Javascript MESS likes Javascript for some reason.

 


The Internet Arcade —

When work began on JSMESS a couple years ago, I knew that it was probably somewhat easy to do all this conversion work for MAME (the arcade side) as it was for MESS (the computer and console side). I specifically chose not to, because I was not interested in a pile of work just to make another game platform. This was about software history, and it’s not that hard to get MAME up and running for the game or games you want to play.

Over the next few years, we got JSMESS working, and working pretty darn well – in a lot of cases, you can boot into a wide range of computers in your browser and it’s working great. There’s hiccups here and there, and we have work left to clean it up, but on the whole.. the proof exists. You can browse the historical software collection and the software library and wow, thousands of programs instantly there!

So, earlier this year, I decided to futz around with our build environment (which, it must be absolutely stressed, the other JSMESS team members built, not me), just to ask the question, “And how hard would it be to build arcade games, anyway?“.

It turned out to be easy. Very, very easy.

Months of testing, refinement, improvements and efforts, and this week I have announced the result: The Internet Arcade.

desktop-arcade-1014-de

 

Of the roughly 900 arcade games (yes, nine hundred arcade games) up there, some are in pretty weird shape – vector games are an issue, scaling is broken for some, and some have control mechanisms that are just not going to translate to a keyboard or even a joypad.

But damn if so many are good enough. More than good enough. In the right browser, on a speedy machine, it almost feels perfect. The usual debates about the “realness” of emulation come into play, but it works.

This is the week it’s been dropped. Not a huge announcement (unless you count this weblog entry), no parties or fanfare beyond, yes. There it is:

http://archive.org/details/internetarcade

So then begins the question that I ask myself more and more in this endeavor: Now What?

Obviously, a lot of people are going to migrate to games they recognize and ones that they may not have played in years. They’ll do a few rounds, probably get their asses kicked, smile, and go back to their news sites.

A few more, I hope, will go towards games they’ve never heard of, with rules they have to suss out, and maybe more people will play some of these arcades in the coming months than the games ever saw in their “real” lifetimes.

And my hope is that a handful, a probably tiny percentage, will begin plotting out ways to use this stuff in research, in writing, and remixing these old games into understanding their contexts. Time will tell.

Until then, game is on.