ASCII by Jason Scott

Jason Scott's Weblog

Dancing on Magnolia’s Grave: Fuck the Cloud II —

When I wrote out a quick little note to give my impression of cloud computing, I got a hell of a lot of feedback in the same way that an exploding oil tanker is getting a lot of feedback. So much anger and distaste, displeasure and for that matter some “hell yeahs” drowning out there among the pitchforks. What the fuck, people.

What I should have realized was that some people have hitched their wagons to this whorish little star, and anybody indicating the star is not a bright and shiny little piece of sky but an overblown crapfest term to be used however the overselling weasels wish to use it, would be pilloried.

Naturally, responses were all over the map, but I was particularly struck by this tool:

Funny, weren’t the same things said about eCommerce when it first hit? “I’d never trust a site taking my credit card.” Sure, there are still issues with security and privacy as related to eCommerce. But wait now, could we live without our Amazon’s or other eCommerce-enabled sites? Hmmm. Also, your œunderstanding of Cloud Computing is very shallow. Sure, Gmail is the cloud but it’s much broader and deeper than that. Of course you can bash and bad mouth things you don’t fully understand and raise the FUD factor. New isn’t bad, new is different and takes time to be fully vetted, then either adopted or discarded. It’s your call. In the meantime, you can live in “history” but you might want to check out: http://www.nohardware.com as it sheds a different perspective on this with a bang!

See how nice I am? I even leave in the link to his sensationalist, misdirecting, all-action-no-traction website, which features you “blowing up” servers because you don’t have to think about them. Of course, now he has to think about them and you better hope how he thinks about them is the same general realm and care that you were going to care and think about them… and naturally, you’ll want to spend some time considering whether this is all just like a taxi service calling itself nocar.com and blowing up cars because you won’t have to worry about them… might be good, might suck..

I am especially entranced by his position that I am a craven ignoramus, unable to fully comprehend the amazing cool things around me, and maybe with some effort I might just do so.. but then he compares it to the revolution of ecommerce, and implies that that problem is solved.

We’re not going to call that problem solved, are we? There’s still the occasional tiny flare-up in that realm and the concerns are ongoing, and valid. Saying “ppft” and “don’t worry your pretty little head” isn’t going to hide the fact that a lot of people, and stick with me here, a lot of people continue not to use ecommerce or online payment and for very valid reasons. Some people buy my documentaries via check and cash and money order. Some of my friends have websites and services that, because some component of them have untoward information or images, are flat-out censored by major credit card firms (of which there are very few) from doing any transactions whatsoever on the front or back end. Oh, and the minor issue where some links in the chain between credit card, vendor and credit firm would be transmitted in poor encryption or cleartext? That kind of didn’t go away, either. But this is nitty gritty stuff, stuff based in facts and history. Let’s ignore that and blow up some servers. Woo hoo!

Anyway, so soon after that article and the founding of the delightfully fun and exciting Archive Team project, a little event happened that I like to think shows where I’m coming from about this hue and cry, and perhaps mitigates the relatively unpleasant names I’ve been called for a couple months.

That event is the death of Magnolia.

Magnolia, if you missed it, was what you’d call a social bookmarking service. You would connect to it, put in bookmarks of sites and web objects you liked, and then others could share them or comment on them. This idea is not a single one; there’s a solid number of “social bookmarking” sites out there. Some are owned by big daddies now, while others continue to flump along in the cold world of independence. Magnolia was independent.

In January, something went wrong. Oh, so very wrong. Magnolia died. Died big, and died hard.

The message on the front, replacing years of data, explained that something had gone wrong and they were working on it, and utilizing data recovery. This should have been the time to go So, why not just restore from backup? Because that’s what you do when shit dies: you go to the backup. Not keeping a backup is up there with jabbing kids with the same needle during innoculations. You’re saving a relatively small amount of money for a needless and possibly inevitable risk.

Magnolia very obviously didn’t have a backup. The twittering about this is deliriously sad and lost: people talking about how years of work was missing and they couldn’t wait to get back on there so they could finally pull off their data.

Well, guess what. Turns out it was completely lost.

Now, let me talk a little about Larry Halff.


Citizen Garden Episode 11: Whither Ma.gnolia? from Larry Halff on Vimeo.

OK, so stick this out with me:

FUCK LARRY HALFF. He is the poster child for everything I’m talking about. Go ahead and listen to his mincing, wimpy, terrible description of how this site went down. Listen to the brilliant technical explanation he tries to give in explaining the technical issues. Enjoy the giggling as they talk about “lessons learned” and Larry’s smiling helpful advice that you should have good backups, and how it’s all very difficult to do all this, and that you shouldn’t do it yourself because it’s very hard. And if you have any history with computers, listen how he explains how way back in 2005 all this was just so very difficult to do things like have a functional IT infrastructure.

His ability to sit in a little room and talk about all this does not earn him praise. Oh wait, I mean it doesn’t earn any praise from me and people who have technical knowledge. Plenty of other people, their data in tatters and their histories gone, are more than willing to give him a hug, give him a helpful set of you-sure-tried-little-mister chin taps and let him off scot free.

What’s that, you say? Don’t put Fuck Larry Halff in bold letters in your weblog, because I haven’t met him, haven’t hung out with him, haven’t gotten to know him? Fuck knowing him. Hundreds of thousands of people use my sites and don’t know me. Some are better off for it, too.

Let’s be clear here. They took money from people. They’re refunding it, but they took money for a “premium” service. What does “premium” mean to you? Does it mean, I don’t know, baseline functionality?

The podcast I embed above minces about how we need to understand that it is so simple to set up a website and service and we should remember that, because then we misunderstand how these are just a few people behind a site. The flip side of this is that it is often equally easy to set up functional, solid code, done by single people, in their spare time. This isn’t discussed.

His explanations, terrible and useless as they are, cut to the heart of what I’m talking about. Surprise, kids, the service you were depending on turns out to be just a guy and some people working on a couple servers. Magnolia allowed exporting and had functionality to allow it, so they get a pass on that contingency, but apparently they didn’t find people exporting/sharing all that much, because there’s an awful lot of folks out there who have lost a lot of stuff, permanently. The outage was a wake-up call that turned out to be a bullet to the head.

So do this for me, readers. The next time you put stuff into a website, pay money to use a website’s premium function, find yourself spending hours a week on a site putting things into it, just use this magic word:

“Magnolia.”

And you will cast upon yourself a spell to at least attempt to do an immediate backup of your work on there, and if you don’t find a way to do a backup, demand one be made to you. You could also quiz the owners/maintainers of the site what their backup policies are, but bear in mind they’re probably going to lie to you, often without knowing it.

The party ended at magnolia. I hope you didn’t think it was a home.


Categorised as: housecleaning

Comments are disabled on this post


12 Comments

  1. beoba says:

    I’ve used webhosts for a long time, and I’ve gotten into the habit of keeping a periodically-updated local copy of everything I put there. For “backups”, I just run an rsync command every couple months or so, since the webspace isn’t updated frequently. It’s easy to set up, easy to run, and protects against contingencies like me deleting files on accident. However, my method requires SSH access, which isn’t available on most of these sites.

    Maybe there should be a site whose purpose is to periodically back up your accounts on other sites :)

  2. Dave Ross says:

    @beoba:

    Lots of sites offer a way to export data. The problem is what to do with it once you have it. Usually it’s in some XML representation of their internal database structure, which needs to be translated for whatever application you want to import into.

  3. jan says:

    so a tool to backup multiple services and convert between different formats would be the little helper to fix it. allthough, again, to be really helpful, it’d need your usernames… and to trust any (possibly free) service that much … that just initiates a new circle of the same problem

  4. jan says:

    >And…y’know, exactly what ELSE were they supposed to DO?

    backups?

  5. Decius says:

    WTF does nohardware.com do? It seems that they exist solely to glorify the mentality of “I pay professionals to handle my hardware needs, so if I pay them, my hardware doesn’t ever fail or need downtime or anything!”

    From what I can find of cloud hosting via google, it seems to be a scaled ‘pay for what you actually use’ service rather than a ‘pay for so much and if you get /.ed you go down’.

  6. Fake Rake says:

    I usually agree with most of what you write, but this seems a little off-target. If Magnolia had been using a cloud service (e.g., Amazon cloud computing and storage) then they would have avoided this issue. The cloud service’s job is to store your data and keep it backed up, so you don’t have to worry about it.

    If the Magnolia people were just running a few servers themselves, then that means they weren’t using cloud computing or storage. They were just running a web site, and your argument is not to trust some random website with important information. That’s a perfectly valid argument, but doesn’t really have anything to do with cloud computing.

    • Jason Scott says:

      My point is that the term “cloud” is so meaningless that even services like Magnolia are considered to be part of “cloud computing”, even though it’s a technologically feeble amateur blowing scripts onto a messed-up infrastructure. In all these cases, I am trying to stop people from wholeheartedly endorsing something based on the pleasant clean graphics and the simple sign-up screen.

      I see the term “cloud” used so much, I think the use of it is now a signifying danger sign. I appreciate that an amount of more subtle observation will determine the actual danger, but the trend is towards less observation, not more.

    • Terry Cloth says:

      Umm, I thought what he meant was that Magnolia was the cloud, from the point of the people keeping their stuff there. And the cloud screwed them over.

  7. Jon says:

    Holy shit. That video. Fail just fail hard. You didn’t test your fucking backups??? WTF were you doing in this business in the first place?

    1TB MySQL DB??? Yea, get a fucking real DBA. Nao.

  8. pabstsmear says:

    I have been nauseated by the capitalist(spend your money on nonsense), idea that is cloud computing. Whats really interesting about the term is how it is now being overused to encompass everything that was previously called the Internet. I understand that to some people it sounds like your saving money if you need less local resources, but I think thats malarkey. What it really sounds like to me, is that these companies want more control over their closed source software. so if they barricade the software behind their secured access point, it becomes much more difficult to debug it and find/write exploit code… Also making it almost impossible to crack without hacking the box, which will most likely be diligently administrated… Meaning now, not only can they charge you to access a program(which they could only do one time if it were on your box), but they can charge you per click. Sure this saves you the trouble of having all those wasted features taking up space on your box locally, but for the slight convenience of saving you, very minimal space(which its not really doing because you still need a pretty heavy duty client side program to connect to these features). Also lets say they said you clicked 400 times, but you think you only clicked 389 times, who mediates that dispute? Sounds like more corporate holding action so they can screw you out of money for software you probably could get free if you would run a linux distro. This is all before you even touch the ideas corresponding to the actual feasibility of migrating to the cloud. Issues like bandwith. They talk all this shit about how great their high-speed bandwith is GOING to be. As things are, try filling out an online application to work retail, or even in a number of other feilds… you fill out 200 pages of personal questions, “if you caught your mother stealing you publically cut her hands off to show your loyalty to the company… strongly agree…” Then after laboriously beleaguering yourself with this nonsense… Your brain dead… apparantly so is your computer, because you hit submit and somewhere between your access point and their server your packet gets dropped, maybe because the bandwith was too high, maybe your tcp connection timed out. maybe, there was an nop sled before the payload and it got confused for a malicious packet… maybe a million things… Lets say 500,000 people need to access the same application simultaneously… Is it possible that there could be an issue with packets getting dropped by access points?

    They don’t care, all they see is a way to make more money off the same shit, and as said here before, if you buy in your a sucker.

  9. gwern says:

    Fake Rake: that’s not true. Magnolia could *easily* have screwed itself over while using Amazon S3 for everything. Amazon S3, so far as I am aware, does not keep incremental revisions of each and every file you upload or modify; to do so would represent a massive increase in cost, especially for heavily modified files.

    All Magnolia has to do is accidentally delete its files on S3, or write junk over them – and it’s gone.

    This is the same reason that ‘rsync’ is insufficient as backup: oops, you just rsynced a bad version of some folder? Hope you had history – oh wait!