Comments on: Nice Try, Archiver-Hater

By: Sarah

Sarah — Tue, 28 Apr 2009 14:36:13 +0000

I think both people have valid points. Often people seriously abuse wget and strip whole sites… often for their own, money grabbing (and cost you money in terms of bandwidth).

But simply blocking wget stops a lot of archive systems people are using.
Perhaps revisiting a 100% kill wget policy after the problem has gone away so that it throttles the connection would be better. Who can say, most people apply the bandaid and leave it at that.

But there is no real need for righteous indignation from either side.

By: steven

steven — Tue, 25 Mar 2008 01:07:31 +0000

Gosh, you are so much cleverer and cooler than me, the mere author of “some academic nib-nob”. (Er, it’s not actually academic, but whatever.) I am shivering in awe.

What was actually going on was that some idiot was wgetting the same file from the same IP address about 50 times a minute for several hours, thus costing me money. I actually didn’t know how to do anything more complicated than ban the user-agent (which worked for this idiot), and didn’t have time to find a better solution. So please aim your self-righteousness somewhere else.

By: Josef Kenny

Josef Kenny — Wed, 13 Feb 2008 23:29:42 +0000

I just wgot ascii.textfiles.com It’s quite a good idea actually. Next time my hard drive dies (yup, maxtor) I can entertain myself by flicking through the archives in Ubuntu.

By: Richard Warezwolf

Richard Warezwolf — Wed, 13 Feb 2008 20:50:07 +0000

There is one on-line community that finds it hilarious to “archive” content on websites that they find offensive to their hive mind (using wget). They in fact use the word ‘archive’ to get around their own self-imposed rule against fucking with other sites.

how i hate them

–RW

By: Matt Brubeck

Matt Brubeck — Wed, 13 Feb 2008 20:01:29 +0000

Maybe he's just trying to keep out Richard Stallman.

By: Gene Buckle

Gene Buckle — Wed, 13 Feb 2008 18:07:57 +0000

Our Archives _never_ forget.

By: Chris Barts

Chris Barts — Wed, 13 Feb 2008 18:05:32 +0000

Too bad "Trigger Happy" was written by someone who can't even get basic facts straight. The PDP-1 was not a mainframe and it was not hulking. In fact, the main, defining feature was that it was small and cheap enough MIT would let students like Greenblatt and off-the-wall professors like Minsky play with it. That is the sole reason people remember it as fondly as they do.

Oh, yeah, great article otherwise. I'd emphasize the fact people like you can adapt to just about any server-side restriction and that wget alone is flexible enough to look enough like a human software can't really filter it (even to the point of inserting random delays into its sequence of requests so it looks like a human instead of a program).

By: Jason Scott

Jason Scott — Wed, 13 Feb 2008 17:26:35 +0000

If one falls, another will take our place.

We are archivers. We are legion.

By: GWB

GWB — Wed, 13 Feb 2008 17:21:01 +0000

Or, more properly, the archivers might not outlast anyone–but the archives will.