« TeeVee | Main | Buy Buy Buy »

Script: A Lot of Little Things

Let's move away from the documentary to talk a little about my big love: collecting. There's a mental fallacy that I and others of my ilk fall into, where we believe that every single thing we collect has to be a few small clicks away, ready to go, gassed up and waiting in the garage to go out at a moment's notice.

The problem with this is that you end up with files. A lot of files. A TON of files. And if there's one thing a system doesn't like, it's lots of files. Once you get to the point that there's thousands of little one-off images, textfiles, or the like, most systems start to get a little sick, not unlike opening your silverware drawer and finding all the silverware randomly scattered in a big pile. It's just bad.

This required me to make a realization. 99 percent of the stuff I keep on my computers should have the following label:

"A Pile of Neat Crap I Collected At Once And Stuck On My Drive For Later"

Aware as I am of the complete transient nature of websites, when I see one, I wget it, entirely, every piece. I put it into a subdirectory and forget about it. When someone mentions a cool song, or a neat movie someone made, or anything of a "neat thing" nature, I grab a copy.

A good site full of, say, really good drawings or neat 3-D renderings could go into the dozens or hundreds of images. And like I said, my system doesn't like that. So the right thing to do is just stick it all in an archive.

But then we hit the big problem that comes after the "tons of tiny files" problem: you come back to your hard drive after a few weeks, and you wonder what the hell www.doofusnet.com.zip is. Or AMAZING-EYE-SHACK.zip, or, my favorite: "woah.rar", which stared at me at one point.

Hence, I now create two files when I take an archive of images:

filename.zip
filename.zip.jpg

The .jpg is a gallery of every image file in the zip. This takes a collection from being hundreds of individual items to a mere two. I use rsync like cars use gas, so this has enormous benefit for me. I can then look at the gallery image, know what's in the thing, and then go off and unpack the archive if I actually want to look at the stuff inside. In the total sum of this file's life, that will likely be less than one tenth of one percent of the time it exists in my collection. Maybe a lot less. So it makes sense to add the extra unpacking step.

I do everything with scripts. Often ones I write. Here's the script I use. I call it GALLERATE.

Note that I'm leaving in my hysterically informal and non-professional status and error messages with this script; it's how I work with all my programs and scripts, and brightens up what would be an otherwise dreary bit of programming.

#!/bin/sh
# GALLERATE: Turn a zip of images into a gallery.
if [ -f "$1" ]
   then
   rm -rf .galleryworld
   echo "[%] Preparting to squat out $1...."
   mkdir .galleryworld
   cd .galleryworld
   unzip -j "../$1"
   unrar e -ep "../$1"
   echo "[%] WHY DOES IT HURT!!!!"
   montage +frame +shadow +label -tile 7 *.JPG *.GIF *.gif *.jpg *.bmp *.png *.PNG *.BMP "../$1.jpg"
   cd ..
   rm -rf .galleryworld
   ls -l "$1.jpg"
  else
  echo "No such file, assmaster."
fi

Gallerate works by taking the like GALLERATE [filename], where [filename] is a .zip or .rar file that contains images. It creates a temporary directory called .galleryworld, then unpacks all the images into that directory.

Going inside, it uses a great (but complicated) program called "ImageMagick", which has a sub-program called "montage", and then tells it to make a nice gallery, 7 across, of all the images. These gallery images can become large (megabytes big) but it sure beats hundreds of little files all around.

Notice that it tries both .rar and .zip files at once, and tries to deal with both contingencies.

So, why am I mentioning this somewhat technical information?

Like it or not, a lot of people don't learn new ideas by reading manuals or scanning documentation; they look around for little stories, little tales written by people in the thick of things, and then use the morals learned by those writers. It's the nature of learning for some. And by putting this online, maybe down the line someone gets the spark of an idea for a new direction for their own maintenance of images.

So here's my little story.

Once upon a time I had thousands and thousands of files on my hard drive. Now I have a few hundred in pairs. The End.

Comments

Back in the BBS days I (and many others) used a program to sort out the entire archiver mess. Throw just about any type of archive at it and it would sort out the details. Trouble is, I can't remember what it was called now. I have an idea it was SHROOM, but another part of my brain is telling me that shroom was an on the fly memory manager that work brilliantly with doors... guess I'll be digging into the archives when I get home...

I have other scripts for attacking archives and sorting the stuff within. I use this for the artscene.textfiles.com site, which is composed of a lot of archives I don't want to split, but need to describe.

> Notice that it tries both .rar and .zip files at once, and tries to deal with both contingencies.

There's an easier way to do it: http://hentailord.net/cdisplay

That's great! I'll integrate that into my script!

Your comment about "learning things by little tales" is very interesting, because it jives with something that I've noticed about my flavor of engineering (I'm in the aerospace field.)

Despite the management's wonderful rhetoric about process improvement, and big thick binders full of acronyms like CMMI and ISO, and all of that rot...engineers learn how to do things by going down the hall and asking our buddies who have already done them. Programming, how to use the CAD system, who to call when the copier goes on the fritz, designing a structure to safely dissipate static charge built up when flying in geosynchronous orbit...all of this stuff is, in a way, passed on through oral tradition.

I'm wondering if it might not work better just to throw everyone with an MBA out on their ass, and go back to the master/journeyman/apprentice system for technical stuff. If nothing else, the people who are competent and experienced would be given rewards commensurate with their value, rather than told that the only way to be rewarded for being a good engineer is to stop being an engineer and start being a manager.

Post a comment