This statement is posted as-is, with no changes or editing, by Jason Scott. Comments by Jason will be in the comments section.
My Perspective on the PKARC / ARC Controversy
by Dean W. Cooper
Nov 30th, 2009
Who Am I
I am the author of the DWC archiver which was created around the same time that Phil Katz created PKARC. I corresponded at the time with several archiver authors and eventually engaged in a friendly competition with Phil to see who could create the fastest and smallest compressing archiver. At one point, Phil offered to have me work for him, but for legal reasons that never worked out.
My perspective then is as the only person to have ever matched what Phil did in optimizing and improving the LZW compression algorithm that we all copied from a magazine article.
How I Got Into This
I was intrigued by an article I read in a magazine on LZW compression and thought I might be able to write code that compressed better. I knew about the ARC program at the time and so I grabbed the compression code from the magazine and in two weeks programmed an ARC-like wrapper around it.
I was pleased with my results, as my DWC archiver ran much faster than ARC, so I decided to get onto some BBS’s and see if anybody else would like to use my program. Unfortunately, I quickly discovered that Vern Buerg and Phil Katz had beat me to it. Not only did PKARC go much faster than DWC, it was also compatible with the ARC file format, and that made a huge difference.
I hadn’t realized how important file compatibility was for BBS sysops, and so my initial mistake in giving DWC its own file format resulted in it never being seriously considered as a viable contender in the archiver competition. Nevertheless, it bothered me that Phil’s program was faster than mine, so I spent my time instead working to speed up my program.
Optimizing Code
Now I happen to love optimizing code, and I rarely meet programmers who do it well, so to compete with somebody like Phil who was as fanatical as I was at optimizing was sheer joy. I worked on my code until it ran faster than his, and then he would work on his until he could beat my code, and so on until neither of us could make our code run any faster.
This took many, many hours and weeks of work. At first I had to optimize my C code to run as fast as possible, and then I switched the core routines into assembler and hand optimized them using every trick imaginable to eke out ever smaller gains in speed and compression size. We used self-modifying code and would meticulously go over ever single instruction trying to think of ways to simplify things.
It’s a bit hard to explain what it’s like working on a single instruction, worrying about a few clock cycles and trading one instruction over another because of even a single cycle in speed improvement.
I eventually beat Phil, though not by much. But the hours I spent slaving on that code, knowing Phil had to be doing exactly the same thing, ingrained in me just what level of effort Phil had put into his code and how truly unique and original it was.
And in the end, it was the speed of Phil’s code over ARC that made all the difference and why people wanted PKARC over ARC.
Portable C
But Thom Henderson had a different view of the matter. One of the things Thom claimed was that he wrote ARC to be portable and that was the reason for ARC’s lackluster speed. But Rahul Dhesi designed ZOO to be even more portable than ARC, and ZOO ran considerably faster. Likewise, my C code also ran much faster, and moreover, the assembler code was interchangeable with the C code. It was a simple matter to compile using the assembler code on MS-DOS machines and the C version on other machines. Why couldn’t Thom had done the same?
In fact, when I eventually took a look at Thom’s LZW compression code, I found he had changed it little from the code he obtained from Kent Williams. No wonder ARC was so slow. Thom apparently had never spent much time to speed it up – even though the primary reason Phil was eating into his sales was all because of PKARC’s speed.
Core Engine
For me, the significant and critical work Phil did was in his core compression code. Being ARC-file compatible made a big difference in gaining acceptance, but I suspect that if Phil had switched the file format to an incompatible format that PKARC would still have taken off like it did. Why? Simply because the speed and compression of PKARC was so much better.
I was repeatedly told by BBS sysops at the time that they would switch over to DWC – if only it was significantly faster or better at compressing. But since I was competing with Phil and not Thom, that just never was the case for me. I could only achieve a slight increase over PKARC. Phil didn’t have that problem. PKARC was clearly faster and compressed better than ARC.
So was it the name “PKARC” that made the difference, or was it the user interface Phil used? Of course not. It was PKARC’s compression size and speed.
Unfortunately, Thom seems to believe that Phil only made marginal improvements. But given that Thom never attempted to do what Phil did, it is easy to see how he simply doesn’t understand what Phil did.
My point is fairly simple. All of us (Thom, Phil and I) started with the same publicly available LZW code. But Phil and I both reworked that code over and over to such an extent that not one line of the original code remained. So while the algorithm was still LZW at heart, the implementation was entirely an original work. And given that it is the compression engine that made all the difference, it struck me as outlandish that Thom would sue Phil for copying his code.
It’s like we had all copied plans for a Pinto engine which Thom simply stuck under the hood of ARC, while Phil and I reworked and rebuilt our engines until there wasn’t anything recognizable in any aspect from the original. And given that our engines now ran like finely tuned Ferrari engines, for Thom to claim Phil stole his code was laughable.
Stealing Code
But didn’t Phil steal Thom’s code?
Well frankly, I don’t know. I never actually saw Phil’s code. I have to take Thom’s word at face value that they found his comments in Phil’s code, misspellings and all. And really, it does make some sense that Phil likely did take some of Thom’s code when it came to the file format. Perhaps he didn’t realize the legal jeopardy he was getting himself into at the time. Clearly, if Phil even took some code, no matter how minor, it was a huge mistake on his part.
Even still, the ARC format should have been an open standard, given that it was in the public’s best interest to have compatible formats among the various archivers. More striking to me is the fact the somebody could sue over the use of compatible bolts and nuts when it’s the Ferrari engine that makes all the difference.
Think of this. If Thom would have merely argued that Phil stole his file format, he would never have survived the political backlash from sysops and users. It was his allegation that Phil stole substantial aspects of his code that made Phil look like a pirate who had unfairly ripped him off.
Consider that to achieve the speeds Phil and I did, we were forced to design fairly complex file handling logic. Our compression engines were fast, but they also had to be fed as fast as we could possibly feed them. This required doing a lot of tricks with how we loaded files. Something I’m quite sure Thom never bothered to do.
In other words, I know that Phil’s code had to be very different from Thom’s even beyond the compression engine itself. Thom himself said that he would have never hired Phil after looking at Phil’s code. How is it then that Phil copied Thom’s code if it looks like code he would have never done himself?
Also consider that Thom created ARC by using quite of bit of other people’s code – and not just in the compression engine itself. So sure, the law says that the additions Thom made were his unique copyrightable work, but how significant was Thom’s actual work? What exactly did Phil even need from Thom? Isn’t the only thing that he likely needed was the code that spelled out the file format?
So yes, Thom legally owned that code and it was a mistake for Phil to just use it (presuming he did), but was Phil really suppose to pay Thom royalties just for using the ARC file format? Was Thom suppose to make lots of money off of Phil when it was Phil’s work on that Ferrari engine that is what made PKARC sell?
As it is, the eventual settlement to the lawsuit was that Phil would never again make an ARC compatible archiver. PKZIP wasn’t compatible and history has proven that Phil didn’t need it to be ARC compatible. PKARC only kept the ARC file format alive longer than it would have lasted otherwise. In a sense, Phil indirectly paid Thom royalties by keeping ARC alive and keeping it in business.
User Interface
That Thom also sued Phil over copying the user interface amazed me. We’re talking a command line program! Has anybody else ever been sued over copying the user interface of a command line program?
And ARC didn’t even have a particularly complicated command line. Nor did Phil copy it exactly. In fact, Phil used two separate programs to compress and extract, while ARC was a single program. So what was Thom thinking?
In fact, I made DWC’s command line interface much more identical to ARC. I did so on purpose! I did so because I thought users would prefer that I do so. I never imagined that I was somehow copying the look-and-feel of ARC. Who ever heard of such a thing for a command line program. I still can’t believe it.
Talking With Thom
I had been posting messages to Thom asking questions and at one point he said I should call him up and talk man-to-man. So I did. It was very enlightening.
I wrote up in detail at the time what he told me (see the end of the file here). But it all had to do with his legal rights and how he was obligated to protect them, not only for his sake, but for other shareware authors.
The problem was in how far Thom was willing to go in pressing his legal rights. It was compounded by the fact that he had released the source code to ARC and he believed that if anyone even looked at the source, then they were obligated to obtain a license from him if they then created a work that was “substantially similar” – even if the code was 100% theirs.
Given that Phil admitted to looking at Thom’s code and given that PKARC was substantially similar, that’s all Thom thought he needed to prove his case in court. Since I’m not a lawyer, I don’t know about such things, but it is depressing to me to think the law would crack down on somebody just because they happened to have seen publically available code and then went on to create a similar product – especially when the new product is so clearly better and uniquely created.
Ironies
A funny thing happened after the settlement gave Thom access and use to Phil’s code. Thom turned around and came out with a new version of ARC that at long last substantially improved on ARC’s notoriously poor performance. No wonder, as he was now using Phil’s code.
And yet Thom claimed that Phil’s acceptance of the settlement indicated that Phil felt he had “no legitimate right to [his] program”. After all, Thom asked, “why would he give up everything if he was right?”
Why indeed? Could it be that Phil had worked so long and hard on PKARC, that it had ceased to be an algorithm that he merely obtained from somebody else, that it became a part of him, and that he knew just how capable he was of coming up with an alternative that would leave all this ARC/PKARC mess behind him once and for all?
In other words, Phil had become so skilled at what he was doing, that it didn’t concern him in the least to make the concessions he did to Thom. He knew he was capable of creating a wholly new archiver. And that’s exactly what he did.
Phil’s Offer
Around the time of the settlement, Phil talked to me seriously about working with him. Unfortunately, DWC had never caught on, so I sold “exclusive” rights to the compression engine to another company, and that created potential intellectual property issues if I worked for Phil. After all, I knew in my head what I had just sold exclusive rights to, so how could I work for Phil and not allow that knowledge to somehow “leak” out?
I seriously doubt that this was a real problem, but given all that Phil had just gone through with Thom, he was gun shy to even touch another potentially tricky legal issue, and Phil’s lawyer advised him against hiring me. And so I went on a different path. Which given Phil’s alcoholism, was probably the best for me, but then I can’t say I know what the path working with Phil would have been like.
Final Thoughts
I haven’t worked on my DWC archiver since those years, but I do like to tell people how I once beat the guy who was behind the ZIP archiver. Few know of Phil Katz today. Even fewer know of PKARC (or Thom Henderson and ARC). But I do know what an incredible job Phil did in optimizing and fine-tuning the LZW algorithm.
It is sad to me that Thom felt he had to sue Phil. It is even sadder that Thom appears oblivious to what Phil accomplished.
We all have legal rights that we can fight for. What is sad is when we use those rights without significant cause to attack the highly original and unique accomplishments that are such a benefit to so many.
Finally, given what we now know of Phil’s life, it is perhaps no wonder that he couldn’t get along with Thom. But Thom wasn’t an easy guy to get along with either – well at least if you did something he disagreed with. But by suing Phil, Thom relinquished perhaps his greatest contribution to modern computing. For today, no one talks about ARC files. Everything is ZIP.
Amazing, no?
Dean W. Cooper
Tucson, AZ
dwc@usa.com
23 Comments
