ARC/ZIP Perspective by Dean W. Cooper —
This statement is posted as-is, with no changes or editing, by Jason Scott. Comments by Jason will be in the comments section.
My Perspective on the PKARC / ARC Controversy
by Dean W. Cooper
Nov 30th, 2009
Who Am I
I am the author of the DWC archiver which was created around the same time that Phil Katz created PKARC. I corresponded at the time with several archiver authors and eventually engaged in a friendly competition with Phil to see who could create the fastest and smallest compressing archiver. At one point, Phil offered to have me work for him, but for legal reasons that never worked out.
My perspective then is as the only person to have ever matched what Phil did in optimizing and improving the LZW compression algorithm that we all copied from a magazine article.
How I Got Into This
I was intrigued by an article I read in a magazine on LZW compression and thought I might be able to write code that compressed better. I knew about the ARC program at the time and so I grabbed the compression code from the magazine and in two weeks programmed an ARC-like wrapper around it.
I was pleased with my results, as my DWC archiver ran much faster than ARC, so I decided to get onto some BBS’s and see if anybody else would like to use my program. Unfortunately, I quickly discovered that Vern Buerg and Phil Katz had beat me to it. Not only did PKARC go much faster than DWC, it was also compatible with the ARC file format, and that made a huge difference.
I hadn’t realized how important file compatibility was for BBS sysops, and so my initial mistake in giving DWC its own file format resulted in it never being seriously considered as a viable contender in the archiver competition. Nevertheless, it bothered me that Phil’s program was faster than mine, so I spent my time instead working to speed up my program.
Optimizing Code
Now I happen to love optimizing code, and I rarely meet programmers who do it well, so to compete with somebody like Phil who was as fanatical as I was at optimizing was sheer joy. I worked on my code until it ran faster than his, and then he would work on his until he could beat my code, and so on until neither of us could make our code run any faster.
This took many, many hours and weeks of work. At first I had to optimize my C code to run as fast as possible, and then I switched the core routines into assembler and hand optimized them using every trick imaginable to eke out ever smaller gains in speed and compression size. We used self-modifying code and would meticulously go over ever single instruction trying to think of ways to simplify things.
It’s a bit hard to explain what it’s like working on a single instruction, worrying about a few clock cycles and trading one instruction over another because of even a single cycle in speed improvement.
I eventually beat Phil, though not by much. But the hours I spent slaving on that code, knowing Phil had to be doing exactly the same thing, ingrained in me just what level of effort Phil had put into his code and how truly unique and original it was.
And in the end, it was the speed of Phil’s code over ARC that made all the difference and why people wanted PKARC over ARC.
Portable C
But Thom Henderson had a different view of the matter. One of the things Thom claimed was that he wrote ARC to be portable and that was the reason for ARC’s lackluster speed. But Rahul Dhesi designed ZOO to be even more portable than ARC, and ZOO ran considerably faster. Likewise, my C code also ran much faster, and moreover, the assembler code was interchangeable with the C code. It was a simple matter to compile using the assembler code on MS-DOS machines and the C version on other machines. Why couldn’t Thom had done the same?
In fact, when I eventually took a look at Thom’s LZW compression code, I found he had changed it little from the code he obtained from Kent Williams. No wonder ARC was so slow. Thom apparently had never spent much time to speed it up – even though the primary reason Phil was eating into his sales was all because of PKARC’s speed.
Core Engine
For me, the significant and critical work Phil did was in his core compression code. Being ARC-file compatible made a big difference in gaining acceptance, but I suspect that if Phil had switched the file format to an incompatible format that PKARC would still have taken off like it did. Why? Simply because the speed and compression of PKARC was so much better.
I was repeatedly told by BBS sysops at the time that they would switch over to DWC – if only it was significantly faster or better at compressing. But since I was competing with Phil and not Thom, that just never was the case for me. I could only achieve a slight increase over PKARC. Phil didn’t have that problem. PKARC was clearly faster and compressed better than ARC.
So was it the name “PKARC” that made the difference, or was it the user interface Phil used? Of course not. It was PKARC’s compression size and speed.
Unfortunately, Thom seems to believe that Phil only made marginal improvements. But given that Thom never attempted to do what Phil did, it is easy to see how he simply doesn’t understand what Phil did.
My point is fairly simple. All of us (Thom, Phil and I) started with the same publicly available LZW code. But Phil and I both reworked that code over and over to such an extent that not one line of the original code remained. So while the algorithm was still LZW at heart, the implementation was entirely an original work. And given that it is the compression engine that made all the difference, it struck me as outlandish that Thom would sue Phil for copying his code.
It’s like we had all copied plans for a Pinto engine which Thom simply stuck under the hood of ARC, while Phil and I reworked and rebuilt our engines until there wasn’t anything recognizable in any aspect from the original. And given that our engines now ran like finely tuned Ferrari engines, for Thom to claim Phil stole his code was laughable.
Stealing Code
But didn’t Phil steal Thom’s code?
Well frankly, I don’t know. I never actually saw Phil’s code. I have to take Thom’s word at face value that they found his comments in Phil’s code, misspellings and all. And really, it does make some sense that Phil likely did take some of Thom’s code when it came to the file format. Perhaps he didn’t realize the legal jeopardy he was getting himself into at the time. Clearly, if Phil even took some code, no matter how minor, it was a huge mistake on his part.
Even still, the ARC format should have been an open standard, given that it was in the public’s best interest to have compatible formats among the various archivers. More striking to me is the fact the somebody could sue over the use of compatible bolts and nuts when it’s the Ferrari engine that makes all the difference.
Think of this. If Thom would have merely argued that Phil stole his file format, he would never have survived the political backlash from sysops and users. It was his allegation that Phil stole substantial aspects of his code that made Phil look like a pirate who had unfairly ripped him off.
Consider that to achieve the speeds Phil and I did, we were forced to design fairly complex file handling logic. Our compression engines were fast, but they also had to be fed as fast as we could possibly feed them. This required doing a lot of tricks with how we loaded files. Something I’m quite sure Thom never bothered to do.
In other words, I know that Phil’s code had to be very different from Thom’s even beyond the compression engine itself. Thom himself said that he would have never hired Phil after looking at Phil’s code. How is it then that Phil copied Thom’s code if it looks like code he would have never done himself?
Also consider that Thom created ARC by using quite of bit of other people’s code – and not just in the compression engine itself. So sure, the law says that the additions Thom made were his unique copyrightable work, but how significant was Thom’s actual work? What exactly did Phil even need from Thom? Isn’t the only thing that he likely needed was the code that spelled out the file format?
So yes, Thom legally owned that code and it was a mistake for Phil to just use it (presuming he did), but was Phil really suppose to pay Thom royalties just for using the ARC file format? Was Thom suppose to make lots of money off of Phil when it was Phil’s work on that Ferrari engine that is what made PKARC sell?
As it is, the eventual settlement to the lawsuit was that Phil would never again make an ARC compatible archiver. PKZIP wasn’t compatible and history has proven that Phil didn’t need it to be ARC compatible. PKARC only kept the ARC file format alive longer than it would have lasted otherwise. In a sense, Phil indirectly paid Thom royalties by keeping ARC alive and keeping it in business.
User Interface
That Thom also sued Phil over copying the user interface amazed me. We’re talking a command line program! Has anybody else ever been sued over copying the user interface of a command line program?
And ARC didn’t even have a particularly complicated command line. Nor did Phil copy it exactly. In fact, Phil used two separate programs to compress and extract, while ARC was a single program. So what was Thom thinking?
In fact, I made DWC’s command line interface much more identical to ARC. I did so on purpose! I did so because I thought users would prefer that I do so. I never imagined that I was somehow copying the look-and-feel of ARC. Who ever heard of such a thing for a command line program. I still can’t believe it.
Talking With Thom
I had been posting messages to Thom asking questions and at one point he said I should call him up and talk man-to-man. So I did. It was very enlightening.
I wrote up in detail at the time what he told me (see the end of the file here). But it all had to do with his legal rights and how he was obligated to protect them, not only for his sake, but for other shareware authors.
The problem was in how far Thom was willing to go in pressing his legal rights. It was compounded by the fact that he had released the source code to ARC and he believed that if anyone even looked at the source, then they were obligated to obtain a license from him if they then created a work that was “substantially similar” – even if the code was 100% theirs.
Given that Phil admitted to looking at Thom’s code and given that PKARC was substantially similar, that’s all Thom thought he needed to prove his case in court. Since I’m not a lawyer, I don’t know about such things, but it is depressing to me to think the law would crack down on somebody just because they happened to have seen publically available code and then went on to create a similar product – especially when the new product is so clearly better and uniquely created.
Ironies
A funny thing happened after the settlement gave Thom access and use to Phil’s code. Thom turned around and came out with a new version of ARC that at long last substantially improved on ARC’s notoriously poor performance. No wonder, as he was now using Phil’s code.
And yet Thom claimed that Phil’s acceptance of the settlement indicated that Phil felt he had “no legitimate right to [his] program”. After all, Thom asked, “why would he give up everything if he was right?”
Why indeed? Could it be that Phil had worked so long and hard on PKARC, that it had ceased to be an algorithm that he merely obtained from somebody else, that it became a part of him, and that he knew just how capable he was of coming up with an alternative that would leave all this ARC/PKARC mess behind him once and for all?
In other words, Phil had become so skilled at what he was doing, that it didn’t concern him in the least to make the concessions he did to Thom. He knew he was capable of creating a wholly new archiver. And that’s exactly what he did.
Phil’s Offer
Around the time of the settlement, Phil talked to me seriously about working with him. Unfortunately, DWC had never caught on, so I sold “exclusive” rights to the compression engine to another company, and that created potential intellectual property issues if I worked for Phil. After all, I knew in my head what I had just sold exclusive rights to, so how could I work for Phil and not allow that knowledge to somehow “leak” out?
I seriously doubt that this was a real problem, but given all that Phil had just gone through with Thom, he was gun shy to even touch another potentially tricky legal issue, and Phil’s lawyer advised him against hiring me. And so I went on a different path. Which given Phil’s alcoholism, was probably the best for me, but then I can’t say I know what the path working with Phil would have been like.
Final Thoughts
I haven’t worked on my DWC archiver since those years, but I do like to tell people how I once beat the guy who was behind the ZIP archiver. Few know of Phil Katz today. Even fewer know of PKARC (or Thom Henderson and ARC). But I do know what an incredible job Phil did in optimizing and fine-tuning the LZW algorithm.
It is sad to me that Thom felt he had to sue Phil. It is even sadder that Thom appears oblivious to what Phil accomplished.
We all have legal rights that we can fight for. What is sad is when we use those rights without significant cause to attack the highly original and unique accomplishments that are such a benefit to so many.
Finally, given what we now know of Phil’s life, it is perhaps no wonder that he couldn’t get along with Thom. But Thom wasn’t an easy guy to get along with either – well at least if you did something he disagreed with. But by suing Phil, Thom relinquished perhaps his greatest contribution to modern computing. For today, no one talks about ARC files. Everything is ZIP.
Amazing, no?
Dean W. Cooper
Tucson, AZ
dwc@usa.com
Categorised as: computer history | documentary
Comments are disabled on this post
First of all, thank you to Dean for taking the time to write this essay on my request after he first contacted me. Whether this causes people to launch into a heated debate or it simply gets added to the files of history, I personally believe it’s critical to collect as many first-person testimonies and narratives as possible, even if they conflict. I have posted it with no changes or edits, per his request (and I didn’t link to any relevant documents, so that it didn’t break up the writing).
I think most people I’d discussed the situation with during BBS Documentary production thought both Phil and Thom were brilliant, and of course I’d been talking with people who’d experienced the whole fight/mindshare battle in the past, so nobody brought up this additional tragedy: that personality and actions ensured these two talents worked against, and not with each other.
Whether Henderson was overzealous in taking on Katz in court depends on your perspective of when legal means are necessary in bringing about what you see as an injustice. In the Henderson/SEA perspective, Katz was utilizing code of theirs and selling it as his own, and additionally was intentionally booking advertising near SEA to indicate his was a superior product, i.e. he was positioning himself as a competitor and using their code.
Atari v. Philips (1982) had already begun the concept of “look and feel” in a computer program as being a legally defensible copyright dispute. Whether this expanded to command line usage was precisely what SEA vs. PKWARE was bringing into focus. This was, after all, 1988, and legal bounds on a lot of computer actions and activity had not yet been determined in court. So while I can understand sadness and puzzlement that look/feel was one of the challenges being issued in court, I don’t think it’s unreasonable to attempt.
I am appreciative to have this document from Dean – it shows the heart of the sadness, confusion, and issues that ARC/PKWARE brought at that time, so long ago.
Compression is my favorite episode of the BBS Documentary, and I find Dean’s comments highly illuminating, as we were able to hear from Thom on the documentary, but obviously not Phil.
It’s hard to see either side as a hero or a villain, and it’s unfortunate that things unfolded as they did. Good ideas are good ideas, and hopefully since the ARC/PKWARE days we’ve gotten closer to a software development environment that rewards good ideas and recognizes that the names attached to a piece of software are real people with real feelings.
Thanks, Dean.
As someone who understands compression, optimization, and assembler programming on the 808x, *and* as someone who used both ARC and PKARC on 808x hardware at the time, I completely agree with Dean. Katz did not make mere incremental improvements to ARC; the engine was orders of magnitude faster. I encourage people to try both out on vintage hardware with a stopwatch.
What Thom was suing for was the ARC-specific file structure code. I suppose he had that right, but as history shows, he shot himself in the foot by doing so.
I do and I’ve done a lot of enterprise consulting, where you do a big project that you “own” for a bit and then hand off the code to someone else. (And then you clearly don’t own it anymore.) The legal encumbrances on the code that you bring it and take out are a very big deal. I’ve been peripherally involved with a “extremely tense business negotiation between two companies lawyers where one party was forced to give up hundreds of thousands of consulting fees because of the intellectual property carelessness of one person”, and this was over something much more trivial than file handling code. (Think “StringUtil”.) It wasn’t ever a lawsuit, though people got fired, but it was a tool that one hostile company used to get leverage over another. I vaguely knew that such a thing was legally wrong but I certainly never realized that it was such a big deal if I hadn’t seen it blow up the consulting company like that.
Now that we live in the days of Open Source and DMCA and everything obviously we’re talking about different eras. But if you’re business is in contracting — especially F500 stuff and doubly especially finance stuff — the questions to these two questions:
“So yes, Thom legally owned that code and it was a mistake for Phil to just use it (presuming he did), but was Phil really suppose to pay Thom royalties just for using the ARC file format? Was Thom suppose to make lots of money off of Phil when it was Phil’s work on that Ferrari engine that is what made PKARC sell?”
Is 100% clearly, obviously, capital letters, Yes and Yes. So don’t ever do that without being very very clear on who owns what code; the company that buys the work, the company that does the work, or the person that does the work. (And this concludes our public service announcement.)
Now what this has to do with the shareware world, I have no idea, let alone that “copy code out of a magazine and fuck around with it” world. As an engineer myself I’m sympathetic to the idea that the hand-tuned core algorithms are more fundamental than mere command/file interface, but this was way before Open Source and you could see how these basic differences of assumptions could make for some serious misunderstandings.
Anyway, I was just thinking about the other side of the compression story just a few days ago. I had always thought of it as “SEA was wronged but PK cleverly won over public opinion and personally crushed SEA’s very soul”, but that picture of some ruthless machiavellian stealing code, then profiting, and then pouring it all away in drink struck me as weird. (These characterizations are one of the many, many reasons I don’t attempt to make object documentaries…) Hmm, I says, I wonder want the other side of the story is like. I wonder if there’s stuff in the BBS archive footage where people talk about it but there wasn’t enough for Jason to really present the other side of the story.
So it’s totally amazing to see this story now. Very weird (but cool) timing.
Will, it heartens me to hear you refer to “SEA was wronged” as the canonical situation, because before the BBS Documentary came out, the exact opposite was true. The story of “there was this guy who made a program and then a big company came and tried to crush him but we’ll show that big company” was the script and that was that, with a few peeps from the darkness saying otherwise. And additionally, the story itself and all the players were fading into obscurity so that it was becoming meaningless to anybody what was even being fought over, or why it mattered. So for that, I am happy the episode had such an effect.
This is why I was careful, so very careful, to collect as much at-the-time primary source material about the controversy as I could at http://www.bbsdocumentary.com/library/CONTROVERSY/LAWSUITS/SEA/ – because I definitely think reading the reactions and discussions in the contemporary context are the key to seeing what was going on. (And a thanks, again, to Bruce Wilson, who was involved in the debates and had saved many files for me to put up).
Part of the reason so many people remember PKWARE/SEA is because it was such a shock at the time – there were horrific battles in many arenas of computer technology over the past 25 years, of course, but I personally think, reading the primary material, that the concept of the “family” or “scene” instigating a lawsuit brought out very intense feeling from all involved. People were shocked it got this far, and that a lawsuit was going to take place, even after all the fervent debate going on.
Software patents are discussed in this page very well, and show that in the 1980s this was still unusual territory.
But I would contend that what we’re seeing here is a company fighting for its life – you can debate whether it had a chance in taking this tactic or what the ethical boundaries were in terms of what was shareable and what was not, but that’s what was at stake here: a family business that saw the heart of its core business being scooped out with advertisements in magazines and journals it was also advertising in. The statement from Ben Baker frames it this way – that it was the moving directly into the space that SEA was relying on income for that caused them to threaten and ultimately sue.
As an aside, Thom Henderson also created programs that phoned home – utilized a modem that would verify payment by checking in with a BBS/computer in the SEA offices. This was also both forward thinking and, to people so inclined to think so, a change in the nature of software that presaged later DRM approaches. But at the time, it could make total sense that this would be a way to check against usage of a program and ensure compensation.
I want to make it clear – there’s no smoking gun in the cutting room floor regarding “the other side of the story” other than the types of cuts I included in the documentary episode: people who said to me, basically, “I didn’t know any of the parties, I didn’t know the exact story, but here’s what I heard.” I have footage in which the Hendersons are markedly more bitter about the whole event, and I might have footage in which Ben Baker might make more redundant statements (already covered by Thom and Irene) that I didn’t include, but there’s nobody I have where they say “I knew Phil Katz, here is his side.” Hence I had to put in that very clear, very distinct disclaimer at the beginning, to say we are missing someone.
I’ve discussed this situation with Dean, and he knew Phil as a fellow craftsman of software and engineer, but was not privy to any specific insider information. Bob Mahoney and I had phone conversations, and he did know Phil, but declined to be interviewed. I will not paraphrase Bob’s discussion of the event, other than to say he understood both sides in the event and was sad that it had such a damning effect on both men (Thom/Phil). Maybe he’ll write more someday, if he feels like it needs to be said.
I just also wanted to be state it again: this was the shareware/software world of 1987-1988, which is completely different than the open source/Linux flood of the 1990s, when a lot of issues hit directly and boldly at the heart of what code was for, how it could be used, and who had rights to what. Regardless of what Eben Mogen may say, the GPL hasn’t been tested in court – it’s been settled every time. Which is good, but it just shows that the grappling with this situation 20+ years was, to our modern eyes, full of weirdness and questions as to what everyone was thinking.
I do agree with what many people have said: Thom won the battle and lost the war with this action. But he really did feel, and continued to feel when I interviewed him, that he did what logic and sense told him to do at the time.
I read the statement from Ben Baker and it is becoming more clear to me what Thom’s perspective on this was. To him, Phil simply downloaded his code, put some assembler code in to make it fast and then started advertising directly against ARC. What was Thom to do but sue to stop this pirate? How interesting.
My perspective was very different. At the time, I took Phil at his word that he hadn’t used any of Thom’s code. This seemed reasonable to me given that the archiver part of the code is pretty trivial. After all, I had duplicated ARC’s functionality in just two weeks – and that’s while working a full time job!
As I’ve said, the engine is where all the hard work was done – the many weeks and months of it – and clearly, Thom hadn’t done this level of work on ARC.
So my perspective was that Thom was simply trying to bully Phil because he didn’t like competition and didn’t care to (or was incapable of) improving his own program. I mean, I simply couldn’t believe that Thom could let ARC’s lackluster performance stand. Didn’t he care that his program was so slow and compressed so poorly? Didn’t he care to give his users a better program?
But now I see that he apparently thought that ARC was just fine and that Phil hadn’t really done much other than to steal his work. In other words, to Thom, his work on the archiver shell itself is what was significant. He didn’t see the compression engine nearly as significant as Phil and I did.
Now consider that when Thom decided to sue Phil, he didn’t know himself whether Phil had actually used his code. He only knew that Phil had downloaded it and presumably looked at it. Given his perspective on the importance of his own code, I can see now why he presumed that Phil had directly used his code.
But good grief, this is just plain silly. The only truly important thing in ARC was the file format. But now it looks like this isn’t what was troubling Thom. He thought Phil had stolen much more. And maybe Phil did. But Phil certainly didn’t need to. ARC was easy to duplicate.
Still, something doesn’t make sense here. If Phil had stolen so much of Thom’s code, then why did Phil use two programs and why did Thom himself tell me that after looking at Phil’s code he would have never hired Phil?
That sounds more like Phil only used small parts of Thom’s code, or perhaps a better explanation was that Phil started with Thom’s code but changed it so much that only a little of the original remained.
But the picture we’re getting of Thom is that he was so hurt to have his code pirated and used to compete against himself. But the competition wasn’t just a copy of ARC, the competition was a program that ran circles around ARC, a program that had little to do with Thom’s code. So, while Thom apparently had legal rights to stand on, I sure don’t see how he can complain about being beaten by somebody using his code. He was beaten by very original code.
I so wish Thom had decided instead to actually work on ARC and make it competitive. But I’m beginning to think he was incapable of that. I guess optimization just wasn’t his thing. How sad.
Consider this. When I heard this talk of how upset Thom was to see Phil’s ads placed next to his, I thought this was bizarre. If somebody has a competing program, why wouldn’t they place an ad in the same publications? Why wouldn’t they claim theirs is better than the standard program?
But then I inherently see Phil’s program as a truly innovative and unique program. Thom, on the other hand, saw it as a pirated program that did little except add assembly code to his work.
And I think this is the crux of the problem. Thom never saw what an amazing thing Phil did. And Phil apparently made the terrible mistake of using parts of Thom’s program.
One last point. I eventually did look at Thom’s code, in part to try to understand this controversy better. But I was quite afraid to look at it, even though it was well after I wrote my archiver. And I hate that feeling. Frankly, I wish Thom had studied my code well. I would be glad if he had learned something. I was always confident I could do better and had nothing to fear from competition.
In other words, I take pride from the fact that I think I can always outdo somebody else if I just work at it. Let them use my code if they want. I’ll just outdo myself.
I mean think about it for a second. Thom thinks somebody took his code and is now selling a pirate program next to his. But gee, the pirated program runs much faster. So even if you decide you must sue, why not at least upgrade your own program so that people have no incentive to use the pirated program? Instead, it looks like you want to force your users in to being stuck using your slow program while you legally drive the superior competitor out of the market.
Hmm. Somehow, that sounds very much like the behavior of some other company that people love to gripe about.
Dean, at the risk of belaboring the discussion, I have to point out that in your speculative writing about Thom’s motivations and thought process, if Thom truly believed that he was being pirated by Katz/PKware, then he, as a business, would have had no choice but to negotiate/warn off another company that was directly lifting code, even minor code. And, if the company that had been warned off failed to respond or otherwise left the lifted code in, a lawsuit might be in order.
Engineering, pure engineering for the sake of best and most efficient and whatever other marks of quality apply, are always but one variable in the equation of a successful business. To indicate that the solution if you suspect someone is utilizing some of your product’s engineering is to simply continue to R&D more and more well-engineered products is to enter a death spiral. Unless I’m mistaken, DWC was never a business product, and its scope and freedom of movement was markedly different than that of a sold product. I’m not saying this is good or bad, just a situation where the stakes and priorities were different.
This is why so many companies utilize a “chinese wall” of building new code with a clean set of developers unrelated to a previous environment – not because it will result in better code, but to prevent accusations of theft or liability. The potential for weaker engineering or reduced quality because of chinese walls has been well documented – but the variables of the rest of the equation result in a product that can survive the legal test.
I watched the BBS Documentary video on the ARC/ZIP controversy again — the one linked to in Thom’s web statement about Phil — and I’m seriously thinking Thom is talking about me when he remembers talking to a guy on the phone about the whole affair (starting at 17:10 in the video).
It is certainly the case that I talked to him for quite some time and that he went through the entire thing as he says. He certainly did mention all the various implications, and I’m sure I basically said the legal issues didn’t matter to me. What mattered to me was that Phil had done an incredible job and I had a hard time seeing how Thom should sue over periphery issues, especially ones like the command line look-and-feel.
I don’t think I said I “didn’t care about right or wrong”, but perhaps that is how Thom took it. Given his perspective on the issue, I am seeing more how he would take my perspective as basically just that.
Wow, if I am the guy he is talking about, I had no idea I affected him as much as I did. At the time, I really wanted an honest discussion with him. I wanted to understand his perspective and I hoped he could see mine.
Now it seems that our two perspectives were so far apart that we just couldn’t help seeing each other in a bad light. How sad.
The truth is, I very much care about right and wrong. Injustice is one of the things that has personally bothered me more than anything else. Only, from my perspective, it was Phil who was having an injustice done to him.
And just like Thom who was clearly effected by this, so was I.
Of course, I had no idea if Phil actually used Thom’s code at the time, and even today I sure would like to see both sets of code to see just how much Phil really did copy.
I am always a little dismayed when I hear someone saying this as if it were a strike against the GPL.
Going to court only happens if the person challenging the license (in this case) feels he can win and is willing to put a lot of his own money and time on the line to ensure the victory. Settling out of court means the challenger thinks, or has been advised by his trained legal counsel, that victory is a long shot at best and that it is better to try and get what he can by asking nicely.
Regarding the GPL being tested in court, I do honestly believe that settling out of court is often an economic move, and not a proof-the-plantiff’s-case-was-awesome move. It is possible to go cheap and take the settlement for ease of the situation with the validity of the GPL being left untested for another day. Egen has always had a habit of saying it’s all a done deal, that the GPL is valid because the GPL is valid, quid pro quo. (I do disclaim that this is what he used to do somewhere in the realm of 5-10 years ago; I am unsure of his current approaches/interests.)
Regardless, my point is that even the GPL, a paragon of open-source licensing and providing a way for parties to share code and promote innovation, is as of yet unseen at the conclusion of a case, meaning that SEA/PKWARE was decades previously in new waters. And that this situation of new waters can sometimes be overlooked looking at the event with jaded 21st century eyes.
Oh, I need to mention something before I forget: The BSD world was embroiled in something similar to this from 1992 to 1994, when USL sued BSDi and others over the ‘Unix’ trademark and, allegedly, because BSD Unix still contained code and trade secrets that could not be released.
The lawsuit put BSD Unix in a legally precarious position precisely when Linux was beginning to shape up into a usable system; the common wisdom is that if it hadn’t been for the lawsuit, the BSDs would be where Linux is now and Linux would have faded away into obscurity. The SCO vs. Linux lawsuits were, at worst, a bizarre parody of what the BSD world went through in the first few years of the 1990s.
My point in bringing it up here is that Open Source licensing does not automatically protect you against lawsuits, even though some licenses, such as the GPL, have never been seriously challenged in court.
Here is first-hand information from Marshall Kirk McKusick. More can easily be found online.
This is very interesting. Thank you for sharing, Dean. About Free Software and Copyrights: I’ve been reading Christopher Kelty’s Two Bits, and in Chapter 6, about the Stallman-Gosling dispute that preceded the start of the GNU project, Kelty makes the point that “software copyrights” were a very new and untested thing in the 1980s, so, er, I guess, what Jason said.
Jason, I don’t get why, if you are a business, that there is “no choice” but to legally pursue somebody if you suspect them of lifting your code. Minor code lifting is just that, minor. It means it is inconsequential, that the person could have easily done it without lifting your code.
Pirating, on the other hand, is when somebody takes unfair advantage of you by taking what you’ve worked on without doing the hard work themselves. I don’t see how this applies to Phil given he worked so much harder at his program than Thom appears to have on his, and the parts Thom did work on were fairly trivial things.
It’s not like Thom had some original invention that was hard to do without looking at his code. He did have the original idea to put the various pieces together, but then he didn’t file for a patent on his invention so that is not what is in question here.
Look, I know companies utilize “chinese walls” to avoid the legal accusations. But this is because businesses play tough and use whatever advantages they have to squash or slow down their competition. But this doesn’t make it right.
Ultimately, I don’t care about the legal issues, and I can’t argue them given I’m no expert. But I do care about what is right and wrong. Thom does to. That comes across very clearly in the documentary. In fact, I am truly surprised to find out how much I effected him and came across as not caring about right and wrong. Because I do care very much about that.
But it comes down to this. I could have easily been in Phil’s shoes. If I had simply made DWC compatible with the ARC file format, I’m sure I would have started running ads too, and certainly that would have started cutting into Thom’s income, and so no doubt I would have been pressured by him to pay royalties too.
I suspect I would have responded somewhat like Phil and basically said “no way”, I’m not paying royalties when this is entirely my program. And so I presume Thom would have sued me as well.
The main thing he had on Phil was that he knew Phil had downloaded his code. I can’t remember when I did, but I certainly downloaded it at some point myself. So, in my view, I’m just as guilty as Phil. But for what? I didn’t do anything. Is it right that I get sued just because Thom gave out his source code and I happen to build something similar (without having looked at his source)?
Try thinking about what the right thing to do is, not just the legal thing. If Phil crossed the line morally, where was that line at? When he downloaded the code? When he glanced at the code? When he studied the code? When he used minor parts of the code? Or when?
And don’t we have some obligation to the public? I mean, isn’t in the best interest of the users that we create compatible programs that use compatible file formats and similar command lines? Why do the legal rights of business always trump what is in the best interest of the public? Don’t we as programmers have some moral obligation to the public?
Yeah sure, what really matters is the law. That’s depressing to me. I would hope we could do what’s right.
Consider that the law does not work in isolation. People are behind each and every lawsuit. And in cases like these, lawsuits are done because a person feels victimized and feels they have an obligation to defend their property rights, and to not allow others to be victimized as they have been.
Clearly, Thom felt that Phil victimized him by unfairly pirating his code. He felt this even before he had seen Phil’s code. He felt it because he knew Phil had downloaded his code and he assumed that all Phil did was to replace some if his core compression code with assembly code. Given that perspective, I would feel victimized myself.
But his perspective was wrong.
He was wrong because he didn’t realize how trivial it was to duplicate what he had done, and how hard it was to duplicate what Phil had done. Phil didn’t need Thom’s code, and therefore there shouldn’t have been an assumed connection between downloading code and creating a similar product. Thom shouldn’t have felt aggrieved to begin with.
But then Thom wasn’t in the business of creating optimized compression code. He was in the business of creating utilities, pieced together using code licensed from other people. This is what gave him a very different perspective on an issue like this. His utilities were his livelihood, not his compression code.
For Phil and I, the important thing was that compression engine. The shell containing it was secondary. And it is that engine that made Phil’s programs popular and launched him into a multi-million dollar a year business. It was the engine that nobody could compete with, not the shell. In fact, PKWARE would have been much, much larger if Phil had just created better shells.
But that gave us a very different perspective. We both knew that Thom had nothing to do with the engine Phil created. Thom didn’t even try to compete. With anybody at that level.
It was his shell that he was proud of and felt victimized about, not the engine.
And this is what I see as silly. The shell was next to nothing compared to the engine. Sure, the shell has some important features, but anybody could create the shell. And in very little time. It was the engine that mattered.
But Thom didn’t see this. So to me, it is Thom’s ignorance that led him to feel victimized and ultimately led him to sue. And that’s a tragedy in my book — or documentary if I was writing the script for one. But don’t worry, I’m not. 🙂
wow, thanks for this.
i’m from wisconsin and i remember this whole thing SO well.
i dont believe SEA was wronged. i think it’s too bad phil is dead so he couldn’t tell his side of the story.
i believe thom was a jackass and people saw him for who he was.
i believe thom used the court’s ignorance to his advantage.
if he believes phil copied his code he should have posted both so people could have saw
it for themselves.
sure, i believe phil used some of it, and left in the comments because he didn’t care enough to take them out. he was about getting it done.
that also is my fav part of the bbs documentary. i still dont like it that it was so one sided due to phil’s death. i’m sure Thom licked his chops at the chance to get the final word in for the last time.
btw, thom is a bitter man, as is his family.
perhaps because in the end, phil DID win.
.zip is the most widely used format, due to katz
So if the ZIP format being used is the measure that we’re using to say that Katz “won”, I guess the format is more important than the algorithm is after all? 🙂
I’m just kidding really. Thanks for writing your thoughts on this Dean. The idea of “intellectual property” is a weird concept, trying to use all these laws about property protection that is designed to help encourage and protect investors exploiting natural resources and applying it to intangible concepts. And it doesn’t really fit. (And it certainly doesn’t differentiate between Real Complicated Things and mere plumbing needed to make it work, like my StringUtils fiasco above.) I think you are perhaps putting your frustrations for the failures of the legal system onto Thom, rather than him “not having perspective”. The system isn’t really about right or wrong, it’s more about order and growing the economy.
But anyway, bitterness or bewilderment aside, I think I understand a little more about why everyone behaved as they did. It’s tragic; it would have been better for everyone if they found a way to cooperate.
To understand this issue a little better, you need to realize just how rigid Thom was concerning his rights under the law. On November 17th, 1988, I called Thom up (at his suggestion) and talked to him at length in order to get his side of the story. I seriously believe this is the conversation he talks about in the BBS Documentary that upset him so much. In effect, he gave me his best shot at explaining why he felt he was right, and so when I didn’t buy it, it appears to have upset him way more than I realized at the time.
But I wrote up what Thom told me, at least as I understood it. And while I may have misunderstood him some, he was quite clear on certain aspects. And this is where you can see his rigidity on the subject of his legal rights. Here is a portion of what I wrote on November 18th, 1988:
———-
To Thom, the law is clearly on his side. He’s simply protecting his rights as allowed by the copyright law. To him, the law says that if a person has access to your copyrighted code, that then you can assume that your code was used. If that person then creates a “substantially similar” product, then it is by definition a derivative work. It doesn’t matter whether or not the product is substantially faster. It doesn’t really even matter if that product really contains any of his code (although he believes that PKARC does).
Thom’s contention is that he has an intellectual property and that by looking at it, you’ve learned some things that are his original creation, and you therefore need to get permission to use what you’ve learned if you create a product that is substantially similar to his. To Thom, it doesn’t matter that his program borrows heavily from others. He did what was proper and obtained those authors’ permissions. All software is, after all, built on other people’s efforts. He simply sees that he made original modifications and those are what he claims to own. It’s irrelevant how major or minor those original modifications are, they are still copyrighted.
I raised the hypothetical situation where I went commercial with DWC and made it ARC compatible. I suggested that in that case I might get a license from them even if I felt I hadn’t violated their copyright just to make sure they wouldn’t sue me. Thom said someone else had actually said the same thing to them, and that he said he would not be a party to extortion or taking pay-offs. If a party doesn’t agree that their work is a derivative work, then SEA won’t license them even if the party is willing to pay the royalties.
In another point I suggested that Thom could have at least warned people who looked at his source that they then couldn’t create similar products without obtaining permission from SEA. But Thom said that to say anything more than just the normal copyright message would potentially cause him to lose rights. And after all, to Thom, the copyright law is the law and people should know it, especially if they’re going commercial.
———–
To me, it was clear that by Thom’s understanding of the law, my archiver also violated his legal rights, even though I didn’t study his code nor use any of it in writing my program. He wouldn’t even accept me obtaining a license from him unless I agreed that my work was a derivative work. To me, this is astonishing. Given Thom’s views on his legal rights, how was Phil to have ever cooperated with Thom? I mean I understand that Thom truly believes these are his rights and that he is doing what is good and proper, but it sure comes across like he is a bully.
Who could have possibly competed with Thom and not eventually been sued, or had to pay him royalties? Only people behind a “chinese wall”? And for what? For the ARC file format? For code that was worthless when it came to speed and compression size? For code that just pieced together other people’s code? For code that could easily be written in days? For code you didn’t even look at? Good grief!
Now it appears Phil actually did look at the code and even used some of it. So as you can see, given Thom’s view on his legal rights, that was a HUGE mistake on Phil’s part. But Thom apparently didn’t see how easy it would be for Phil to abandon ship, rewrite his program and come out with an entirely new archiver. I’m just surprised Thom didn’t sue him for PKZIP. But maybe the settlement they reached precluded that, or at least the cost would have.
But what has Thom learned from this? From the BBS Documentary it appears he feels a large injustice was done to him. To me, given his view of the law, this was inevitable. I don’t think I could have ever cooperated with Thom either. And I like to think I could cooperate with most anybody.
Put yourself in Phil’s shoes. What would you have done? Could you have cooperated with Thom?
Notice also how Thom’s sense of what is “right and wrong” seems to be very much based on the law. If the law grants him legal rights, then that is what belongs to him and he will fight for his rights. By defending himself, I presume he believes he is upholding right and wrong.
But the law doesn’t make things morally right. It just makes them legal.
The law allows authors to protect their intellectual property, and for good reason. There are thieves who will steal what belongs to you and that took you great effort to create. A person who copies DVD’s and sells them didn’t do all the work to make the movie, they just steal it. And that’s wrong.
But for me, except for the issue of the ARC file format itself, Thom used legal technicalities in an attempt to stop legitimate competition. Phil certainly made mistakes, but in my view, he didn’t really steal anything (especially given that I view the file format itself more as public property).
And in that case, it wasn’t morally right for Thom to defend his intellectual property to the extent he did, regardless of what the law may or may not have actually allowed him to do.
Does that make sense?
Thanks for the reminiscing. It’s interesting to hear what was happening behind the scenes.
I remember when it was happening, and seemingly en masse, the BBS operators, and early internet hosts switching over to zip from arc. I cheered for Phil then, and nothing I have read makes me think in hindsight that it was not the right thing to do.
I had another thought on the documentary, which goes to great length to point out that while Thom may have had legitimate reasons to sue Phil, he still lost in the end in part because of a PR battle that made SEA come across as a “big company” while Phil was just the “little guy”.
But consider that SEA was big enough to file a serious lawsuit against Phil. That alone made them appear larger than typical shareware authors were at the time.
But the crucial factor for me is simply that PKARC compressed better and ran much faster than ARC. People naturally loved PKARC for just those reasons. It was obvious then to anybody that Thom was losing income simply because ARC wasn’t competitive. And what do big mean companies do who can’t compete? Why of course they sue to squash the little guy.
In other words, Phil didn’t have to do *anything* to wage an effective PR campaign, other than make a program that was clearly better and then get slapped with a lawsuit for things like his program’s name, the command-line interface and supposedly stealing code. All three of those claims seemed laughable, at least to people who weren’t steeped in the law, and that made Thom appear to be a bully. Compounding the laughable claims was the very slowness of ARC that made Thom come across as a lumbering Goliath, while the zippy PKARC made Phil look like a fleet and nimble David.
Moreover, there was the perceived injustice of it all, for it was clear that Thom was incapable of speeding of ARC. Thus, it looked for all the world as if Thom resorted to suing simply because he couldn’t write blazing fast code like Phil could.
Now this would have been much different if this had been just a lawsuit between consultants. But it wasn’t. ARC and PKARC were used by the public and Phil was inherently seen as the upstart who through hard work had created a better product that people wanted. Everyone knew Phil had started out creating PKARC by himself (not as part of a company). Doesn’t that make him small by definition, even if his success allowed him to grow after that?
In short, while the documentary makes it appear that Phil somehow was able to wage a better PR campaign than Thom, that PR campaign didn’t require deception or better “framing” of the issue to succeed. It needed only a *single* aspect — that Phil had created a better product on his own, along with the perception that the established and dominate SEA was trying to squash him for it.
So yes, Thom’s mistake was looking at this issue as an engineer and not as a PR issue. But he didn’t lose the battle because Phil waged a better PR campaign. He lost the PR battle because it looked like he was incapable of making ARC better, and chose to file a lawsuit instead.
The *only* part of Thom’s lawsuit that most people felt a measure of sympathy for was the allegation that Phil had stolen his code. Nobody likes somebody stealing somebody else’s work. But in this case it was just too hard to believe. Phil’s program was just so much better. The instinctive thought was why would Phil even *want* to steal Thom’s code?
And thus the lawsuit ultimately came across as implausible and people’s sympathies largely went with Phil. Right or wrong as that may be.
But I don’t think Thom to this day understands that. In effect, he’s still losing the PR battle. Amazing.
Consider this. Regardless of the lawsuit, if Thom had only worked to speed up ARC and kept it competitive, PKZIP may have never taken off. ARC was the standard after all. It was Thom’s to lose, and lose it he did simply because he didn’t seem to care to make his program better. He lost literally millions because he didn’t do that — not because Phil stole his code.
He didn’t even need to outdo Phil, just come close, and then most people would have never had a reason to switch.
I, on the other hand, should have made my archiver ARC compatible. I may have ended up paying Thom royalties in that case, but perhaps I would have been the one to make the millions then. Alas.
[…] long ago: The ARC/PKARC war. ASCII by Jason Scott / ARC/ZIP Perspective by Dean W. Cooper Reply With Quote + Reply to Thread « Previous Thread […]