Looks like this is a rant night...
The article lead to from Beating the Averages (with Lisp) lit a fire in my brain. Oddly, that is basically what it is calculated to do but I did not freak for the reasons as you might expect... or that the author might expect.
As background, I am a long time programmer. I write fairly fluently in Perl, SQL, Basic (don't bother upcasing it anymore), Pascal, ForTran and can get by -- rather well some days -- in C, C++, Lisp, Scheme, and a few others. Perl is currently my favorite language to work in and Lisp is my favorite to force others to learn. I honestly believe that the well rounded programmer must get a little lambda calculus in their soul along with recursion and other handy ways of describing solutions.
The article (I read the PDF version) promotes Lisp as being the bestest language in the whole world and the very reason for their company's success in the big bad world. I rather suspect that he may be fairly right there in the first part, just not the way he means it. In fairness, I also believe that Lisp was first to many concepts that are now common in today's languages.
Unfortunately for him, history doesn't support that being for "superior" reasons, rather that they were under severe restrictions in attempting to write a language that could simulate some fevered imaginings about how the inner mind operates. Garbage collection is a consequence of an ad-hoc data type system and self-modifing code objects. It was SLOOOOW back then, every one hated it and cursed it as evil but you needed it since the code in Lisp was a morphable data object and the code had to be able to contain itself. Worse for his case, Lisp as it originally existed was sorely lacking in fundamentals for common programming. Those were mostly bolted on by borrowing them from other languages. The bolting-on was so bad that the whole language had to be refactored into Scheme so as to protect many people's sanity. Most of what Lisp was first to, it was first to not as a goal but as a consequence of the mathematics of what they wanted to test. Necessity was the mother of these inventions not purity.
Most computer languages are created with a specific problem domain in mind. Cobol was for business, ForTran for maths, Basic for teaching, C for operating systems coding, Lisp for exploring one mathematical aspect of artificial intelligence. Perl was built to do heavy-duty system administration, mostly linear tasks and text reporting, with file modifications and tests, and even some lightweight prototyping thrown in. It is a rare languge that is built from the ground up as a general purpose language. Even the modern examples of just such a thing all wind up having some specific point to rub in (Python's white space blocking, Eiffel's expect'ness, Ruby's kitchensink'ness, Haskell's function'ness, Ada's OO/package/perfection dream)
Anyway, none of this is more than passingly relevant to my real rub with the essay. We'll get down to brass tacks now. The assertions he makes are:
- What [Eric Raymond] says about Lisp is pretty much the conventional wisdom. But there is a contradiction in the conventional wisdom: Lisp will make you a better programmer, and yet you won't use it. He even quotes Eric before getting it all wrong: "Lisp is worth learning for the profound enlightenment experience you will have when you finally get it; that experience will make you a better programmer for the rest of your days, even if you never actually use Lisp itself a lot." See, what Eric is saying is there is something to learn from Lisp. It doesn't make better code, it makes better coders. The thing you learn from it is functional style programming, ala lambda calculus. You learn to think outside the box more. You shouldn't condemn yourself to clinging to the outside of the box for the rest of your life; there are plenty of things best done with the in-the-box way.
- If you can use any language, which do you use? We chose Lisp. For one
thing, it was obvious that rapid development would be important in this market.
We were all starting from scratch, so a company that could get new features done
before its competitors would have a big advantage. We knew Lisp was a really
good language for writing software quickly, and server-based applications magnify the effect of rapid development, because you can release software the minute it ’s done. Read this, "We knew Lisp already and were good at it. We can work faster if we don't switch and we keep doing it the way we like to." The advantage they had in their business was flexible, smart minds who were confortable and confident with their tools from the start. There is a company out there that is sucessfully running their entire web business with Cobol-CGI. Seriously. There are plenty of rapid development languages, hire people who all like one and let them use it.
- I ’ll begin with a shockingly controversial statement: programming languages vary in power. Add "moreso by far in their chosen problem domain" to the end of that and I'll agree wholeheartedly. The problem is general programming language isn't in a single problem domain and once you hit a critical limit of features and capabilities what sets a language apart isn't its "bestness" but its focus on you and your classes of problems. Perl and Lisp are both fantastic for rapid prototyping of systems and both pretty much suck for rigid correctness "proveability" that languages like Ada, Eiffel, and C++ all claim they strive for. He even states his real problem domain more than once in the article, speed of development. Java, Perl, Python, Lisp and C do indeed fall at different levels in expressiveness but I'd suggest that they differ by domain by an order of magnitude over their difference in power.
- When we switch to the point of view of a programmer using any of the languages higher up the power continuum, however, we find that he in turn looks
down upon [fictional language]. How can you get anything done in [fictional languge]? It doesn’t even have y.
By induction, the only programmers in a position to see all the differences in
power between the various languages are those who understand the most powerful one. (This is probably what Eric Raymond meant about Lisp making you a
better programmer.) You can’t trust the opinions of the others, because of the
[fictional language] paradox: they’re satisfied with whatever language they happen to use,
because it dictates the way they think about programs. Sigh. No, by induction, you are just as blind as the people you look down on. You can't just say "by induction" and then make a blanket statement. By induction means following a trend. The trend you describe is ignorance. Note that he now miss-presents what Eric meant again. Eric means, and I think this is pretty clear, that knowing multiple ways of doing things stretches your mind. Also, real programmers aren't ever satisfied with their language of choice. I've grown to hate every language that I've ever used for what I miss from the last language I learned to hate. And I learned to hate and love Lisp early. =)
- A suspicious person might begin to wonder if there was some correlation
here. A big chunk of our code was doing things that are very hard to do in other
languages. The resulting software did things our competitors’ software couldn’t
do. Maybe there was some kind of connection. I encourage you to follow that
thread. There may be more to that old man hobbling along on his crutches than
meets the eye. Feh, taking a whole paragraph to cheekily say "so there" isn't sporting, man. =)
Still, I don't think that the special features of his chosen language are nearly as important as his comfort with them. These are the same claims made by people enamored with OOP and RAD and many other technologies. I still say his abilities are the deciding factor, not his language features. More importantly, he picked the right language for himself and the problem domain he faced. That doesn't prove he found the skeleton key to all locks, he just found the right key for the door he wanted to open.
In the end, my main complaint was the same I have with every pundit of this sort. Being successful in a business doesn't make you an expert in success, it makes you an expert in what you did. That sort of ego trip in confusing history with understanding invariably leads to what I call the Tony Robbins Phenom. You get someone who got lucky and worked hard and who has a dramatic sense of self decide that their personal demons and quirks must be the true way and go out and promote that inner vision as truth.
I helped a company not die and am in the process of trying to do it again. Most of the lessons I've learned figure around how to fail quickly and stopping myself from doing that early enough. I make no pretense that I know how to be successful but I'm getting pretty good at spotting obvious signs of failure. One is believing that how you did it last time is how it must be done next time. You wind up with "lucky socks" style thinking.
Say the boss got on a health kick and made everyone do 30 minutes of exercise every morning at 10am. Now, 5 years later you take your millions from the last company and start your own. You implement the 30 minute workout at 10am and see no benefit for years from it. The difference? Maybe it wasn't the exercise but the way he barked questions at random people about their job, letting everyone know who was responsible for what in the company? Maybe it was the quick, insprirational speeches everyday that you suck at but he had the charisma to pull off? Maybe 70% of that company was 25ish and active and 50% of your workforce is over 35 and fucking tired as hell at 10am? Maybe it was your lucky socks all along?
Cause and effect is rarely as obvious as people seem to want to believe, especially when there is subtle self-flattery involved.
I'm just compelled to add to Maud'Dib's post (man is it weird typing two apostrophes in one word!) below. There are some juicy bits in the forbidden paper that need airing even if you don't hustle your behind off like a good hacker to read the whole paper. And the software Grinch's MP3 collection grew 5 times that day...
from the letter that told them not to publish...
In addition, because public disclosure of your research would be outside the limited authorization of the Agreement, you could be subject to enforcement actions under federal law, including the DMCA. The Agreement specifically reserves any rights that proponents of the technology being attacked may have "under any applicable law, including, without limitation, the U.S. Digital Millennium Copyright Act, for any acts not expressly authorized by their Agreement." The Agreement simply does not "expressly authorize" participants to disclose information and research developed through participating in the Public challenge and such disclosure could be the subject of a DMCA action.
We recognize and appreciate your position, made clear throughout this process, that it is not your intention to engage in any illegal behavior or to otherwise jeopardize the legitimate commercial interests of others. We are concerned that your actions are outside the peer review process established by the Public Challenge and setup by engineers and other experts to ensure the academic integrity of this project. With these facts in mind, we invite you to work with the SDMI Foundation to find a way for you to share the academic components of your research while remaining true to your intention to not violate the law or the Agreement. In the meantime, we urge you to withdraw the paper submitted for the upcoming Information Hiding Workshop, assure that it is removed from the Workshop distribution materials and destroyed, and avoid a public discussion of confidential information. Nice, "please, please, pretty please don't make us sue your asses" there SDMI!
The SDMI challenge offered a small cash payment to be shared among everyone who broke at least one of the technologies and was willing to sign a confidentiality agreement giving up all rights to discuss their findings. The cash prize amounted to the price of a few days of time from a skilled computer security consultant, and it was to be split among all successful entrants, a group that we suspected might be significant in size. We chose to forgo the payment and retain our right to publish this paper. "Fuck their piddly cash, we live in the publish or perish world! Plus, it'll piss them off if we publish!"
It was at this point that we considered a patent search, knowing enough about the data hiding method that we could look for specific search terms, and we were pleased to discover that this particular scheme appears to be listed as an alternative embodiment in US patent number 05940135, awarded to Aris corporation, now part of Verance [5]. .... It also spurred no small amount of discussion of the validity of Kerckhoffs's criterion, the driving principle in security that one must not rely upon the obscurity of an algorithm. This is, surely, doubly true when the algorithm is patented.
Hey, why politely point out that they are wrong when you can kick them in the nuts and shout "stooopid" at them? As a reminder, Kerckhoff knew better back in 1883. Geeks just call this the "security through obscurity fallacy" since we like rhyming and the fact that "fallacy" sounds like a dirty word. when in fact obscurity is the dirty one. =)
For the moment, let us assume that the hash function used in Technology D has only 16 bits of output. Given the number of distinct CDs available, an attacker should be able to acquire almost, if not all, of the authenticators. We note that at 9 kilobytes each, a collection of 65,536 files would fit nicely on a single CD. Many people have CD collections of 300+ discs, which by the birthday paradox makes it more likely than not that there is a hash collision among their own collection.
One disk to crack them all, One disk to find them,
One disk to copy them all, and in the geekness trade them.
Heh. Oh yeah, the birthday paradox is the easy proof that the average Joe on the street's intuition for statistics is sorely flawed. Quick now, in a room with 30 people, what are the odds that some one shares your birthday? Now, what are the odds that some two people in the room share a birthday? The answers are a less than 10% chance that anyone shares yours and around a 70% chance that some two people match up!
Using the techniques we have presented here, we believe no public watermark-based scheme intended to thwart copying will succeed. Other techniques may or may not be strong against attacks. For example, the encryption used to protect consumer DVDs was easily defeated. Ultimately, if it is possible for a consumer to hear or see protected content, then it will be technically possible for the consumer to copy that content. OTOH, we could be wrong. You might be able to work up some continuous mind scanning technique that can find us "criminals" before we "strike"...
Oh yeah, now that the SDMI group is done with the HackSDMI.org domain, can we geeks have it? I'm pretty sure we're the ones who are going to need it...
©
