Self
The Problem with Problems with Patterns
Commentators are making statements like this:
In certain programming cultures, people consider Design Patterns to be a core set of practices that must be used to build software. It isn’t a case of when you need to solve a problem best addressed by a design pattern, then use the pattern and refer to it by name. It’s a case of always use design patterns. If your solution doesn’t naturally conform to patterns, refactor it to patterns.What Reg originally said was "Most people consider Design Patterns to be a core set of practices that must be used to build software." Most? Really most? I challenged this remarkable claim, and Reg subsequently re-worded it as you see above. Quite who's in this certain culture I don't know. And apparently neither does Reg:
We could argue whether this applies to most, or most Java, or most BigCo, or most overall but not most who are Agile, or most who have heard the phrase "Design Patterns" but have never read the book, or most who have read the book but cannot name any pattern not included in the book, or...But if that "certain culture" doesn't exist, then what's the problem? And if it does exist, then why is it so hard to pin down?
But it seems off topic. So I have modified the phrase to be neutral.
There are anecdotes:
When I was [...] TA for an intro programming class [...] I was asked to whip up a kind of web browser "shell" in Java. [...] Now, the first language that I learned was Smalltalk, which has historically been relatively design-pattern-free due to blocks and whatnot, and I had only learned Java as an afterthought while in college, so I coded in a slightly unorthodox way, making use of anonymous inner classes (i.e., shitty lambdas), reflection, and the like. I ended up with an extremely loosely coupled design that was extremely easy to extend; just unorthodox.So, maybe there are some teachers of programming (an emphatically different breed from industry practitioners) who over-emphasize patterns. That I can believe. And it even makes sense, because patterns are a tool for knowledge management, and academics are in the business of knowing stuff.
When I gave it to the prof, his first reaction, on reading the code, was...utter bafflement. He and another TA actually went over what I wrote in an attempt to find and catalog the GoF patterns that I'd used when coding the application. Their conclusion after a fairly thorough review was that my main pattern was, "Code well."
It's where Patterns came from
But what's this? "Smalltalk [...] has historically been relatively design-pattern-free due to blocks and whatnot" That's simply not true. In fact, the idea of using Alexander's pattern concept to capture recurrent software design ideas comes from a group that has a large intersection with the Smalltalk community. There's a Smalltalk-originated pattern called MVC, you may have heard of it being applied here and there. Smalltalk is pattern-free? Where would such confusion come from? I think that the mention of blocks is the key.That Smalltalk has blocks (and the careful choice of the objects in the image) gives it a certain property, that application programmers can create their own control-flow structures. Smalltalk doesn't have
if and while and what have you baked in. Lisps have this same property (although they do tend to have cond and similar baked in) through the mechanism of macros.Now, this is meant to mean that patterns disappear in such languages because design patterns are understood to be missing language features, and in Lisp and Smalltalk you can add those features in. So there shouldn't be any missing. The ur-text for this sort of thinking is Norvig's Design Patterns in Dynamic Programming.
The trouble is that he only shows that 16 of the 23 GoF patterns are "invisible or simpler" in Lisp because of various (default) Lisp language features. This has somehow been inflated in people's minds to be the equivalent of "there are no patterns in Lisp programs", where "Lisp" is understood these days as a placeholder for "my favourite dynamic language that's not nearly as flexible as Lisp".
But there are
And yet, there are patterns in Lisp. Dick Gabriel explains just how bizarre the world would have to be otherwise:If there are no such things as patterns in the Lisp world, then there are no statements a master programmer can give as advice for junior programmers, and hence there is no difference at all between an beginning programmer and a master programmer in the Lisp world.Because that's what patterns are about, they are stories about recurrent judgments, as made by the masters in some domain.
Just in passing, here's a little gem from Lisp. Macros are great, but Common Lisp macros have this property called variable capture, which can cause problems. There are a bunch of things you can do to work around this, On Lisp gives 5. Wait, what? Surely this isn't a bunch of recurrent solutions to a problem in a context? No, that couldn't be. As it happens, it is possible to make it so that macros don't have this problem at all. What you get then is Scheme's hygienic macros, which the Common Lisp community doesn't seem too keen to adopt. Instead, they define variable capture to be a feature.
Now, it's tempting to take these observations about the Lisp world and conclude that the whole "no patterns (are bunk because) in my language" thread is arrant nonsense and proceed on our way. But I think that this would be to make the mistake that the anti-patterns folk are making, which is to treat patterns as a primarily technical and not social phenomenon.
Actually, a little bit too social for my taste. I've been to a EuroPLoP and work-shopped (organizational) patterns of my own co-discovery, and it'll be a good while before I go to another one–rather too much of this sort of thing for my liking. But that's my problem, it seems to work for the regulars and more power to them.
Contempt
Look again at Reg's non-definition of the "certain programming cultures":[...] most, or most Java, or most BigCo, or most overall but not most who are Agile, or most who have heard the phrase "Design Patterns" but have never read the book, or most who have read the book but cannot name any pattern not included in the book,[...]doesn't it seem a little bit as if these pattern-deluded folks are mainly "programmers I don't respect"?
This is made pretty explicit in another high-profile pattern basher's writing
The other seminal industry book in software design was Design Patterns, which left a mark the width of a two-by-four on the faces of every programmer in the world, assuming the world contains only Java and C++ programmers, which they often do. [...] The only people who routinely get excited about Design Patterns are programmers, and only programmers who use certain languages. Perl programmers were, by and large, not very impressed with Design Patterns. However, Java programmers misattributed this; they concluded that Perl programmers must be slovenly [characters in a strained metaphor].Just in case you're in any doubt, he adds
I'll bet that by now you're just as glad as I am that we're not talking to Java programmers right now! Now that I've demonstrated one way (of many) in which they're utterly irrational, it should be pretty clear that their response isn't likely to be a rational one.No, of course not.
A guy who works for Google once asked me why I thought it should be that, of all the Googlers who blog, Steve has the highest profile. It's probably got something to do with him being the only one who regularly calls a large segment of the industry idiots to their face.
And now don't we get down to it? If you use patterns in your work, then you must be an idiot. Not only are you working on one of those weak languages, but you don't even know enough to know that GoF is only a bunch of work-arounds for C++'s failings, right? I think Dick Gabriel's comment goes to the heart of it. If patterns are the master programmers advice to the beginning programmer, codified, then to use patterns is to admit to being a beginner. And who'd want to do that?
What "Ivory Tower"?
I recall walking through the grounds at Imperial, heading for the first XpDay in the company of Ivan Moore. As we passed by he noted that there was an ivory tower for me to move into. Now, I am prone to a little bit too much theoretical musing on things, it's true, but I though this comment was a bit rich coming from someone with a degree, MSc and PhD in Computer Science.
Anyway, this "ivory tower" notion is an curious one and seems to be very much a Yes Minister sort of thing: I pragmatically balance theory and practice, you pay too much attention to unnecessary detail, he lives in an Ivory Tower. The phrase has strong overtones of academic detachment. Strange, then, that this posting should suggest that Smalltalk was ever in one. Smalltalk was developed in an industrial research lab paid for by a photocopier company. An early (maybe the earliest) adopter of Smalltalk, where some of the folks responsible for spreading certain ideas from the Smalltalk world more widely worked, was an electronic engineering firm.
Currently (mid 2007) Smalltalk is (still) being used commercially in the design of gearboxes, it's being used in the pricing of hairy financial instruments, it's being used to do hard stuff for money. I've even earned money for writing Smalltalk myself, within the last 5 years.
Lisp, now, Lisp did have to escape from an ivory tower. And that didn't work out too well, so it tried to get back in. But the door had been closed behind it. Ouch.
Well, if, as is suggested, "Ruby is a unix dialect of Smalltalk" then it would seem that being unixified is not such a completely good thing. Really, spelling blocks with {}s instead of []'s is neither here nor there (although having that invisible magic parameter for them is really bad). the theory is that being being now outside the "VM" (I think that Giles probably means "image") then Ruby-is-Smalltalk plays much better with others. That's true enough. Like certain other refugee systems taking shelter in the unix world, Smalltalk really wants to be your whole world. But we kind-of know, and certainly the unix way is, that that's not a great model.
What's a shame, though, is that if Ruby is Smalltalk then it is Smalltalk with the most important thing taken out: life. And life comes with the image, and not from objects (or worse yet, merely the instructions for buliding the objects) being trapped in files. Sorry, but that's the way it is. So, until there's a ruby environment as lively as a Smalltalk image, with all its browsers and such, I can't see the ruby-is-smalltalk metaphor doing anything but spreading confusion and disappointment.
It's Alive! [flash of lightning, crash of thunder]
- Systems should never reboot [and so in orer to be useful]
- systems must be able to grow without rebooting [thus]
- world-class software systems always have an extension language and a plug-in system
- [as a bonus] Great systems also have advice, [...]hooks by which you can programmatically modify the behavior of some action or function call in the system
- [in order for 4 to be possible and 3 to be easy] great software systems are introspective
- [and] every great system has a command shell [so that you can get at the above easily and interactively, see below for the importance of this]
Wot no database?
Steve give examples of systems that exhibit various of these properties to varying degrees, but one whole class of system is missing from his list: RDBMSs. It's possible to get very much carried away with what relational datastores can do, and on the other hand, relegate them to being not much more than file managers. Either way, it's strange that Steve doesn't mention even one RDBMs in his lists of "favourite systems", since the industrial-strength RDBMs exhibit all of the properties that he ascribes to good systems (possibly for small values of "plug-in system"). Certainly they do much more so than, say, the JVM, which is on his list.
But then, when was the last time you saw someone with a terminal open doing real work by composing ad-hoc SQL queries (which is where RDBMs really shine)? Why have we, int eh IT industry, robbed our users of this?
Steve goes on to talk about rebooting as death, and muses about the concious/unconcious boundary and what it might mean to reboot a system on the concious side of it. Highly speculative, but then I did once have a conversation down the pub with someone who reported that someone else (sorry about this) was researching what it would take for self-aware AI to be benevolent towards us. An intruiging notion: those who subscribe to the idea of a creator seem always to assume that the creator is both infinitely superior to the creature and well disposed towards it (except for this one), or at worst indifferent. A self-aware, self-modifying AI might be expected to rapidly become superior to us. And if we're in the habit of killing its siblings as standard operating procedure...
But I digress. So does Steve, and I find that in amongst all the amature philosophising he kind of misses his own point. Towards the end he starts banging on static type systems in general and on Hindley-Milner in particular. This provoked the Haskell folks over on Reddit to show that Haskell can support the sort of systems he describes. Which goes to show how relatively uninteresting the kind of "life" that Yegge talks about is. Frameworks that support hot-swappable executables can be built in Haskell, of course they can. But doing so seems like a stunt.
Back to the Future
Thinking about persistance reminds me of an event: in a previous job I had for a time the title of "Head of Research" with a very open brief to investigate new ways of doing things. The business liked applications to be presented through a web browser so I built a prototype for one system that I had in mind using Seaside. One of the production developers looked at this prototype and asked me how it was persisted. It's persisted, I said, by never turning this box (kicks box hosting the prototype) off. That's not an acceptable solution for a production system, although something that simulates the same effect can be. And Ralph Johnson has this to say about that:
[clang!] just a moment... Oh, the irony: that blog is down (18 Jan 2007), the posting is here, and here's the google cache. Anyway, what Ralph says his own squeak list post that the blog post points to is:
I did this in Smalltalk long before Prevayler was invented. In fact, Smalltalk-80 has always used this pattern. Smalltalk programs are stored this way. Smalltalk programs are classes and methods, not the ASCII stored on disk. The ASCII stored on disk is several things, including a printable representation with things like comments that programmer need but the computer doesn't. But the changes file, in particular, is a log and when your image crashes, you often will replay the log to get back to the version of the image at the time your system crashed. The real data is in the image, the log is just to make sure your changes are persistent.
It's a shame that it has to be when your image crashes, not if, but never mind. This is a pointer to the big win that Yegge misses, in my opinion (and his pro-Haskell commentators, too) The interesting thing is not hot-swapping components without a restart. It's about being enabled to interact with computational objects ("object" in the broadest possible sense) in the same way you interact with physical objects: they should be ready-to-hand not present-at-hand.
A Smalltalk image is a long-lived thing (it's been claimed that some of the standard Smalltalk-80 image survives into contemporary Squeak images) but it has, along with a long lifetime, also a sense of liveness. The objects are just there, beavering away, or perhaps hanging out waiting for you to ask something of them. And they respond immediately. And Smalltalks have this sort of liveness built in by default, and assumed all the way down to a very low level. Working in such systems is a very different thing from working on or with others-and in my experience a much more productive and (the two go together) enjoyable one.
Smalltalk is not unique in this. Common Lisp implementations tend to be this way (and Scheme ones not, which is a shame). Self is this way. Subtext (not the one you're likely thinking of) is very much this way. And, to a limited extent, an Excel spreadsheet is this way. VisualAge for Java tried really hard to be this way, but this seemed to confuse the majority of that new breed of folks "Java Programmer".
Lots more systems should be this way too. Funnily enough, I believe that the best candidate around at the moment for getting this sort of immersed computation experience into the mainstream is JavaScript hosted in web browsers, if only more people would take it seriously.
Keywords, Magic and (E)DSLs
aJanitor open: aDoor with: aKey.Pretty sweet. How might the implementation of
open:with: look? Like this:self insert: aKey into: aDoor; turn: aKey; push: aDoor.Imagine how productive you could be if you could write code this way. Imagine how easy it would be to maintain code written this way. It's so clear what this code achieves (though maybe not quite how it works) that I'm not even going to explain it. This is Smalltalk.
A big part of the reason why Smalltalk is so great is this keyword message syntax (blocks help a lot, too, as we shall see). It allows code like the above, which look very much like natural language. If you are more familiar with curly bracket languages or similar, it can take a little effort to get used to reading code like this, but it's worth it.
No Magic
There's more to it than just ease or reading and writing, though. Smalltalk has no magic. Well actually, it has a lot of magic, but it enables all programmers to be magicians, whereas certain other languages hide their magic away. The language implementers grant themselves wizardly status, but deny it to the language user.One kind of magic is a little bit of laziness around choosing alternatives. In Java we choose between two alternatives like this:
if(condition){
this.doOneThing();
} else {
this.doAnotherThing();
}
and we can be confident that one thing or the other will happen, but not both. That's actually quite clever, although the order that Java is usually taught in hides this. Imagine that if were a method on some object...ah yes, turns out that Java isn't really object oriented after all. Java has objects that instantiate classes that declare methods, it's true, but the code inside those methods, with if and switch and the rest, isn't object-oriented. Such a shame, as we shall see.Anyway, imagine that Java were an OO language and that
if, therefore, were a method. There would be a potential problem (assuming the near universal eager evaluation of function arguments). Our imagined ObjectJava code might look like this:if(condition, {doOneThing();}, {doAnotherThing();});
which kind-of suggests that both the one thing and the other would get done. And this is the magic of the keyword if and it's funny syntax. What are those things with the {}'s? They're little lumps of code (possibly with local variables declared in them), and one of them gets run and one doesn't, under programmatic control. That's a pretty powerful feature, too powerful for the Java wizards to allow we working programmers to wield.The syntax of our invented ObjectJava is pretty bad there,
);}); isn't a thing of beauty, although it's not much worse than the way some real Java looks. The Smalltalk equivalent is much neater. Continuing with our janitorial example:aDoor isAlarmed ifTrue: [self disarm: aDoor] ifFalse: [self openNormally: aDoor].The
[], conspicuously unlike the {} of Java, creates a new object, a so-called block, which contains the code to run. And the interesting thing about this is that ifTrue:ifFalse: is just a method. It's a method of the class Boolean. And Boolean has two subclasses: True and False, each of which is a Singleton. The sole instance of True is called true, and similarly for False.The implementation of
ifTrue:ifFalse: in True is(+):ifTrue: t ifFalse: fand in
^ t value
False it isifTrue: t ifFalse: f(Note:
^ f value
^ is how Smalltalk spells "return" and value is a method on blocks that returns the value of the code inside the block).This is pretty much exactly the implementation of Booleans used in the Lambda Calculus, and that fact reveals one aspect of the close relationship between OO and functional programming.
(Embedded) Domain Specific Languages
Perhaps most astonishing about the Smalltalk approach is that Boolean values and selecting different actions based upon them is not part of the language. These are facilities provided by methods of classes in the Smalltalk standard library! This library (the "standard image") turns out to contain a large number of overlapping Embedded Domain Specific Languages--one of which, provided by the classesBoolean, True and False is specific to the domain of two-valued logic. That's a very remarkable thing. Most remarkable is what it says about the business of writing programs in Smalltalk.There is no mechanism available to the Smalltalk programmer to create programs other than the creation of EDSLsEven the Smalltalk programmers who write Smalltalk itself don't have any other (*) mechanisms available to them. And that's the benefit of No Magic: anything the language implementers can do, you can do too. And you end up doing what they language implementers do, that is, implement (a) language(s). Our example,
self insert: aKey into: aDoor; turn: aKey; push: aDoor.is nothing more (and emphatically nothing less) than a statement about janitors, doors and keys written in an EDSL that knows about...janitors, doors and keys.
Dot Dispatch language
What about those of is who are not fortunate enough to be able to use Smalltalk in our work. What can be done in the languages where regular programmers are second-class citizens?The best example that I've seen of an EDSL in Java is the language used to describe expectations in jMock. This language supports a certain style of programming that many folks are finding valuable. jMock allows us to write programs to talk about how other programs will behave. Like this:
mock.expects(once()).method("m").with( or(stringContains("hello"),
stringContains("howdy")) );again, I think this is clear enough without explanation. You would use code like this within a programmer test to ensure that the test failed if some other object collaborating with our mock doens't call into the mock in the expected way.It's worth digging into the implementation of jMock, not only because it is delightful code, but also to see the amount of heavy lifting that has to go on behind the scenes to make it possible for Java programmers to write code of the clarity seen above.
Productivity?
Building DSLs is the bread and butter of Smalltalk (and Lisp) programming, but is a bit of a struggle in the Java (and similar) worlds. The big vendors are attempting to fix this through the use of mighty tools in the interests of supporting a new-but-old-but-new model of development, a rather fishy proposition at best.This is symptomatic of one way in which the industry has decayed. The message of Smalltalk (and Lisp) is that the route to productivity is to use simple tools with few features and allow everyone interested to build upon them. The favoured route at the moment is to encode every good idea into an all-singing all-dancing "solution", take it or leave it.
Once, the computer itself was locked away, ministered to by a priestly class who mediated your desire to perform computation. Then the (personal) computer revolution began to start and we could all gain direct access to our computing power, and grow our own way of using it. But the something went wrong and a new priestly class--the tool builders in the corporate software vendors--arose to try and put the genie back in the bottle (to mix an increasingly muddled metaphor). They must not be permitted to succeed.
(+) Well, kinda. If it were, then it would be, but for practical reasons it isn't. But for our purposes, it's exactly as if it is. Don't worry about it.
(*) Well, they only have the one: they can hack the virtual machine itself. But then so can you