The Problem with Problems with Patterns

There seems to be a new venom in the informal blogsphere-wide movement against patterns. I'm not the only one to notice it.

Commentators are making statements like this:
In certain programming cultures, people consider Design Patterns to be a core set of practices that must be used to build software. It isn’t a case of when you need to solve a problem best addressed by a design pattern, then use the pattern and refer to it by name. It’s a case of always use design patterns. If your solution doesn’t naturally conform to patterns, refactor it to patterns.
What Reg originally said was "Most people consider Design Patterns to be a core set of practices that must be used to build software." Most? Really most? I challenged this remarkable claim, and Reg subsequently re-worded it as you see above. Quite who's in this certain culture I don't know. And apparently neither does Reg:
We could argue whether this applies to most, or most Java, or most BigCo, or most overall but not most who are Agile, or most who have heard the phrase "Design Patterns" but have never read the book, or most who have read the book but cannot name any pattern not included in the book, or...

But it seems off topic. So I have modified the phrase to be neutral.
But if that "certain culture" doesn't exist, then what's the problem? And if it does exist, then why is it so hard to pin down?

There are anecdotes:
When I was [...] TA for an intro programming class [...] I was asked to whip up a kind of web browser "shell" in Java. [...] Now, the first language that I learned was Smalltalk, which has historically been relatively design-pattern-free due to blocks and whatnot, and I had only learned Java as an afterthought while in college, so I coded in a slightly unorthodox way, making use of anonymous inner classes (i.e., shitty lambdas), reflection, and the like. I ended up with an extremely loosely coupled design that was extremely easy to extend; just unorthodox.

When I gave it to the prof, his first reaction, on reading the code, was...utter bafflement. He and another TA actually went over what I wrote in an attempt to find and catalog the GoF patterns that I'd used when coding the application. Their conclusion after a fairly thorough review was that my main pattern was, "Code well."
So, maybe there are some teachers of programming (an emphatically different breed from industry practitioners) who over-emphasize patterns. That I can believe. And it even makes sense, because patterns are a tool for knowledge management, and academics are in the business of knowing stuff.

It's where Patterns came from

But what's this? "Smalltalk [...] has historically been relatively design-pattern-free due to blocks and whatnot" That's simply not true. In fact, the idea of using Alexander's pattern concept to capture recurrent software design ideas comes from a group that has a large intersection with the Smalltalk community. There's a Smalltalk-originated pattern called MVC, you may have heard of it being applied here and there. Smalltalk is pattern-free? Where would such confusion come from? I think that the mention of blocks is the key.

That Smalltalk has blocks (and the careful choice of the objects in the image) gives it a certain property, that application programmers can create their own control-flow structures. Smalltalk doesn't have if and while and what have you baked in. Lisps have this same property (although they do tend to have cond and similar baked in) through the mechanism of macros.

Now, this is meant to mean that patterns disappear in such languages because design patterns are understood to be missing language features, and in Lisp and Smalltalk you can add those features in. So there shouldn't be any missing. The ur-text for this sort of thinking is Norvig's Design Patterns in Dynamic Programming.

The trouble is that he only shows that 16 of the 23 GoF patterns are "invisible or simpler" in Lisp because of various (default) Lisp language features. This has somehow been inflated in people's minds to be the equivalent of "there are no patterns in Lisp programs", where "Lisp" is understood these days as a placeholder for "my favourite dynamic language that's not nearly as flexible as Lisp".

But there are

And yet, there are patterns in Lisp. Dick Gabriel explains just how bizarre the world would have to be otherwise:
If there are no such things as patterns in the Lisp world, then there are no statements a master programmer can give as advice for junior programmers, and hence there is no difference at all between an beginning programmer and a master programmer in the Lisp world.
Because that's what patterns are about, they are stories about recurrent judgments, as made by the masters in some domain.

Just in passing, here's a little gem from Lisp. Macros are great, but Common Lisp macros have this property called variable capture, which can cause problems. There are a bunch of things you can do to work around this, On Lisp gives 5. Wait, what? Surely this isn't a bunch of recurrent solutions to a problem in a context? No, that couldn't be. As it happens, it is possible to make it so that macros don't have this problem at all. What you get then is Scheme's hygienic macros, which the Common Lisp community doesn't seem too keen to adopt. Instead, they define variable capture to be a feature.

Now, it's tempting to take these observations about the Lisp world and conclude that the whole "no patterns (are bunk because) in my language" thread is arrant nonsense and proceed on our way. But I think that this would be to make the mistake that the anti-patterns folk are making, which is to treat patterns as a primarily technical and not social phenomenon.

Actually, a little bit too social for my taste. I've been to a EuroPLoP and work-shopped (organizational) patterns of my own co-discovery, and it'll be a good while before I go to another one–rather too much of this sort of thing for my liking. But that's my problem, it seems to work for the regulars and more power to them.


Look again at Reg's non-definition of the "certain programming cultures":
[...] most, or most Java, or most BigCo, or most overall but not most who are Agile, or most who have heard the phrase "Design Patterns" but have never read the book, or most who have read the book but cannot name any pattern not included in the book,[...]
doesn't it seem a little bit as if these pattern-deluded folks are mainly "programmers I don't respect"?

This is made pretty explicit in another high-profile pattern basher's writing
The other seminal industry book in software design was Design Patterns, which left a mark the width of a two-by-four on the faces of every programmer in the world, assuming the world contains only Java and C++ programmers, which they often do. [...] The only people who routinely get excited about Design Patterns are programmers, and only programmers who use certain languages. Perl programmers were, by and large, not very impressed with Design Patterns. However, Java programmers misattributed this; they concluded that Perl programmers must be slovenly [characters in a strained metaphor].
Just in case you're in any doubt, he adds
I'll bet that by now you're just as glad as I am that we're not talking to Java programmers right now! Now that I've demonstrated one way (of many) in which they're utterly irrational, it should be pretty clear that their response isn't likely to be a rational one.
No, of course not.

A guy who works for Google once asked me why I thought it should be that, of all the Googlers who blog, Steve has the highest profile. It's probably got something to do with him being the only one who regularly calls a large segment of the industry idiots to their face.

And now don't we get down to it? If you use patterns in your work, then you must be an idiot. Not only are you working on one of those weak languages, but you don't even know enough to know that GoF is only a bunch of work-arounds for C++'s failings, right? I think Dick Gabriel's comment goes to the heart of it. If patterns are the master programmers advice to the beginning programmer, codified, then to use patterns is to admit to being a beginner. And who'd want to do that?


Anonymous said...

thank you, for combatting this constant FUD that design patterns are anything more than a common descriptive vocabulary.

Anonymous said...

I think the criticism is a reaction to the millions of programmers that use Design Patterns instead of thinking about their problems and where they may need a degree of flexibility, like hacks reuse and abuse idioms and old metaphors.

The results are invariably verbose, anemic, unreadable, and frequently full of logical errors, just like bad prose. They can fool the beginner into thinking there's substance there, and even encourage him to create his own meaningless tripe. In fact, I've seen more than one fledging "architect" measure the eliteness of his code by the number of patterns used.

keithb said...

@rubinelli: millions? See, this is exactly the sort of hyperbole that got me thinking about this topic. Where are these millions?

If this sort of behavior were really as widespread as (some) patterns critics seem to claim, indeed to make the foundation of their critique, I'd have expected to have come across it, and come across it a lot, in my decade and more as a professional programmer. But I simply haven't. And so I'm very skeptical about this line of argument.

Anonymous said...

There are two main times I see patterns (in general). First with junior programmers who have just discovered them. Suddenly their code is full of Singletons, Observers and Visitors. They needlessly complicate otherwise simple problems because they are junior and patterns provide them with a defensible solution.

Second I find them in conversations with experienced developers when trying to communicate the general design they have created. It is nice to be able to say "This class functions like a Visitor to this class" or "I delegate to this class through this object". Often the implementation is conceptually similar to the pattern described although it may not be a canonical example.

It is my opinion that the recent rants are mainly about the former and not the latter. When they are ranting about the latter it is to lament some languages inability to elegantly allow a simple implementation of a desired behavior (whether or not it is pattern based).

Anonymous said...

One additional problem is, that it is not so much the smart people that gain most attention, but more those that can rant "best".

Hopefully someone smart can make an analysis of patterns in general (including human thinking patterns) - but without ranting, i.e. combining it to ranta gainst Java (we all already know Java su**s due to it's abhorrent verbosity *grin*)

keithb said...

@James I think I agree with your distinction between those two groups of pattern users. I'd like to believe that the pattern ranters are indeed complaining about the former and not the latter, but I remain unconvinced that they are so refined in their criticism: not least because I don't see any of them drawing this distinction.

Anonymous said...

A very good post. I too have never understood why "best tool for the job" only seems to apply to IDEs and programming languages but not to "code templates" (i.e. design patterns).

Anonymous said...

Most people appear to identify patterns with specific code structures, but the class diagrams in the GOF book are not the patterns.

It's exactly as you (and Martin Fowler) say, patterns are "recurrent solutions to a problem in a context".

When talking about a design, it is quite sensible and understandable to say things like "look, there you can register observers and over there that's a singleton". But these code structures are not patterns, they are the typical manifestations of applying those specific patterns to design problems.

The pattern encompasses a problem, possibly conflicting forces, and a solution that trades off these forces in some way. Anyone who thinks otherwise ought to go back to the Introduction chapter of Design Patterns.

Anonymous said...

I wish "millions" was a hyperbole, but it isn't. Sun estimates there are more than six million professional Java programmers in the world. The top million is composed of good or great programmers, while the bottom million is dead weight. That still leaves millions of average or inexperienced programmers that are likely to fall into a design-pattern cargo cult mentality. And we're ignoring C# and C++ programmers here.

Keith, if you haven't found this behavior it's probably because (1) you are a good software developer working with other good people and (2) you or someone you trust made most of the design decisions. Had you worked for a big-five consulting firm for 7 years, like I did, your perspective would be different.

And as a last argument, I think that "design patterns aren't all that special" is better than "design patterns can be dangerous in untrained hands" if you intend to influence people to see them as the tools they are and not as the cool passphrases of the architectural priesthood.

keithb said...

Rubinelli, thanks for getting back to me on that. Do you have a source for this Sun estimate? It's not that I don't believe you, I believe that you heard that from Sun—I'm interested to know what they think a "professional Java programmer" looks like (they probably think that I'm one, for instance, not a happy thought) and how they arrived at that number.

On reasonable statistical grounds indeed we would expect most of those millions to be pretty average (ie poor) programmers. Less clear to me is that we should expect those programmers to be pattern cargo-cult devotees.

Winterstream said...

For my part, I've never taken criticism of design patterns to mean that all patterns are bad. Lisp's macros and Smalltalk's neat syntax allow you to capture patterns, after all.

I believe that these authors all refer to the lack of any way to capture such patterns in Java (using macros or something similar) and C++; all that you can count on are (GoF-like) design patterns to help ease the pain.

Maybe I'm a mediocre programmer, but I agree with them (but I don't think that people who use patterns are morons).

GM said...

Was that last paragragh supposed to be sarcastic? The words themselves ring very true, but the tone sounds like you're trying to exaggerate it to emphasise some kind of absurdity that I'm not catching.

keithb said...

@Greg—yes, it is meant to be sarcastic (I'm British, it's our national sport). Sorry that it's not clear. I'm in a bit of a bind now, though, because I don't want to "steal the a-ha!" from a follow-up piece that I'm developing.

Oh what the hell, I doubt many people will get this far in the comments anyway: I'm coming round to the opinion that the reason that the broader patterns movement has never really taken off (and what of the violent reaction to the success of GoF isn't explained by those who do go pattern-crazy for a bit) is that to want to use patterns in your work speaks of a certain humility in one's approach to ones' work that is increasingly alien to the mainstream programmer.

Anonymous said...

First of all, thanks for that last link. I enjoyed that read, and feel I must work to develop my "beginner mind".

Secondly, I suppose it is possible that there are millions of developers who needlessly complicate their code with design patterns when they aren't needed. But like you, I suspect that the average/poor/beginner programmer is more likely to have not even read GoF. Millions sounds inflated to me, but it is simply my intuition, and I have no source. Then again it is quite possible I'm one of those average/poor programmers, but I don't think so.

Anonymous said...

I agree to some extent that many programmers use design patterns just to be used but on the other side I think it would be worth without using them at all.

Doug said...
This comment has been removed by the author.
Doug said...

References for Sun's Java programmer counts:
6 million (2007)
5 million (2006)
4.5 million (2005)
4 million (2004)
three million (2003)

keithb said...

@ Doug, thanks for finding those links. I stand corrected!

Anonymous said...

I'm rather late to the conversation, but my impression is that the anti-pattern sentiment is not so much that there exist no patterns, but rather, that if you have recurrent patterns that are not easily expressed in your language, then your language sucks.

It's (thankfully) been a long time since I wrote any C++, but iterating on a loop looks something like:

for (Foo::Iterator it = list.iterator; it != NULL; it = it->next) { something(it.thing); }

And in a a slightly less awful language it looks like:

list.each { |thing| something(thing) }

Both implementations are absolutely patterns, but one of them isn't moronic. I think that is the point the anti-pattern posts are really about...

keithb said...

Hi Justin. On the intertubes it's always now.

I agree that languages where you can't define your own control structures are weak. When the day comes than everyone who programs is using one of those, will there be no need for patterns?

No. Because patterns are a social and not a technical phenomenon. They are about mining a community for its tacit knowledge and making that knowledge explicit. Even in the world of Lisp, say, if you don't know that a bunch of other people have come across the same solutions in similar contexts as you, then the fact that they wrote some killer macro to do it is no help.

There again, external iteration is a soft target. Soft verging on straw man, because both C++ and the patterns world have changed almost beyond recognition since 1994.

A pattern like Process-Related Test Message is going to take a bit more than a pinch of meta-programming pixie dust to fade into the background (although it would help a lot).