Innovation

Last year Luke Hohmann demonstrated some of his innovation games at an XtC event hosted at the Zuhlke Ltd office in London.

One theme was that innovation is different from invention. That's a topic close to my heart. Zuhlke has a division named "Product Innovation" and indeed they innovate like crazy (for example, repurposing the optical sensor for a mouse to better control sewing machines [pdf]) but they rarely invent anything (although they do from time to time, for example a newly patented waterless urinal trap [pdf]). Knowing a bit about those folks and what they do has helped make me very sensitive to abuses of the term "engineering" as (mis-) applied to software development. But that's a story for another time.

The Shock of the Old

I've just finished reading The Shock of the Old. It goes on a bit, it's a bit repetitive, it's highly polemical and a bit repetitive. Pretty good, though. The key observation is that the history of technology as usually presented (for example, in institutions like the Science Museum) is largely bunk. This history for the most part ignores use and so also ignores folk technology and what Edgerton calls creole technology. He presents numerous case studies to show that what has been made to look like invention is really innovation and diffusion.

Quick quiz: when was the first document transmitted by fax? Answer: depending on quite what you think a fax is, sometime between 1843 and 1865.

But the average householder couldn't go and buy a fax machine until about a century later. When faxes became available to the general public that wasn't invention, it wasn't even innovation, it was diffusion.

The Revolutionary Period of Big Innovation

Bruce Eckel has come to realize that "software development has stalled". He says,

in recent years it has started to look like we're moving out of the revolutionary period of big innovation, and into a phase of relative stability.

I don't believe this for a minute.

I think the revolutionary period of big innovation in the tools of programming ended about the time that Sun dropped Self. I'd say that the gold standard development environment for mainstream languages right now is Eclipse. As Dave Ungar explains towards the end of this video about the history and influence of Self, Eclipse represents the continuation of the tool-based approach to building a programming environment developed in Smalltalk. I'd say that Eclipse, even the best-of-breed Java environment built with it, still isn't as good as the best Smalltalk environments for ease of use, productivity and fun.

Sun pulled the plug on Self about fifteen years ago. Ironically, they had to buy back the technology for making dynamic dispatch in a dynamically typed language on a VM go fast from Self project staff who had left Sun.

What has happened since then is a steady diffusion of features from good object-oriented development environments of the 80's and 90's into the mainstream. Sadly, few features of the very best object-oriented environment of that time (Self, of course) have made it through.

Sources of Diffusion

Where else are ideas diffusing from? From the mother of all demos. Unfortunately, that seam is about worked out. Alan Kay is supposed to have said "I don't know what Silicon Valley will do when it runs out of Doug's ideas." We may be about to find out. Further diffusion (disguised as innovation) in the field of living with computers may well require actual invention.

Fortunately, that seam is about worked out. We, the public (at least in high-income countries with stable governments) do now finally live in the world that Englebart invented in 1968.

And what of software development tools?

Bruce says:

no matter how good and powerful our software tools get, we are only getting a fraction of the leverage out of them that we could get.

Programming tools are no longer where the greatest potential lies.

We will get the biggest leverage, not just in programming but in all our endeavors, by discovering better ways to work together. [emphasis in original]

and I think he's right. I wonder what it is about the world that Bruce works in that has hidden this from him for so long. After all, in 2001 one group said:

We are uncovering better ways of developing
software by doing it and helping others do it.
Through this work we have come to value:

Individuals and interactions over processes and tools [etc]

Behind the manifesto is a use-based story about the history of technology. Notice that it's uncovering by doing, and not inventing. I'd like to think that Edgerton would approve. It is also a story of diffusion. In Agile and Iterative Development Larman quotes Weinberg:

We were doing incremented development as early as 1957, in Los Angeles, under the direction of Bernie Dimsdale. He was a colleague of John von Neumann, so perhaps he learned it there, or assumed it was totally natural. [...] the technique used was, as far as I can tell, indistinguishable from XP. [...] much of the same team was reassembled [...] in 1958 to develop Project Mercury, we had our own machine [...] whose symbolic modification and assembly allowed us to build the system incrementally, which we did, with great success.

What has happened is that techniques that were once restricted to research projects at the cutting edge of a technological nation's story of existential survival are now available to the mainstream.

What, I would like to know is, next?