tag:blogger.com,1999:blog-239124782024-03-08T02:23:35.090+00:00peripatetic axiomSoftware, systems and management thereofkeithbhttp://www.blogger.com/profile/14314542307822401015noreply@blogger.comBlogger159125tag:blogger.com,1999:blog-23912478.post-79589078387997447452012-07-21T22:07:00.003+01:002012-07-22T09:20:05.966+01:00ICSE 2012 RoundupSeveral folks have asked me what I thought of <a href="http://www.ifi.uzh.ch/icse2012/" target="_blank">ICSE 2012</a>. Here's what.<br />
<br />
Full disclosure: ICSE 2012 was sponsored in part by Zuhlke Engineering AG, a sister company of my employer Zuhlke Engineering Ltd, and on that basis I was an “invited industrial speaker” at the conference. However, what I write here are purely my own opinions and do not necessarily represent the opinions or policies of any of the <a href="http://www.zuehlke.com/de/zuehlke-group.html" target="_blank">Zuhlke Group</a> companies.<br />
<br />
Phew. Well, am I glad I went? Yes. Would I go again? Not in a hurry. <a href="http://2013.icse-conferences.org/" target="_blank">Next year is San Francisco</a>—not interested. <a href="http://icse2014.acm.org/" target="_blank">2014 is Hyderabad</a>—very interested!<br />
<br />
To begin at the end, at the closing plenary <a href="http://www.computer.org/portal/web/awards/lionel-briand" target="_blank">Lionel Briand was awarded the IEEE Computer Society's Harlan D. Mills award</a>. In his address he appealed for software engineering research to come out of the shadow of computer science and function as a stand-alone discipline. This crystallised a thought I had half-formed during the conference which was that many of the papers presented did seem a bit like second–rate computer science. And that's not right. Software engineering should no more be cheap'n'cheerful CS than mechanical engineering should be bargain basement physics.<br />
<h3>
</h3>
<h3>
Economics?</h3>
It seems to me that the difference between pure science and applied science is that applied science has to be <i>useful</i> as well as interesting, and the difference between engineering and applied science is that engineering has to <i>make money</i>. A solution is good engineering if it solves the problem at hand and <i>makes a profit</i>. Various people are credited with saying something like<br />
<blockquote class="tr_bq">
an engineer is someone who can do for $STEREOTYPICAL_SMALL_SUM _OF_MONEY what any fool can do for $STEREOTYPICAL_LARGE_SUM_OF_MONEY. </blockquote>
So, let's look at <a href="http://www.ifi.uzh.ch/icse2012/fileadmin/downloads/ICSE2012_conference_program.pdf" target="_blank">the programme of ICSE 2012</a>[pdf] and search for “econmic”. There are 4 hits, all in two of the keynotes. Either <a href="http://www.saskiasassen.com/" target="_blank">Saskia Sassen</a>'s talk on<i> Digital Formations for the Powerful and Powerless</i> (which was met with derision, so far as the conference twitter feed suggests) and Frank–Dieter Clesle's talk on <i>Supporting Sustainability with Software</i>. No papers. Not one.<br />
<br />
Here is a <a href="http://twitpic.com/9rz0f2" target="_blank">word cloud</a> prepared by <a href="http://scg.unibe.ch/staff/adriankuhn" target="_blank">Adrian Kuhn</a> of the words that are in the papers at ICSE. To me, the big words look very closed, very inward–facing, rather self–centred.<br />
<br />
Contrast this with <a href="http://www.asmeconferences.org/Congress2011/TechnicalProgramOverview.cfm" target="_blank">the programme at the ASME 2011 International Mechanical Engineering Congress</a> (at the time of writing the 2012 programme was still being worked out). Here are some sessions titles:<br />
<ul>
<li><blockquote class="tr_bq">
An Integrated Energy, Carbon, and Economic Analysis of Reclaimed Water Use in Austin, Texas</blockquote>
</li>
<li><blockquote>
Economic Feasibility of Nuclear Desalination for Abu Dhabi</blockquote>
</li>
<li><blockquote>
Technoeconomic Analysis of Biomass Gasification System Utilizing Geothermal Steam for Processing</blockquote>
</li>
<li><blockquote>
Energetic and Economic Performance of a Compressed Air Energy Storage
Facility in Texas as a Function of Technical and Cost Parameters</blockquote>
</li>
<li><blockquote>
A System Performance and Economic Analysis of IGCC with Supercritical
Steam Bottom Cycle Supplied with Varying Amounts of Coal and Biomass
Feedstock </blockquote>
</li>
</ul>
And so on. This year's American Institute of Chemical Engineers annual meeting has “economic” in the very strapline, and <i>pages</i> of hits for <a href="https://aiche.confex.com/aiche/2012/webprogram/start.html?words=economic&method=and&filenamerestrict=all&pge=&action=search&source=webprogram&webprogrammode=default&submit=Search#srch=words%7Ceconomic%7Cmethod%7Cand%7Cfilenamerestrict%7Call%7Cpge%7C2%7Caction%7Csearch%7Csource%7Cwebprogram%7Cwebprogrammode%7Cdefault" target="_blank">session abstracts that mention economic issues</a>. Compared to these the ICSE program, for all the good intentions and hard work that have gone in to the material, looks a bit...immature, a bit amateurish.<br />
<h3>
</h3>
<h3>
Papers I Liked</h3>
Laura Plonka, Helen Sharp and Janet van der Linden examined <a href="http://oro.open.ac.uk/33135/" target="_blank"><i>Disengagement in Pair Programming. Does it matter?</i></a> Answer: often yes. They show why your pair can tune out, why that's a problem, and offer some techniques for dealing for it.<br />
<br />
Felienne Hermans, Martin Pinzger, and Arie van Deursen built tools for <a href="http://www.win.tue.nl/ipa/activities/springdays2012/contributions/hermans.pdf" target="_blank"><i>Detecting and Visualising Inter-worksheet Smells in Spreadsheets</i></a> (slides <a href="http://www.slideshare.net/Felienne" target="_blank">here</a>). <a href="https://twitter.com/Felienne" target="_blank">Hermans</a> was one of the stars of the conference, as this <a href="http://www.felienne.com/?p=345" target="_blank">word cloud of tweets about the conference</a> shows. And rightly so. This paper alone is a lovely piece of work. It takes ideas from the world of programmers' programming and applies them to the much wider, and very important, world of end–user, reactive, data–flow oriented programming known as “spreadsheets”.<br />
<br />
David Budgen and Sarah Drummond ask <a href="http://dro.dur.ac.uk/9666/" target="_blank"><i>What Scope is There for Adopting Evidence–Informed Teaching in SE?</i></a> One would hope, quite a lot. This is part of the effort to promote <a href="http://www.dur.ac.uk/ebse/" target="_blank">Evidence Based Software Engineering</a>, the rather radical notion that perhaps software engineers should do what works, <i>because</i> they know it works. The paper provides a survey of SE research literature that might be used to teach students what's known to work.<br />
<h3>
</h3>
<h3>
And the opposite of that</h3>
Conversely, <a href="http://semat.org/?p=398" target="_blank">the SEMAT panel proudly announced</a> that while the SEMAT kernel has yet neither been (1) completed nor (2) tested in industry, it is already being taught to students. This would seem to be the opposite of evidence–informed teaching. Speculation–informed, perhaps. This is enormously disappointing, as is SEMAT.<br />
<br />
While SEMAT is advertised as something like an attempt to reboot (in the “Spiderman” sense) the whole discipline of software engineering (such as it is), it seems to have ended up as an exercise in flogging the same old dead horse. Now with checklists.<br />
<br />
Take a look at the SEMAT logo. See the spring–bow compasses standing in for the letter ‘A’? The compass is a powerful symbol. Compasses are found in<a href="https://www.google.co.uk/search?q=freemason&hl=en&safe=off&prmd=imvnsab&tbm=isch&tbo=u&source=univ&sa=X&ei=gxcLUIjuDMq50QX4m7mxCg&ved=0CHUQsAQ&biw=1096&bih=792&sei=kxcLUOvvA4bF0QX19eC2Cg" target="_blank"> the symbolism of the Freemasons</a>. And of the former <a href="http://en.wikipedia.org/wiki/Coat_of_arms_of_East_Germany" target="_blank">Deutsche Demokratische Republik</a>. According to wikipedia they stand for “precision and discernment” which I guess is what SEMAT want to invoke. They also stand for an idea of what engineering is all about tools, on intermediate work products, on the accidental instrumentality of what engineers do. Here's Alastair Cockburn's <a href="http://alistair.cockburn.us/A+Detailed+Critique+of+the+SEMAT+Initiative" target="_blank">Detailed Critique of the SEMAT Initiative</a> from 2010 which covers a lot of what concerns me about SEMAT very well. Nothing much seems to have improved since then. The ‘T’ in SEMAT is meant to stand for “theory” and a speaker from the floor observed that there didn't seem to be any, yet. I also don't believe that there is any ‘E’–for–“engineering” either. Certainly not in the sense I identified above.<br />
<h3>
</h3>
<h3>
The economics of research</h3>
The researchers may not be paying much attention to economics but it has an influence on them. While I was talking about these papers at a lunchtime
session back at the office my boss, who used to be a full–strength
professor of computer science, observed that my choice of interesting
papers were all from Europe, and that my sense of “2nd rate comp. sci.”
was likely fuelled by American papers.<br />
<br />
American
researchers are driven by their funding structures to pump out a
continual stream of not–very–good papers, the so–called <a href="http://en.wikipedia.org/wiki/Least_publishable_unit" target="_blank">Least Publishable Unit</a>.
Whereas Europeans have a bit more breathing space, and in particular
have the time to, as in the case of Plonka, go spend time with
practitioners, or with people who don't realise that they even are
practitioners, as Hermans did, or to take a long view of the industry,
as Budgen is doing. <br />
<br />
<h3>
A Strange Feeling</h3>
During several of the sessions at ICSE I was struck by a strange, uncomfortable feeling which I subsequently identified as a consequence of having a feminist moment (if you'll allow the conceit).<br />
<br />
Many, but certainly not all, speakers—mainly the 2nd rate CS types—spoke earnestly about these people called “practitioners” but spoke of them in a way which suggested that (1) there couldn't possibly be any of these people in the room and (2) they are <i>other</i>, a strange and mysterious group with peculiar motivations and interests who need to be studied, and handled, carefully and (3) they need someone strong and capable to come and help them out—all in their own best interests, of course. Ah! practitioners! Dear, silly, practitioners! You have to love them but (as <i>we</i> all know, don't we?) they can't really be trusted with important jobs. Too flighty and erratic. Energetic enough in a blundering sort of way, but so muddled and confused.<br />
<br />
It took a while to realise that they meant <i>me</i>.<br />
<br />
<br />keithbhttp://www.blogger.com/profile/14314542307822401015noreply@blogger.com3tag:blogger.com,1999:blog-23912478.post-57021530049993604942011-10-03T17:53:00.002+01:002011-10-04T15:48:08.320+01:00Hiring in London: Lead and Principle Consultants...Principal consultants take responsibility for particularly challenging solutions and in demanding organisational environments. They closely interact with senior project managers, customer representatives at all levels including senior management, and guide project teams. Together with the responsible project managers, they lead technical and strategic initiatives to success, ranging from critical consulting mandates to complex delivery projects. Together with business development and business unit managers, they actively expand Zuhlke’s business and develop new opportunities. This can involve taking the leading technical role in large bid<br /><br />Lead consultants take decisions and provide advice regarding complex technical systems. They closely liaise with the software development team, the project manager, and customer representatives, often with a technical background. They ensure that sound technical decisions are made and subsequently realised in state-of-the-art solutions by the project team. They can take the leading role in technical consulting assignments within their specialisation area.<br /><br />The role is based in London and the majority of the work takes place in UK but on occasion training and consulting engagements may be delivered anywhere in the world.<br /><br />The competitive package includes 20 days of professional development time per year.<br /><br />The successful applicant will be a skilled in iterative, incremental development using at least one of Java/J2SE, C#/.NET and Adobe Flex. They will be skilled at applying test-driven development using both automated unit tests and automated acceptance/functional tests in a continuous integration environment (continuous build and dev-ops experience a plus). <div><br /></div><div>They will be skilled in recognizing code smells and resolving them through refactoring. The successful applicant will be able to elicit requirements as User Stories (experience of “story mapping” is a bonus) and estimation of effort using Planning Poker and Wideband Delphi. They will be skilled in coaching and supporting a development team to effectively self-organize while maintaining transparency and control of rapid development using techniques from Scrum, Kanban, Extreme Programming and other Lean or Agile approaches. </div><div><br /></div><div>Experience in a leading role like Scrum Master is required. They will be experienced in coaching individuals and teams in effective use of current good practices in all the areas mentioned above.</div><div><br /></div><div>Experience delivering classroom training in technical practices such as TDD or build management techniques and a track record of technical publications and conference presentations a plus.<br /><br /><a href="http://www.linkedin.com/jobs?jobId=2029148&viewJob=">Apply via LinkedIn</a>, or drop me a line.</div>keithbhttp://www.blogger.com/profile/14314542307822401015noreply@blogger.com4tag:blogger.com,1999:blog-23912478.post-58625188522623848712011-07-26T19:04:00.001+01:002011-07-26T19:05:41.348+01:00New blog for complexity studyMy continuing investigation into the effect of TDD on code complexity (and maybe other things, too) will continue on<a href="http://cumulative-hypotheses.org/"> this new blog</a>.keithbhttp://www.blogger.com/profile/14314542307822401015noreply@blogger.com0tag:blogger.com,1999:blog-23912478.post-2261407946665378762011-07-25T14:03:00.004+01:002011-07-25T14:34:00.669+01:00Agile has little to say about ProjectsAt the <a href="http://www.agilecoachesgathering.org/wiki/index.php/Home">2011 UK Agile Coaches Gathering</a> at <a href="http://www.bletchleypark.org.uk/">Bletchley </a>last week I convened a session about the desire for (larger) organizations to execute "projects" but the desire of people working in them to use "agile" techniques. I perceive a contradiction there.<br /><br /><div style="margin: 0 0 10px 0; padding: 0; font-size: 0.8em; line-height: 1.6em;"><a href="http://www.flickr.com/photos/keithbraithwaite/5973248423/" title="Agile has little to say about Projects"><img src="http://farm7.static.flickr.com/6142/5973248423_73ee8bbd28.jpg" alt="Agile has little to say about Projects by Keith Braithwaite" /></a><br /><span style="margin: 0;"><a href="http://www.flickr.com/photos/keithbraithwaite/5973248423/">Agile has little to say about Projects</a>, a photo by <a href="http://www.flickr.com/photos/keithbraithwaite/">Keith Braithwaite</a> on Flickr.</span></div><p>For the purposes of the discussion I define a "project" as a management structure with a start date, and end date, a budget and a goal. (It says "goal", but subsequent scribbling makes it look as if it says "goat", prompting one of my colleagues to quip: pick which one you want to sacrifice!)</p><p>My contention is that when people talk about managing Agile work they generally mean Scrum or these days Kanban (or something very much like them) and that these techniques talk mainly about the steady state of product development and how to control that. They have relatively little to say about what happens at the beginning of a "project". How to get one signed off, for example. Kanban seems particularly weak on this. Nor what happens at the end of a "project" when a set of features go into production. Scrum seems particularly weak on that. As a side note, when I first started working with banks it took me a long time to properly grasp that when they talk about the "delivery" phase of a project they don't mean the relatively trivial bit at the front where the programmers write code.</p><p>So, the conclusions of the group were that projects exist as a management structure to mitigate a certain kind of risk in a certain way. As such, I would not agree with <a href="http://twitter.com/#%21/OlafLewitz/status/94781616232218625">this tweet</a> (in reply to one I sent about the very session:</p><p></p><blockquote>The concept [of a project] is superfluous as it does not add business value. </blockquote><a href="http://twitter.com/#%21/search?q=%23project" title="#project" class=" twitter-hashtag" rel="nofollow"><span class="hash"></span><span class="hash-text"></span></a><p></p><p>the concept seems to persist as (some people) believe that having projects allows them to protect money. They might be mistaken: thinking about risk in software development is often wrong, but the idea of a project cannot be dismissed out of hand.</p><p>The group observed that different stakeholders have different concerns at different size and time scales, and these are nested. Scrum and Kanban talk mostly about the innermost few layers of the onion.<br /></p><p>There was a concrete recommendation: when communicating from the inner part of the onion (smaller scale, shorter time frame) to the outer, do not talk about "iterations" or "cadence" or much of anything time based. Talk about money: pay something in this range, get something in that range.<br /></p><p><br /></p>keithbhttp://www.blogger.com/profile/14314542307822401015noreply@blogger.com7tag:blogger.com,1999:blog-23912478.post-86408891797164998252011-07-25T11:16:00.007+01:002011-07-25T14:06:22.987+01:00Agile CharlatansAt the <a href="http://www.agilecoachesgathering.org/wiki/index.php/Home">2011 UK Agile Coaches Gathering</a> at <a href="http://www.bletchleypark.org.uk/">Bletchley </a>last week I convened a session which was prompted by comments like <a href="http://www.reddit.com/r/programming/comments/ge2am/save_your_cleverness_take_the_shortcut_build_the/c1mwim2">this one</a> from reddit user <a href="http://www.reddit.com/user/grauenwolf">grauenwolf</a>:<br /><blockquote>I used to think people like you were quacks, but now I see that there are teams that really need your services.</blockquote>Grauenwolf and I have both been on reddit or a good long while and we've had a few...full and frank exchanges of opinion about the merits of this thing called "Agile". The issues I wanted to address in the session were: what causes competent professionals like grauenwolf to think that the sort of people who would go to something like an "Agile Coaches Gathering" are quacks, and what changes their mind? How might we do less of the former and more of the latter?<br /><br />The output is captured here:<br /><br /><div style="margin: 0 0 10px 0; padding: 0; font-size: 0.8em; line-height: 1.6em;"><a href="http://www.flickr.com/photos/keithbraithwaite/5973249039/" title="The "Agile Charlatan" Problem"><img src="http://farm7.static.flickr.com/6005/5973249039_be90a010a9.jpg" alt="The "Agile Charlatan" Problem by Keith Braithwaite" /></a><br /><span style="margin: 0;"><a href="http://www.flickr.com/photos/keithbraithwaite/5973249039/">The "Agile Charlatan" Problem</a>, a photo by <a href="http://www.flickr.com/photos/keithbraithwaite/">Keith Braithwaite</a> on Flickr.</span></div><p></p>It needs some explanation, though.<br /><br />"Quack" is an evocative word. "Snake-oil salesman". "Charlatan". We felt that these words appeal to an analogy something along these lines: Agile consultants are like bogus medical practitioners.<br /><br />The poster shows a 2-by-2 matrix (naturally!): well or ill against conventional medicine or alternative therapy. The model is that conventional medicine is mainly for people who really are ill and need to get better, while alternative therapy is mainly for people who are basically well but feel entitled to be "well-er".<br /><br />Yes, that is quite cynical. We did talk about the fact that some alternative therapies do work, for the things they work for, but when that happens it's not for the reason that the practitioner thinks. And we talked about how<a href="http://www.cochrane.org/about-us/evidence-based-health-care"> some conventional medicine is not nearly as well founded</a> as many people would like (you) to think. But, bear with me.<br /><br />So, the top right and bottom left cells of the chart are the exemplars. Bottom right is the really bad place: trying to treat a mangled road traffic accident victim with a very, very, very dilute solution of London Bus, for example. Top left is pretty bad too: pushing pills for restless leg syndrome and things even less needing of allopathic intervention.<br /><br />It seems that there is a problem with some kinds of Agile coaching or consulting or training or certification practices. They could be trying to address a really serious organizational problems with techniques that cannot possibly help (and maybe they even know it), or they can be barging in to teams that are functioning perfectly well and crashing about the place changing things that don't need to change.<br /><br />How could we recognise these cases? Maybe this way:<br /><ol><li>conventional medicine often leads to treatment regimes that are quite unpleasant</li><li>people often need to be talked into consulting a conventional medic (partly because of [1])</li></ol>vs<br /><ol><li>alternative therapies are often really rather enjoyable</li><li>people often self-refer to an alternative practitioner (partly because of [1])</li></ol>How to avoid breaking a working team by spuriously making them "Agile"? Watch out for one-size-fits-all solutions. Watch out for inflexibility, a rejection of new ideas (on the part of the consultant or coach).<br /><br />How to increase the chances that you're really helping? Look for objective evidence of improvement, change your behaviour based upon that evidence.<br /><br />The final question we came up with extends the analogy. If Agile coaches are a bit like certain kinds of medical practitioners, then what is the equivalent for us of <span style="font-style: italic;">public health</span>?keithbhttp://www.blogger.com/profile/14314542307822401015noreply@blogger.com5tag:blogger.com,1999:blog-23912478.post-82596196957329965302010-10-25T22:54:00.003+01:002010-10-25T23:24:38.919+01:00No More Than TwoLondon’s “convenience” stores are replacing staffed checkouts with self-service robots. Some of these offer lessons in user interface design painful to behold.<div><br /></div><div>This evening I watched someone attempt to use one to buy paracetamol. The particular self-service robot checkout that I saw presented the user with a message very much like “paracetamol sale: you may only buy two packs” above a yes/no button pair. What is the casual paracetamol buyer to make of this? I watched with interest as my co-emptor pondered this. And then she did the only reasonable thing: she went and got a second pack. After all, it does say you may <i>only</i> buy two packs. </div><div><br /></div><div>Quite what anyone is supposed to make of the yes/no buttons following a statement not a question I don't know.</div><div><br /></div><div>The really sad thing is that, apart from the shame of confusing your customer, the behaviour I saw here—which I believe is the only reasonably response to the message from the robot—is <i>the exact opposite</i> of what is intended. </div>keithbhttp://www.blogger.com/profile/14314542307822401015noreply@blogger.com1tag:blogger.com,1999:blog-23912478.post-8708039790709424862010-05-22T21:19:00.004+01:002010-05-22T21:36:23.030+01:00fixed-length iterations: a transitional practiceI find it had to think of a development practice that isn't almost certainly a transitional practice. Configuration management, maybe. <div><br /></div><div>Anyway, Scrum, XP, and all the rest I've come to understand as each a record of a reaction against some bad condition, with transitional practices to get a team away form that condition to a better one. Fo example, as Michael Feathers says in<a href="http://michaelfeathers.typepad.com/michael_feathers_blog/2010/04/zenolength-iterations.html"> this post</a>, regular, fixed-length Iterations require (and enable) a certain kind of discipline, force a certain set of tradeoffs. And maybe doing that can help a team a lot. And maybe not, as the case may be. </div><div><br /></div><div>That doesn't mean that, once the use of that practice has taken the team away from the bad condition then no further change to practices can be beneficial. In the specific case of regular fixed-length iterations I mainly see the application being to teams moving from a condition where no-one ever has any idea at all when anything is going to be delivered to a condition where everyone always knows a date when something is going to be delivered. In many settings that would be considered a major improvement by those paying the team. And consistently working that way is a great way for a team to gain the trust of the business. </div><div><br /></div><div>Once that trust is established, and once the conditions are in place for frequent delivery, what need for the fixed-length iterations? I suspect that what worries a lot of people who've seen the transition from chaos to iterations is that they can't imagine a world without iterations which is not also a (return to) chaos. Michael suggests an experiment:<blockquote>Suppose that you had an iteration of one week, followed by an iteration of 2 days, followed by an iteration of 1 day, followed by an iteration of one-half a day, and so on. If you still had your sanity at the end of this process, would you have learned anything? I haven’t tried it with a team yet, but here’s the thing that I hope would come across: if you apply enough ingenuity and you’ve acquired enough skill, you can deliver business value in shorter times than you can currently imagine. </blockquote>That would be cool. Of course, the Kanbanista's seem to suggest going straight to that world in one step. And maybe that can work in a certain setting, and maybe not. The idea scares me, whe I look at most of the teams I help.</div><div><br /></div><div>The aspect of this sort of thing that really interests me, though, is this: if what the Certified Scrum Masters say about software development living on the <a href="http://www.controlchaos.com/seminal-articles/">“edge of chaos”</a> is right and if what the <a href="http://en.wikipedia.org/wiki/Cynefin">Cynefin</a> people say about that being exactly the place where “emergent practice” lives then by their own argument, we would expect Scrum to consist of mostly transitional practices.</div><div><br /></div><div>Fixed-length iterations (excuse me, “sprints”) seem like a good candidate to be one.</div><div><br /></div><div><br /></div>keithbhttp://www.blogger.com/profile/14314542307822401015noreply@blogger.com1tag:blogger.com,1999:blog-23912478.post-77911568225795474952010-05-20T15:04:00.003+01:002010-05-20T17:58:23.537+01:00Government IT projects: who can politicians listen to?As you may have heard, the UK has at the time of writing a rather confusing new government. We, and I suspect, they are still trying to understand what this means.<div><br /></div><div>One thing that it might mean is an opportunity for government departments to change the way they deal with their IT suppliers. Recently (ie during the Labour administration we had since 1997 until 2010), it hasn't gone well.</div><div><ul><li>Accenture build a system for the RPA: not fit for purpose, <a href="http://www.zdnet.com/blog/projectfailures/uk-rural-payments-agency-rpa-it-failure-and-gross-incompetence-screws-farmers/536">£46.5 million overspend</a></li><li>BT, Fujitsu and others build NPfIT for the NHS: not fit for purpose, <a href="http://www.zdnet.co.uk/blogs/security-bullet-in-10000166/campaigners-criticise-10bn-nhs-it-overspend-10014501/">£10.4 billion (with a “b”) overspend</a></li><li>Fujitsu build an information system for magistrates courts: <a href="http://www.computerweekly.com/Articles/2009/11/20/239392/More-than-half-of-16319bn-overspend-on-government-projects-due-to.htm">£342 million overspend</a></li><li>Cap Gemini build PRISM for the FCO: not fit for purpose, £34.5 million overspend</li></ul><div>And so on. It has been estimated that the ten worst IT project failures under Labour cost the country around <a href="http://www.eweekeurope.co.uk/news/news-government-it/cost-of-labours-botched-it-projects-exposed-3076">£26 billion</a>. That's half the annual budget for schools. </div><div><br /></div><div>So, a new government setting out to tame a spectacular deficit might want to bring these projects under control. Unfortunately, they get their advice from places such as Fujitsu, the very firms who do so well form these failed projects. <a href="http://www.computing.co.uk/computing/analysis/2263194/labour-legacy-live-haunt">Says Fujitsu's marketing director Simon Carter</a> of discussions held with the Conservatives when they were still the shadow cabinet:<blockquote>[the Conservatives] began to take on some of our suggestions, as they came to better understand government IT. For example, their proposal to cut IT contracts into smaller and shorter chunks was dropped as they realised they would have to act as system integrator to each of these smaller projects.</blockquote>What is the taxpayer to do in the face of this sort of thing? Particularly the well–informed taxpayer who knows full well that in no way whatsoever is this argument from Mr Carter valid. </div><div><br /></div><div>At this year's Spa conference there was a Birds of a Feather session about this issue. Some of the signatories of <a href="http://petitions.number10.gov.uk/ITProcessReview/">this petition</a>—a petition only 62 signatures away, as I write, from the 500 needed for it to have anyone inside 10 Downing Street pay any attention to it. If you are a UK resident and would rather that the new government were not wasting money on entirely avoidable IT project failures I urge you to sign the petition and to urge others to.</div></div>keithbhttp://www.blogger.com/profile/14314542307822401015noreply@blogger.com4tag:blogger.com,1999:blog-23912478.post-57565280846503788712010-05-16T08:59:00.004+01:002010-05-16T09:44:42.291+01:00T-shaped designersBBC 2 is running a series of documentaries called <i>The Genius of Design</i>. Programme 2 is "Design for Living" and discusses, amongst other things, the <a href="http://www.bauhaus-dessau.de/index.php?en">Bauhaus</a> and its influence. It's getting on for a century since the hay-day of the Bauhaus and it's always worth being reminded of the influence it had—got any tubular steel furniture in your house or office? Bauhaus. Any lighting fixtures with push–on/push–off switches? Bauhaus. Got a fitted kitchen? Bauhaus. <div><br /></div><div>The segment on the fitted kitchen was interesting. A fitted kitchen seems like a natural and obvious thing now, but the idea had to be <a href="http://en.wikipedia.org/wiki/Frankfurt_kitchen">invented</a>. The discussion of the Frankfurt Kitchen in the programme was the start of an interesting thread. Users of the kitchen tended to be a bit ill-disciplined. Certainly the tended to disregard the labels permanently attached to the custom–made drawers and put any old thing in them. Users found that the kitchen was built to support well only certain workflows, workflows that they didn't like, didn't understand and couldn't change. Workflows devised by an architect <i>who couldn't cook</i>.</div><div><br /></div><div>Meanwhile, another Bauhaus architect, Le Corbusier, is being given free reign to redesign entire cities, up to the point of making models, anyway. Filling them with great towers full of his “machines for living in”. And <a href="http://en.wikipedia.org/wiki/Pruitt-Igoe">we know how that worked out</a> once people started taking it seriously. </div><div><br /></div><div>If you are a regular reader of this blog you probably know where I'm going next.</div><div><br /></div><div>Commentators on software development often seem to exhibit a lot of discipline envy. Two common themes are that 1) our projects should exhibit the <a href="http://news.bbc.co.uk/1/hi/uk/4735812.stm">reliability</a> of those in the “established” branches of engineering, and 2) our projects should exhibit the conceptual integrity attained by building architects.</div><div><br /></div><div>That conceptual integrity can be a dangerous thing. Lihotzky's kitchens had a lot of conceptual integrity (and a lot of research to back that up), Corb's vision of mass housing (and its implementation by later architects) had a <a href="http://en.wikipedia.org/wiki/Modulor">really astonishing amount of conceptual integrity</a>. Neither leads to much in the way of joy for users (<a href="http://peripateticaxiom.blogspot.com/2010/05/t-shaped-designers.html#corb">*</a>). The Bauhaus architects designed a lot of chairs, none are comfortable to sit in.</div><div><br /></div><div>One of the designers interviewed in the programme explained the problem along these lines: architects tend to be ‘I’ shaped, by which he means they have a very deep knowledge and skill in their craft, but not a lot else going on. Designers tend to be ‘T’ shaped, deep in craft but also with a breadth that touches many other disciplines. And from that breadth comes the ability to design objects that people can comfortably incorporate into their lives.</div><div><br /></div><div>I think that the application of this thought to the software world is clear.</div><div><br /></div><div>(<a id="corb">*</a>) The very few dense housing projects that Le Corbusier himself built have proven to be resilient and popular. It's the shoddy works inspired by his ideas and executed without his art that are the problem.</div>keithbhttp://www.blogger.com/profile/14314542307822401015noreply@blogger.com2tag:blogger.com,1999:blog-23912478.post-13638680673502948242010-04-27T22:37:00.003+01:002010-04-27T22:39:29.603+01:00Tests vs checksTrying to spread the good word on "testing" vs "checking" in <a href="http://www.testmagazine.co.uk/2010/04/automation-your-check-mate/">this article</a> for T. E. S. T. Magazine.keithbhttp://www.blogger.com/profile/14314542307822401015noreply@blogger.com0tag:blogger.com,1999:blog-23912478.post-14083284406092732132010-03-23T07:26:00.004+00:002010-03-23T17:42:27.100+00:00Software Engineering?I wish that the people in the software industry who bang on about a need for "software engineering" showed more evidence of ever having met an engineer. The latest run at it, <a href="http://www.semat.org/bin/view">SEMAT</a>, seems to be making the same old mistakes all over again.<div><br /></div><div>It's not all bad. I'm pleased to see Ivar Jacobson repeat his call to move beyond "process" to "practices". On the other hand, I'm dismayed to see the SEMAT programme described at it's highest level by these streams: <i>definitions</i>, <i>theory</i>, <i>universals</i>, <i>kernel language</i>, <i>assessments</i>. </div><div><br /></div><div>Really? Definitions, theory, universals? Are these really the things that the software industry is lacking? The problem here, I think, is that just as at the original "Software Engineering" conferences in the 60's the SEMAT folks have confused the retrospective coherence with which engineering (that is, the mechanical, chemical, electrical, electronic and other flavours) is described with how engineering is actually done.</div><div><br /></div><div>Similarly, the presence of a spring bow compass in the SEMAT logo worries me. The 60's effort at SE also confused the contingent artefacts produced by engineers (which at the time were actual drawings produced with things like spring bow compasses) with the essence of what engineers <i>do</i>. With this logo SEMAT are associating themselves not just with tools and outputs rather than principles and practices, but with mightily outdated tools. They might as well have put a slide rule in their logo.</div><div><br /></div><div>It really seems as if an historic mistake is about to be repeated. I need to study SEMAT more, but for now <a href="http://alistair.cockburn.us/Alistairs+January+2010+proposed+agenda+for+the+SEMAT+software+engineering+initiative">Alistair Cockburn's commentary</a> resonates strongly with me.</div><div><br /></div><div>He urges that SEMAT does these to things:</div><blockquote><ol><li>Look at what engineers <strong>‘do’</strong>, not what they <strong>build.</strong></li><li>Catch up with the state of the art in what is conventionally called <strong>engineering.</strong></li></ol></blockquote>I can hope.<div><br /></div><div><br /></div>keithbhttp://www.blogger.com/profile/14314542307822401015noreply@blogger.com4tag:blogger.com,1999:blog-23912478.post-69085273415826226202010-02-01T14:18:00.007+00:002010-02-01T16:38:41.671+00:00InnovationLast year <a href="http://www.lukehohmann.com/blog/index.php">Luke Hohmann</a> demonstrated some of his <a href="http://en.wikipedia.org/wiki/Innovation_game">innovation games</a> at <a href="http://xpday-london.editme.com/XTC20091117">an XtC event </a>hosted at the <a href="http://www.zuehlke.com/en/jobs/vacancies/detailview/job/principal-consultantsolutions-architect/jobbpid/142/">Zuhlke Ltd office in London</a>.<div><br /></div><div>One theme was that innovation is different from invention. That's a topic close to my heart. Zuhlke has a division named "Product Innovation" and indeed they innovate like crazy (for example, <a href="http://www.zuehlke.com/uploads/tx_zepublications/pn_344_e_bsr_web.pdf">repurposing the optical sensor for a mouse to better control sewing machines</a> [pdf]) but they rarely <i>invent</i> anything (although they do from time to time, for example <a href="http://www.zuehlke.com/uploads/tx_zepublications/pn_328_e_waterless_urinals_web.pdf">a newly patented waterless urinal trap</a> [pdf]). Knowing a bit about those folks and <a href="http://www.zuehlke.com/fileadmin/pdf/flyers/fl_055_e_pep_zrh.pdf">what they do</a> has helped make me very sensitive to abuses of the term "engineering" as (mis-) applied to software development. But that's a story for another time.</div><div><br /></div><h2>The Shock of the Old</h2><div>I've just finished reading <a href="http://www.amazon.co.uk/Shock-Old-Technology-Global-History/dp/1861973063/">The Shock of the Old</a>. It goes on a bit, it's a bit repetitive, it's highly polemical and a bit repetitive. Pretty good, though. The key observation is that the history of technology as usually presented (for example, in institutions like the <a href="http://www.sciencemuseum.org.uk/">Science Museum</a>) is largely bunk. This history for the most part ignores use and so also ignores folk technology and what Edgerton calls <i>creole</i> technology. He presents numerous case studies to show that what has been made to look like invention is really innovation and diffusion. </div><div><br /></div><div>Quick quiz: when was the first document transmitted by fax? Answer: depending on quite what you think a fax is, <a href="http://www.hffax.de/html/hauptteil_faxhistory.htm">sometime between 1843 and 1865</a>. </div><div><br /></div><div>But the average householder couldn't go and buy a fax machine until about a century later. When faxes became available to the general public that wasn't invention, it wasn't even innovation, it was diffusion.</div><div><br /></div><h2>The Revolutionary Period of Big Innovation</h2><div><a href="http://www.mindviewinc.com/Index.php">Bruce Eckel</a> has come to realize that <a href="http://www.artima.com/weblogs/viewpost.jsp?thread=281005">"software development has stalled"</a>. He says, <blockquote>in recent years it has started to look like we're moving out of the revolutionary period of big innovation, and into a phase of relative stability.</blockquote> I don't believe this for a minute. </div><div><br /></div><div>I think the revolutionary period of big innovation in the tools of programming ended about the time that Sun dropped <a href="http://selflanguage.org/">Self</a>. I'd say that the gold standard development environment for mainstream languages right now is <a href="http://www.eclipse.org/">Eclipse</a>. As <a href="http://en.wikipedia.org/wiki/David_Ungar">Dave Ungar</a> explains towards the end of <a href="http://www.youtube.com/watch?v=3ka4KY7TMTU">this video about the history and influence of Self</a>, Eclipse represents the continuation of the tool-based approach to building a programming environment developed in <a href="http://www.smalltalk.org/main/">Smalltalk</a>. I'd say that Eclipse, even the best-of-breed Java environment built with it, still isn't as good as the best Smalltalk environments for ease of use, productivity <i>and fun</i>.</div><div><br /></div><div>Sun pulled the plug on Self about fifteen years ago. Ironically, they had to buy back the <a href="http://java.sun.com/products/hotspot/whitepaper.html#method">technology for making dynamic dispatch in a dynamically typed language on a VM go fast</a> from <a href="http://www.cs.ucsb.edu/~urs/">Self project staff who had left Sun</a>. </div><div><br /></div><div>What has happened since then is a steady diffusion of features from good object-oriented development environments of the 80's and 90's into the mainstream. Sadly, few features of the very best object-oriented environment of that time (Self, of course) have made it through.</div><div><br /></div><h2>Sources of Diffusion</h2><div>Where else are ideas diffusing from? From <a href="http://sloan.stanford.edu/MouseSite/1968Demo.html">the mother of all demos</a>. Unfortunately, that seam is about worked out. <a href="http://www.vpri.org/html/people/founders.htm">Alan Kay</a> <a href="http://unrev.stanford.edu/introduction/introduction.html">is supposed to have said</a> "I don't know what Silicon Valley will do when it runs out of Doug's ideas." We may be about to find out. Further diffusion (disguised as innovation) in the field of living with computers may well require actual invention.</div><div><br /></div><div>Fortunately, that seam is about worked out. We, the public (at least in high-income countries with stable governments) do now finally live in the world that Englebart invented in 1968.</div><div><br /></div><div>And what of software development tools?</div><div><br /></div><div>Bruce says: <blockquote>no matter how good and powerful our software tools get, we are only getting a fraction of the leverage out of them that we <em>could</em> get.<p><em>Programming tools are no longer where the greatest potential lies.</em></p><p>We will get the biggest leverage, not just in programming but in all our endeavors, by discovering better ways to work together. [emphasis in original]</p></blockquote><div>and I think he's right. I wonder what it is about the world that Bruce works in that has hidden this from him for so long. After all, in 2001 <a href="http://agilemanifesto.org/">one group</a> said:</div><blockquote> We are uncovering better ways of developing<br />software by doing it and helping others do it.<br />Through this work we have come to value:<br /><br />Individuals and interactions over processes and tools [etc]</blockquote></div>Behind the manifesto is a use-based story about the history of technology. Notice that it's <i>uncovering by doing</i>, and not inventing. I'd like to think that Edgerton would approve. It is also a story of diffusion. In <i><a href="http://www.amazon.co.uk/Agile-iterative-development-Craig-Larman/dp/0131111558/">Agile and Iterative Development</a></i><a href="http://www.craiglarman.com/wiki/index.php?title=Main_Page"> Larman</a> quotes <a href="http://www.geraldmweinberg.com/Site/Home.html">Weinberg</a>:<blockquote>We were doing incremented development as early as 1957, in Los Angeles, under the direction of Bernie Dimsdale. He was a colleague of John von Neumann, so perhaps he learned it there, or assumed it was totally natural. [...] the technique used was, as far as I can tell, indistinguishable from XP. [...] much of the same team was reassembled [...] in 1958 to develop Project Mercury, we had our own machine [...] whose symbolic modification and assembly allowed us to build the system incrementally, which we did, with great success. </blockquote><div>What has happened is that techniques that were once restricted to research projects at the cutting edge of <i>a technological nation's story of existential survival</i> are now available to the mainstream.</div><div><br /></div><div>What, I would like to know is, next?</div>keithbhttp://www.blogger.com/profile/14314542307822401015noreply@blogger.com0tag:blogger.com,1999:blog-23912478.post-6063269795232582982010-01-26T13:40:00.004+00:002010-01-26T13:54:42.115+00:00UK Goverment IT Failures<a href="http://www.independent.co.uk/"><i>The Independent</i></a> recently <a href="http://www.independent.co.uk/news/uk/politics/labours-computer-blunders-cost-16326bn-1871967.html">reported</a> on the very, very sorry state of Government IT projects in the UK. It's a mess:<blockquote>[...]the total cost of Labour's 10 most notorious IT failures is equivalent to more than half of the budget for Britain's schools last year. Parliament's spending watchdog has described the projects as "fundamentally flawed" and blamed ministers for "stupendous incompetence" in managing them.</blockquote><div>I thought that the article missed a trick and wrote to tell them so. My letter didn't make it to the paper paper, but has appeared on <a href="http://opinion.independentminds.livejournal.com/1628420.html">their blog</a>. It's somewhat hidden, though, so I reproduce it here. </div><div><br /></div><div>It would be easy to conclude that large government IT projects fail primarily for technical reasons. Technical mistakes certainly are made, but the root cause of many failed projects seems to be the procurement process with which they begin. The question which is asked of the suppliers is not one that allows for the kind of answer that leads to success. It is well known in the private sector that very large monolithic projects have little chance of success, but this is the only kind for which the Civil Service seems to know how to ask.</div><div><p>I have no doubt that ministers are, as <a href="http://muckrack.com/michaelsavage">Michael Savage</a> puts it, "too easily wooed by suppliers". The suppliers which we see winning government contracts again and again might also find it too easy to woo a minister. Particularly a minister and a department which would rather launch a high-profile, high-budget, high-risk project than adopt the smaller scale, incremental approach that creates few headlines when launched, but also has a much better chance of creating few headlines when it does not fail.</p></div>keithbhttp://www.blogger.com/profile/14314542307822401015noreply@blogger.com0tag:blogger.com,1999:blog-23912478.post-79765756287498443072010-01-22T03:17:00.004+00:002010-01-22T05:19:30.214+00:00It's the ScreamingTim Bray <a href="http://www.tbray.org/ongoing/When/201x/2010/01/02/Doing-It-Wrong">observes</a>: <blockquote>The community of developers whose work you see on the Web, who probably don’t know what ADO or UML or JPA even stand for, deploy <em>better systems at less cost in less time at lower risk</em> than we see in the Enterprise. [emphasis in original]</blockquote>That's a pretty bold claim. <div><br /></div><div>Oh, certainly, the Enterprise makes a botch of IT projects on a titanic scale, and Tim gives a partial list of very high profile failures. I think this data is tainted. Firstly, these project failures (such as the UK NHS <a href="http://en.wikipedia.org/wiki/NHS_National_Programme_for_IT#Criticisms_of_the_programme">NPfIT</a> programme) are, well, very high profile—but they aren't representative. Plenty of corporate IT projects do very OK. Secondly, the specific examples Tim gives are actually government projects. Government IT is the <i>reductio ad absurdum</i> of Enterprise IT. These might not be the most informative examples to look at.</div><div><br /></div><div>On the other side of the matter, Tim cites the successes of Facebook, Google and Twitter. These are also not representative. Quickly, name three failed web 2.0 startups...that's a hard ask because those startups that don't succeed sink without trace. Which, if you believe in the <a href="http://ycombinator.com/about.html">Y Combinator</a> argument, is the point. A web 2.0 startup needs so little initial investment that western civilisation can afford to throw them away in bulk without noticing. The result is a sort of <a href="http://en.wikipedia.org/wiki/Monte_Carlo_method">Monte Carlo method</a> for product innovation. I wonder how that's working out for them now that money is no longer so cheap.</div><div><br /></div><div>But still, something <i>is</i> very wrong with enterprise IT. Would the "web 2.0" way of working help?</div><div><br /></div><div>Maybe, maybe not. I think Tim has made the mistake of assuming that a technique that he's seen work in one context will necessarily work in another. And should be deployed there post-haste. The IT industry collectively makes this mistake about every seven years.</div><div><br /></div><div>Some commentators have noted some of the differences between the Web and the Enterprise. In summary: web 2.0 startups don't have to integrate with a heterogeneous ecosystem of legacy systems older than the combined ages of their founders. Web 2.0 startups don't have to operate within multiple tight and complex regulatory regimes. Some were even so unkind as to observe that <a href="http://en.wikipedia.org/wiki/Tim_Bray#XML">Tim is one of the devisors of XML</a> and so some of the chest–deep mud that Enterprise developers have to slog through is his fault.</div><div><br /></div><div>I think there are two more issues that Enterprise groups have to deal with that web 2.0 startups don't, and don't have a good story for.</div><h2><br /></h2><h2>Scale</h2><div>I'm going to say that enterprise IT faces a much tougher scale problem than do web developers. Part of the web 2.0 model is that the initial user base is the founders plus their <a href="http://en.wikipedia.org/wiki/Dunbar's_number">immediate circle</a>. The problem that startups face is that if they are one of the (very) few that make it, their user base may grow very quickly. Their problem is <i>scaleability</i>. This is a tough problem.</div><div><br /></div><div>The problem for the Enterprise is <i>scale</i>.</div><div><br /></div><div>Citigroup, Bank of America and JP Morgan Chase all have <a href="http://www.bankspider.com/topbanks.php?sel=employees">slightly less than a quarter of a million employees</a>. If you are deploying a "corporate IT app" at one of those organisations then you have to plan for all quarter of a million of them to hit your app at 9am their local time tomorrow. And all day and everyday thereafter. That's a different problem. And by the standards of corporate IT this is no–where near as bad as it can get.</div><div><br /></div><div>You know that NPfIT project that Tim holds up as an exemplar of corporate IT failure? In 2008 the NHS had <a href="http://www.ic.nhs.uk/statistics-and-data-collections/workforce/nhs-staff-numbers/nhs-staff-1998--2008-non-medical">1.12 million non–medical staff</a>, <a href="http://www.ic.nhs.uk/statistics-and-data-collections/workforce/nhs-staff-numbers/nhs-staff-1998--2008-medical-and-dental">99,000 medics and dentists</a> and <a href="http://www.ic.nhs.uk/webfiles/publications/nhsstaff2008/gp/Bulletin%20Sept%202008.pdf">34,000 General Practitioners</a>. Imagine that they are all going to start using your app tomorrow. To help them treat patients. Of which there are several million a week handled by NHS services. Putting up the <a href="http://fredericiana.com/2009/08/01/why-wikipedia-might-need-a-fail-pet-and-why-mozilla-does-not/">fail whale</a> isn't going to cut it.</div><div><br /></div><div>And that's the real problem that corporate IT have the deal with: <i>the screaming</i>.</div><h2><br /></h2><h2>Screaming</h2><div>Let's say that your web 2.0 startup thingy that you put together in record time in your buddy's mum's spare room falls over. What's the worst thing that can happen? A few snarky tweets? The odd complaining blog post? The whole point of your business model is that your service (and company) are disposable, at least in the early days. It's not as if you have a revenue stream to protect. As if!</div><div><br /></div><div>Now lets imagine what happens when your FX app you just deployed into Mammon Inc's trading floors falls over. What happens then is that a head of desk in Farawayvia phones you up at 3 am your time and <i>will not stop screaming at you</i> until you get it fixed. I exaggerate for comic effect, but not by much. If you are a Web 2.0 type then you have the luxury of fixing stuff up in something like your own time. If you are a Corporate IT type then you might well have someone who believes that you, personally, are trying to steal their <a href="http://www.telegraph.co.uk/finance/newsbysector/banksandfinance/6958528/Record-bonus-pot-at-JP-Morgan.html">bonus</a> from them raging at you until you get the damn thing working again. </div><div><br /></div><div>Or that you, personally, are trying to kill their patient.</div><div><br /></div><div>If you were a founder of a web startup and the users of your service had your personal mobile number and believed that five minutes of downtime could cost them personally millions of bottletops (the currency of Farawayvia, ticker symbol BTP), or lead them to a malpractice suite over a dead patient, do you think you would behave the same way that other founders do?</div><h2><br /></h2><h2>Solutions?</h2><div>So, what's the answer? I don't there is any "the answer". Not a technical one, anyway. There are a bunch of tools and techniques that I've come to think will help a lot, and I encourage my clients who happen to be in corporate IT to use them. I fact, Tim mentions some of them in his post. He notes that startups get a lot of help from:<blockquote> dynamic languages and Web frameworks and TDD and REST and Open Source and NoSQL at varying levels of relative importance</blockquote> Too right. But, as <a href="http://www.geraldmweinberg.com/Site/Home.html">Gerry Weinberg</a> says: it's always a people problem. It always is. It's the screaming. </div><div><br /></div><div>Get people into a place where they believe they can do the right thing and they mostly will. The technology can then, to a surprising extent, look after itself. This is the really big advantage that startups have, I think: no–one to tell them not to do the right thing. It's not that corporate IT is full of people saying "don't do the right thing" (although in the worst case that does happen). Rather it's that the social context inside any organisation big enough to call itself an "enterprise", without any actual malice on the part of any individual, works against the right thing.</div><div><br /></div><div>I think it was <a href="http://xprogramming.com/blog/">Ron Jeffries</a> that I first heard say that most of the advice given to software developers to help them do better is about as useful as telling them to be taller. Telling folks who work in corporate IT to behave more like the folks in web startups seems no more useful than that to me.</div><div><br /></div><div>On the other hand, if one does end up working in a really toxic environment that does work against the right thing it can be worth looking around. Look around to see who, exactly, is holding the gun to your head to make you put up with it. </div><div><br /></div><div><br /></div>keithbhttp://www.blogger.com/profile/14314542307822401015noreply@blogger.com7tag:blogger.com,1999:blog-23912478.post-79142390970645790262010-01-17T20:33:00.104+00:002010-04-05T20:38:23.650+01:00Complex Domains: playing with AlloyBefore digging in to more Bayesian ideas I want to take a step back. The <a href="http://peripateticaxiom.blogspot.com/2009/12/bayesian-testing.html#example">example in this previous post</a>, which I worked through with Laurent, has an interesting property: Each subsequent test halves the remaining uncertainty about the correctness of the system under test. This is a property of the problem domain, not the solution. For a next example I want to look at a more complex domain. But what makes a domain complex (in the sense of <a href="http://www.bigvisible.com/asroka/essential-complexity-part-one/">essential complexity</a>)? That's not a rhetorical question.<div><br /></div><div>And what tools are appropriate to handling a complex domain? Let me tell you a story... (if you want to skip the story and go direct to the techie stuff, it's <a href="#techie">here</a>)</div><div><br /></div><h2>Microcell Predictor</h2><div>Back in the day I worked on a tool used by radio engineers to design mobile phone networks. </div><div><br /></div><div>In particular I worked on a so–called "microcell predictor". This would take a description of a dense urban environment and a proposed low–power base station location and calculate the expected signal strength at various points in the area. The input was a file containing a bunch of polygons describing building footprints and some materials data (steel and glass, masonry, etc) and the base station location and properties (antenna design and so forth). The output was a raster of predicted signal strengths. This could overlay the building polygons and generate a map that the engineers could first eyeball and then if necessary analyse more closely to help them optimise the base station placement. This was a lot faster and cheaper than putting up a temporary antenna and then driving around in a vehicle with a meter measuring the actual signal strength, which was the way the very first networks were planned.</div><div><br /></div><div>The requirement for this came from the radio specialists in the form of pages of maths describing various "semi-empirical" models of microwave propagation and how these interacted with buildings. Let's say we are looking at GSM 900. If a 900 MHz microwave photon were moving in free space it would have a wavelength of approximately 3×10<sup>9</sup> ms<sup>-1</sup>/9×10<sup>8 </sup>s<sup>-1 </sup>or around 33mm. This makes such photons quite good at seeming to go around the corners of "human scale" structures by diffraction. To calculate that exactly would be very messy and on <a href="http://obsolyte.com/sun_ss5/">the boxes we used</a>, impracticable. So we had these other methods which hid a lot of the details and gave results that the radio experts deemed good enough and that we could compute with. The input didn't have to be especially large or complicated for the prediction to take long enough that the user would give up, but the point of microcells is that they only cover a small area in a city centre anyway so that was OK. This was fifteen years ago, the techniques used now are a lot more sophisticated.</div><div><br /></div><h2>Semi-formal</h2><div>We used a development process called <a href="http://www.syntropy.co.uk/syntropy/">Syntropy</a>. It's an unusual day in which if I spend any time at all thinking about software I don't use ideas from Syntropy to good effect. Amongst other things Syntropy combines a graphical notation for object structures much like <a href="http://en.wikipedia.org/wiki/Object-modeling_technique">OMT</a> with a textual notation for facts about them much like <a href="http://en.wikipedia.org/wiki/Z_notation">Z</a>. Some (but not nearly enough) of these ideas made it into UML, particularly the <a href="http://en.wikipedia.org/wiki/Object_Constraint_Language">OCL</a>. </div><div><br /></div><div>So, we had these mathematical requirements and we produced from them mathematically supported specifications and designs, full of ∀'s and ∃'s, and we had to turn these into working software. I learned a great deal about the art of doing that there, other parts of which are a story for another time. </div><div><br /></div><div>The main thing for my current purpose, though, is that when I think back to those times I'm stunned by the amount of effort we put into determining if those specifications and designs were <i>correct</i>. The only way we knew how was to round up a bunch of seriously smart people (which luckily we had) and check these models manually. Management were smart about it and paid for us to be trained in <a href="http://en.wikipedia.org/wiki/Fagan_inspection">Fagan inspection</a> techniques, which helped a lot. But the expense! six or eight top-flight programmers in a room for a couple of hours is not a trivial investment. And to do that many times per document. Sometimes many times per page of a document, over the years that we worked on this thing. </div><div><br /></div><div>But that was then. As it happens, more–or–less exactly <i>then</i> another group in the UK were using much more advanced formal methods to address a much trickier problem.</div><div><br /></div><h2>This is Now</h2><div>In <a href="http://www.spaconference.org/spa2007/programme.html">2007</a> Sir <a href="http://en.wikipedia.org/wiki/C._A._R._Hoare">Tony Hoare</a> delivered a keynote at the <a href="http://www.spaconference.org/">Spa conference</a>. He talked about the effort required to prove (really, <i>prove</i>) that the <a href="http://www.mondex.com/">Mondex</a> electronic money system was secure. The thing about Mondex is that the money is actually on the card, rather than being in the network with the card acting as a credential to allow the money to be moved. This made the Bank of England <a href="http://www.bankofengland.co.uk/education/museum/exhibitions/past2.htm">very nervous</a> (Mondex was developed in the UK). Developing that 200 page proof was very expensive. <a href="http://www.springerlink.com/content/w5x622t55187u2l5/">This effort</a> has become something of a <a href="http://vsr.sourceforge.net/Activities.htm#The%20Mondex%20case%20study">celebrity</a> amongst the Verified Software community. </div><div><br /></div><div>The folks who worked on the Mondex proof were, almost certainly, much smarter than my colleagues and I who worked on the microcell predictor (sorry guys), but they seem not to have known a better way to proceed than manual checking, either. In fact they, so they say, they said at the time<blockquote>mechanising such a large proof cost–effectively is beyond the state of the art</blockquote></div><div>Hoare's keynote explained that between then and now, in fact in that year 2007, the problem had been re–addressed. The goal was to discover to what extent the state of the art had moved on in ten years and whether mechanisation had become cost–effective. Hoare suggested strongly that through improvements in theory and hardware that cost–effectiveness is within reach.</div><div><br /></div><div>One of the things that came out of that effort was a <a href="http://vsr.svn.sourceforge.net/viewvc/vsr/projects/mondex/alloy/">model</a> written in <a href="http://alloy.mit.edu/community/">Alloy</a> (note: the link was dead at the time of writing but the Alloy site <i>is</i> actively maintained).</div><div><br /></div><div>Alloy is actually what I wanted to write about. Alloy seems to live at an interesting place: the intersection of <i>proof</i> and <i>examples</i>. What Alloy does is help you develop a proof of various properties of a specification by on the one hand generating examples (if the specification is consistent) or counter–examples (if it isn't).</div><div><br /></div><h2>Testing</h2><div>Over time, and quite naturally, our focus changed while working on the microcell predictor. We became less interested in demonstrating that our code conformed to a design that conformed to a specification that conformed to a requirement. We became more interested that the code conformed to the users' needs. We showed this through intensive automated testing. </div><div><br /></div><div>My boss at the time insisted that we write fully automated tests for every function we wrote. He had an automated testing framework that he carried around in his head and regenerated at each new place of work. I think he had learned this from previous boss of his and the framework had, IIRC, originally been written in Pascal. So we crated a C++ version and off we went writing tests and I can't begin to mention the number of times that writing the tests, and running them, again and again and again, was crucial to overcoming what would otherwise have been show–stopping problems.</div><div><br /></div><div>It was especially interesting that one of the guys on the team, a real code–basher and much better programmer than I am, built a graphical test runner (polygons, remember) that let you see what the code was doing as a test ran. See the building footprint polygons, see the triangulation of the line–of–sight region, see the first–order rays from the antenna to the corners of buildings, see the second–order virtual sources, see the triangulation of their line–of–sight, and so on. See it in all these various scenarios, each devised specifically to check that some particularly interesting feature of the <i>problem</i> was dealt with correctly. At one time I had several sheets of big flipchart paper covered in the tiniest writing I could manage describing all the ways I could think of that a line segment could meet a set of polygons. I missed a few.</div><div><br /></div><div>Something like Alloy would have helped <i>so much</i>.</div><div><br /></div><div>These animated tests became the premier way of explaining what the microcell predictor did. Even to customers.</div><div><br /></div><div>Notice that my work on microcells, with the intensive automate testing, and the original Mondex proof took place more–or–less contemporaneously with the discovery of Extreme Programming. I think I recall a lunchtime conversation during the microcell work to the effect that there was this <a href="http://c2.com/cgi/wiki?ChryslerComprehensiveCompensation">mad project</a> going on where they had automated tests for everything (as we did), but they <i>wrote the tests first!</i> I think I recall some comment along the lines that this was fine only so long as you knew the requirement in great and final detail, but in practice you never do. I'm now pretty confident that the converse is true: while we hardly ever do have great and finally detailed requirements, this is exactly when writing the tests first does help. </div><div><br /></div><div>I'm glad that I dropped off the "models" path to correctness and onto the "test first" one. And I'm glad that I had the experience of doing the "models" approach. I find it interesting to look back over the fence sometimes, and see how those folks are getting along.</div><div><br /></div><h2>Alloy</h2><div>And so to Alloy<A id="techie"></A>. I have Alloy 4.1.10 here on my MacBook Pro (2.4 GHz Core 2 Duo, 4GB ram). I'm going to try to develop a formal model of the points on a Goban. If you'd like to play along, there's <a href="http://bitbucket.org/keithb/goban-alloy/">an hg repo</a>.</div><div><br /></div><h3>Points</h3><div>Alloy models are essentially relational although the syntax is deliberately chosen to be as familiar as possible to users of "curly bracket" OO languages. I begin by writing a kind of test. This takes the form of a predicate called <code>correct</code> which says<pre>pred correct {<br /> there_are_such_things_as_points<br />}</pre> and I can ask Alloy to run this test<pre>run correct</pre>and Alloy tells me that <code>The name "there_are_such_things_as_points" cannot be found.</code> which is excellent news. I'm well on the way to using the familiar TDD cycle. Not compiling is failure and here is a failing test. I can make the test fail in a slightly more informative way by defining <code>there_are_such_things_as_points</code> like so<pre>pred there_are_such_things_as_points{<br /> #Point > 0<br />}</pre>which says that the size of the set named <code>Point</code> (which is the set of all tuples conforming to the signature <code>Point</code>—it's a relational model, remember) is strictly greater than zero. Of course I haven't defined that signature yet so Alloy tells me that <code>The name "Point" cannot be found.</code> I define <code>Point</code> like so<pre>sig Point {}</pre>and now Alloy reports that</div><div><br /></div><div><code>Executing "Run correct"</code></div><div><code><span class="Apple-style-span" style="white-space: pre; "> </span>Sig this/Point scope <= 3</code></div><div><code><span class="Apple-style-span" style="white-space: pre; "> </span>Sig this/Point in [[Point$0], [Point$1], [Point$2]]</code></div><div><code><span class="Apple-style-span" style="white-space: pre; "> </span>Solver=minisatprover(jni) Bitwidth=4 MaxSeq=4 SkolemDepth=2 Symmetry=20 </code></div><div><code><span class="Apple-style-span" style="white-space: pre; "> </span>18 vars. 3 primary vars. 23 clauses. 183ms.</code></div><div><code><span class="Apple-style-span" style="white-space: pre; "> </span>Instance found. Predicate is consistent. 41ms.</code></div><div><br /></div><div>There's a lot of information there. The important part for now is that Alloy could find an <i>instance</i> (that is, a bunch of tuples) that conforms to the model and of which the predicate is true. Therefore the model and the predicates I have defined are <i>consistent</i> (that is, contain no contradictions). I can also ask Alloy not to run my predicate but instead to <i>check</i> it. The news here is not so good.</div><div><br /></div><div><code>Executing "Check check$1"</code></div><div><code><span class="Apple-style-span" style="white-space: pre; "> </span>Sig this/Point scope <= 3</code></div><div><code><span class="Apple-style-span" style="white-space: pre; "> </span>Sig this/Point in [[Point$0], [Point$1], [Point$2]]</code></div><div><code><span class="Apple-style-span" style="white-space: pre; "> </span>Solver=minisatprover(jni) Bitwidth=4 MaxSeq=4 SkolemDepth=2 Symmetry=20 </code></div><div><code><span class="Apple-style-span" style="white-space: pre; "> </span>18 vars. 3 primary vars. 24 clauses. 11ms.</code></div><div><code><span class="Apple-style-span" style="white-space: pre; "> </span>Counterexample found. Assertion is invalid. 18ms.</code></div><div><br /></div>It turns out although my model is consistent it is not <i>valid</i>. It is possible to construct instances of the model for which the predicate is not true.<div></div><div><br /></div><div>Notice the lines in the reports about <code>sig this/Point</code>. In working with my model Alloy has made some instances of <code>Point</code> form which it has then constructed instances of the model. By default it chooses to make up to 3 instances of a signature. Here is a graphical (in both senses) representation of the instance of the model which Alloy built</div><a onblur="try {parent.deselectBloggerImageGracefully();} catch(e) {}" href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi4aNE7gc24bGZq9fVsBSxGNm50W3xFS5hfJV7MgqrDHpCjjdkSYEj3gxHIi4V3NpKRDvARMF3TTp6CfiE0i_nfL3DD6Sl086p3ZIC1BkMQ_bA8XeIaIu_XXbKbJwVcrH1H71Dwsg/s1600/instance.png"><img style="display:block; margin:0px auto 10px; text-align:center;cursor:pointer; cursor:hand;width: 92px; height: 66px;" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi4aNE7gc24bGZq9fVsBSxGNm50W3xFS5hfJV7MgqrDHpCjjdkSYEj3gxHIi4V3NpKRDvARMF3TTp6CfiE0i_nfL3DD6Sl086p3ZIC1BkMQ_bA8XeIaIu_XXbKbJwVcrH1H71Dwsg/s200/instance.png" border="0" alt="Point$0" id="BLOGGER_PHOTO_ID_5456636386277868834" /></a>Do you see this instance named in the array of three instances which Alloy reported it had created? Clearly the predicate is satisfied. Alloy will also produce a graph of the counterexample which it found—which is empty. (Well, strictly it's a message telling me that "every atom is hidden" in a "this page intentionally left blank" sort of way).<div><br /></div><div>There is nothing in the model which says that there are any points, only that there possibly is such a thing as a <code>Point</code>. The problem domain can help us here, as it turns out that some of the points on the board have <a href="http://senseis.xmp.net/?Tengen">names</a>. </div><div><pre><br />pred tengen[p : Point]{}<br /><br />fact tengen_exists {<br /> one p : Point |<br /> tengen[p]<br />}</pre>Here I state a fact, which is very much like a predicate, except that it is information for Alloy to use not a question for it to ask of the model. </div><div><br /></div><div>Read the fact <code>tengen_exists</code> like this: "it's true of exactly one instance, named <code>p</code>, of the signature <code>Point</code> that the predicate <code>tengen</code> is true of <code>p</code>". The predicate itself is parameterised on an instance of <code>Point</code> but does not depend upon that instance. Which seems as if it should smell.</div><div><br /></div><div>Running the predicate as before finds that same instance of the model with one point in it. I can pop up an evaluator on that instance and ask for the value of <code>tengen[Point$0]</code> which is (of course) <code>true</code>. If I ask Alloy to check the model it now reports <code>No counterexample found. Assertion may be valid. 69ms. </code>Note the "may be" there. Alloy can't be absolutely sure because it only instantiates a small number of tuples for each signature. This is a manifestation of the Small Instance Hypothesis (sometimes "small model" or "small scope") which claims that if your model is bogus then this will show up very quickly after looking at a small number of small examples—exhaustive enumeration of cases is not required.</div><div><br /></div><div>So now I have a model, however feeble, which is consistent and cannot be shown (using up to three <code>Point</code>s) to be invalid. I'll check in.</div><div><br /></div><h3>Refinement</h3><div>I'm not very happy with this model. I said that some points are named, such as tengen, but that's not really very well expressed. There's that smelly predicate which doesn't depend upon its parameter. If some instances of Point have names, then we can say that. After going around the fail-pass loop (trust me, I am doing that but I'm not going to write it out every time) the model looks like this</div><pre>enum Name { Tengen }<br /><br />sig Point {<br /> name : lone Name<br />}<br /><br />pred tengen[p : Point]{<br /> p.name = Tengen<br />}<br /><br />fact tengen_exists {<br /> one p : Point |<br /> tengen[p]<br />}</pre>Several new Alloy features are used here. Since Alloy 4 doesn't support string literals I use an atom (an instance of a signature with no further structure). The <code>enum</code> clause creates quite a complex structure behind the scenes but gets me the atom <code>Tengen</code>. The signature of <code>Point</code> is extended to have a field named <code>name</code> which will, in a navigation expression such as <code>p.name</code> resolve to an instance of signature <code>Name</code>, or to <code>none</code>, as shown by the cardinality marker <code>lone</code>. These navigation expressions look like dereferencing as found in OO languages, but are actually joins.<div><br /></div><div>This looks a lot healthier to me, and the model is still both consistent and not demonstrably invalid. Here's the new instance of the model. I'll check in.</div><a onblur="try {parent.deselectBloggerImageGracefully();} catch(e) {}" href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEitW9RC4EpvGacf1Rj3U8vTqhAIB6GH6F7Tjks2MNVnaOnuNvFHvRjGC6e5JwOEdRJMZE1SHbabECQ9VHHUy-9gU6K-lUOj0Wd2UT4pbl0qEuhVg3BOsOe2Dy9NI3OXzCKtLWBlZQ/s1600/instance.png" style="text-decoration: none;"><img style="display:block; margin:0px auto 10px; text-align:center;cursor:pointer; cursor:hand;width: 97px; height: 163px;" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEitW9RC4EpvGacf1Rj3U8vTqhAIB6GH6F7Tjks2MNVnaOnuNvFHvRjGC6e5JwOEdRJMZE1SHbabECQ9VHHUy-9gU6K-lUOj0Wd2UT4pbl0qEuhVg3BOsOe2Dy9NI3OXzCKtLWBlZQ/s200/instance.png" border="0" alt="" id="BLOGGER_PHOTO_ID_5456649947323609090" /></a><h3>Directions</h3><div>There are many points on a goban, however. And these points stand in a certain relationship. Specifically, every point on the board has some neighbours. Tengen has four neighbouring points, one in each of the four directions I will call N, E, S and W. I start with N and obtain this <i>invalid</i> model <pre><br />enum Name { Tengen }<br /><br />sig Point {<br /> name : lone Name,<br /> neighbour : Direction -> lone Point<br />}<br /><br />pred tengen[p : Point]{<br /> p.name = Tengen</pre><pre>}<br /><br />fact tengen_exists {<br /> one p : Point | tengen[p]<br />}<br /><br />enum Direction {N}</pre> which admits a counterexample which does not satisfy this predicate<pre>pred tengen_has_a_neighbour_in_each_direction{<br /> let tengen = {p : Point | p.name = Tengen} {<br /> not tengen.neighbour[N] = none<br /> }<br />}</pre> The counterexample looks like this<a onblur="try {parent.deselectBloggerImageGracefully();} catch(e) {}" href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjnlTmJvBCa5wcPQcY39yc_m6Z-nezzzv3XF5vi6HNCYqlX3dA_c5EIXV1SgYmUcKvNnq0KHwAVLECPZBVnAt23qO40MlLg2hDx5c0hl8RucvFGh8HMgyyu-G8VBpemABpW8ByM7A/s1600/counter_example.png"><img style="display:block; margin:0px auto 10px; text-align:center;cursor:pointer; cursor:hand;width: 159px; height: 166px;" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjnlTmJvBCa5wcPQcY39yc_m6Z-nezzzv3XF5vi6HNCYqlX3dA_c5EIXV1SgYmUcKvNnq0KHwAVLECPZBVnAt23qO40MlLg2hDx5c0hl8RucvFGh8HMgyyu-G8VBpemABpW8ByM7A/s200/counter_example.png" border="0" alt="" id="BLOGGER_PHOTO_ID_5456668690600929314" /></a><div>New Alloy features are the mapping <code>Direction -> lone Point</code> which is pretty much the same as a typical "dictionary" and the <code>let</code> form and its binding of the name <code>tengen</code> to the value of a comprehension. The comprehension should be read as "the set of things <code>p</code>, which are instances of <code>Point</code> of which it is true that the value of <code>p.name</code> is equal to <code>Tengen</code>" Some sugar in Alloy means that we don't need to distinguish between a value and the set of size 1 who's sole member is that value.</div><div><br /></div><div>Running the model produces the somewhat surprising result that it is consistent. Looking at the example shows that this is a red herring. <a onblur="try {parent.deselectBloggerImageGracefully();} catch(e) {}" href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhh-bt57qOFLwMaUj_GaQocBhc9CbqLzHwidjHW1zdNt93Mb61_GMMjdwB3Ryu5X_hzksn_I5iMMQopzlrO5FFJTq_yVKg7h75-VAyjGMiOjG22BBjH55hXoFePEnae09WW_NUXyA/s1600/instance.png"><img style="display:block; margin:0px auto 10px; text-align:center;cursor:pointer; cursor:hand;width: 200px; height: 155px;" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhh-bt57qOFLwMaUj_GaQocBhc9CbqLzHwidjHW1zdNt93Mb61_GMMjdwB3Ryu5X_hzksn_I5iMMQopzlrO5FFJTq_yVKg7h75-VAyjGMiOjG22BBjH55hXoFePEnae09WW_NUXyA/s200/instance.png" border="0" alt="" id="BLOGGER_PHOTO_ID_5456673069017567810" /></a>This is an interesting state of the world, so I check in with a suitable caveat in the message.</div><div><br /></div><div><h3>A YAGNI Moment</h3></div><div>The invalid aspect of the model comes from the cardinality on <code>Point.direction</code>. There are points on a Goban which do not have four neighbours, one in each direction. But I haven't mentioned any of them yet. There's a good chance that eventually points will need to have optional neighbours, but right now YAGNI.<br /><br />As the (so far, incomplete) predicate's name suggests, tengen really does have four neighbours. The cardinality should be <code>one</code>. Making that change produces a model which no cannot be shown to be invalid. However, the example instance is still bogus. I know from TDD practice what to do: write a test that will fail until the problem is fixed. Here it is<pre>pred points_are_not_their_own_neighbour {<br /> all p : Point |<br /> not p in univ.(p.neighbour)<br />}</pre>The construction <code>univ.r</code> for any relation <code>r</code> evaluates to the range of the relation.<br /><br />As I hoped, this test fails. Although running the predicates can produce an instance in which the northerly neighbour of tengen is not tengen, checking can also still produce an invalidating counterexample in which it is. I must add a predicate to apply to all <code>Point</code>s forcing them not to be their own neighbour<pre><br />sig Point {<br /> name : lone Name,<br /> neighbour : Direction -> one Point<br />}{<br /> not this in ran[neighbour]<br />}</pre>Here I use the function <code>ran</code> imported from the module <code>util/relation</code> to state the constraint on the range of <code>neighbour</code>. The conjunction of the predicates listed in curleys immediately after a sig are taken as facts true of all instances of that signature. The model is now consistent and not demonstrably invalid, but a glance at the example reveals that all is not well<a onblur="try {parent.deselectBloggerImageGracefully();} catch(e) {}" href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgnYHbIGirGC3Gh_kSbPH1QC1jN6SW8BZ86-34wfmBSuJaTu29ajLluRkCA5IYes1g3bHxB5RwbCziuOl7y-HBSzFEYobBQGephLZTlwzZeGcd6YVHuHyKeb1FJ1xJrhbl5fisq8Q/s1600/instance.png"><img style="display:block; margin:0px auto 10px; text-align:center;cursor:pointer; cursor:hand;width: 200px; height: 140px;" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgnYHbIGirGC3Gh_kSbPH1QC1jN6SW8BZ86-34wfmBSuJaTu29ajLluRkCA5IYes1g3bHxB5RwbCziuOl7y-HBSzFEYobBQGephLZTlwzZeGcd6YVHuHyKeb1FJ1xJrhbl5fisq8Q/s200/instance.png" border="0" alt="" id="BLOGGER_PHOTO_ID_5456681048624326082" /></a></div><br />This is interesting, so I check it in.</div><div><br /><h3>Complementary Directions</h3>Once again, I need to strengthen the tests. If a point is the northern neighbour of tengen, then tengen is the southern neighbour of the that point. Directions on the board come in complementary pairs.<pre>pred directions_are_complementary{<br /> N.complement = S<br /> S.complement = N<br />}</pre> and now I have to de-sugar <code>Direction</code> in order to insert the <code>complement</code> relation. And now we see how enums work<pre>abstract sig Direction{<br /> complement : one Direction<br />}{<br /> symmetric[@complement]<br />}<br /><br />one sig N extends Direction{}{complement = S}<br />one sig S extends Direction{}<br /></pre> In the fact appended to <code>Direction</code> I say that the relation <code>complement</code> (the <code>@</code> means that I'm refering to the relation itself and not its value) is symmetric using the predicate <code>util/relation/symmetric</code>. Thus I do not have to specify that <code>S</code>'s complement is <code>N</code> having once said the converse. The same pattern applies to <code>E</code> and <code>W.</code><br /><br />The instance is now a spectacular mess.<a onblur="try {parent.deselectBloggerImageGracefully();} catch(e) {}" href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgBODJBeavJKZ8G1rFC3jpPAOFS09pKdjhFR6Wa86Ulkl40qEvNI7WKxEhLxBlRqKmEihaOuNqak6le-GN6-QJOWjpDhJ2FAsDAJndLawgnI6Ym2QUl9q85iiEiFH8xwrbhVCWKOg/s1600/instance.png" style="text-decoration: none;"><img style="display:block; margin:0px auto 10px; text-align:center;cursor:pointer; cursor:hand;width: 320px; height: 152px;" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgBODJBeavJKZ8G1rFC3jpPAOFS09pKdjhFR6Wa86Ulkl40qEvNI7WKxEhLxBlRqKmEihaOuNqak6le-GN6-QJOWjpDhJ2FAsDAJndLawgnI6Ym2QUl9q85iiEiFH8xwrbhVCWKOg/s320/instance.png" border="0" alt="" id="BLOGGER_PHOTO_ID_5456692073024424290" /></a>I will check in anyway.</div><div><br /><div><h3>Distinct Neighbours</h3></div><div>What I might like to say is<pre>pred neighbours_are_distinct{<br /> all p : Point |<br /> all disj d, d' : Direction |<br /> p.neighbour[d] != p.neighbour[d']<br />}</pre>The nested quantification uses the disj modifier and should be read "for all distinct pairs of <code>Direction</code>, named <code>d</code> and <code>d'</code>..." </div><div><br /></div><div>This immediately renders the model (seemingly) inconsistent. More specifically the problem is that Alloy can no longer find an instance. I don't think that this is because the model contains contradictions so much as that it now requires more than three points in order to satisfy it. I can increase the number of instances available when the predicates are run like this <code>run correct for 5 Point</code> The resulting example is a rats nest of dodgy looking relations (it's in the repo as instance.dot if you want a look)</div><div><br /></div><div>A less extravagant predicate is<pre>pred neighbours_of_tengen_are_distinct{<br /> let tengen = {p : Point | p.name = Tengen} |<br /> all disj d, d' : Direction |<br /> tengen.neighbour[d] != tengen.neighbour[d']<br />}<br /></pre>and with this I see a much less tangled, but still wrong, instance (instance1.dot). And the model is also demonstrably invalid. I'm going to make a significant change to the model. One I've been itching to do for some time. I check in before this.</div><div><br /></div><div><h3>A Missing Abstraction?</h3></div><div>I feel as if I'm missing a degree of freedom, which is making it hard to say what I want. I'm going to promote tengen to be a signature, or rather to create a signature of which tengen will be the only instance at the moment: <code>InteriorPoint</code>. I remove the predicates about distinct neighbours and introduce <code>InteriorPoint</code><pre>sig InteriorPoint extends Point{}</pre>and can then quite happily say<pre>pred interior_points_have_a_neighbour_in_each_direction{<br />all p : InteriorPoint {<br /> not p.neighbour[N] = none<br /> not p.neighbour[E] = none<br /> not p.neighbour[S] = none<br /> not p.neighbour[W] = none<br /> }<br />}<br /></pre> and this gets me back to a consistent, not demonstrably invalid (although still wrong) model. I check in. </div><div><br /></div><div>Now I can say<pre>sig InteriorPoint extends Point{}{<br /> #ran[neighbour] = #Direction<br />}<br /></pre>and leave other kinds of point to look after themselves. If the range of the neighbour relation (which is a set) is the same size as the set of Directions, then there must be one neighbour per direction. </div><div><br />This leaves me with the model in this state<pre>sig Point {<br /> neighbour : Direction -> lone Point<br />}{<br /> not this in ran[neighbour]<br /> all d : dom[neighbour] |<br /> this = neighbour[d].@neighbour[d.complement]<br />}<br /><br />sig InteriorPoint extends Point{}{<br /> #ran[neighbour] = #Direction<br />}<br /><br />fact tengen_exists {<br /> #InteriorPoint = 1<br />}<br /><br />abstract sig Direction{<br /> complement : one Direction<br />}{<br /> symmetric[@complement]<br />}<br /><br />one sig N extends Direction{}{complement = S}<br />one sig S extends Direction{}<br /><br />one sig E extends Direction{}{complement = W}<br />one sig W extends Direction{}<br /></pre>a little bit of tidying up and I check in. The model is consistent and cannot be shown to be invalid. <a onblur="try {parent.deselectBloggerImageGracefully();} catch(e) {}" href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEha4ndraw3KitUbGYwJyAwCparOmT9RI_Ixa3423hSidZI7gO6DLWt23ro-KHw60S8XhIPtRzZ0-Ip3hAD1sEykjQcduQnwo2Rub9ui0woMUm2tT-IB2HhXMqEoV-o6ss5ll2eJQQ/s1600/instance.png"><img style="display:block; margin:0px auto 10px; text-align:center;cursor:pointer; cursor:hand;width: 320px; height: 213px;" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEha4ndraw3KitUbGYwJyAwCparOmT9RI_Ixa3423hSidZI7gO6DLWt23ro-KHw60S8XhIPtRzZ0-Ip3hAD1sEykjQcduQnwo2Rub9ui0woMUm2tT-IB2HhXMqEoV-o6ss5ll2eJQQ/s320/instance.png" border="0" alt="" id="BLOGGER_PHOTO_ID_5456728602333684130" /></a>The example instance looks respectable too—so long as we focus on the interior point and don't worry about how its neighbours relate to one-another, which is clearly wrong. But there are no tests for that. This diagram has been cleaned up in omnigraffle to focus on the interior point but the .dot of the original is checked in. Maybe another time I'll sort out the regular points.</div><div><br /></div><h2>Thoughts</h2>Wow, that was hard work. Took a long time, too (although not so long as the timestamps make it look, I was also doing laundry and so forth during the elapsed). Does that make what is after all merely a rectangular array of points a "complex domain"? No. I'm out of practice with this kind of thing, and not fluent with the tool. Even so, I'm impressed by how good a fit the TDD cycle seems to be for this formal modelling tool. I even got into a bit of trouble towards the end but was rescued by recalling the TDD technique of making the tests dumb and repetitive—<i>but concrete and clear</i>. And how the same subtle trap of thinking too far ahead applies here too.<div><br /></div><div>Would this have helped with the microcell predictor? Maybe not. Alloy doesn't do numbers at all well, and that was an intrinsically numerical problem. Could this approach help with other things? I think so. This tiny model(ling) problem has turned out to be harder and more time-consuming than I expected it to be, but it is my first go with the tool. I'm going to play around with it some more, as time allows, and see what comes up. </div><div><br /></div><div>I'm certainly impressed with the tool. Alloy comes very close to making powerful models an everyday tool for the working programmer, but I don't think its quite there yet. The gulf, as it always was, is between that very succinct model and working code. How to bridge that gulf in a useful way I don't yet know. </div></div>keithbhttp://www.blogger.com/profile/14314542307822401015noreply@blogger.com1tag:blogger.com,1999:blog-23912478.post-2008983528647574242010-01-03T19:03:00.002+00:002010-01-03T19:24:48.846+00:00Observations on SpotifyI use <a href="http://en.wikipedia.org/wiki/Spotify">Spotify</a>, but not quite enough to want to pay for the ad–free premium service just yet. I'm getting pretty good at not paying any attention to the ads (which seems as if it could be an increasingly useful life skill in the apparently inevitable entirely ad–funded on–line world to come), but a couple just now caught my...ear.<div><br /></div><div>Spotify make a big noise about their targeted ads: better value for advertisers, less annoying for listeners. And yet here I am half way through the third act of <i>Tristan</i> and it's choosing to tell me about a campaign called <a href="http://www.dance4life.com/">dance4life</a> which encourages clubbers under 25 to "start dancing and stop aids". A worthy goal, but unless Spotify knows something about Wagnerians that I don't this seems <i>a little odd</i>. </div><div><br /></div><div>I also notice that while playing music in its peer–to–peer mode Spotify is gratifyingly parsimonious of bandwidth, but when the ads are streaming in my network suddenly jump up to 125KB/s and stay there for the duration of the ads. I wonder why that should be.</div>keithbhttp://www.blogger.com/profile/14314542307822401015noreply@blogger.com0tag:blogger.com,1999:blog-23912478.post-63491045143742665122009-12-28T14:09:00.040+00:002009-12-30T19:42:38.418+00:00Bayesian Testing?<h2>Introduction</h2><div>I'm tossing this idea out into the world. It's half-formed and I'm learning as I go along. It may be invalid, it may be old news, it may not. What I'm hoping for is that someone who knows more about at least one of testing and Bayesian inference than I do will come and set me straight. </div><div><br /></div><div><b><span class="Apple-style-span" style="color:#FF0000;">UPDATE:</span></b> Laurent Bossavit turned out to be that person. The results below have be adjusted significantly a a result of a very illuminating conversation with him. Whatever virtue these results now have is due to him (and the defects remain my responsibility). Laurent, many thanks.</div><div><br /></div><div>In addition, a bunch of folks kindly came along to an <a href="http://xpday-london.editme.com/TestsAsEvidence">open space session</a> at <a href="http://www.xpday.org/">Xp Day London</a> this year. <a href="http://dolsonagile.wordpress.com/2009/12/08/xpday-day-2/">Here is the commentary of one</a>. From that already the idea became better formed, and this article reflects that improvement, thanks all. If you want to skip the motivation and cut to the chase, go <a href="http://peripateticaxiom.blogspot.com/2009/12/bayesian-testing.html#example">here</a>.</div><div><h2><br /></h2><h2>Evidence</h2>You may have <a href="http://faculty-staff.ou.edu/W/Jonathan.D.Wren-1/The%20Fine%20Art%20of%20Baloney%20Detection.htm">read</a> that <i>absence of evidence is not evidence of absence</i>. Of course, this is exactly wrong. I've just looked, and there is no evidence to be found that the room in which I am sitting (nor the room in which you are, I'll bet: look around you right now) contains an elephant. I consider this strong evidence that there is no elephant in the room. Not proof, and in some ways not the best reason for inferring that there is no elephant, but certainly evidence that there is none. This seems to be different from the form of bad logic that Sagan is actually criticising, in which the absence of evidence that there isn't an elephant in the room would be considered crackpot-style evidence that there <i>was</i> an elephant in the room.</div><div><br /></div><div>You may also have <a href="http://www.cs.utexas.edu/users/EWD/ewd02xx/EWD249.PDF">read</a> (on page 7 of that pdf) that <i>program testing can be used to show the presence of bugs, but never to show their absence!</i> I wonder. In the general case this certainly seems to be so, but I'm going to claim that working programmers don't often address the general case.</div><div><br /></div><div>Dijkstra's argument is that, even in the simple example of a multiplication instruction, we do not have the resources available to exhaustively test the implementation but we still demand that it should correctly multiply any two numbers within the range of the representation. Dijkstra says that we can't afford to take even a representative sample (whatever that might look like) of all the possible multiplications that our multiplier might be asked to do. And that seems plausible, too. Consider how many distinct values a numerical variable in your favourite language can take, and then square it. That's how many cases you expect the multiplication operation in your language to deal with, and deal with correctly. As an aside: do you expect it to work correctly? If so, why do you?</div><div><br /></div><div><a id="example">In this post I want to explore an approach that seems as if it might help us to decide how much confidence in the correctness of some code we should have, given the test results we can obtain about it.</a></div><div><h2><br /></h2><h2>A Small Example of Confidence</h2> Let's say that we wish to write some code to recognise if a stone played in a game of <a href="http://en.wikipedia.org/wiki/Go_(game)">Go</a> is in atari or not (this is my favourite example, for the moment). The problem is simple to state: a stone with two or more "liberties" is not in atari, a stone with one liberty is in atari. A stone can have 1 or more liberties. In a real game situation it can be some work to calculate how many liberties a stone has, but the condition for atari is that simple. </div><div><br /></div><div>A single stone can have only 1, 2, 3 or 4 liberties and those are the cases I will address here. I write some code to implement this function and I'll say that I'm fairly confident I've got it right (after all, it's only an <code>if</code>), but not greatly so. Laurent proposed a different question to ask from the one I was asking before—a better question, and he helped me find and understand a better answer.</div><div><br /></div><div>The prior probability of correctness that question leads to is 1 ⁄ 16. This is because there are 16 possible one-to-one onto mappings from {1, 2, 3, 4} to {T, F} and only one of them is the correct function. Thus, the prior is the prior probability that my function behaves identically to some other function that is correct by definition.</div><div><br /></div><div>How might a test result influence that probability of correctness? There is a <a href="http://spreadsheets.google.com/pub?key=tnjLDV9k47GnGraUnKmTD8Q&single=true&gid=0&output=html">spreadsheet</a> which shows a scheme for doing that using what very little I understand of <a href="http://en.wikipedia.org/wiki/Bayesian_inference">Bayesian inference</a>, slightly less naïvely applied than before. </div><div><br /></div><div>Cells in the spreadsheet are colour–coded to give a guide as to how the various values are used in the Bayesian formula. The key, as discussed in the XpDay session is how to count cases to find the conditional probabilities of seeing the evidence. </div><div><br /></div><div>The test would look something like this:<br /><style type="text/css">table.test td{ border-style:solid; border-width:1px; border-color:gray} .sig{background:LightGray} .pass{background:LightGreen} .fail{background:LightPink}</style><table class="test"><thead><tr><th>One Liberty Means Atari</th></tr></thead><tbody><tr><td class="sig">liberties</td><td class="sig">atari?</td></tr><tr><td>1</td><td class="pass">true</td></tr></tbody></table></div><div><br /></div><div> The posterior probability of correctness is <b>0.125 </b><span class="Apple-style-span" style="color:#FF0000;"><br /></span><h2><br /></h2><h2>Adding More Test Cases</h2>Suppose that I add another case that shows that when there are 2 liberties the code correctly determines that the stone is not in atari.<br /><table class="test"><thead><tr><th>One Liberty Means Atari</th></tr></thead><tbody></tbody><tbody><tr><td class="sig">liberties</td><td class="sig">atari?</td></tr><tr><td>1</td><td class="pass">true</td></tr><tr><td>2</td><td class="pass">false</td></tr></tbody></table></div><div>Using the same counting scheme as in the first case and using the updated probability from the first case as the prior in the second then it seems as if the updated probability of correctness with the new evidence is increased to <b>0.25</b> as <a href="http://spreadsheets.google.com/pub?key=tnjLDV9k47GnGraUnKmTD8Q&single=true&gid=1&output=html">this</a> sheet shows.</div><div><span class="Apple-style-span" style="color:#FF0000;"><br /></span></div><div>But suppose that the second test actually showed an incorrect result: 2 liberties and atari true. <table class="test"><thead><tr><th>One Liberty Means Atari</th></tr></thead><tbody></tbody><tbody><tr><td class="sig">liberties</td><td class="sig">atari?</td></tr><tr><td>1</td><td class="pass">true</td></tr><tr><td>2</td><td class="fail">true</td></tr></tbody></table>Then, as we might expect, the updated probability of correctness falls to <b>0.0</b> as shown <a href="http://spreadsheets.google.com/pub?key=tnjLDV9k47GnGraUnKmTD8Q&single=true&gid=2&output=html">here</a>. And as the formula works by multiplication of the prior probability by a factor based on the evidence, the updated probability will stay at zero no matter what further evidence is presented—which seems like the right behaviour to me.</div><div><span class="Apple-style-span" style="color:#FF0000;"><br /></span></div><div>This problem is very small, so in fact we can exhaustively test the solution. What happens to the probability of correctness then? Extending test coverage to these cases<br /><table class="test"><thead><tr><th>One Liberty Means Atari</th></tr></thead><tbody></tbody><tbody><tr><td class="sig">liberties</td><td class="sig">atari?</td></tr><tr><td>1</td><td class="pass">true</td></tr><tr><td>2</td><td class="pass">false</td></tr><tr><td>3</td><td class="pass">false</td></tr></tbody></table> gives an updated probability of <b>0.5</b> as shown <a href="http://spreadsheets.google.com/pub?key=tnjLDV9k47GnGraUnKmTD8Q&single=true&gid=3&output=html">here</a>.<br /></div><div><span class="Apple-style-span" style="color:#FF0000;"><br /></span></div><div>One more case remains to be added:<br /><table class="test"><thead><tr><th>One Liberty Means Atari</th></tr></thead><tbody></tbody><tbody><tr><td class="sig">liberties</td><td class="sig">atari?</td></tr><tr><td>1</td><td class="pass">true</td></tr><tr><td>2</td><td class="pass">false</td></tr><tr><td>3</td><td class="pass">false</td></tr><tr><td>4</td><td class="pass">false</td></tr></tbody></table> and the posterior probability of correctness is updated to <b>1.0</b> as shown <a href="http://spreadsheets.google.com/pub?key=tnjLDV9k47GnGraUnKmTD8Q&single=true&gid=4&output=html">here</a>.<br /><br />That result seems at to contradict Dijkstra: exhaustive testing, in a case where we can do that, does show the absence of bugs. He probably knew that.<br /><img src="http://spreadsheets.google.com/oimg?key=0AgiIblVYHDgsdG5qTERWOWs0N0duR3JhVW5LbVREOFE&oid=1&v=1262156070431" /></div><h2><br /></h2><h2>Next?</h2>My brain is fizzing with all sorts of questions to ask about this approach: I talked here about retrofitted tests, can it help with TDD? Can this approach guide us in choosing good tests to write next? How can the structure of the domain and co-domain of the functions we test guide us to high confidence quickly? Or can't they? Can the current level of confidence be a guide to how much further investment we should make in testing?<br /><div><br /></div><div>Some interesting suggestions are coming in in the comments, many thanks for those.</div><div><br /></div><div>My next plan I think will be to repeat this exercise for a slightly more complex function.</div>keithbhttp://www.blogger.com/profile/14314542307822401015noreply@blogger.com13tag:blogger.com,1999:blog-23912478.post-74456272644357651882009-11-25T11:26:00.002+00:002009-11-25T11:27:58.184+00:00New article for BCWIf you read this blog then there's likely little new for you in <a href="http://www.businesscomputingworld.co.uk/?p=1618">this article</a> for Business Computing World, but it might amuse.keithbhttp://www.blogger.com/profile/14314542307822401015noreply@blogger.com0tag:blogger.com,1999:blog-23912478.post-91490630432246562622009-11-12T15:16:00.003+00:002009-11-12T15:24:53.816+00:00Innovation GamesNext Tuesday there will be a special <a href="http://xpday-london.editme.com/eXtremeTuesdayClub">XtC</a> event at Zuhlke's office in London. <a href="http://www.lukehohmann.com/blog/index.php">Luke Hohmann</a> will be demonstrating his <a href="http://www.innovationgames.com/">innovation games</a> for Agile teams. Should be good. <div><br /></div><div>Details <a href="http://xpday-london.editme.com/XTC20091117">here</a>.</div>keithbhttp://www.blogger.com/profile/14314542307822401015noreply@blogger.com0tag:blogger.com,1999:blog-23912478.post-72265289796656934052009-11-09T10:17:00.003+00:002009-11-09T10:21:57.785+00:00Places remain at XP Day London 2009<a href="http://www.xpday.org/">XP Day London</a> is filling up, but places remain. The <a href="http://www.xpday.org/programme">programme</a> is looking very good. Register <a href="http://booking.xpday.org/registration.php">here</a>.keithbhttp://www.blogger.com/profile/14314542307822401015noreply@blogger.com0tag:blogger.com,1999:blog-23912478.post-91983547322979738652009-11-03T17:51:00.004+00:002009-11-04T09:54:37.159+00:00SketchesOne of the things I like to do in my free time is to dabble, in the most unschooled fashion imaginable, in music composition. Composing is hard. About as hard (and remarkably similar to) programming. <a href="http://www.schoenberg.at/default_e.htm">Arnold Schoenberg</a> offers this "advice for self-criticism" to students of composition:<blockquote>6. MAKE MANY SKETCHES<br />Join the best sketches to produce others and improve them until the result is satisfactory.<p>To make sketches is a humble and unpretentious approach toward perfection. </p>—<em>Fundamentals of Musical Composition, Ch XII</em></blockquote>I think that this applies equally well to programming.keithbhttp://www.blogger.com/profile/14314542307822401015noreply@blogger.com0tag:blogger.com,1999:blog-23912478.post-70550258671149650922009-10-19T22:14:00.002+01:002009-10-19T22:19:46.873+01:00XP Day London 09: ProgrammeAfter a lot of wrangling the almost-but-not-quite final <a href="http://xpday.org/programme">programme for XP Day London</a> is now available. Because of illness and other asynchronous distractions some of the presenters had to change at the last minute we still have to nail down one session, but this will be pretty much it.<div><br /></div><div>This year we have a lot of excellent experience reports from a range of practitioners who've been doing exiting new things and some really <a href="http://xpday.org/2009/keynotes">outstanding keynotes</a>.<br /><div><br /></div><div><a href="http://xpday.org/register">Register now!</a></div></div>keithbhttp://www.blogger.com/profile/14314542307822401015noreply@blogger.com0tag:blogger.com,1999:blog-23912478.post-9567638385704677652009-10-18T13:51:00.010+01:002009-10-18T18:13:37.561+01:00Scheduling by value?David Peterson has started <a href="http://www.kanbanblog.com/">a new blog</a> on Kanban (and snaffled a very tasty URL for it). He presents <a href="http://www.kanbanblog.com/article/time-at-bottleneck.html">this discussion</a> of scheduling features into a development team. The case the David presents is related to a behaviour I sometimes see with inexperienced teams who's just had someone go learn Scrum. Comes the next planning meeting and this idea pops up that the backlog needs to be ordered by "business value" so that the "most valuable" features can be delivered earliest.<div><br /></div><div>This can easily lead to some very nasty scenes where the Scrum Master demands that the Product Owner produce a "value" for each story—actually write a number on the card. The problem comes to a head when it turns out that the Product Owner not only doesn't know the value of the stories they are putting on the backlog, but they also have no way of finding out what they value of a story is. And this isn't because they are stupid, nor incompetent, nor malicious. It's because finding that value is far, far too difficult and time consuming an activity. And there's a good chance that any answer that came out of it would be so well hedged as to be meaningless.</div><div><br /></div><div>Sometimes the Product Owner does know, or can find out at reasonable cost, a value for a story or feature. Being able to trade a new asset class probably can be valued. Changing a flow to give 10% high conversion probably can be valued. Improving a model to get 1% higher efficiency in the machines it's used to design can probably be valued. These valuations will be functions of time time and various other parameters. If you really have to, you could get a number of them that's valid today (and perhaps only today). David makes the point that even if you do know that number for a feature, scheduling the next one simply on the basis of highest value might not be the smartest move. There are other variables to consider. </div><div><br /></div><div>There is a case to be made that within the context of a project value isn't the best figure of merit to use anyway, since someone should have made a go/no-go decision at some point that the planned budget and planned value seemed reasonable. That decision should be re-assessed frequently (far too little of this goes on) based on progress to date, and action taken if the actuals have come too far adrift, but in-between those times trying to optimise on value is perhaps not worth it. </div><div><br /></div><div>Another option is to indeed demand (and obtain) those value numbers and then <a href="http://legalizeadulthood.wordpress.com/2009/09/04/agile-roundtable-notes-september-3rd-2009/">schedule work </a><i><a href="http://legalizeadulthood.wordpress.com/2009/09/04/agile-roundtable-notes-september-3rd-2009/">primarily</a></i><a href="http://legalizeadulthood.wordpress.com/2009/09/04/agile-roundtable-notes-september-3rd-2009/"> on the basis of business value</a> and dispense with effort estimates, so-called "<a href="http://agiletoolkit.libsyn.com/index.php?post_id=400364">naked planning</a>". This has <a href="http://alistair.cockburn.us/Internal+versus+commercial+value">caused eyebrows to be raised</a>. The underlying claim is that</div><blockquote>value varies along an exponential scale while development costs vary along a linear scale. Therefore delivering the most valuable features trumps any consideration of whether or not the most valuable feature is cheap or easy to develop</blockquote>whihc, if true of your environment, might give pause for though. How this interacts with the desire to schedule so as to maximise throughput at the bottleneck is an open question, for me at least.keithbhttp://www.blogger.com/profile/14314542307822401015noreply@blogger.com3tag:blogger.com,1999:blog-23912478.post-29423355476061770562009-09-09T19:51:00.005+01:002009-09-09T20:25:37.060+01:00Service-Oriented ArchitectureI'm currently embroiled in the long and fraught process of having telephony and data services installed in a certain location. One supplier steadfastly and consistently refused to respond to my offers to become a paying customer, so I selected another who were very responsive at first, but have become less and less so over time. In fact, it's about two months since I signed and still no service has been provided (although bills have been sent). <div><br /></div><div>Part of my frustration with this is that it's very hard to find out what's going on. The company, a British telecoms provider and let's leave it at that, was once a monolithic monopoly but now has been dissected into multiple different business units, components, we might almost call them, each—I suppose—focussing on its so–called Core Competence (and more on that in a later post). Each of these components has its various workflows that it does and one or more contracts with other components for services it supplies or consumes and the components communicate by passing electronic messages to one another. Sometimes they pass electronic messages to me, complete with the URL of some other component where I have to go and do some action. It's all very slick and automated and orchestrated and, indeed, seems to have a mind of its own. </div><div><br /></div><div>For instance, the putting-in-wires component received a message telling it to come to my location and do just that. Unfortunately, the agent of the no-I-mean-really-putting-in-wires component to which they delegated implementation of that action was not able to complete it. He sent a message saying so and various exception flows kicked off, requiring a <i>lot</i> of manual intervention, oh yes.</div><div><br /></div><div>Meanwhile, the arrange-for-telephony component turned out to have a clock running and when a certain (unpublicised) duration had elapsed without it receiving a notification of success from the putting-in-wires component (which was busy with some recovery actions on the no-I-mean-really-putting-in-wires component) it triggered a flow that cancelled my original request to have some services. A notification was received by one (but not all) of the taking-money-off-you components and one of them sent me a message telling me that my order for some services had been cancelled. A good thing, because otherwise I would have been blissfully unaware of the situation. On the other hand I am now angrily aware of the situation.</div><div><br /></div><div>Now, here's the fun bit: irrespective which component sends me a message, no agent working for that component can explain to me what the message means, because whatever it means that meaning belongs to whatever other component sent the earlier message that lead the the message I received being sent. And no, they can't put me through to an agent in that component. There is no interoperability layer.</div><div><br /></div><div>Today I spoke with five agents in three different components. One of them gave me quite the run–around because although I had contacted him through the callback given in the message I'd received from his component I had mis–configured part of my message header leading to my message being dispatched to the wrong agent because I had misunderstood the published specification for that header which he freely admitted was itself a shoddy piece of work with unreasonable and misleading contents but it was still my problem that I'd botched the message send.</div><div><br /></div><div>Also, I've learned that to get to speak to an agent at all I have to go twice around the loop of failing a handshake because I can't provide a piece of data that the protocol requires but that I won't get until the request succeeds. After two failures in a row a supervisory process notices and I'm failed over to a more generic service through which I can contact an agent, but that service is not exposed on a public URL.</div><div><br /></div><div>To all of which I say: bring back the mainframe.</div>keithbhttp://www.blogger.com/profile/14314542307822401015noreply@blogger.com1tag:blogger.com,1999:blog-23912478.post-82510691717363872382009-09-08T10:09:00.008+01:002009-11-12T15:40:50.889+00:00Observations on EstimationTeams following a process like Scrum tend to estimate the "size" of stories as an aid to figuring out a commitment for a sprint. My view is that this is a transitional practice, and that the aim should be to learn how to make stories all roughly the same size so that commitments (also a transitional practice) can be determined by counting.<br /><br />While all of that is going on teams that want to use a numerical scale to estimate (rather than, say, "t-shirt" sizing) tend to choose a scale, a sequence of licit values from which estimates must be drawn. The various planning tools that demand a numerical field be filled in tend to force this issue.<br /><br />I've noticed a tendency for "expert" level practitioners to want to use some clever non-linear scale, maybe Fibonacci numbers (1,2,3,5,8,13), maybe a geometric series (1,2,4,8,16) and they will have some sophisticated reason why this or that series is preferred. And I've noticed that a lot of teams aren't comfortable with this. They want to use a linear scale.<br /><br />It seems to be traumatic enough that the estimates don't have units, or even dimensions. The idea that estimates are dimensionless but also structured can be a double cause of confusion.<br /><br /><span style="font-weight: bold;">Anecdote:</span> a team had been estimating and planning and delivering consistently for a good long time. Their velocity was fairly constant, but drifted over time (fair enough). One day it turned out that their velocity happened to be numerically equal to the number of team members times the number of days to the next planning horizon. Someone noticed this and with a huge sigh of relief the team concluded that these mysterious "units" in which they estimated were actually man-days in disguise. Now they finally understood what they were estimating! And they promptly lost the ability to estimate: their next planning session was all over the place and it took some time for their planning activities to converge again. My inference was that it's actually quite important that estimates are dimensionless.<br /><br /><span style="font-weight: bold;">Anecdote:</span> a User Experience expert at a client had been involved in some research whereby (as a side effect) members of the general public had to create a scale that made sense to them within which to rank the usability of features. These folks were presented with different generic objects and asked to give them a "size", and then to give a corresponding "size" to some other generic objects in order to create a scale that made sense to them, which would then be applied to the merit of the system features that were the actual target of the research. They created linear scales. <div><br /></div><div>[After seeing this he added the observation that this process was in aid of avoiding what often happens with the strongly disagree, disagree, no preference... type of scale which is either polarised or bland results, neither of which is that useful]<br /><br />That surprised me at first, since I know that the physics of our sensory apparatus are generally non-linear, and memory is non-linear and so forth. But thinking about it some more I realised that our <span style="font-style: italic;">experience</span> tends to seem to be linear, even if the underlying phenomena aren't.<br /><br />Meanwhile, if one did want to use a particular scale for estimating the size of stories, why not use one of the series of <a href="http://en.wikipedia.org/wiki/Preferred_number#Renard_numbers">prefered values</a>? They are very well established in engineering and product design and offer interesting error-minimising properties. On the other hand, it might be a real struggle to get a team to decide if a story was a 1.6 or a 3.15<br /><br />I don't have a grand narrative into wich to fit these observations, but <a href="http://blogs.msdn.com/cellfish/archive/2009/10/23/never-convert-your-story-points-to-time.aspx">here</a> is another related anecdote about estimation.</div>keithbhttp://www.blogger.com/profile/14314542307822401015noreply@blogger.com0