Bayesian Testing?

Introduction

I'm tossing this idea out into the world. It's half-formed and I'm learning as I go along. It may be invalid, it may be old news, it may not. What I'm hoping for is that someone who knows more about at least one of testing and Bayesian inference than I do will come and set me straight.

UPDATE: Laurent Bossavit turned out to be that person. The results below have be adjusted significantly a a result of a very illuminating conversation with him. Whatever virtue these results now have is due to him (and the defects remain my responsibility). Laurent, many thanks.

In addition, a bunch of folks kindly came along to an open space session at Xp Day London this year. Here is the commentary of one. From that already the idea became better formed, and this article reflects that improvement, thanks all. If you want to skip the motivation and cut to the chase, go here.

Evidence

You may have read that absence of evidence is not evidence of absence. Of course, this is exactly wrong. I've just looked, and there is no evidence to be found that the room in which I am sitting (nor the room in which you are, I'll bet: look around you right now) contains an elephant. I consider this strong evidence that there is no elephant in the room. Not proof, and in some ways not the best reason for inferring that there is no elephant, but certainly evidence that there is none. This seems to be different from the form of bad logic that Sagan is actually criticising, in which the absence of evidence that there isn't an elephant in the room would be considered crackpot-style evidence that there was an elephant in the room.

You may also have read (on page 7 of that pdf) that program testing can be used to show the presence of bugs, but never to show their absence! I wonder. In the general case this certainly seems to be so, but I'm going to claim that working programmers don't often address the general case.

Dijkstra's argument is that, even in the simple example of a multiplication instruction, we do not have the resources available to exhaustively test the implementation but we still demand that it should correctly multiply any two numbers within the range of the representation. Dijkstra says that we can't afford to take even a representative sample (whatever that might look like) of all the possible multiplications that our multiplier might be asked to do. And that seems plausible, too. Consider how many distinct values a numerical variable in your favourite language can take, and then square it. That's how many cases you expect the multiplication operation in your language to deal with, and deal with correctly. As an aside: do you expect it to work correctly? If so, why do you?

In this post I want to explore an approach that seems as if it might help us to decide how much confidence in the correctness of some code we should have, given the test results we can obtain about it.

A Small Example of Confidence

Let's say that we wish to write some code to recognise if a stone played in a game of Go is in atari or not (this is my favourite example, for the moment). The problem is simple to state: a stone with two or more "liberties" is not in atari, a stone with one liberty is in atari. A stone can have 1 or more liberties. In a real game situation it can be some work to calculate how many liberties a stone has, but the condition for atari is that simple.

A single stone can have only 1, 2, 3 or 4 liberties and those are the cases I will address here. I write some code to implement this function and I'll say that I'm fairly confident I've got it right (after all, it's only an if), but not greatly so. Laurent proposed a different question to ask from the one I was asking before—a better question, and he helped me find and understand a better answer.

The prior probability of correctness that question leads to is 1 ⁄ 16. This is because there are 16 possible one-to-one onto mappings from {1, 2, 3, 4} to {T, F} and only one of them is the correct function. Thus, the prior is the prior probability that my function behaves identically to some other function that is correct by definition.

How might a test result influence that probability of correctness? There is a spreadsheet which shows a scheme for doing that using what very little I understand of Bayesian inference, slightly less naïvely applied than before.

Cells in the spreadsheet are colour–coded to give a guide as to how the various values are used in the Bayesian formula. The key, as discussed in the XpDay session is how to count cases to find the conditional probabilities of seeing the evidence.

The test would look something like this:

One Liberty Means Atari
liberties	atari?
1	true

The posterior probability of correctness is 0.125

Adding More Test Cases

Suppose that I add another case that shows that when there are 2 liberties the code correctly determines that the stone is not in atari.

One Liberty Means Atari
liberties	atari?
1	true
2	false

Using the same counting scheme as in the first case and using the updated probability from the first case as the prior in the second then it seems as if the updated probability of correctness with the new evidence is increased to 0.25 as this sheet shows.

But suppose that the second test actually showed an incorrect result: 2 liberties and atari true.

One Liberty Means Atari
liberties	atari?
1	true
2	true

Then, as we might expect, the updated probability of correctness falls to 0.0 as shown here. And as the formula works by multiplication of the prior probability by a factor based on the evidence, the updated probability will stay at zero no matter what further evidence is presented—which seems like the right behaviour to me.

This problem is very small, so in fact we can exhaustively test the solution. What happens to the probability of correctness then? Extending test coverage to these cases

One Liberty Means Atari
liberties	atari?
1	true
2	false
3	false

gives an updated probability of 0.5 as shown here.

One more case remains to be added:

One Liberty Means Atari
liberties	atari?
1	true
2	false
3	false
4	false

and the posterior probability of correctness is updated to 1.0 as shown here.

That result seems at to contradict Dijkstra: exhaustive testing, in a case where we can do that, does show the absence of bugs. He probably knew that.

Next?

My brain is fizzing with all sorts of questions to ask about this approach: I talked here about retrofitted tests, can it help with TDD? Can this approach guide us in choosing good tests to write next? How can the structure of the domain and co-domain of the functions we test guide us to high confidence quickly? Or can't they? Can the current level of confidence be a guide to how much further investment we should make in testing?

Some interesting suggestions are coming in in the comments, many thanks for those.

My next plan I think will be to repeat this exercise for a slightly more complex function.

New article for BCW

If you read this blog then there's likely little new for you in this article for Business Computing World, but it might amuse.

Innovation Games

Next Tuesday there will be a special XtC event at Zuhlke's office in London. Luke Hohmann will be demonstrating his innovation games for Agile teams. Should be good.

Details here.

Places remain at XP Day London 2009

XP Day London is filling up, but places remain. The programme is looking very good. Register here.

Sketches

One of the things I like to do in my free time is to dabble, in the most unschooled fashion imaginable, in music composition. Composing is hard. About as hard (and remarkably similar to) programming. Arnold Schoenberg offers this "advice for self-criticism" to students of composition:

6. MAKE MANY SKETCHES
Join the best sketches to produce others and improve them until the result is satisfactory.

To make sketches is a humble and unpretentious approach toward perfection.

—Fundamentals of Musical Composition, Ch XII

I think that this applies equally well to programming.

XP Day London 09: Programme

After a lot of wrangling the almost-but-not-quite final programme for XP Day London is now available. Because of illness and other asynchronous distractions some of the presenters had to change at the last minute we still have to nail down one session, but this will be pretty much it.

This year we have a lot of excellent experience reports from a range of practitioners who've been doing exiting new things and some really outstanding keynotes.

Register now!

Scheduling by value?

David Peterson has started a new blog on Kanban (and snaffled a very tasty URL for it). He presents this discussion of scheduling features into a development team. The case the David presents is related to a behaviour I sometimes see with inexperienced teams who's just had someone go learn Scrum. Comes the next planning meeting and this idea pops up that the backlog needs to be ordered by "business value" so that the "most valuable" features can be delivered earliest.

This can easily lead to some very nasty scenes where the Scrum Master demands that the Product Owner produce a "value" for each story—actually write a number on the card. The problem comes to a head when it turns out that the Product Owner not only doesn't know the value of the stories they are putting on the backlog, but they also have no way of finding out what they value of a story is. And this isn't because they are stupid, nor incompetent, nor malicious. It's because finding that value is far, far too difficult and time consuming an activity. And there's a good chance that any answer that came out of it would be so well hedged as to be meaningless.

Sometimes the Product Owner does know, or can find out at reasonable cost, a value for a story or feature. Being able to trade a new asset class probably can be valued. Changing a flow to give 10% high conversion probably can be valued. Improving a model to get 1% higher efficiency in the machines it's used to design can probably be valued. These valuations will be functions of time time and various other parameters. If you really have to, you could get a number of them that's valid today (and perhaps only today). David makes the point that even if you do know that number for a feature, scheduling the next one simply on the basis of highest value might not be the smartest move. There are other variables to consider.

There is a case to be made that within the context of a project value isn't the best figure of merit to use anyway, since someone should have made a go/no-go decision at some point that the planned budget and planned value seemed reasonable. That decision should be re-assessed frequently (far too little of this goes on) based on progress to date, and action taken if the actuals have come too far adrift, but in-between those times trying to optimise on value is perhaps not worth it.

Another option is to indeed demand (and obtain) those value numbers and then schedule work primarily on the basis of business value and dispense with effort estimates, so-called "naked planning". This has caused eyebrows to be raised. The underlying claim is that

value varies along an exponential scale while development costs vary along a linear scale. Therefore delivering the most valuable features trumps any consideration of whether or not the most valuable feature is cheap or easy to develop

whihc, if true of your environment, might give pause for though. How this interacts with the desire to schedule so as to maximise throughput at the bottleneck is an open question, for me at least.

Service-Oriented Architecture

I'm currently embroiled in the long and fraught process of having telephony and data services installed in a certain location. One supplier steadfastly and consistently refused to respond to my offers to become a paying customer, so I selected another who were very responsive at first, but have become less and less so over time. In fact, it's about two months since I signed and still no service has been provided (although bills have been sent).

Part of my frustration with this is that it's very hard to find out what's going on. The company, a British telecoms provider and let's leave it at that, was once a monolithic monopoly but now has been dissected into multiple different business units, components, we might almost call them, each—I suppose—focussing on its so–called Core Competence (and more on that in a later post). Each of these components has its various workflows that it does and one or more contracts with other components for services it supplies or consumes and the components communicate by passing electronic messages to one another. Sometimes they pass electronic messages to me, complete with the URL of some other component where I have to go and do some action. It's all very slick and automated and orchestrated and, indeed, seems to have a mind of its own.

For instance, the putting-in-wires component received a message telling it to come to my location and do just that. Unfortunately, the agent of the no-I-mean-really-putting-in-wires component to which they delegated implementation of that action was not able to complete it. He sent a message saying so and various exception flows kicked off, requiring a lot of manual intervention, oh yes.

Meanwhile, the arrange-for-telephony component turned out to have a clock running and when a certain (unpublicised) duration had elapsed without it receiving a notification of success from the putting-in-wires component (which was busy with some recovery actions on the no-I-mean-really-putting-in-wires component) it triggered a flow that cancelled my original request to have some services. A notification was received by one (but not all) of the taking-money-off-you components and one of them sent me a message telling me that my order for some services had been cancelled. A good thing, because otherwise I would have been blissfully unaware of the situation. On the other hand I am now angrily aware of the situation.

Now, here's the fun bit: irrespective which component sends me a message, no agent working for that component can explain to me what the message means, because whatever it means that meaning belongs to whatever other component sent the earlier message that lead the the message I received being sent. And no, they can't put me through to an agent in that component. There is no interoperability layer.

Today I spoke with five agents in three different components. One of them gave me quite the run–around because although I had contacted him through the callback given in the message I'd received from his component I had mis–configured part of my message header leading to my message being dispatched to the wrong agent because I had misunderstood the published specification for that header which he freely admitted was itself a shoddy piece of work with unreasonable and misleading contents but it was still my problem that I'd botched the message send.

Also, I've learned that to get to speak to an agent at all I have to go twice around the loop of failing a handshake because I can't provide a piece of data that the protocol requires but that I won't get until the request succeeds. After two failures in a row a supervisory process notices and I'm failed over to a more generic service through which I can contact an agent, but that service is not exposed on a public URL.

To all of which I say: bring back the mainframe.

Observations on Estimation

Teams following a process like Scrum tend to estimate the "size" of stories as an aid to figuring out a commitment for a sprint. My view is that this is a transitional practice, and that the aim should be to learn how to make stories all roughly the same size so that commitments (also a transitional practice) can be determined by counting.

While all of that is going on teams that want to use a numerical scale to estimate (rather than, say, "t-shirt" sizing) tend to choose a scale, a sequence of licit values from which estimates must be drawn. The various planning tools that demand a numerical field be filled in tend to force this issue.

I've noticed a tendency for "expert" level practitioners to want to use some clever non-linear scale, maybe Fibonacci numbers (1,2,3,5,8,13), maybe a geometric series (1,2,4,8,16) and they will have some sophisticated reason why this or that series is preferred. And I've noticed that a lot of teams aren't comfortable with this. They want to use a linear scale.

It seems to be traumatic enough that the estimates don't have units, or even dimensions. The idea that estimates are dimensionless but also structured can be a double cause of confusion.

Anecdote: a team had been estimating and planning and delivering consistently for a good long time. Their velocity was fairly constant, but drifted over time (fair enough). One day it turned out that their velocity happened to be numerically equal to the number of team members times the number of days to the next planning horizon. Someone noticed this and with a huge sigh of relief the team concluded that these mysterious "units" in which they estimated were actually man-days in disguise. Now they finally understood what they were estimating! And they promptly lost the ability to estimate: their next planning session was all over the place and it took some time for their planning activities to converge again. My inference was that it's actually quite important that estimates are dimensionless.

Anecdote: a User Experience expert at a client had been involved in some research whereby (as a side effect) members of the general public had to create a scale that made sense to them within which to rank the usability of features. These folks were presented with different generic objects and asked to give them a "size", and then to give a corresponding "size" to some other generic objects in order to create a scale that made sense to them, which would then be applied to the merit of the system features that were the actual target of the research. They created linear scales.

[After seeing this he added the observation that this process was in aid of avoiding what often happens with the strongly disagree, disagree, no preference... type of scale which is either polarised or bland results, neither of which is that useful]

That surprised me at first, since I know that the physics of our sensory apparatus are generally non-linear, and memory is non-linear and so forth. But thinking about it some more I realised that our experience tends to seem to be linear, even if the underlying phenomena aren't.

Meanwhile, if one did want to use a particular scale for estimating the size of stories, why not use one of the series of prefered values? They are very well established in engineering and product design and offer interesting error-minimising properties. On the other hand, it might be a real struggle to get a team to decide if a story was a 1.6 or a 3.15

I don't have a grand narrative into wich to fit these observations, but here is another related anecdote about estimation.

Real Engineers

There's a recurring fashion for beating up those who would build software for a living with tales of how "real" engineers do it. This started with the woefully misguided "Software Engineering" conferences of the late 1960's and continues on-and-off to this day.

As a response to this I like to collect examples of "real" engineers screwing up. Not out of malice, but out of a desire to ground certain aspects of my professional life in something resembling fact. Here are some reported facts about the Lockheed Martin F-22 "Raptor" fighter aircraft:

1. the aircraft has recently required more than 30 hours of maintenance for every hour in the skies


2. the canopy needs refurbishing after 331 hours of flying, less than half the design goal of 800 hours, and this costs $120,000 a pop


3. the aircraft is almost twenty thousand dollars more expensive to fly per hour than its predecessor (which costs an already eye-watering 30+ grand per hour—what do they run on, Chanel No 5? Single malt whisky?)

And so on. Just to add to the fun, the development cycle for this aircraft was so long that the class of mission for which it was designed no longer takes place.Is this the fault of the folks at Lockheed Martin? Not really. It's the fault of a system that put politics ahead of engineering, a circumstance under which no-one can be successful.

Change

We aim to keep all our offices ISO 9000 certified. This means updating the procedures to reflect our increasing tendency to run software development projects under an Agile process. It's fallen to me to update our existing "Change Control" procedure which of its kind is very good, but does build up from the assumption that to have a change to control is an exceptional (although expected) event.

I keep wanting to turn that inside out and say—deal with every requirement this way, make this the default, make the threshold of cost/schedule/risk impact for triggering change control be 0.0 in all cases, have the Change Control Board meet every week, have them assess the impact of every item and decide whether or not to schedule it, have no other way to schedule any activity than via the change log. And bingo! Your project would pretty much be agile.

In fact, I might just do that.

De–skilling through Technology: friend and foe

Once, and for a mercifully short time, I lived on the very western edge of Bournemouth. You might recall that Ford considered his article for the Guide describing how to have a good time in Bournemouth to be one of his finest pieces of fiction. So, I was often wanting to go away, farther away than I could sensibly go on my pseudo-vintage motorcycle in what time was available. That meant going from the County Gates to Bournemouth railway station. This could be done on the cramped, slow local bus or at great expense by taxi.

But I noticed that the long–distance coaches coming in from the even further south and west of England made one last stop before Bournemouth Coach Station (adjacent to the railway station) in Westbourne, just around the corner from me. These coaches are large, have luggage space and go fast on major roads. And the on–line booking system did not blink an electronic eye at me buying and downloading to print out a 90 pence ticket to ride from Westbourne to Bournemouth. The drivers of the coaches did blink when I got on there, but I had a ticket so they shrugged and went about their business. That's nice. Imagine the counter–arguments that would likely come from a human ticket agent who knows only too well the various alternatives to doing so preposterous a thing as riding a long–haul coach half–way across the town where you live.

Now, much more recently I was on the very edge of the very agreeable city of Charlotte, North Carolina. I had been down–town to check out the excellent Mint Museum. Being a citizen of the Socialist People's Republic of Europe I naturally tried to do the journey by public transport. Charlotte has a pretty good public transport system, C.A.T.S. However, it turned out that the bus service that would otherwise run from exactly the last stop of the light rail system (the "Lynx") to exactly my hotel was not running that day. Nice try, but I have to do the last few miles by cab. Now, I am a stranger to the city and am in any case in a fairly obscure part of it. I call a cab company and while I can explain with a certain degree of accuracy where I want to go, I don't know where I am.

And that's a problem.

The cabs are guided by GPS, so they need an exact street address for pickup and set down. This is so that the drivers do not need to know their way around. That's a pretty shocking concept for a resident of London, where cab drivers know their way around so well it changes their brains. Troublesome de–skill #1.

All I know is that I'm at the last stop of the Blue Line, but I don't know where that is. And neither does the dispatcher at the cab company—after all, all they do is pass on the co–ordinates of two places they don't know the location of to drivers who don't know the route between them. De–skill #2. Luckily for me, the dispatcher thought that they knew someone who might be able to figure out where this Lynx station was. I should call back in a few minutes. I do so, and by virtue of the elevated situation of the station I can call off enough landmarks (that is, names of shopping malls) for this person at the other end to work out where I am. My hotel, for some reason, they can look up very easily.

Some time later a cab rolls up. And it turns out that all this marvellous de–skilling has been wasted because the guy is a veteran driver and knows the district like the back of his hand.

Also, once I'm seated and strapped in, his first question is: so, where are we going?

Finish-start Dependencies: just say no!

I'm working with a team that have (as do we all) a tendency when under stress to fall back upon what they are comfortable with—in particular, to serialising their activities. "We can't x because they haven't finished y because they're waiting for z..."

To help them remember that this isn't the best move they can make, I've devised this symbol to display in the team area. You are welcome to use it yourself.

No Serialised Activities by Keith Braithwaite is licensed under a Creative Commons Attribution-Noncommercial-Share Alike 2.0 UK: England & Wales License.
Based on a work at 1.bp.blogspot.com.

Learning from Architects

From dog–houses to skyscrapers, the discipline of the building architect has long been a rich source of metaphor for system architects. I don't disagree, indeed in one of my contributions to the 97 Things Every Software Architect Should Know I recommended that software architects should learn from architects of buildings. So, you can imagine that my eye was caught by Matthew Frederick's 101 Things I Learned in Architecture School. This small book contains the eponymous count of hints, tips and tricks from a qualified architect to architecture students.

This is a good resource for those of us who would learn from architects of buildings, as it is advice to would–practitioners from a practitioner and as such relates to what building architects actually do, rather than what someone who isn't a practitioner thinks they must, surely, do. This latter is a common failure mode of software professionals seeking inspiration from other disciplines—all the way back to the very first Software Engineering conferences of the late 1960's, in which an hallucinatory notion of what "engineers" do was foisted upon us. But I digress.

Some of the 101 are very low level and very specific (eg Number 90 "Roll your drawings for transport or storage with the image side facing out"), others are much broader and seem to me to have relevance for any sort of design work.

Number 15 tells us that "A parti is the central idea or concept of a building." Wikipedia tells us that parti is from the French prendre parti "to make a decision". The parti captures, presents and summarises the highest level decision that has been made about the organising principle of an entire building or building project, and examples are given where the parti expresses all that in one, highly abstract, diagram.

A parti sounds to me a lot like a system metaphor.

Number 100 tells us that the parti should have a name, such as "half–eaten donut" or "meeting of strangers". Could the nominal (or de facto [*]) architect of the system that you are working on draw such a diagram? Would it tell anyone anything if they did? Could they name the parti, the very highest level design decision from which the rest of the system design flows?

Number 28 tells us that a good designer isn't afraid to throw away a good idea. Notice: a good idea. An idea can be good and not fit with the parti, in which case it has no place in the design. We are advised to "save [...] good but ill–fitting ideas for another time and project—and with the knowledge that they might not work then, either." When was the last time you (or your team) threw out a good idea?

Number 46 tells us to "Create architectural richness through informed simplicity or an interaction of simples rather than through unnecessarily busy agglomerations". Frederick warns particularly against "busying up a project with doodads because it is boring without them; agglomerating many unrelated elements without concern for their unity because they are interesting in themselves." What interesting doodads does your current project have?

One reason to follow the guidance of Number 46 is given in Number 51, which observes that "Beauty is due more to the harmonious relationships among the elements of a composition than to the elements themselves". One achieves this beauty through a design process, or system.

Number 77 cautions that "No design system is or should be perfect. Designers are often hampered by a well–intentioned by erroneous belief that a good design solution is perfectly systematic [...] but nonconforming oddities can be enriching, humanising aspects of your project." 77 also observes that "exceptions to the rule are often more interesting that the rules themselves."

Number 81 notes that "Properly gaining control of the design process tends to feel like one is losing control of the design process" 81 advises that the designer should "accept uncertainty. Recognise as normal the feeling of lostness that attends to much of the process. Don't seek to relieve your anxiety by marrying yourself prematurely to a design solution; design divorces are never pretty" No, they never are.

Number 99 can help. It says "Just do something. [...] don't wait for clarity to arrive before beginning to draw. Drawing is not simply a way of depicting a design solution; it is itself a way of learning about the problem you are trying to solve." I think that much the same can be said for coding. Are the design procedures in your team aligned with these principles?

[*] Even the most self–organised, most cross–functional, most Agile, most collectively–code–owning software development team will have one individual who knows most about (and likely has most influence over) the architecture of the system. You can pretend otherwise, or you can take advantage of it.

Plaudit

Apparently, this is the 141st most "top" blog for developers (as of Q2 2009). That's probably pretty far out along some long tail of topness. Ah well.

The metric used captures something like degree of interest of the rest of the blogosphere. So, thanks for your interest.

Let's talk about feelings

I seem to recall that back in the days when he was prone to wild outbursts of public self-examination (this even before blogging and twitter) John Cleese gave an interview in part about his early experiences with therapy, with a "talking" cure. His therapist would begin, "How do you feel?" and John would say "Well, I think..." and his therapist would interrupt "No. How do you feel?" and John would say "Well, I think..." and his therapist would interrupt "No. How do you feel?" And so on. You can see the problem. And he couldn't, I suppose. Which was the problem.

More and more these days I want to ask people how they feel about their code. Here's part of why.

Ivan Moore and "the other" Mike Hill have this conference session that they do called Programming in the Small [pdf]. I love it. They put little examples of code in front of folks and invite them to refactor, just a little bit. And then reflect on the refactoring. One of the things they've noticed that programers tend to do under these conditions is futz around with the code before getting down to actually improving the design.

Mike and Steve Freeman do a similar session called "Programming without Getters" This is in the so-called "dojo" format, where a revolving pair of programmers is invited to take some fairly typical "enterprise" class code and refactor it to remove the getters. There are those of us that believe that there can be something quite special about OO code with that property, but the session didn't really get there. That's because the pairs couldn't bring themselves to do the refactoring as asked, they couldn't even start to remove any getters, until they'd futzed around with the code first. And since a new person rolls into the pair every five minutes or so, it's pretty much a non-stop futz-fest.

Linda Rising saw a talk I give about this design metric that I've been playing with over the last couple of years (I have a day job, so progress has been slow). She was struck by my observation that folks who've tried this metric on their own code have reported that refactoring which made them happier with the code also increases the value of the metric. Linda wanted to know if they really said "happier". They really do.

It turns out that Linda did some research back in the day on a design metric of her own. During this work she had noticed that in general, programers like to futz around with code before they get down to work on it. If I recall correctly, she asked Dick Gabriel about this and he said that "programmers do that", along with some allusion to that metaphorical aphorism about the flavour of soup and pissing in it. I'm sure a lot of that goes on. But what Linda (again, this is as well as I recall) further noticed was that they tended to do this much less with code that scored well on her metric. And they described themselves as being more comfortable working on this code than on code that had a low score.

Now, I've nothing against metrics in principle (after all, I seem to be in the middle of inventing one) but I'm rather dubious about all this dashbordery and piechartism that's going on these days, all this getting of the CI server to dish up trends of metrics and blah, blah, blah. It's nice to have the numbers, it's nice to see a healthy trend, but.......

How do you feel about your code? Does it make you happy? Is it comfortable?

XP Day London 09: Call for Sessions

Submissions are now open for programmed sessions at XpDay London 2009, to be held 7th and 8th December 2009. http://www.xpday.org/

You are invited to propose a session for the first day of the conference. We are particularly interested in the following

• Experience reports—share your stories of challenge and success with Agile and Lean techniques. Experience reports will be intensively shepherded by experienced practitioners.
• Hands-on technical sessions—share techniques and practices in practical sessions: workshops, tutorials, simulations
• Practitioners' advances in the art—share the techniques of expert Agile and Lean practitioners, work with them to move the craft forward.

The second day of the conference will be an OpenSpace session with topics selected at the end of the first day. Programmed sessions are most suitable for topics requiring some set up or extensive preparation.

To submit a session, please go to http://xpday-london.editme.com/XpDay2009Submissions

Submissions will be accepted until Friday 14th August.

Flow: are two dimensions enough?

Inspired by a comment by Joseph at the recent Agile Coach's Gathering I've been experimenting with the use of this model in mid-year reviews for my team.

One of the guys observed that it seems to have a missing dimension. It seems that it's possible to have a case whereby an individual is working on a solidly challenging problem, well up the y-axis, and they have the high level of skill to meet that challenge, well along the x-axis...and when all's said and done they'd really rather be doing something a bit more enjoyable.

Missing axis: fun.

Bridging the Communication Gap

Call it automated acceptance testing, functional test driven development, checked examples or what you will—the use of automatic validation is one of the most effective tools in the Agile developer's kit.

It's a large and involved field, touching on almost every aspect of development. Handy, then, that Gojko Adzic has published a very comprehensive guide to the use of automated acceptance tests in contemporary Agile development practice, Bridging the Communication Gap: Specification by example and agile acceptance testing.

This book is a much-needed checkpoint in the on-going adventure to discover (and re-discover) how to write software effectively. Gojko is a very energetic enthusiast for these ideas, and a very experienced practitioner of them. His knowledge and expertise is present on every page.

A strong theme runs through the book—that the reason we capture examples of required behaviour and automate validation of them is to improve communication. Examples turn out to be a very powerful way to understand a problem domain and to explore a solution suitable for that domain. There turn out to be fascinating reasons for why this is true, but Gojko quite reasonably focusses on practical advice.

The main body of the book tells a story, a story of understanding, finding, and using examples to create shared understanding across a team. Gojko gives very concrete advice in a series of short chapters and explains how to do this. How to organise a workshop to find examples, how to find good examples, how to use tools to automate validation, how to use the resulting tests to guide development. Each chapter ends with a handy bullet list of key points. Together with other material on the best use for developers to make of such checked examples, and how to fit example discovery and capture into a typical Agile development process Bridging the Communication Gap provides as close to a vade mecum for newcomers to the discipline of functional test driven development as we are likely to see.

Gojko draws informative parallels with other techniques more or less strongly aligned with the Agile development world. This places the practice of Agile acceptance testing in context, and as a team-wide activity, reinforcing the cross-functional nature of the tool. Always the emphasis is on helping the various stakeholders in a development project communicate better.

There is a survey of tools available for this kind of work, which I might wish were slightly broader in scope and a little more detailed, but it does give a good overview of the market leaders. "Market leaders" in the weakest sense, since it turns out that the best tools for this kind of work are all FOSS: big-ticket corporate testing tools really aren't in this game.

Various points regarding writing and using tests are illustrated with (of course) illuminating examples. Also described are limitations of these techniques and some pitfalls to watch out for, something that more promoters of development techniques should provide.

The book is self-published and my copy was printed by Lightening Source. Books produced this way are getting better all the time, but are still not presented at the level of quality one would expect from a commercial publishing house. The pages seem very full and with the choice of font made the text a very dark colour which I don't find easy to read. The section and sub-section headings are sometimes over long and are not laid out well, a combination that I found made the book less easy to navigate than it might have been.

I will be using with book with clients and recommending it to them for future reference. A boon to the community.

The Quality of non-declining Velocity

I'm supposing that anyone reading this blog will be familiar with the stuff-left-to-do vs time chart used by many Agile teams to track the progress and (with due care an attention) predict the outcome of a development episode:In his Zurich Lean/Scrum/Agile show keynote Ken Schwaber presented an interesting slant on this chart (apparently he got this from Ron Jeffries). The argument is that if you have poor internal quality—in particular if your "definition of done" is not strong enough and does not require you to keep your code in tip-top condition—then there will be a hidden accumulation of stuff-left-to-do. So your chart is in effect more like this:

This is interestingly different from the technique that some use to show scope added during an episode, with a stepped baseline. Here, additional work is accumulating, invisibly, inside your code. Work that you will have to do at some point to get a releasable increment. This has the effect of making a burn-down line more shallow than it appears, suggesting that there will be either unanticipated under-delivery of scope or (worse yet) a need to slip delivery.

If the causes of poor internal quality are not rectified, then this effect will repeat, and over successive episodes the team in question will get slower and slower (or deliver less and less)
Talking about this with Karl Scotland and Joseph Pelrine in the bar afterwards we tossed around the idea that this shows internal quality (traditionally a hard thing to measure) to be something like the first derivative of project velocity with respect to time.

And now to stretch the metaphor to breaking point. For that to make sense quantitatively it seems as if what we're really saying is that if the internal quality Q is less than some threshold Q_cv (the quality of non-decreasing velocity) then the velocity V will decrease over time:

∂V/∂t∝ Q-Q_cv

Well, it seems reasonable that this will hold for low quality, when Q - Q_cv is negative. But what about when Q - Q_cv is positive? Is it possible to take a team that is writing code at the level of quality required for non-decreasing velocity—that is, not accumulating hidden extra work—and then increase velocity by increasing quality?

I think it is.

I think that a team can push really hard on internal quality and have it turn out that there is less work to do than they thought to get finished.

And maybe that's obvious—and maybe it isn't—but certainly I now feel as if I have a much better handle on how to explain to someone why (as the Software Craftsmanship folks say) the only way to go fast is to go well.