Life in APL, programming in a live environment

This gorgeous screencast of an APL session shows an implementation of Conway's Game of Life being derived in APL.

It's a very delicious demonstration of what can be achieved in 1) a "live" computational environment (rather than a mere language 'n' platform) with 2) a language that works by evaluating expressions (rather than merely marshalling operations) and 3) already knows how to show you what complicated values look like (because it understands its work to be symbolic not operational).

Ponder what the Dyalog folks (no affiliation) are showing you there, ponder just how far towards that same goal you'd get in whatever programming system you use at work in 7 minutes 47 seconds (even if you'd done as much rehearsal as I'm sure they did) and then ponder the state of our industry.

Then have a stiff drink.

Agile Coaches Gathering

I'm going to the Agile Coaches Gathering at (and partly in aid of) Bletchley Park in May. I commend the event to you.

The Lives of Others, the Lives of Ourselves

I've kept to a rule with this blog, that the posts on it should be of broadly technical and mostly professional nature. I'm going to break that rule now.

Some time ago I watched the film Das Leben der Anderen, a bitter-sweet offering from Florian Henckel von Donnersmarck. It's a love story, of several kinds, and a thriller and little bit of (and little bit of a response to) an Ostalgie comedy. In one scene Hauptmann Wiesler, a Stasi interrogator and surveillance officer, teaches a class of prospective secret policemen. He puts great emphasis on the importance of the handling of the seat cover after an interview. This was a real thing: during interrogation a cloth would be used absorb the personal odours of the subject and then sealed in a jar. When, not if, the Stasi needed to find you the jar would be unsealed and used to give tracker dogs your scent.

I recall hearing about this in a John Peel piece many years ago. He went to interview some punks in the former DDR and they spoke of this and many other horrors of living in an authoritarian surveillance society. And this sort of thing was considered a genuine horror. A fine example of the reason why it was worth the grinding fear of the Cold War, to avoid living in that sort of society.

Between the time of watching that film and writing this piece I got into a conversation with a Daily Mail type of fellow, who was complaining that "the UK is becoming a complete police state. Like[sic], on the level of Nazi Germany". This is well known to be a bad way to argue. My response was that it wasn't really, not even on the level of Communist Germany, go watch Das Leben der Anderen to see what a real police state looked like.

Today I'm thinking that I might owe that guy a little bit of an apology. This report [pdf] from the Rowntree Reform Trust makes dismaying reading:
  • A quarter of the public-sector databases reviewed are almost certainly illegal under human rights or data protection law
  • Fewer than 15% of the public databases assessed in this report are effective, proportionate and necessary, with a proper legal basis for any privacy intrusions
  • The benefits claimed for data sharing are often illusory.
Well, joy.

Top of the list of problematical databases is the National DNA Database. A little background: England and Wales (Scotland and Northern Ireland have their own arrangements) are Common Law countries and like other such used to have a distinction between summary offences, misdemeanours and felonies. We don't any more (much as we don't have Grand Juries, either) we only have summary vs indictable offences. But that only applies if you are brought to trial.

We used also to have a distinction between arrestable offences (effectively, felonies) and non-arrestable offences. Over the years so many more offences have been created that this distinction was seen as unhelpful and at first it was blurred and eventually, in the Serious Organised Crime and Police Act 2005, abolished.

Under section 110 of that act a constable may arrest, without warrant, anyone he or she "has reasonable grounds for suspecting to be about to commit an offence" (any offence at all, remember, however minor) if this is "necessary". For example "to enable the name of the person in question to be ascertained" (and for various other reasons listed in the Act). What this amounts to is a summary power of arrest, of anyone, at any time. This was spectacularly under-reported at the time. And here's the punchline: everyone who is arrested is obliged to give a DNA sample and that goes on the National DNA Database.

And it stays there. It stays there if you are released without charge. It stays there even if you are charged, go to trial and are acquitted. This was already known to be in breach of the European Convention on Human Rights. In that unanimous ruling the European Court of Human Rights made a number of interesting statements:
The Court was struck by the blanket and indiscriminate nature of the power of retention in England and Wales. In particular, the data in question could be retained irrespective of the nature or gravity of the offence with which the individual was originally suspected or of the age of the suspected offender; the retention was not time-limited; and there existed only limited possibilities for an acquitted individual to have the data removed from the nationwide database or to have the materials destroyed.
And indeed it turns out to be non-trivial to get the DNA of innocent persons removed from the database.

The Court noted that England, Wales and Northern Ireland appeared to be the only jurisdictions within the Council of Europe to allow the indefinite retention of fingerprint and DNA material of any person of any age suspected of any recordable offence.
The Court expressed a particular concern at the risk of stigmatisation, stemming from the fact that persons in the position of the applicants, who had not been convicted of any offence and were entitled to the presumption of innocence, were treated in the same way as convicted persons.
So we are, here in the UK, faced with a police system that in this regard at least makes no distinction between the innocent and the guilty. And on a grand scale. The full version[pdf] of the RRT report states that
Over half a million innocent people (people not convicted, reprimanded, given a final warning or cautioned, and with no proceedings pending against them) – including over 39,000 children – are now on the database
and to no good effect
doubling the number of people on the database from about 2m to about 4m has not increased the proportion of crimes solved using DNA, which remains steady at about 1 in 300. Indeed, in 2007 the number actually fell slightly
I'm no longer sure that I know how this is much different from the Stasi bottling seat covers.

Except maybe that we do still have a chance to do something about it. A shame that we must appeal to the European institutions for protection from our own executive. Time to join Liberty. Hope I'm not too late.