s2e03: Any Old Iron

by danhon

0.0 Sitrep

At mumble mumble thousand feet on the way to Chicago and then Madison on a family vacation to visit friends and attend a friends-of-family wedding. It is, bizarrely, going to be significantly warmer and less rainy in Portland than it’s going to be in Chicago and Madison (a difference of a good 12ish degrees Fahrenheit and 6ish degrees Celsius).

I’ve been doing significantly less travel this year than last year. Last year was crazy: getting started with Code for America meant regular short trips down to San Francisco and a team visit to Aviation House to spend some time with the Government Digital Service. My consulting work with Premier Boxing Champions[1] meant regular trips down to the base in Las Vegas as well as all over the country to visit, assess and recommend Original Device Manufacturers to develop what Fast Company called Connected Gloves[2]. That was the work-work; in amongst all of that was a whole bunch of conference speaking that saw me head out to Washington, DC, Chicago and Sydney, Australia and explore in keynotes what I’ve been thinking about empathy in design over the last year. During all of that I kind-of-failed-well-actually-I-did-fail-because-I-haven’t-finished-it-yet to produce a book proposal on said topic of empathy in design despite two invitations to do so. So, you know. No pressure or anything.

Pretty much all of the first half of this year, then, has been swallowed up by the full-time job for Code for America, and it turns out that while remote working is hard, remote-managing a team that’s based in a headquarters is even harder. Like, ridiculously hard. After having spoken to some people who’ve gone through this before, it practically feels impossible, and there feels like some sort of end-state that people gravitate towards that essentially involves having 1:1 meetings *all* *the* *time*, and if not all the time, then pretty much spending the entire time in video conference calls with other people, despite liberal usage of Slack. And this is all against a background of everyone *knowing* that remote working is hard (whether managing or not) and knowing (explicitly, at least) that the strict dedication to communication so remote working can work results in better working for everyone.

But hey: here I am again. Because, I think, writing is healthy for me. So I’m still trying.

[1] Premier Boxing Champions
[2] Connected Gloves and “Bullet Time”: NBC Thinks Technology Can Make Boxing Cool | Fast Company | Business + Innovation

1.0 Any Old Iron

First, an old article about the quality of car manufacturer Toyota’s software quality[1, 2], referenced recently from Hacker News due to what you might call “increased chatter” about software reliability, which is sort of funny if you’re a bit old and jaded about things, especially the sorts of things that the stereotypically younger and less experienced crowd at Hacker News get excited about.

This feels interesting because you can kind of tell a weird story about how Toyota is really good at *some* bits of engineering (ie: the car-making bit) but not about other bits of engineering (ie: anything to do so with software). That first article is damning: Toyota’s code is *really, really bad*. Like, I’m not sure I remember exactly how libel works, but there might have been an expectation that the kind of engineering organization that had played a large part in the evolution and “perfection” of Total Quality Management[3] would apply it to *everything* they made, and not just the physical components, but now you can kind of snarkily say that Total Quality Management actually means Just The Bits You Can Touch With Your Fingers Quality Management. (Although, he snarks, maybe Toyota was also applying Just In Time[4] delivery to their software projects as well).

I mean, the code in question in the 2013 Toyota Unintended Acceleration suit contained “thousands” of global variables as well as what one of the expert witnesses termed a “kitchen-sink task”, overloaded with functions that included throttle control and managing cruise control.

My instinct here is to apply the mediocrity principle[5] again and thus wring my hands at the precarity of this software-enabled world that we live in. In other words: the quality of Toyota’s software for the 2005 Camry isn’t special. It’s not a bad apple. It’s that we can assume that most software (you know, everything from the open-source encryption software behind a significant part of the internet[6] to medical devices that deliver drugs[7] to, say, internet-enabled home automation apps that didn’t really have a good way to quickly remove access to previously authorised users) just isn’t that good. Or that what looks good enough is in fact hiding a mass of dark passages of No Good, Very Bad, Horrible Security And Usability Problems.

So in *one* compartment of my brain, there’s a whole bunch of this stuff. Like: will this ever get as bad as tobacco? Surely, software isn’t killing people in the way that lung cancer kills smokers and second-hand smokers. I mean, it isn’t, right? It’s not like we’d be able to count. The country I live in doesn’t even count how many people the police have killed, intentionally or not! (Cheap shot, I admit). But on the other hand (never mind the click-wrap licenses that we all blithely paw through with their carefully worded indemnities and exclusions of liability and occasional wordings of THIS IS NOT FOR USE IN A NUCLEAR POWER STATION), it’s not like people are reminded that the software that’s shipped and used MIGHT NOT BE SAFE or MIGHT NOT BE SECURE. That’s just a fact of life. You know. Buildings just fall down sometimes when, er, you send them the wrong kind of text message[8, 9].

Anyway. All of that’s in one part of my brain, slowly being turned over by whatever unconscious processes I have, whatever background tasks that churn things over. In the other corner, in the day job, lots of thinking about Procurement, which is a big long word for How Governments (and companies that are big enough to be Governments, I suppose) Buy Things, and in particular (because this is my day job), the specific example of How Governments Procure Technology, which as soon as you look at it that way starts to feel a bit silly.

There are a whole bunch of issues with Government Procurement. So many that if you tried to fix it, you’d take one look at the whole mess and decide to just have a lie-down and promise that you’d make a start on it tomorrow, after you’d had a bit of a rest and you’d cleared your head, maybe gone for a walk or a run or that kind of thing. So you’d have a look the next day and it wouldn’t just be the same mess, it would be a slightly more detailed mess, or a mess that had grown in a new way, or a mess that had decided, whilst you weren’t looking (or sometimes, whilst you *are* looking*) to try to be more intractable in some sort of way that it hadn’t been before. And that’s before you even really get into Technology Procurement.

I mean, we know *why* Procurement is the way it is. It is because we are humans and we have a tendency to do bad things and when bad things happen we write them down so we don’t do them again. It is easier — less work — to write all the bad things down and check to see if we’re not doing them — than it is to figure out what it is we want to do and figure out the Non-Stupid-Human way of doing that thing and, fundamentally, get people who aren’t going to do Stupid Human things either on purpose (in the worst case) or through a lack of having enough time to think things through (in the best case). In our case, of course, we invert all these Bad Things that have happened, so all our rules about procurement are about You Must Do This Thing instead of Don’t Do The Thing Steve Did, which sometimes results in a wording that’s a bit weird and not necessarily clear about the Typical Human Thing someone must have done once that we’re trying to not do again.

Anyway. Government technology procurement. The sort of thing that typically involves buying “solutions” because who doesn’t want a good solution? I want a good solution. I’d love a solution. Solutions are great because you can take some powered software and then add some water and then give it a stir and then you have a solution that you can give to everyone in the department and then when they’ve drunk it (you know, like when everyone in your team or corner of the building does a cleanse together and the toilets in that part of the building are unusable for a month – I’M LOOKING AT YOU, CERTAIN ACCOUNTS IN THE WIEDEN+KENNEDY PORTLAND OFFICE). I digress: solutions are big things, and the problem with using a word like solution (and even a word like system, which might marginally be better) is that the former definitely implies, well, an end. With a solution, you get an answer. The answer. There is no improvement, and certainly not any Continuous Improvement. A solution is, as it were, the Final One. A system, as well, because these things that are Procured are often Delivered, which is why I got all excited when I was in a work Slack channel talking about this stuff and about writing a white paper (where it all fell down a bit, but I hope to make it public) called From Procuring Technology To Delivering Services which, if I’m being perfectly honest, probably gets you *at least* two drinks if you’re playing the UK Government Digital Service Drinking Game (of which: there probably is one, right?).

So. In this corner of my brain: the giant, old technology systems, historically (because we’re in America, and anything older than twenty five years is Historical) deployed on giant mainframe platforms that have been Just Working, Kind Of, for the last twenty-to-thirty odd years and at this point still do what they were supposed to, when they were specified in the late 80s/early 90s, but when you sit them next to the New Shiny Stuff, look a) embarassing, b) slow in response time and slow in workflow, and c) as if, sometimes, they’re actually trying to obstruct what you’re trying to do, and why would they do that? They’re just bits of code running on a humongous heavy piece of old iron.

I mean, look at all these apps. And these phones. Aren’t they great! And the Facebook. The Facebook didn’t cost that much. And loads of people use The Facebook. And from their computers and their phones. And they can take loads of pictures with it. And talk to each other. It does so many things! Why doesn’t $OUR_PROCURED_THING do that?

Well, yeah: because $OUR_PROCURED_THING was designed and delivered in the late 80s and early 90s when there were only so many ways for five hundred people to use a thing at the same time, and now, a good 26 iterations of Moore’s Law later – which is 26 *doublings* of processing capability – it turns out it isn’t that hard or that expensive to do things for 500 people. Or 10,000 people.

So, now: thinking about how you get from a here that has a 20-30 year old legacy, to there, which has the New Stuff that keeps getting better. Which is interesting because in the meantime, the procurement rules have evolved and changed precisely to stop the horrible old legacy system from happening again. Will they? Who knows?! (Actually, we do know. They probably won’t).

No, the big thing about such Systems and Solutions is that to get what technology has been promising (Better workflow! Doing more with less! More communication amongst our staff and between teams!) involves nothing less than a wholesale transformation and *that* isn’t just about delivering software or buying technology. You don’t buy a system – whether off-the-shelf or that someone else is using – that is going to be the magic band-aid, that’s going to change things that much when what you’re fundamentally looking for is technology to deliver massive change management and absolute clarity and focus on *delivering a service*.

[1] Toyota Unintended Acceleration and the Big Bowl of “Spaghetti” Code | Safety Research & Strategies, Inc.
[2] Toyotas Unintended Acceleration and the Big Bowl of “Spaghetti” Code (2013) | Hacker News
[3] Total quality management – Wikipedia, the free encyclopedia
[4] Just-in-Time Manufacturing – Wikipedia, the free encyclopedia
[5] Mediocrity principle – Wikipedia, the free encyclopedia
[6] Heartbleed Bug
[7] Drug Pump’s Security Flaw Lets Hackers Raise Dose Limits | WIRED
[8] New iOS Bug Crashing iPhones Simply by Receiving a Text Message [Includes Fix] – Mac Rumors
[9] Skype falls victim to text chat bug – BBC News

OK. That was longer than I thought it was going to be. A bit rambly. Maybe I’ll write again tomorrow.

Best,

Dan