Episode Sixty Four: Computer Says No, Snow Crashing

by danhon

0.0 Station Ident

As I write this, I’m nestled in a status-upgraded first-class normalcy field at an altitude of around 30,000 feet with a couple megabit’s worth of connectivity, a Diet Coke (naturally) a banana and enough computing power around my person that it’s mostly just idling playing music or displaying text. If it could talk, it would say something like “a brain the size of a planet”. I am on my way back from Missouri to Portland.

1.0 Computer Says No

A side-effect of our current family situation (geographically distributed, divided to conquer) is that whilst my wife and son are with her family I am left on my own in Portland to fend for myself with our cat. It means that since I’m sans fourteen month old, I suddenly find myself with an expanse of free time that, in my evolution-optimised, short-termist brain looks for all the world like a limitless green savannah of movies, books, video games and TV shows, even though my bachelor period is only about a week and a half.

I’ve started on Daniel Suarez’s books[1], which I’ve been meaning to get around to for a while ever since seeing Daemon come out in 2006. Of which: 2006! Eight years ago! That’s got to be at least in the top ten for finally-getting-around-to-read.

I have to admit, Daemon reads like a first, self-published novel, which is fortunate, because it is Suarez’s first, self-published novel. Freedom, the sequel of which I’m only about a third of the way through, is already a significantly improved read. Daemon suffers a lot from Tom Clancy-itis – a meticulously researched and technically accurate thriller that descends into at times a practical step-by-step guide to compromising a computer or network. It’s interesting, because the equivalent in Trek is technobabble (“Geordi’s theory was that if we reprogrammed the warp reaction chamber with a new intermix ratio, and configured the main deflector dish to emit a .47 nanosecond gamma ray pulse, tuned to the precise coordinates and the inverse phase of the harmonics Data detected from subspace rift, we would be able to seal the rift before it got too big and wiped out the inhabitants of Rigel Alpha and still make it back to the Starbase in time for Riker’s jazz recital”) which I find to be equally offensive as Suarez:

“Since he didn’t have the luxury of time, he opted for an attack that was effective against a wide range of devices: SNMP— a buffer overrun that exploited a known vulnerability in unpatched implementations of Simple Network Management Protocol. This service was present on the target, and it was worth a shot. He switched to the command console and quickly keyed in the commands, pointing his exploit code to port 161 on the target machine. If the target was running an unpatched OpenBSD, he’d get to root pretty quick. He executed the command, waited, and in a moment he got a return instructing him to telnet to port 6161 at the target IP address. He sighed in relief.”
– Suarez, Daniel (2009-01-08). Daemon (pp. 154-155). Penguin Group US. Kindle Edition.

Here’s some of the themes in the Daemon/Freedom duology that I found interesting:

Computers control everything these days

Daemon was first published in 2006 and it shows quite a bit. There’s a bit where one of characters uses Netstumbler and I have to admit, part of me squeed a bit and did a bit of a Lexi from Jurassic Park (“Netstumbler! I know this!”). Unsurprisingly, Suarez’s emphasis on technical accuracy is one thing that dates the novel, but I think we can all agree that in a post-STUXNET[2], post-Snowden world everything we’d dreamed about is finally coming true. Apart from, obviously, the jet packs bit. For a book that’s happy to throw down phrases like “unicode traversal exploit” and words like “Netbios”, it’s interesting to note than back in 2006 it’s possible that SCADA hadn’t yet infiltrated Suarez’s consciousness as an in-place framework for a lot of what the Daemon accomplishes.

These days, knowledge of SCADA has spread outside of its initial domain, and Stuxnet[2], discovered a mere three years later, is probably the best example of a multi-vector, doggedly single-minded example of software reaching out and having a defined offensive physical effect. It’s perhaps telling that it was much easier (in the sense that it’s been attempted and, apparently, succeeded) to design a single-purpose software-eats-real-world-centrifuges piece of malware, and that it undoubtedly requires the resources of state actors than it is to design a piece of (admittedly fictional) general purpose distributed software that while dumb is capable of achieving goals through multiple redundancy. What I guess I’m saying is this: Sobol might have had cancer, but he was pretty damned busy (or he kept his two associates super busy) during the final years of his life.

Or, more simply: software for a specific purpose is always much easier than software generalised. Acting on the user need to “disable Iran’s centrifuge capability” is much easier to plan for than “destabilise and reboot civilisation”.

So my understanding is that SCADA’s great and all but it doesn’t necessarily provide a homogenous attack surface and that if you want to do what the Daemon does and take over a lot of materiel quickly, you need custom modules and a lot of preparation.

Narrow AI Is Sufficiently Complicated

Suarez gets a lot better at describing exactly what Daemon is – at the start of the book, Daemon is the equivalent of a Sufficiently Advanced If This Then That Recipe masquerading as variously artificial intelligence or the uploaded consciousness of a dead man. And yes, while all the pieces do exist, kind of, and the idea of a whole bunch of malware, botnet-driven distributed components has *in theory* been possible at least since Sun starting trying to get us excited, so many years ago, about portable network computing and remote invocation, we haven’t really gotten anything quite so complex yet. Later on, in Freedom, we see Daemon described as a “narrow AI” which is a pleasant way of saying that while we might not have achieved the frothy proclamations of the academic AI gang, we do have a bunch of specialised subsystems that do provide some degree of intelligence, if not consciousness.

The bit about using the machinery of an MMORPG for command-and-control of a network, or as a Bond-villain-esque backdoor is a bit genius though, and does set Suarez up for having to explain to countless characters in the books a) what video games are, b) why people play them, and c) that the skills they’re learning by playing those games aren’t useless. One feels like Jon Ross would exasperatedly point Sebeck at Jane McGonigal’s Reality Is Broken and not listen to him until he’d finished it.

Perhaps the hardest to stomach part of the Daemon is its semantic analysis of the news. There’s reference to the mechanical *how* of it – distributed RSS newsreaders and scrapers scanning for keywords – but what we don’t see any of in the book are false positives. I’m not entirely sure how the scrapers would work in practice: we can throw Bayesian filters and train them to recognise spam (which we’ve recognised as spam), but can we similarly train such filters to look for news reports about WACO-style massacres? This is the bit where Sobol’s planning comes in to play: did he define activation keywords for each and every single triggering event that the Daemon required? There’s no mention of self-modification, so we needn’t go down the route of the Daemon generating keyword lists itself (that would be a step too far, for me) but this is the luxury of fiction: it just works.

The Invisible

This is a bit more developed in Freedom than it is in Daemon, but roughly Suarez explores the idea that unseen forces shape the physical world that we live in. All of this is somewhat couched in the language of gaming and Suarez deftly uses augmented reality to bring to life and bring sight to he who couldn’t see. This idea of a data shadow, a clickstream plume that is emitted as a result of your activity and never really dissipates, has been gaining a lot of steam lately, if only because a) it’s true and b) I think it legitimately preys upon peoples’ contemporary fears of not really being in control. While it’s true that there *is* a lot going on out there that isn’t visible or given a tangible presence, ie. that data moves in mysterious and opaque ways, where I feel the books get a bit fantastical is in wrapping all of that data in a user-friendly queryable interface. I think it’s fair to say that if there were an easy way to aggregate, query, disambiguate and sort through all the chaff, our security services would probably be doing a better job than they are. Quite frankly the most fictional, implausible aspect of all of this is that someone’s figured out a way to let regular people conduct accurate, queries across numerous stupendously large datasets.

Follow the instructions to be part of a team

There’s a whole set of examples of people blindly following instructions (and the variant, of people choosing to follow opaque instructions) in Daemon.

What we see early on is a fictional illustration of the predisposition people have to follow along with instructions and a narrative, just because, well, there’s instructions and a narrative. Almost a sort of games-as-Milgram experiment. We’ve seen this before in games like BioShock that (spoiler!) remind us exactly how good, and how willing, we are to follow instructions when the surrounding circumstances are constructed just right. Unfortunately, BioShock never really went that far in exploring the relationship between a computer “user” and the system encapsulating her, but later games like Stanley Parable pushed the genre a little further. Structurally, I like the way Suarez has set this up: we have Graggs playing through an Over The Rhine map that’s a good example of 3D environmental storytelling that has been honed to a fine art by Valve and latterly the Call of Duties of our time. That then gets followed through with the mechanic of our characters responding to Daemon generated prompts that are only really answerable with a yes or no, and the equivalent of characters who are in such controlled physical environments (the garage that Gragg encounters comes to mind) that all they have to effect their surroundings is the equivalent of the spacebar USE key.

There’s a side-alley that I feel Suarez could’ve gone through here: essentially, he’s talking about control and the way in which we can be influenced without our executive function necessarily catching on quick enough. The canonical TED-talk examples for this are things like supermarket grocery layout design and the sight-lines that you get in controlled environments for different purposes, like Disneylands and Disneyworlds. There’s a way in which Suarez could have made a point, if he wanted to, about putting people in stressful environments and showing how easy it is to manipulate them, but he didn’t really have to do that. The garage sequence with Gragg, though, where Gragg inexorably follows instructions to the point that he kicks himself immediately after following them because the instinct to follow is just that: an instinct that is difficult for him to consciously override, is a great example of how a videogame level designer might think about real-world architecture in producing an action/influence funnel.

Toward the end of Daemon, there’s a nicely choreographed set-piece that could’ve looked interesting on film that a nice one-take Alfonso Cuaron shot would’ve done justice. At this point, the Daemon has spread to a not insignificant proportion of the population, and orchestrates a sort of deniable hit on spammers, directing and controlling tens or even hundreds of human agents at once to deliver and assemble the assassination weapon. This is the kind of thing I’d like to see a DARPA Grand Challenge-style event for: for something like that level of orchestration to work, it feels like you need pretty granular location services and wayfinding, all operating in realtime, and all without your battery dying on you. Of course, by this time, Daemon is fabbing a whole bunch of high-spec Glasses, and we’ve entered into the slightly more Star Trek toy universe. The key word here I think is choreography, and that’s perhaps a clearer way to think about what’s being described. Imagine a ballroom and everyone wearing earpieces and a choreographed dance happening, in realtime, but through voice prompts, where none of the participants have advance knowledge of the routine (and may well have had no training, either). Anyway, if you love that sort of stuff you really should’ve read Bruce Sterling’s Maneki Neko[3] by now.

In Freedom, there’s a little Falling Down-esque moment in a fast food joint with Sebeck nearly losing it due to employees literally following what the computer’s telling them to do to the letter, in an Americanised Computer Says No[4] moment. You get to illustrate this quite a lot with the enough-truth-to-them-to-not-be-entirely-apocryphal stories of people blindly following GPS instructions (because, of course, we’ve been taught that Computers are Perfect Computing Machines and that a bug or error is when the screen futzes with static or blue-screens, not returns inaccurate data) or even through casual reading of the RISKS digest[5]. Again, this doesn’t necessarily tell us anything about computers: it tells us about how we interact with machines and the way we’re encouraged to trust them.

At least from the beginning third of Freedom, it’s good to see the more nuanced (in parts – it’s still a screed against capitalism, I think, but also firmly planted in the Californian Ideology of technology saving the day) plot. It’s an interesting perspective to see how widespread AR might develop from a gaming-first community. There’s a lot of CI tear-it-down-to-build-it-up “disruption” going on (which is why I had a feeling this might inevitably turn into a set of opinions about Captain America: The Winter Soldier, of which more later this week). That said, there’s a lot of handwaving, too: the way to bootstrap a new civilization is by plundering an old one, which Suarez amply points out is the way we’ve always done things. It may not be nice and it may not be what we want, and the devil will always be in exactly how much tearing down needs to take place. It’s not really disruption if it’s managed, is it.

[1] Here’s a list:
Daemon (Amazon: http://amzn.to/QCyg3L, Powell’s: http://www.powells.com/biblio/9780525951117, Abe Books: http://www.abebooks.co.uk/Daemon-Daniel-Suarez/9555111019/bd)
Freedom TM (Amazon: http://amzn.to/1gP4vTX, Powell’s: http://www.powells.com/biblio/1-9780451231895-2, Abe Books: http://www.abebooks.co.uk/Freedom-Daniel-Suarez/11483682581/bd)
Kill Decision (Amazon: http://amzn.to/1fnBYo4, Powell’s: http://www.powells.com/biblio/1-9780451417701-1, Abe Books: http://www.abebooks.co.uk/servlet/SearchResults?isbn=0525952616&cm_sp=mbc-_-0525952616-_-all)
Influx (Amazon: http://amzn.to/1k5m3z8, Powell’s http://www.powells.com/biblio/1-9780525953180-0, Abe Books: http://www.abebooks.co.uk/servlet/SearchResults?isbn=9780525953180&cm_sp=mbc-_-9780525953180-_-all)
[2] http://en.wikipedia.org/wiki/Stuxnet
[4] https://www.youtube.com/watch?v=WOdjCb4LwQY
[5] http://catless.ncl.ac.uk/Risks/

2.0 Snow Crashing (6)

It’s been over ten episodes since I’ve done a Snow Crashing! So here’s a new one, which is exciting, because in this installment we get our introduction to Snow Crash, the drug.

So I think there’s something interesting about black-and-white-rendered photocopy dude, the guy who’s stood in the throng at the entrance to the Black Sun. For starters, Stephenson asserts that “Street protocol states that your avatar can’t be any taller than you are,” which strikes me as a) completely missing the point of VR in the first place, and b) ridiculously unenforceable. Much easier to make everyone the same height, or only within a user-definable range, than to actually stipulate that your height in a virtual representation may not exceed *your actual physical height*, which instantly renders everyone going through puberty on the ‘street with an interesting problem.

Anyway: photocopy dude who’s slightly taller than everyone else in the crowd for a reason that doesn’t stack up *manages to grab Hiro’s attention*. Which is another way of saying: look at the attention economics of this place. This isn’t a crowded inbox and Hiro’s not clicking on a phishing scam. This is someone who has to, in a way, be physically present in order to project presence, an interesting enough presence at the right time, at the right place, to attract Hiro’s attention to get him to try Snow Crash.

Snow Crash, as Stephenson tells us, gets its name from computer lingo – describing a low-level hardware failure so bad that it makes the electron gun in your CRT skitter wildly across the screen, giving you the colour that the sky above the port’s turned to. Unfortunately, time hasn’t been kind to Snow Crash: we’re not really using CRTs anymore, and even in Hiro’s world, he’s not using CRT’s either, really: the goggles that he’s using are using RGB lasers to paint directly onto his retinas.

There’s a bit of a disconnect too, in that Hiro’s weirdness detector goes all the way up to eleven because of photocopy dude’s “utterly calm solid presence” that nonetheless is affixed to an avatar that “[breaks] up into jittering, hard-edged pixels”. And then we get to the next exciting bit, because Stephenson has just introduced us to Hypercards.

The card that pusher hands to Hiro is a Hypercard – as Stephenson explains, a hypercard is “an avatar of sorts” – by which we’re explained it’s an avatar in the way that it exists in the metaverse and it’s a representation of information, just the same way that an avatar is a representation of a human. The hypercard is a representation of data, such data being anything that can be digitised. What’s interesting to me here is that Stephenson’s clearly played with Hypercard-the-real-software, and the description of Hypercard in Snow Crash means that what he took as being transformative about it was the packaging of multimedia. A Hypercard in Stephenson’s world is a stack – a set of digitised information that can be thought of as a collection and navigated. Back then, the notion of linking sounds like it wasn’t as clear: mainly because you’re linking between resources inside one meta-resource as opposed to resources located on different parts of the network. Which is why Stephenson was *so close* to describing something like the internet because he had what a lot of people consider in some ways to be a grandparent to the web with his hypercard analogue, and also a global telecommunications network with stupendously low latency and high bandwidth. But all that was missing was the humble link anchor.

It’s 10:46pm on the West Coast, which means in Missouri, where I was this morning, it’s past midnight. I’m a little behind on replies to the notes that you’ve sent in, but I really do appreciate them. And if you’re new to the newsletter, check out the archive at http://danhon.com/newsletter/, because there’s a bunch of other stuff I’ve waffled on about. And drop me a note and introduce yourself.