s19e11: Your Model is Naive and it’s Not The Territory
0.0 Context Setting
Friday, May 30, at the Code for America Summit in a hotel in Washington, D.C.
I am in, fittingly, a hallway. I am also jetlagged. There is hotel carpet underneath me.
0.1 Events
At the time of pressing send, Hallway Track 12 The Thing About COBOL is full. I think you can check back every so often if there happen to be cancellations. There isn’t a great way for me to let people know about spaces opening up because of cancellations with Calendly.
1.0 Some Things That Caught My Attention
Just one thing today that I needed to vomit out through my fingers.
1.1 Your Model is Naive and it’s Not The Territory
In his piece What DOGE gets wrong about tech and government1, Don Moynihan has successfully nerd-sniped me into writing about DOGE again. For anyone who’s not interested in that, or by now is tired of it, I am very sorry. You can just skip this one?
I think DOGE ran into a The Map Is Not The Territory problem2. This is the whole thing where you want to make a map of something (duh) and then at some point you make a more accurate map because your map is missing something, and you keep doing that until your map is the thing you are mapping. Maps are abstractions and artifacts of decision-making as to what’s important. They’ll reflect what’s important over time. All of you map nerds reading this will hopefully be nodding along, and I will say again that this is me speaking off the cuff. I imagine that there is extensive literature on the subject of what the entire deal is with humans and maps. (I mean, not all the maps we make are for humans!)
The thing with a certain kind of technologist is that the power of software works when you can reduce something down to a model. When the model is sufficiently simple, then it’s easy (and fast) to get the model to do something. That might make you feel powerful.
The other thing about a certain kind of technologist, especially the kind that tends to hang out on orange websites, is they will say things like “hey, check out the Twitter I built over the weekend for fifty bucks” or “hey check out the vaccine appointment website I built over the weekend for fifty bucks”.
In the first case, they did not build “a Twitter”. They built a toy model of a Twitter, a thing that looks like Twitter but -- just like the relationship between a map and the territory, the toy model is not the thing, in part because it just doesn’t do what Twitter does that’s not visible. Building a Twitter includes, I don’t know, also building all of the trust and safety infrastructure. As soon as you start qualifying, the comparison is, for these particular purposes, useless.
Kind of the same for the “I built a vaccine appointment website” thing. This is not saying that the vaccine appointment website wasn’t useful. Not at all. But I am saying that it’s an example of a smaller model that is a sort of abstraction of the more complicated thing.
What with the apparent post-Musk comms onslaught about how he’s totally chastened and everyone can celebrate that he’s “left” and DOGE will be different now, I thought a couple things were worth picking out: he said something along the lines of “turns out government was bigger and more complicated”.
Between that, my conversations with journalists about what DOGE was planning or wanted to do, and with people formerly in the Federal government, all of these things stuck in my head.
The kind of person who says that you can rebuild social security in 3 months is the kind of person who also says “twitter is easy, I can do it in a weekend”. It is a fundamental misapprehension of the scope of the problem and complexity that the thing (“social security”, or more accurately, “the things the social security administration does”) addresses and is charged with dealing with. “Twitter” does not reduce down to “microblogging”. Not anymore.
Government is complex because at various points, it has to map to the territory. And even then, the part that’s interacting with the territory at that point is a part of an abstracted part that’s a policy -- which in itself is based on a model in the first place.
Government is big because the fact that it’s supposed to serve everyone means it is probably the closest model:territory ratio you’re going to get. Ever, maybe? At least for some people, the remit and scope of government is supposed to be everyone. It’s clear that for a whole bunch of other people -- i.e. fascists -- government applies to different people in different ways, and explicitly has ways of declaring, and with the ability to do so arbitrarily, people as non-people.
There are policy levers where you can decide it’s important to simplify the model you need to make. These are ideological. It would be cheaper, easier, and faster to just give people money in the form of a universal benefit or credit than it would be to include various forms of means testing. Those forms of means testing exist just because some people don’t deserve it. You can cut out a bunch of complexity on the model end.
But you can never cut out the complexity on the reality end. There will always be edge cases; it’s like humanity and life has only gotten to where it is because of edge cases. The “government is for everyone” part is figuring out how to meet the needs of those edge cases (and efficiently so, depending on your placement of efficiency of implementation and administration versus efficiency of outcome), and that will require modeling to meet complexity.
I didn’t want to and didn’t think that this would turn into an AI rant, but I guess everything does these days. I mean, people see “AI” as a magic bullet that’s being promoted to solve everything, and especially things that have previously been hard. And it appears to solve those things in a magic way, too!
Part of the concern about AI is that it abstracts complex models away. The architecture of certain neural networks literally has the concept of hidden layers. The models are complex enough when encoded in legislation and then a business rules engine, but now the models are potentially implicitly encoded into your LLM by you “just” writing a prompt and setting out your criteria.
People like AI because it acts like a sponge, a gas, a perceived infinitely malleable material that adapts to whatever shape is presented to it. It will fill a space in the same way that cats do. It will meet your reality in the way you want it to, or at least it will appear to, and seduce you with the promise of you not needing to do all that thinking about edge cases, in part because Sam Altman’s religion is that if you give it enough data, it will understand how the world works.
But anyway.
You go into government thinking you understand the point of something and you have a model of it in your head. “It’s just a case management system!”. And that’s true, and at the same time, shit is fractally detailed. You can handle that well because you know that shit is fractally detailed, or you can handle it in a very bad way where you keep building onto a shit mountain that at the very least works.
The world is not the model. The reason why (the good) people in tech keep talking about user research and actually talking to and listening to users is because that gets you closest to the territory.
If you’re a naive technologist coming in, a young grad who’s smart and can figure things out from first principles, then your model is going to be wrong because you don’t know what level of fidelity is needed and where. A whole bunch of people thought they did and flamed out.
(I say this about government, it’s not really about government. It’s about everything).
Hello, I am between sessions (actually, hanging out before my panel) at the Code for America Summit. How are you doing?
Best,
Dan
How you can support Things That Caught My Attention
Things That Caught My Attention is a free newsletter, and if you like it and find it useful, please consider becoming a paid supporter.
Let my boss pay!
Do you have an expense account or a training/research materials budget? Let your boss pay, at $25/month, or $270/year, $35/month, or $380/year, or $50/month, or $500/year.
Paid supporters get a free copy of Things That Caught My Attention, Volume 1, collecting the best essays from the first 50 episodes, and free subscribers get a 20% discount.