0.0 Context Setting

Monday, July 27 in Portland, Oregon. That’s it. That’s all the context I have for you. Oh, I guess it’s 4:50pm and I started writing this on something like Wednesday last week.

0.1 Events: How People Work, Live!

How People Work, Live! is the live show about how to people. It’s based on my How People Work, my workshop that helps teams deliver better, faster, which itself is built on my last 20 years of working in tech.

Each show I’m joined by a very special guest and any number of interesting people (that’s you) to find out how to get hard results by learning and applying soft skills. Why? Well, because this marketing copy says that success isn’t just about what you do and whether you’re good at it -- it’s also about how well you work with other people.

Spoilers: other people are EVERYWHERE. I mean, come on.

The show page has got previous recorded episodes (you can see those on YouTube, too), as well as details of upcoming shows you can register for.

Coming up next is How People Work, Live! ... with Russell Davies. Who’s Russell? Why, let me tell you:

Russell Davies is a writer and strategist who has spent over 50 years thinking about what happens when marketing and services meet the internet. He's worked on communications and digital strategies for organisations like Honda, Nike, Microsoft, Apple, the Government Digital Service and the Co-op. He's currently Marketing and Product Director for The Modern House.

So come see Russell and me on Friday 31 July at 9am Pacific, 12pm New (“He’s Just Doing Things?”) York, and 5pm “Madchester”.

There’s lots more after that, too. Check out the show page for more.

1.0 Some Things That Caught My Attention

1.1 The Irritating Aspect of Spiciness

I’ll start by saying that the technology exists. I agree that it has been borne of an offense against society in how the technology was developed that can be seen as (and is, in some peoples’ experience) moral injury. I agree that the technology was created in an unjust economic environment that is harming people. All of that is true.

I also agree that there are genies that can be put back in bottles, but we cal also agree that it is exceedingly difficult (perhaps even impossible!) to completely revert something, but that there are reversions that result in significant societal benefits. One example that crops up is the collective benefit of banning smoking in public places, of which there were numerable benefits. I cannot honestly be bothered to go into whether there was a net negative economic impact, I suspect there wasn’t, and in any event would consider suspect the scope of any such studies insufficient. I mean, sure the tobacco industry might have taken a hit and there been subsequent unemployment, sure local businesses selling cigarettes might have taken a hit but again in my ivory tower-esque position I will betray that I simply do not care. I did not really know of people with lung cancer apart from the nationally loved Roy Castle¹, but what I did appreciate at the time was that my clothes and I did not stink of smoke after going out to the pub.

Terrence Tao is one of this generation’s leading mathematicians. In 2006, he was awarded the Fields medal² (which you get for like, being, stupendously good at maths and also (weirdly?) for also being under 40 years old) and became a MacArthur Fellow.

My (internet?) friend Kelly Link is also a MacArthur fellow, she writes amazing fiction and also runs a small press and a bookstore. I love how the MacArthur foundation gives people the space for this. Imagine if everyone had etc etc etc, sigh.³

Tao recently shared the conversation he had with ChatGPT⁴ about the Jacobian conjecture counterexample. For the purposes of this newsletter, let’s just say that it’s a maths things that’s very maths and clearly the kind of thing that someone maths a lot is into.

It’s interesting to see someone who’s clearly accomplished (did you not see all the credentials just there?) use what’s described as spicy autocomplete talk through and do substantial work on a problem. I’ve seen it described in places where it’s all fine for spicy autocomplete to suggest (mediocre, average) writing for your email or update deck, but an entirely qualitatively different thing when someone is validly spicily autocompleting an erdos problem!

What adds insult to injury on top of that is that to date a bunch of work has been done in terms of waves hands prompt engineering, which I’ve always found amusing because it’s the fuzzy human language equivalent of trying to describe what you want to achieve. In the early days of language models and especially in image generation you had people selling prompt collections that you could use, literally magical incantations that would include phrases like “realistic 4k HDR” to get images that look a certain way. Just that practice is also hilarious/interesting because in the way the models work by ingesting a ton of data and then doing maths on it in order to do operations on concepts and so on, humans are doing the same thing in reverse in order to figure out/fuzz the inputs to the machine in order to get the results that they want.

IN OTHER WORDS, this is just the lossy process of communication when you’re working with a black box, otherwise known as “another human” or, I don’t know, sufficiently complex and opaque tool. It is weird to people because computers are supposed to be programmable and “smart” but the thing to understand here is that when you ask an LLM to count the number of letter Rs in strawberry, it is not doing what you’ve previously expected and experienced a computer doing, which is running a, uh, program (Dan, isn’t a program just the same bunch of mathematical operations over data as an LLM is? I mean YEAH but also NO) over a bunch of data. LLMs don’t do that! They really are, at the highest most naive level trying to predict based on their training data.

The thing is, they got better -- or should have gotten better -- at counting letters and so on because of tool use, which is to say an LLM could generate text that is far more accurate in counting letters in a phrase if “it” used software that did that and the software was written in a way to past tests for, you know, actually counting letters in phrases and validly did so.

Where was I. Right, prompting.

The disturbing thing that is a fact, a thing that happened, a thing that is true, is somehow this statistical bag of tricks is now able to solve a previously unsolved mathematical problem using a prompt like this:

Construct a counterexample to general (non-planar) case of Dinitz Garg Goemans conjecture. You should do a breakthrough and find a structured counterexample.⁵

“You should do a breakthrough” is genuinely hilarious in the context of what was previously understood to be necessary for “crafting” prompts to get the results you want.

What’s going on here? Getting down to such a simple prompt feels startlingly like being able to instruct a tireless maths phd to, like, just solve the problem? I want to acknowledge here that sure you could try the phrase “you should do a breakthrough” on older models and see if that works -- and I think in some of the cybersecurity examples, older models have been shown to match the newest ones -- you don’t always need the newest capabilities. And I don’t know either if using the same model you could’ve gotten the same result without the phrase “you should do a breakthrough”. It’s almost funny it’s insulting?

So I think we need to accept that it’s unhelpful to characterize these systems as spicy autocomplete. I mean yes that’s what they do in some situations but now I see that phrase as more like an expression of frustration at the entire socioeconomic system that led to these systems and the power structures around them, and the power structures that they are being used to perpetuate.

This goes back to the jagged capabilities of the systems. There are more examples of them. One might wonder ok sure but what are the civilian applications⁶ of doing maths breakthroughs and I will happily tell you that I don’t know and that it likely depends on, like, the exact maths?

What I’m getting at here, the irritating part is that there is an incredibly large domain of tasks to which people are putting generative AI to, to the extent that you could just treat them as a giant wall and people seeing what will stick. This throwing tactic is unsurprising given the sheer amount of marketing involved and the jaggedness. You just don’t know how the application will turn out, and maybe you’ll be surprised?

It is not surprising, then, that the areas in which generative AI is having the most success is in tightly defined, provable domains. Maths is one of them, and honestly I’m sad to say that it’s plain inaccurate that generative AI doesn’t generate working code.

You could drive a truck through what “working code” means, though, and I think an easy way of understanding the utility in maths-vs-can-it-code is that can-it-code has so much more of a surface area in terms of “for humans”. As soon as you involve humans and move outside of the abstract “solve this maths conjecture” domain then you end up hitting “did it solve the problem” and “did it do it safely” and “how updatable is it”, which is to say that all of those answers lie on a continuum and they all have to do with the other human(s) on the end of the software.

The funny thing about the software stuff is that say you can instruct the generator to come up with a plan and in the same instruction say “and then go through the plan and improve it” and... the plan will be improved? This is another example of expectations being confused: you’d expect a computer to be able to get a plan right the first time, but this isn’t a computer, this is a bag of statistical tricks that is getting increasingly good at producing useful output for people. It seems reasonable that getting the bag of tricks to iterate over a plan and find ways to improve it is what you’d instruct a human to do too.

Anyway. I still think the writing generation is terrible. But see here, I think this might be interesting? I keep seeing more and more slop writing. One of the last pieces I saw was someone who posted to the cognitive science subreddit something that was clearly generated. The top reply was “ok, but what do you think?” and the reply to that from the original poster acknowledged that the original post was indeed generated. But then their reply to “ok but what do you think” was also generated. The original poster justified that by saying they are super busy, don’t have much time etc but they did check the text and it was generated based on their opinion. (I thought it was still bad text).

I learned a thing as a baby lawyer that you can sign a thing as pp, where indicate that you are signing something on somebody else’s behalf. It’s a clarifying signal to me:

• the text is to be treated as “official”
• the text was authored/generated/provided on behalf of a certain individual

Say you’re super annoyed at something and you’ve got the money to tell a lawyer to tell the person you’re annoyed at to stop doing the thing. You get your lawyers to write the letter, but it would, you feel, be more effective if it came from the lawyer. The lawyer could send the letter on their letterhead and the particular lawyer -- or the firm! -- could sign it pp on your behalf, so your name is still on it.

I think it would be nice to treat AI generated text that way. That reddit poster thought what they were doing was fine. Some readers clearly think it is useful signal to know the provenance of a text/how it was generated, whether it was an autotext or a generated text. I know there are shades and degrees here, let’s just agree that if I’ve used autocorrect then I’ve still written it, but at a certain threshold an amount of predictive text chaining together or authoring eg sentences or phrases sufficiently tips the scale.

Because look: some people have trouble with writing! That is not their fault. Writing clearly is difficult. The intent of writing can be to clearly communicate what’s in your head into another head, and if that’s hard that could be frustrating. Help doesn’t have to be a bad thing. And! I think it’s helpful here to start delineating idea from execution. To be able to say “I had these thoughts, and they are being expressed via this tool” is an important signal. That’s where a lot of split is happening anyway, with the “I had this creative idea, and then I used this tool to express it”. I do think there’s a difference between “I had an idea for this, and here’s the output from a tool” versus “here is the film I made”. Sure, we can talk about photography later.

I know all I’ve done here is talk about authorship. But the connection back to signing-on-behalf caught my attention. It is not like we would be adding something radically new here.

1.2 Improving Pharmacies Somewhat

This is based on a short thread/rant I made on Bluesky:

on the one hand, it would be good to know how long the queue is before I go to the pharmacy

on the other hand, total information awareness and private/public late stage capitalism surveillance⁷

See the thing here is that knowing that little bit of information would be useful. It’s like knowing how long the queue is to get your drivers license renewed if you need to go to the DMV in the U.S. and you don’t live in a country or state where you can simply do that online. It’s a nice Customer Relationship Management thing. There’s probably a whole book about how frog and IDEO and Disney worked on it and how queue management can be applied to Your Business through the magic of Design Thinking.

Anyway.

There are ways you could do this! You could point a camera at the queue. That would be a thing someone would think of. Then you’d run some vision processing on it, see if you can count the people, put all the bounding boxes on them, all that jazz. Then publish that somewhere.

But what you’d have to do is you’d have to resist the pressure to do even more with that data because, well, you have it now, right? And now that you have it you have to justify the investment in gathering it in the first place and processing it. So you may as well keep it, too. And now that you have it and you kept it, well now you can supply it to other interested parties. It’s all data. You could cross it with the cameras you have at the other bit of the grocery store for the self-checkout or even at the entrance. But you don’t even need to do that because you’re already tracking a customer profile and phone number for the loyalty points. But hey, why not, right?

But maybe you don’t do it that way. Maybe you’ve got BLE beacons around the store so you can log what phones have been where and disambiguate those and tie them back to identities and maybe you can log the dwell time around the pharmacy that way.

Or maybe you don’t even do it that way but you still have cameras. You just need to recognize whether there’s a line and then measure how long the line is, no need to recognize anyway.

And all this for want of a “it would be helpful if I knew how long I’d have to wait to pick up my meds and whether I should go now or later”. And anyway, that problem is solved by delivery.

And! You, the store or pharmacy operator, I bet you could share that data with Google and Google could figure it out for you because Google Maps and Search totally wants to be able to help your customers know whether this location is Busy at This Time. I bet Google would do it for free for you! I wonder why. Wait, no I explained why already.

Why do you even want to know whether it’s a good time to go to the pharmacy? Oh, you’re probably busy. Got other things to do. Probably working on your grit or resilience or whatever during These Times. It would be nice for it to not matter, hm?

All because it might be convenient to know that little bit of information: how long is the queue right now? And then it explodes into a surveillance apocalypse. Thanks, entire economic environment.

That’s it! That was a lot. I didn’t mean to. I’m sorry. It made sense?

How are you doing?

Best,

Dan

PS. My consulting dance card is opening up. Do you know someone who really needs my help? Let me know.