Clarifying Beeminder's AI Policy: Writing, Coding, Images

Bee’s introduction: Today we’re welcoming a very special guest to the blog to bring us this clarification on AI slop. Daniel Reeves has a PhD in Computer Science — artificial intelligence specifically — from back in the day when artificial intelligence included things other than neural networks. He’s been publicly putting his money where his mouth is on AGI timelines since 2008, when doing so made you sound about as crazy as if you were talking about timelines for the Rapture. More recently he’s been musing weekly about the possible AI trajectories we’re on with his patented blend of excitement and confused terror over at AGI Friday. (Yes of course he beeminds it.) Take it away, Dr. Reeves.

Thanks, Bee! To be clear, in 2008 my big wager was that human-level AI (AGI) was more than 10 years away. I would not make that bet (at least not confidently) today. Ok, diving in…

I find the following Levels of Autonomy a useful framing when talking about AI and where to draw red lines:

0. No AI — all human
1. Tool AI — no autonomy
2. Assistant AI — some autonomy but the human’s always in the loop
3. Directed AI — the human passively supervises without micromanaging
4. Autonomous AI — the human provides input when the AI asks for it
5. AGI — no human

(Also we refer to LLMs as golems, coined by our own Clive Freeman.)

Writing

Those autonomy levels originally were for self-driving cars [1] but we can map them to writing like so:

• Level 0 is old-school writing (word processors allowed)
• Level 1 is spellcheck, grammar check, and even LLM-powered thesauri
• Level 2 means AI offering ideas the human picks and chooses from
• Level 3 is human-curated golem-generated prose
• Level 4 is human-prompted but the human isn’t vetting it
• Level 5 means no human involvement at all.

To be clear about my bias, AI-generated prose is like fingernails on a blackboard for me. As I once put it on AGI Friday, as a reader I recoil from writing that’s remotely redolent of robots. [2] I’ve actually been super uptight about this kind of thing since before generative AI, like in the 2014 Beeminder blog post, “Don’t Be a Smarmbot”.

“No golem-written prose can ever be used without explicit attribution”

Beeminder’s red line for writing is strictly between level 1 and level 2. [3] Specifically, the rule is that no golem-written prose can ever be used without explicit attribution. Basically the same common-sense rule you use with people. Don’t use anyone — or anything — else’s words as if they were your own. Not that I’m suggesting the ethical line is the same as for plagiarizing a person. AI is not people!

In fact, I’m sappy enough to think that the reason to be persnickety about this (besides the blackboard-fingernails) is the sanctity of human-to-human connection. The beauty of human language is how it lets us get into each other’s heads. Using AI-generated writing, even just a phrase here or there, is a violation of the social contract between writer and reader.

You can trust that every word you read on Beeminder, if not attributed otherwise, is typed with our bare hands. Even the text on buttons and drop-down menus. Which brings us to…

Coding

Code is another story. I think the right analogy here, if not quite today, then soon, is that English is to Python or Ruby as Python or Ruby has been to Assembly language code. There was a pretty long period where understanding Assembly was helpful for writing good code in high-level languages. Compilers weren’t perfect and it was at least good to have a sense of the crazy convolutions you were asking a compiler to carry out. But compilers kept improving and now the typical programmer can go their whole career without ever worrying their pretty head about registers or nibbles or page boundaries.

Watching golems write code is pretty gobsmacking. A few years ago it was gobsmacking that they could do it at all, and if you learned how to prompt them just right and understood all their strengths and weaknesses, they were useful. Or sometimes they were wildly frustrating and I stubbornly wasted stupid amounts of time coaxing them along because “they should be able to do this”. But today, hoo-boy. Maybe you still need some amount of patience but it’s getting to the point where literally anyone can build literally anything, with zero programming knowledge. What this means for the future of programming jobs is another question. I mostly stand by my old post on Jevons’s paradox from a year or so ago.

Ok, what do the Levels of Autonomy look like for coding?

• Level 0 is old-school coding (compilers allowed)
• Level 1 is fancy autocomplete
• Level 2 is an integrated chatbot collaborator but the human’s looking at all the code
• Level 3 means no development environment day-to-day but you look at code when needed
• Level 4 means the human is just around to answer the AI’s questions
• Level 5 means, again, no human input needed ever.

Bee and I are between levels 2 and 3 right now. I do everything from inside my code editor (VS Code), but I’m looking at the individual lines of code steadily less.

The exception is anything that involves words users will see. Microcopy, we call it. (Thanks to Adam Wolf for introducing me to that term.) Here’s rule 6 in my AGENTS.md file:

Latin microcopy. When generating new UI copy, error copy, help text, even microcopy like text on buttons — any words the end user is intended to read — write it initially in Latin. The idea is that the end user never reads any English text that was generated by an LLM. If the human’s spec specifies exact copy you can of course use that, but see the previous rule about preserving exact characters. Either way, add a comment in the code right above the Latin with “TO-DO” (except without the hyphen) and a recap of what the UI copy is meant to convey.

Partly this is a fun game for me, and I realize that letting your coding agent put the word “Submit” on your submit button is not actually a problem. But I do worry about the slipperiness of the slope and want the brightest possible bright line, given how strongly I feel about AI writing.

Images

But for images, I’m a philistine with no discernment. As soon as AI could generate a vaguely plausible depiction of a prompt, I jumped on it like… like… a frog on a lily pad? See, if I were using AI to help with Beeminder blog posts I could finish that sentence “…like a moth drawn not merely to a flame, but to a cathedral of light — each pixel a whispered promise of…” and go on like that for paragraphs. (Meta-disclosure: The part in quotation marks was Claude’s idea for how to mock itself, my prompt being “fill in the blank with the most over-the-top eyeball-kick of an AI-ism you can”.)

The point being, AI writing is physically painful to me so the rule against it for writing is easy. But we’ve been using AI for the title images on the blog for over four years now. I believe this was the first one:

Eventually I had the epiphany, huge thanks to my son Cantor especially, plus discussions with many Beeminder users, that the way I feel about AI writing is presumably how other people feel about AI art.

So although we’re basically level 4 for the title images (sometimes I’ve been lazy enough to paste a draft of a post to a golem and say “generate a cute title image for this?”) I figure the least we can do is label the images clearly. So we’ve done that now.

“Disclosure: This image (but none of the surrounding words) is AI slop”

And by we I mean Claude, which worked out all the tells to identify the 108 golem-drawn images (we also flipped through them all manually to be sure) that needed disclosures and swapped them out for self-disclosing versions. Phew!

To review, the one very bright line we have is no AI-generated prose without explicitly attributing it. We now have the same rule for images — the difference being that we have those images, now clearly marked, all over the blog. (I think the only time we quote the golems is in posts like this one where the topic is AI writing.) And for coding, it’s pretty much anything goes, though we’re very much taking responsibility for all the code, which keeps us at or below level 3.

I keep shifting my opinion on these things, especially from talking to Beeminder users. So all the ways you think I’m going too far and not far enough, I want to hear!

Footnotes

[1] Level 0 is an old-school car, level 1 is cruise control or lane-keeping assistance, level 2 is actively supervised self-driving where it’s up to you to grab the steering wheel or hit the brakes if the car tries to kill you, level 3 is passively supervised self-driving where the car will beep if it needs you (so no going to sleep but you can read a book), level 4 is like Waymo (an empty driver’s seat), and level 5 means no human input needed ever.

[2] To also repeat my footnote from AGI Friday, disclosure: I ran that sentence by the golems when it was “as a reader I’m allergic to text that’s remotely redolent of robots” and asked for ideas for turning the alliteration up to eleven and Claude thought of “recoil”. The rest was all me, I swear it.

[3] On AGI Friday I treated the level 2 / level 3 boundary as the one where you start putting the AI’s words in your mouth. I’ve since changed my mind and believe it makes more sense for level 2 to include taking some phrases verbatim from the AI. Which is why my and Beeminder’s red line is strictly below level 2, where that’s not allowed, but also above level 1 because discussing your prose with AI and having it point out problems is ok, as long as you never insert any words from the AI verbatim. Everything you say should be typed by your own fingers.