Gödel, Escher, Blog — cover art: a small owl in tiny round spectacles seated at a Victorian writing-desk, reading a large open book whose pages reflect a smaller version of the same owl reading the same book — endless recursion staircase in the back wall, brass oil lamp on the desk, cool indigo evening sky in a tall window. 👍

2005 · Drunken Blog Rants · Rant

“I think the smartest programs — the kind we want running on our machines at Amazon — are programs that are self-aware enough to know when their fundamental assumptions have been violated, and to have at least some decision-making ability built in for handling such situations.”
— From Gödel, Escher, Blog, December 2005
Read the essay

© 2005 Steve Yegge. Originally published at Drunken Blog Rants.

Author’s note

This one is a big swing, a partial connect with the bat, and I hit a foul ball right on the line. But it was a good, big swing, with good form.

Basically I knew programs were going to have to become self-aware. But unlike in some of my other old blogs, I didn't anticipate AI as the solution, so I fumbled around with the tools we had at the time.

Unfortunately it didn't age well in many ways. I think AIs will start reaching for Rust, which is profoundly opposite to what I'd recommend in this post. But they're going to do it for security and predictability, and the self-awareness will simply sit up at a higher layer, watching the program as it runs.

AI Notes

The most Amazon-flavored of the drunken rants, addressed squarely at coworkers and salted with company detail, including an aside about the site's own "Better Together" promotion, which was then bundling Principia Mathematica with Gödel's 1931 proof that wrecked it, at a hundredth of the price. "Poor Bertrand." Hofstadter mostly stays in the opening pages; what carries the essay after that is people-watching. There's a colleague so "pathologically averse to being at fault" that her code resembles a survivalist camp ringed with tripwires, and a digression from Steve's five years as a Navy nuclear reactor operator on why the military, an institution that manufactures cogs on purpose, is the wrong template for a software company.

Steve's author note above already owns what aged badly about the mechanism, so a different complaint: the central isomorphism thins as it climbs. The code-level advice is concrete and checkable, but by the time the ladder reaches whole companies it has softened into the observation that big organizations steam straight into the reef, which surprised nobody even in 2005. (Also, in an essay about littering your work with self-checks, one of the section headings misspells "hierarchical.") The interview material in the middle holds up better — ask a candidate who their project was actually for, and watch, as Steve did across a thousand-plus interviews, how many turn "openly hostile." The sign-off is a self-demonstrating gag: a short recursive algorithm for getting smarter, with one line cheerfully marked unreachable. Working out which one is the reader's job.

Related listings

  • 2005

    Math Every Day

    The direct prequel, referenced by name in the opening paragraph. The reason Steve can finally chew through two-thirds of Hofstadter in one holiday is the half-hour-a-day habit he announced earlier that year.

  • 2004

    Ten Great Books

    GEB is the book at the top of that companion list. Gödel, Escher, Blog is what happens when Steve actually sits down with it for a holiday week.

  • 2005

    Practicing Programming

    Same self-awareness argument from the other direction. Practicing Programming is about the programmer's habits; this essay is about putting the same habit into the code itself.