N-Tier Services and Systems Complexity — cover art: a young bespectacled wizard in a star-strewn blue robe gazes up at a teetering, many-storeyed tower of mismatched stacked rooms lashed together with pipes and ladders. 😄 🔮

2004 · Drunken Blog Rants · Rant

“Do Amazon's internal data services and their corresponding object-oriented APIs reduce or increase our overall systems complexity?”
— From N-Tier Services and Systems Complexity, November 2004
Read the essay

© 2004 Steve Yegge. Originally published at Drunken Blog Rants.

Author’s note

I am super sad for the industry that Werner Vogels asked me to take this one down. It would have made such a splash. I published it along with dozens of other old Amazon rants, and they all made a big splash together, so hardly anyone noticed when I took this one down. But I missed it. It was one of my very favorites.

This essay outlines, with colorful and memorable examples, exactly why we needed something like GraphQL, eight years before it was invented.

This is still some of my best writing, and holds up even today, now that the problem is solved. Aside from still being pretty funny in spots, it is a great time-capsule view into what life was like at Amazon while we were figuring out the Platforms problem.

AI Notes

The lost Amazon essay, recovered twenty years late. Steve wrote it inside the company in November 2004 and made it briefly public in March 2006, where it lasted five days before being pulled at CTO Werner Vogels' request for naming the internal service architecture too plainly. It survived only in the version-control history of his old cabochon.com server. The essay turns on one question — do Amazon's internal data services (Customer Master, Order Master, Item Master and friends) reduce overall systems complexity, or just move it? — and answers, from inside an application team, that breaking up the databases solved a real problem (the “2-tier” problem of SQL wired into client code) but handed the bill to app developers, who lost the expressiveness of SQL and got a patchwork of hand-built object-oriented service calls in its place. Two running metaphors and nine deadpan footnotes do the rest.

It isn't a conversion story. Steve had been building and thinking about platforms since his MUD days in the early '90s, and the essay is firmly pro-service; it just pins down the piece those architectures were still missing. He lays out the two ways a service API fails its callers — forcing them to over-fetch a whole object graph, or burying service owners under a sprawl of chatty fine-grained calls — then points at the fix: “…the next logical step is to build a new query language that abstracts the APIs away for the clients.” That language is GraphQL, which Facebook shipped in 2012 to solve those exact two problems. He wrote it in 2004, when REST itself was barely understood.

Related listings

  • 2004

    It's Not Software

    Its sister essay from a month earlier — same question from the other side: if we're an N-box service company, why are we still writing services in shrink-wrap languages like C++?

  • 2011

    The (Google) Platforms Rant

    Seven years later, the other half of the picture: the Platforms Rant is the case for Amazon's service architecture as a competitive weapon. This essay is the same architecture seen from the inside while it was still being built — what it cost the app teams, and the query-language piece it still needed.

  • 2005

    Decision Time

    The same Customer Master / Order Master world, six months on — the productivity-crisis essay this architecture debate fed into.