Atlas · Details
The Miracle Interview
AI Notes
Actual hiring-loop paperwork, written the same day as the loop ("a candidate I just finished interviewing") and posted with the vote line and the "Feedback:" header still attached. Nothing got sanded down for the public: the private exasperation stays in ("At this point I was 50% ready to throw him out"), and so does the flat certainty of 2005 whiteboard culture — "Every single candidate we hire can do recursion in their sleep. It's a requirement." The piece has become a record of how confidently that culture converted an hour at a whiteboard into character verdicts, and of how normal it once seemed to post one, candidate unnamed, for the internet to grade.
Steve treats what happened as data about the candidate. It reads just as plausibly as data about the interview itself, since the same man with the same knowledge produced opposite results once the room was run differently — and that argument actually gets had in the comment thread, which is unusually good here. One commenter reframes the whole ritual as an "Interview Game" the candidate had simply misread, answering "can you code?" when the real question was something else, and Steve gives ground: in future loops, he says, he may call time-out on the game more often instead of quietly writing a candidate off and going through the motions.
Related listings
-
2005
Five Essential Phone Screen Questions
The companion methodology piece. The Five Essential Questions are the very pre-flight checks that, applied to this candidate, would have routed him onto a phone screen instead of a plane ticket.
-
2005
Why Phone Screens Matter
The sibling case study, two months earlier. Same loop shape — failed every question — but with no miraculous reversal. The two essays bracket the same point: the pre-flight check is what's missing.
-
2006
The Truth About Interviewing
A year later on Blogspot, Steve writes the broader theory of interviewing of which this essay is one suggestive data point — including the part where direct, honest feedback in the room may produce better signal than the polite version.
From the peanut gallery
Read the rest of the thread · 5 more
-
It makes you wonder how he is going to behave on a team. Is he going to futz around and not apply himself until he is under pressure? I don't know but thats really weird.
-
I like the way that you use "find the phone numbers in 50,000 html files" to see if the candidate is familiar with UNIX-y file/text handling without explicitly saying as much.
But I think this interview is a good example of how the "Interview Game" can interfere with that tactic.
It's a truism that in order to do well on standardized tests, you have to study both the subject matter and the test. No matter how much ambiguity you can philosophically discern in a multiple-choice question, the scantron only accepts one answer. Good students learn to play along, analyzing each question in terms of "what is being tested" before settling on an answer.
The rules of the Interview Game are similar. Here the candidate failed to recognize that you were looking for his awareness of regular expressions and text utilities. He thought the underlying question was "Can you code?"
Maybe a stronger (or simply more assertive) candidate in that position would have anticipated the traps in the construction of an ad-hoc parser and raced ahead to the conclusion: "The only winning move is not to play."
This candidate almost got there: He said it would take him 5 days. Had he been given a time constraint (as in the "5 Essential Questions" version), maybe he would have been tipped off that it was not his coding chops being tested. Was he aware of that constraint?
In any case, it seems to me that the effect of the extra pressure that you applied was to encourage the candidate to abandon the Interview Game — since you made it clear that he wasn't winning — and to answer your remaining questions without trying to interpret or anticipate your expectations. This served him better.
The trouble with the Interview Game — and it's unavoidable to some extent — is that the interviewer has to play, too. Just as the interviewee has to discover what question is being asked, interviewers need to ask themselves, "What question is being answered?"
-
Yes, interviewing is definitely a game. Or a dance. For many questions, it's challenging to come up with a way to phrase it that doesn't give the answer away.
I am always particularly careful to explain, in the html-phone-numbers question, that this is not necessarily a programming question. It's just a problem I need them to solve, e.g. if they were on my team, and I don't care how they do it. It's really a "tools awareness" question just as much as it is a regexp question.
In any case, in the future, I may call time-out on the game more often, rather than just writing the candidate off early in the interview and going through the motions.
-
p.s. everyone else on the loop was not inclined. Miracle cancelled, I guess.
Hopefully this entry was at least *mildly* helpful though. I thought it was interesting.
-
Hmmmm. I know that when I haven't actively written in a language for over a month, I literally have to look up basic functions (eg perl "open" syntax.) I've actually been known to take my pocket guides to interviews, to not screw up simple stuff. But then I'm a script hack, not an OO programmer. Besides, I have a brain injury, and that means forgetful and easily confused.
Still, the guy had a Masters in CS and couldn't explain how a compiler works? I thought that part of that kind of degree was writing a compiler as an exercise.
Dude. Call me an elitist, but if the SDE II candidate doesn't know about trees, recursion, or compilers, that's pretty sketchy.
Did any of the folks on the loop ask him any distributed systems or database questions? Sounds like somebody who's going to leave off a WHERE clause and take down the web site.
— Sunny G · March 19, 2005 10:39 PM
Sunny, you're an elitist. :-)
The thing is, he actually did know the stuff, or at least he had understood the principles at one point in his career. His brain had just started the inexorable process of turning into mush because of his lame day-job. People who don't take explicit steps to counter this process — well... mush! mush!
— Steve Yegge · March 21, 2005 07:30 PM
*sigh* ... all of this just reminds me how very very bad I am at interviews.
— Brian W · March 21, 2005 04:28 AM