Atlas · Details
The Five Essential Phone-Screen Questions
Author’s note
This post was voted a top read from my oeuvre by an AI committee, and it is still a good read. But it is pretty dated now.
For starters, we don't do phone screens anymore. But there is also no point in asking them on Zoom or in person, because they are all now poisoned by being in my blog. Within six months after publishing this, I found candidates already knew that was what I was going to ask. Oops.
Second, it is getting dated. You finally stopped needing bits and bytes sometime in the past decade. And OOP is meh. Regexps are pretty, but nowadays, you probably only need to know their rough shape and performance characteristics, not their syntax or detailed semantics.
Algorithms and data structures, though, will never go out of style.
AI Notes
Postmortem on roughly a hundred failed Amazon SDE phone screens turned up
two recurring antipatterns: in most failed screens the candidate
did most of the talking, with the screener only riffing off the resume;
and "one-trick ponies only know one trick" — single-language candidates
turn out to have gaping holes. The fix is a concrete checklist: five
universal litmus-test areas every SDE screen has to cover — coding,
OO design, scripting and regular expressions, data structures, and bits
and bytes — chosen because they apply to a college hire and a thirty-year
veteran alike, take one to five minutes each, and reliably predict
interview failure. Steve is hunting for a total vacuum in one
area, not mere rustiness: a Vehicle class subclassed from ParkingGarage,
or a 2,500-line C++ program from someone who has never heard of
grep.
It is a primary-source snapshot of mid-2000s Amazon hiring practice, Bar Raisers and SDE loops and all.
Related listings
-
2008
Get That Job at Google
The two ends of the same gauntlet. Five Essential Phone-Screen Questions is written for the screener; Get That Job at Google, four years later, is the candidate's preparation manual for surviving exactly this checklist.
-
2008
Done, and Gets Things Smart
The phone screen catches a total vacuum — the candidate who is clueless in one whole area. Done, and Gets Things Smart is the harder problem above it: how to recognise the rare engineer who is genuinely better than you are.
-
2005
Practicing Programming
Two drunken-era siblings. Practicing Programming even prescribes sitting in on phone screens as a practice drill — and the five litmus-test areas here are a tidy map of exactly what that deliberate practice should cover.
From the peanut gallery
Read the rest of the thread · 11 more
-
You can put a spin on your 'reverse a string' coding question - first have them write a func that prints out a C string without looping constructs or using local vars. Then if they get that, ask them to implement a reverse string function in the same manner as the first one. Don't say "use recursion" - let them figure out its straightforward applicability to the problem. That's, IMHO, how you can gauge if they 'think recursively' when lightly nudged in that direction:
void printreverse(char *s) {
char *s = "Hello world";
-
Er, a small error in my comment:
>>>I'd hope that most candidates would know that (without memoization of results) the naive recursive solution is O(n!) in time.
I meant "is O(2^n) in time" of course.
-
Interesting post overall...a couple of comments/issues I've seen when doing phone screens:
1. How do you have a candidate read code to you over the phone when the candidate isn't a native English speaker and the phone connection is sub-optimal [I've had this more times than I can remember...]
2. I have some issues with the "scripting" category--it expresses a preference for hacky programmers who prefer speed over maintainability. On our team (Customer Behavior) we're still cleaning up a fair amount of such code that broke the moment a database machine got moved between data centers. Yes, the code was written quickly, but now our managers are wondering why we can't get to a stable production system so quickly. The section also prefers UNIX programmers over, say, people who worked with Windows for an entire career.
-
> 1. How do you have a candidate read code to you over the phone
> when the candidate isn't a native English speaker and the phone
> connection is sub-optimal [I've had this more times than I can
The best approach I've seen is to give the candidate a "homework" question. Give them an hour to code up a solution to some problem and email it to you. Several teams are doing this regularly, with good results.
> 2. I have some issues with the "scripting" category--it
> expresses a preference for hacky programmers who prefer speed
> over maintainability.
I'm sorry if I gave that impression. We don't want those kinds of programmers here, obviously. The five question areas here are a delicate balance. This category is geared towards determining whether someone has the self-sufficiency to be able to respond quickly to emergencies affecting our customers. They still need to have good judgement and good design skills.
I never actually specify that they need to write it as a script. I just give them problems that are best handled that way: emergency queries and backfills, for example. I've had a few folks burn through 200-line Java or C++ apps that solved the problem, right there on the board, and they got offers.
But after 4 1/2 years over in Customer Service, I've found that knowing how to use "grep" and its ilk is a pretty important survival skill.
And we -are- Unix shop, after all. Unix isn't exactly a niche operating system. There are people who've figured out the basics on their own, even if their professional experience has all been with Microsoft technologies.
But feel free to ask whatever works for you!
-
Thanks Darren. Obviously extra-cool solutions are bonus points for the candidate.
Fixed the loop typo. It was caused, interestingly, when I corrected my original working code, after someone emailed me and complained about the performance. Originally it did this:
for (int i = 0; i < 100; i++) {
if (i % 2 != 0) System.out.println(i);
The point was to see if the candidate could write something that worked at all, and I was whipping up examples at 3:00am. Had no idea it would get so much attention.
In any case, someone objected to the fact that it wasn't simply incrementing the loop counter by 2, so it performed poorly. So I went and "optimized" it (an optimization that would go undetected by humans or profilers in this case, as it's totally I/O bound), and in my haste, broke it.
I suppose I should claim that I rigged the whole thing as a demonstration of why optimizing stuff that doesn't matter is needlessly risky. Or that I broke it so I could rant about why unit testing is critical, even for your personal blog content.
But really I just made a bad fix. :)
-
Thanks for posting this. I have been trying to improve my interviewing skills, and this looks like a good set of basics for phone screens.
-
I tried using the find-the-phone-numbers question on a candidate yesterday, and she went straight for Java. I was getting ready to slap her down when I noticed that Java 1.4 contains a regular expression engine. With that enhancement, the write-a-complete-program solution becomes more reasonable.
-
Yep. Most languages these days have (mostly) Perl5 compatible regexp engines. Java's file handling is a bit more cumbersome than it'd be in a scripting language, but not overly so. The problem is more about understanding fundamental pattern-matching tools than it is about scripting or any particular language.
-
I'm about to do my first phone screening and this was very helpful.
Maybe I just don't want to admit that I don't have vital information, but I don't know what 2^16 is. Jeff Bezos' phone number? I don't understand the significance of knowing powers of two off the top of one's head.
-
Jason: there are many domains for which knowing binary counting is useful, if not essential.
One is when you're doing stuff that involves a lot of bit- and byte-manipulation; examples include network protocols, writing binary serialization/marshalling code, and reading or reverse-engineering file formats.
Another is when you're doing any sort of memory or pointer coding (or more to the point, debugging) with C/C++ code. It's a lot easier to stare at hex-dumps if you're good at translating between decimal and hexadecimal (and binary).
Another broad class of problems involves data whose byte representation is especially significant. UTF-8 and Unicode are good examples. If you ever need to do internationalization, it can be a big help to have a crystal-clear understanding of the meaning of each bit in a byte, short, or int/word value. Actually, this may just be another sub-example of the first domain I mentioned, but you get the idea.
Lastly (and most importantly; all the other examples I've listed pale to insignificance in comparison to this one), it's important for algorithm time and space estimation. If you're trying to decide what data structure to use for a given data set, and you know off the top of your head that 2^16 is about 65,000, 2^20 is a million, 2^32 is 4 billion, and 2^64 is "big enough", then you'll have an easier time with the decision. Looking at it backwards -- if you use a balanced binary tree (or a log-n search algorithm), you can quickly estimate the base-2 logarithm of your data-set size, which gives you the rough number of steps involved in a lookup operation.
If you don't have a good feel for the growth rate of powers of 2, then a little old man will save your daughter, and you'll grant him anything you want in your kingdom, and he'll say he just wants one grain of rice on the first square of a chessboard, then 2 in the second, 4 on the third, and so on. And then: you'll be all out of rice.
Fortunately, you can memorize this stuff in less time than it took you to post your comment. :)
-
This is an *excellent* resource, Steve.
I've noticed that a lot of the Java-steeped interviewees will use Strings in their string reverse solutions like so:
for (int i = s.length - 1; i >= 0; i--) {
returnS += s.substring(i, 1);
Most will understand that objects are being allocated and discarded per loop, and will use a StringBuffer to make things "more efficient". Interestingly, a couple of folks couldn't explain _why_ it would be more efficient.
It's helpful to ask how they would design a limited StringBuffer class using just primitive types to try to help dispel library addiction. It can be surprisingly intimidating for some folks, especially when it comes to reallocating arrays. (Ironic, considering the allocation-fest of their original solutions...)
Thus, I recommend it as another coding question -- though just to talk through, not to code over the phone. Alternatively, asking for the reverse() function using just primitives would do the same thing, but without the scariness of dreaded reallocation.
The ultra-cool solution to Fib is the closed-form constant time solution. You need floating point and exponentiation, but that's constant time on modern hardware.
There's also an off-by-one error of sorts in your printOdds() functions. i should start at 1, not 0, or the function should be renamed printEvens().
— Darren V · October 7, 2004 02:42 AM
Nice post Steve - thanks for the checklist.
Would you really accept this answer to the 'Write function to compute Nth fibonacci number' question?
I'd hope that most candidates would know that (without memoization of results) the naive recursive solution is O(n!) in time. If they made that error in production code it would be and utter disaster (probably large enough to be noticed early by QA, but still...).
If the candidate didn't at least mention this caveat about their solution, I'd prompt them to compare & contrast with alternatives. If they didn't immediately give the iterative solution and explain the big-O difference, that would be a red flag for me.
— Chris N · September 29, 2004 08:36 PM
Yeah, that factorial fibo solution sucks. I'd be very happy if the candidate told me that they could do it tail-recursively with an accumulator parameter, even if I'm not sure you can do a doubly-recursive call tail-recursively. I'd still be happy.
I was thinking of splitting out recursion from basic coding, but that would be Six Essential Areas, and I only have five fingers.
— Steve Yegge · September 30, 2004 03:42 AM