The Truth About Interviewing

You say that "all the companies are interviewing like this", but my company doesn't allow quiz-style interviews, for fear of discrimination lawsuits or something silly like that. We pay for it by hiring a lot of duds, but no one wants to admit it. Management just keeps throwing people at the problem, and wondering why the projects don't improve.

Do you have any ideas on how to decide whether to hire someone as a programmer when you're not allowed to ask them to write some code?

First anonymous:
If you really can't ask technical questions of interviewees then why not just pick the top N resumes (where the best resumes are based on projects worked on being similar to the project you are hiring for. Then take the names of the candidates from these top N resumes and write them on slots of a roulette wheel (giving each an equal portion of the wheel).... spin the wheel a hundred times and hire the candidate the ball landed on most often. Mone Carlo hiring! You could even automate the process by writing a program to simulate the roulette wheel selection - no need to manually spin a wheel 100 times! To be honest, I'm not sure this method of hiring wouldn't be just about as effective as the current interview-based method. Just think of how much time it would save both interviewers and interviewees.

Now here's the interesting thing about this proposal: while many candidates complain about being asked 'difficult questions' which they feel are not relevant, many of these same candidates will also be offended if you were to tell them that the hiring selection will be made by an entirely random process.

I think the better answer to the hiring problem in general is an audition-based process where you have the most promising N candidates (based on resumes and a phone screen) come in and actually work for a few days. It may well be that the guy/gal who would do terrible in an interview because of nerves could be the best one for the job based on actual job performance. Conversely, someone who might ace your interview might not actually be very good at real work. Yes, the audition-based hiring process would take longer. If you have 5 promising candidates and you want each of them to audition for 3 days it's going to take at least 15 days to figure out which one you will hire. But maybe it would be time well-spent...

Another thought: I'm terrible at coding on a whiteboard and I suspect I'm not alone. No, I don't mean block diagramming, I mean actual coding - nobody does that in real life. However, I've often been asked in an interview to write the code for a problem on a whiteboard. In real life when a new problem comes up that I need to solve in code I might go to a whiteboard to do a block diagram or a state diagram, but then I go and do the coding on a computer and rely on a compiler to tell me if I've got errors. So how hard would it be Interviewers (hint, hint) to have a laptop handy in the interview room which has a plethora of editors on it (at least vi and emacs ;-) and compilers/interpreters for the languages of interest. And then when you ask the candidate to write code to solve a problem you offer the candidate the use of the laptop. With vi and gcc and maybe gdb I could get through a programming problem a lot faster than I can on a white board and as a bonus, you the interviewer could see that I know how to compile a program and interpret error messages in addition to the actual coding.

The acid test I am currently using is pair programming. If you can have your candidate for a few hours to work together on a section of code you can find out how they think, how quickly they pick up the problem domain, how well they communicate, and if they can contribute.

I'm terrible at coding on the whiteboard too. I was shocked at how bad I was, last time I interviewed. This is a really unfortunate way to interview people (but it's amazingly widespread.)

Do you think interviewers take projects you have done your self into account? Written patches for Open source projects/your own shareware project etc. I am not expecting to walk into a technical programming job without the proper experience but I would like to get a foot on the ladder and make it there someday.

Jon.

I ask candidates to write code on a white board, but I let them warm up to it. I start with questions like, "if I execute these 5 lines, in this sequence, what happens?" Then I move to, "find the bugs in this function". Finally they get, "re-write the function using recursion (or whatever)." By that time they've seen enough code that they aren't starting completely cold.

I spend 90 minutes doing an interview. I couldn't imagine doing it in less and my management wouldn't allow much more.

Great blog, I wish I could write that well.

I dont'know how the situation in the US is, but here in Europe it is pretty relaxed. Currently I am working (knock on wood)for a top tech company. Actually it is in the top 3 of telco's in the world. Not working directly for them, but still after 17 months it has become more then a temp job. They even let me deal with important customers now.

So everything is pretty great. However, their interviewing procedure was a joke. There was a techical guy with a spreadsheet containing all their technical requirements. He asked some simple yes/no questions - like do you know J2SE and that was basically it. Probably you think that this was a junior position or that there is an enormous shortage of programmers, but that is not the case.

Maybe this is just an exception to the rule - don't know. To this day I haven't been asked to solve difficult programming puzzles during an interview.

I wrote an article not long ago for TechRepublic.com titled "The Best Software Developers are Built not Bought" (http://techrepublic.com.com/5100-3513_11-6038687.html).

I mention this because many of the same challenges you deal with in this post are the same ones that I struggle with. How do you quickly and (here's the rub) accurately assess how someone will develop software.

Nice Post.

long but very interesting. i just had a phone screen and it turned out to be aloit less scary than i had thought. considering the top company. just read guy k's blog about it and there is a description on how HP does interviews. very very through... http://blog.guykawasaki.com/ (art of recruiting part2)
in 15 years of it work i have never once been asked to write code during interview. never. i was often perpared to but never happend. but then i never went for the top of the top jobs. it's like steve says. one must know what one wants in the first place...

Last Thursday I was faced with the all too uncomfortable prospect of writing code on a whiteboard for an interview.

I think I did... ehh, okay. Not as good as I would've hoped. I made a number of mistakes across the three technical interviews I did, but I don't think any of them were that terrible.

Some things I decided that I'd do the next time I got a request to do that:

1) Before taking the erasable marker in hand, I'm going to write out my plan on paper before even thinking about writing code. Just a prose description of what the alogrithm is going to do. The reason being that I got up to the board and felt scared because I didn't immediately have the algorithm ready to go. I think it's reasonable to take at least 5 minutes before commiting yourself to fake code to figure out what you want to write and find the fundamental problems with your initial ideass. The other advantage is that if you've written out basically what you want it to do, you might be able to decide on an optimal paradigm, even if that choice is as simple as iterative versus recursive.

2) Probe the test cases of the input that the interviewer gives you. This one inparticular I wish I'd done. I'd come to the last line of the algorithm and then say out loud, "I think I'm done..." then instantly regret it because whoops, there's 5 boundary cases I forgot about in my rush to get the general case written as fast as possible. I don't think it's necessary to write fake code that tests the boundaries, but to just write out some examples of the boundaries in the first place.

3) Don't put types (int, char, etc, etc) on anything until you have to. I managed to do this on the third interview because I've been writing Ruby for the past year and I realized that typing things prematurely is a detriment if you don't know precisely how something is going to work. The one example I did this on worked out very well, because I'm pretty sure I would've made a bunch of typing mistakes if I'd written the types down in order (like I did on the other two that I had to correct). The interviewer wasn't really sure what I was doing while I was doing it, but it all worked out in the end.

I think if I ever end up in an interviewing position, I'd want to try and reduce the anxiety of the situation by telling the candidate to not worry too much about the specifics and that I just wanted to see how they reasoned about code (even if it wasn't true). I wasn't really all that scared, but doing code up on a whiteboard where you can't verify if you're correct because there's no unit testing software to tell you that you've at least found solutions for all the boundaries that you've thought of or can use inverse relations to check your work is a bit like walking a tight-rope.

This very morning during an interview I was asked to write a threaded cache class, implemented through a singleton to access a database with, wait for it, pen and paper. Have you ever tried writing code with pen and paper? Go ahead, try it. One thing they are not testing is coding ability.

The author genuinely believes that the top tech companies do a good job at interviewing but it just isn't true.

In an ideal world, it would be true. Even in this world, top tech companies wish it was true. And, on the whole, top tech companies work very hard to try to make it true. But they fail. Almost completely. The previous commenter who wrote about the "Monte Carlo" method is closer to correct than the author is.

If an applicant fails to get a job with Microsoft, that isn't a dig against the applicant. Take heart: Microsoft is a company run by humans and humans often make bad or random decisions. That's just human nature. If you try again a few times, you might get hired.

On the lighter side, Microsoft and Google are in a race to have the worst recruiting system in the industry. It's strange: they both have terrible recruiting systems but they are entirely opposite. On one hand, Microsoft recruiting is all about bureaucracy. They've got this elaborate web site that pushes NDAs, drug test and background check approvals and a bunch of other bureaucratic red tape at you while a staffer e-mails you maps to the wrong campus and calls you by the wrong name. On the other hand, Google recruiting is all about randomness. They zip you through, give you an offer ... and then withdraw it. If you complain, they offer to interview you again for the same job. I'm not sure who's going to win but we can draw at least one conclusion: being a top tech company isn't any guarantee that they are good at interviewing.

I wonder if there is a correlation between being a top tech company and being bad at interviewing?

Out of curiosity, how do you manage to make them aware of your enthusiasm besides getting doey-eyed and bouncing a lot?

I agree that coding on a whiteboard is just silly. It's one thing if they ask you to whiteboard a system architecture, but coding on a whiteboard is like cooking with a typewriter. My solution: I bring my laptop with me. Instead of depending on the interviewer providing an environment that can demonstrate that I can code, I can just do it right in front of them.

I also try to have some sort of relevant and short presentation at the ready, but I've never needed it.

Don Howard wrote:
The author genuinely believes that the top tech companies do a good job at interviewing but it just isn't true.

How could you have arrived at that conclusion after reading this blog entry? Did you read a different post than the one I wrote? I said interviewing is a terrible mess, and 10% of all interviewers ask the same ridiculous questions to every candidate year in and year out, and I'm not a fan of interviewing the way it's practiced in our industry, on account of it being a "crapshoot", and that even good engineers can wind up not getting offers [because of the inherent randomness and poor selection power of the process most places use.]

Did that mean nothing to you? Interviewing is random! If you get a job, you're lucky! If not, I absolutely agree with you -- it's not necessarily a knock on you.

I can tell you this: at every company I've ever worked at (5? 6 total, maybe), many folks concluded that on a different day, with a different group of interviewers, they may well have failed to get an offer.

Put another way, many of us (myself included) believe that for virtually every employee Y of any sufficiently large company, you could find a group of interviewers A through F who would fail Y in a round of interviews. It's a wonder companies hire anyone at all!

but for some reason there are all these people floating around who think they know everything there is to know about software development

Well, that's because there are many documented cases of totally self-taught developers creating products that sold a zillion copies and made them into zillionaires. This isn't fiction. It's fact, and it's been going on continuously from 1970 to 2006.

I don't know of any successful brain surgeries by self-taught brain surgeons, or any successful 747 landings by completely self-taught airline pilots.

Does this mean Steve's original comparison sucks? Partially. It could mean that our current tech is so primitive that, like brain surgery in 1850, is essentially experimental patient killing.

Thus, it really doesn't matter if a "professional" or "self-taught" brain surgeon is working on you in 1850. Either way, you're gonna die.

If Steve thinks the state of software development has advanced as much as brain surgery has from 1850 to 2006, then he's a far more exuberantly optimistic man than I.

One idea might be to ensure that the job reflects the interview. Some years ago I went through a very encouraging selection process, which included writing a pseudocode solution, for a job that turned out to be so depressing that it ended my twenty year love affair with programming.
I am now a freelance journalist; pays peanuts but it beats working for a company that thinks enforced technical ignorance is a good way to minimise staff turnover.

“The Harsh Reality
They're going to test you on algorithmic complexity analysis, advanced data structures, algorithm design, ...
If you're lucky, that is.”

If you are lucky they will call you on time on a day of your phone interview with them. Not more than half an hour later, that is…

If you are lucky they will even reschedule the interview for a next day if you were called an hour later and you had already left and was unable to answer their call.

If you are lucky they will apologise when they call you next day more than half an hour later than a scheduled time.

And after that they will even send you a courtesy letter saying that you were unlucky… If you are lucky, that is.

Nice post. I worked at Oracle for many years in software development. I had my favorite questions - though they tended to be more logic-driven than code or language specific.

Now I'm doing a start-up. And it's not glamorous. So I challenge the "if money is all your after" assertions in your post. If money is all your after, actually you do belong at Microsoft or Oracle or Google or Company-x-y-z. If you are insane enough to pursue your own dreams & sacrifice a steady income to do it, money is definitely not all your after...

Stevey, I have read the whole article and all your comments and my conclusion is the same as before, I love you!

Signed,
Secret Admirer

Stevey, I have read the whole article and all your comments and my conclusion is the same as before, I love you!

Signed,
Secret Admirer

Stevey, I have read the whole article and all your comments and my conclusion is the same as before, even if you use emacs and I like Vim, I love you!

Signed,
Secret Admirer

Hi Steve,

Yes, I agree with you. The whole interview thing is one big lottery.

I personally think that some of the randomness can be eliminated by making the interview longer. I've seen some people who time their interviews - its like 30 minutes and whatever gets answered in that. But preferable I'd like to have free-form (i.e. no pre-decided questions, at least not an exact number) and free-length interviews. Kinda like seeing how things go and playing along.

Maybe thats because my time is not very "precious", bottom rung software engineer that I am ...

I like the first half of the analogy you draw with airline pilots and so on. I agree that a company wants to be sure that the person they hire is qualified. However, this is only one side of the problem. The other side is where this analogy fall apart. For example, I have passed my driving test, and I've been driving for years, but I am not qualified to test other drivers.

What we have is interviewers, unqualified in testing, attempting to test applicants with untested questions and often looking for a subjective answer heavily coloured by personal preference. We tend to ask technical questions about our pet subjects and tend to hire applicants that know and agree with our opinions (this is the cloning trap.) But if we don't ask technical questions how do we assess applicants?

Interviews have two players and each has a different goal. The candidate's goal is to find a compatible work environment. The interviewer's goal is to find a qualified employee. To win, both need to achieve their goals: it's a collaborative game. Any process applied to an interview as either interviewer or interviewee must move us towards our goal, but must not block the other player from achieving theirs.

Here is an approach, for what it's worth. It still requires applicants to express technical knowledge but doesn't fall foul of the cloning trap. Rather than ask about a technical subject I know well, I ask candidates to talk about a technical subject they know well. Whether or not I know the area well, I look it up after the interview to assess how well the candidate was able to describe the subject to me.

I can do the same in writing, by asking candidates to bring code samples. I can also do it in code, by asking candidates to bring demos. It gives me a clear picture of candidates abilities without inadvertently presenting the company as rigid or bureaucratic.

Hello Steve,
I came across your blog accidentally, and simply love reading it. Making no bones about the fact that I was a terrible programmer (a student currently) - Its mainly helped me to figure out ways I can improve (on the interviewing candidate's side).
I am richer,mentally, for having read your blogs. Great work!
-Mallika Iyer

Hi,
I mean the candidate being interviewed .

Rhialto, as a novice computer scientist I think the best lesson I've gotten so far out of it is that if one approach doesn't work, try another. Because with software, generally, when one approach isn't working, its not because of the language's features.

I think this applies to the real world, as well.

If getting a job at Microsoft doesn't work out, instead of complaining, find a different company. Or get better and try again.