Programming Interview Questions

There's a cottage industry devoted to devising questions to ask folks applying for "programming" jobs, to collating such questions and then posting answers to them on the internet. I was reminded of this recently when an announcement of a new Google tool, moderator, came along. The example issue being moderated was "Interview Questions for Software Engineers". At the time of writing the winner (with 4 upvotes and 0 down votes) is my own proposal:
Would you please sit down here at this workstation and write some code with me?
If you are reading this then you have almost certainly been through many an interview for a job that will require some programming to be done. Think back and ask yourself how many times you were offered such a job without ever having been in the same room as a computer and an interviewer.

In my own experience (I've had 9 jobs involving programming, and many more interviews) it's been quite rare. One has to wonder why that is. I suspect that having a candidate do some programming is considered very expensive—and yet hiring the wrong candidate is unbelievably expensive. Hence the questions, which seem to be aimed at finding out if someone will be able able to program without watching them do any.

The second most popular question on the moderator site right as I write is
Given a singly linked list, determine whether it contains a loop or not
I can see what's being aimed at here, and yet it feels as if, in the stress of an interview setting, all the answer given will tell you is whether or not the candidate has seen the textbook answer before. How does knowing that help?

By the way, this is a systems programming question. The vast majority of working programmers don't come anywhere near any such class of problem. They do applications programming. There seems to be a sort of folk wisdom in a large part of the industry that having the skills (and, see above, mere knowledge) of a systems programmer will suite them well to being an application programmer. It's far from clear to me that this is true: the two areas of programming are quite distinct and require different (albeit overlapping) skill sets. I wonder how many interviewers use questions like this to create the impression (maybe even to themselves) that their organisation is involved in sexy, impressive systems work, whether or not that's true?

A while ago we did an informal survey of folks who work for Zuhlke in the UK, asking what had made them apply to us and what had made them subsequently accept an offer. We (the managers) were surprised by how many said that the very recruitment process itself had encouraged them to accept an offer.

This is what we do: Line mangers (such as myself) read CVs. There is an HR department, they don't come anywhere near recruitment. CVs that seem to describe a credible technical career go to the Chairman of the company. The candidates he likes the sound of are invited in for a first interview (of around an hour) to see if they are firstly, someone we'd be prepared to put in front of a client under our brand and secondly, someone we would enjoy working alongside. There's also a short written test of programming knowledge.

Those who make it through that are invited back for an interview with a small panel of our consultants who explore their technical knowledge in depth. First the candidate is asked to give a technical presentation on an interesting piece of work they did recently: stand at a white-board, explain the architecture, the technology choices, how it worked, how the project worked, what they did etc.

Then we set to finding out what they know. We have a checklist of technologies that we've used in projects and the questions take the form "what do you know about $technology?" No tricks, no puzzles. We do a breadth first pass to eliminate the things that the candidate doesn't know (and the process rewards optimistic answers at this point with a lot of embarrassment later on) and then we drill down into each area until at least one of the candidate's or the interviewers' knowledge is exhausted. This goes on for several hours.

Those who make it through that, and people have been known to walk out, get invited back to do some pair programming. That's the point (and not before) at which we decide if we'd like to make an offer.

And yes, this is all unbelievably expensive (I mean, the Chairman does first interviews?) And very, very effective. Our false positive rate is essentially zero and best of all, clients love our engineers and ask for them back again and again. Which makes it all worth while. I can't think of a cheaper way that would actually work.

PS: yes we are hiring. If you'd like a go a this, drop me a line.

10 comments:

tag said...

To answer your question, I've never had to actually write and execute code on a real computer at an interview! Pencil and paper or pen and whiteboard programming questions have been standard, though. In fact, I'd have reservations about any interview where I wasn't asked to write code.
I agree that your interview question is the best of a poor bunch, but it does beg several other questions: what editor, which language, what version etc.? And in particular, what exactly will you be coding up? A pencil and paper programming test side-steps issues raised by the first set of questions and focuses attention on how exactly the candidate goes about coding a solution to the particular problem. I do think your approach better emphasises communication and team work though.

keithb said...

Hi Tag.
Firstly, we aim to do the pairing in whatever the candidate is comfortable with. We're a consultancy so we can cover a lot of ground: Java in Eclipse, C# in Visual Studio, Flex, Ruby, Erlang, vim, emacs. We prefer what our clients prefer and currently that's Java in Eclipse. In cases where we can't accommodate what the candidate is most comfortable with we will have the local in the pair drive the tool but have the candidate tell them what to do.

Secondly, using the whiteboard to write code isn't as neutral and exercise in problem solving as is sometimes claimed. In fact it constrains the candidate to use a very particular set of techniques: ones that rely on only what you can remember, ones that don't involve looking anything up, ones that rely on not having any feedback from a compiler, not having any feedback from a test suite, or from a running example.

Asking candidates to write code on a whiteboard is asking them to write code using techniques from the 1970's.

I'm not so interested in that because it bears very little relationship to the working life of the C21 programmer.

I want to see how a candidate marshals the vast array of information available to solve a problem. I want to see them take advantage of the very rapid feedback that the machine can give to help them find a demonstrably correct solution quickly. And so on.

tag said...

Thanks for your reply Keith.

I'm convinced. It's hard to argue against real actual live programming being a fundamental part of the interview process. Like you, I wonder why this is so rare?

keithb said...

It is a puzzle, isn't it? I can understand people's motivation for wanting to no do programming in an interview. What I don't understand is why they think they can get away with it.

Chris said...

I worked at a company where they used either a face-to-face technical interview or a pair programming interview. Some of the people who didn't go through the pair programming interview didn't turn out to be too great, in fact, they had to let one go after 2 weeks. However they very useful for writing programs that generate prime numbers on a piece of a paper.

Andy said...

On a marginally related note - I think one of the most powerful questions you can ask is the one the interviewee has no answer for. It's at that point you get to sort the people who can innovate, think on their feet and keep going at a problem.

I often wonder about the effectiveness of programming in interviews, since it must be hard to derive a good example that tests people sufficiently. Clearly it works for you guys.

At the place I'm at the interview period extends over 8 hours. There's no pair programming and the false +ve rate is also very low.

I suspect that it's a companies commitment to doing an exhaustive set of interviews where candidates see a reasonable spread of people from the company and you get lots of viewpoints on a candidate that maybe the dominant factor.

onemind said...

Looking at new technology, i guess computerizing the interview quations can help management in many ways. But can it be a full proof for long term use is still a quastion.
___________________________________
Sara Lee

Cv interview questions

Thiep said...

First of all. Thanks very much for your useful post.

I just came across your blog and wanted to drop you a note telling you how impressed I was with the information you have posted here.

Please let me introduce you some info related to this post and I hope that it is useful for community.

Source: Stress interview questions

Thanks again
Ngo

catania03 said...

Hi

I read this post 2 times. It is very useful.

Pls try to keep posting.

Let me show other source that may be good for community.

Source: It interview questions

Best regards
Jonathan.

Anonymous said...

good collection