A few months ago some members of the team I lead at Toptal’s core team started interviewing programmers.

The fundamental questions that popped up were simple and deep: What should I look for in the interviewee? How do I know I should pass or fail the person?

I really like the priorities my own former supervisor, Timo Roessner, advocates: to look for—in decreasing order of importance—a team player, a good communicator, and a technically excellent person.

The person should ideally be very good at all 3 requisites. The main point, however, is deemphasizing technical wizardry in favor of interpersonal skills.

How to assess team-playing chops in a 90-minute interview with a coding challenge

Team-playing chops and communication chops seem intermingled, and in fact I believe one potentializes the other, but they can be assessed quite separately in software engineers.

As you facilitate a coding challenge or nearly pair-program with the interviewee, try to make it as close to the interviewee’s normal workflow as you can. Don’t use CodePens, have the interviewee use the editor or IDE of her preference. Apart from the technical things you need to evaluate her on, encourage her to use her favorite tools and procedures.

What you need to evaluate the candidate on is up to the needs of your company or team, but in general I prefer to give the candidate a wide-open small project instead of some algorithmic or procedural task. The reason is you will get a chance to role-play being the customer or Product Manager, and you can talk to the interviewee as such and see if she is able to conduct a dialog like this.

Calibrate, as you go along and get a sense of the candidate, how high-level or low-level it’s better to be. That is, is this person more concerned about the coding nitty-gritty or about what the entire project should do? Both are fine and necessary programmer inclinations, but the quicker you spot the candidate’s preferences, the quicker you can develop a rapport.

As the coding challenge starts, pay attention to the questions and shared thoughts. It should be clear to you, given some experience, if the candidate has mileage or a willingness to openly express thoughts. If they don’t say anything at all, probably this person will behave the same way at work.

Avoid, or delay, “helping” the candidate. Generally, in a coding challenge lasting 1 hour, I may interfere 1 to 3 times. But when you do, try to make it very meaningful and make it so that the proposed change of course really takes things in a different direction.

For example you propose the programmer develop a simple game. She starts by crafting the algorithm that will define if the game is over. As the interview goes past half its allotted time, and if the candidate has shown a good direction regarding the solution, propose she create the UI (if she is a frontend developer), or suggest creating a way to store a leaderboard (if she is a backend developer).

The way the candidate reacts to such proposals can be telling of a team player.

Also, very often candidates will do or say things that you don’t know anything about. Tell them you don’t know anything about that and see how they react. Do they explain it to you? Are they surprised? Do they convey a sense that they respect you less for that? But don’t pretend you don’t know something you know—you don’t need any trickery to interview people well.

How to assess communication chops

Assessing communication skills is a bit easier than assessing team-player inclinations. Start with the basics: can you even understand what the interviewee is saying? Often that is not the case. If and when you don’t, tell them that in a gentle way. This happens particularly often in international interviewing when spoken languages are not native to you or the candidate.

The opposite can happen: the candidate does not understand you. You should, of course, try your best to communicate clearly, and if the candidate still has a hard time getting what you are saying or asking, notice if they have the energy to ask you to clarify. If so, that is a good sign.

Being able to listen deeply is what differentiates an ok communicator from a good or great one. Does the candidate flow from what you are saying? In other words, can you notice them using and reshaping pieces of your discourse? Repeating things back to you transformed? Those are good signs. If they just respond things that seem unrelated to what you have been saying, then clearly this is a minus. More often things fall around the middle, and with time you will develop a feel for how good the candidate seems to be, plus the courage to use your intuition when the decision is not clear-cut.

How to assess programming chops

This is the easiest part if you yourself are a programmer, but it’s still challenging sometimes. Granted, it’s easy to reach a conclusion in extreme cases: if the candidate did really poorly or just brilliantly in the coding part.

When things fall more in the middle, which is by far the most typical, your skill can make a huge difference in the decision to say Yes or No to a candidate.

I always like it when a candidate writes, in human language, something like a list of things that need to happen in the program. For example, if the proposed task is to build a search user interface: to jot down that the screen has an input and a submit button, the user will type a string in that input, then press the submit button, and so on. This helps the candidate “think like a computer”, it allows her to expose her thought process to the interviewer and, if her assumptions are off-base, to know early. The worst that can happen is to end up with a little algorithm of what to code.

TDD, or anything similar to that, is a huge positive differential. It is a kind of planning ahead.

I must say it’s very rare that anyone uses TDD in interviews I run, and I’ve run a few hundred of them at the very least. It has never ever happened that a candidate wrote down point by point what should happen in the application.

Finally, good code architecture early on is one of the strongest signs that the candidate is good. If they refactor continuously, improve variable names often, and order things in their codebase without losing their train of thought, you are looking at a hire. And that’s why giving people much harder tasks than they will face in their work is typically not a good idea: you need to see how they can make code understandable to fellow programmers.

So if they start one file and fill it up with disorganized, commented out code and huge blocks of attempts, unless you asked them to do something too hard, in general consider this as a potential flag. But don’t get stuck to this: it’s not uncommon for great developers to seem quite sloppy the first 15 minutes in, and 10 minutes before the hour, to start deleting code and apply a final layer of improvements that leaves the code looking quite wonderful.

If you give the candidate a project that is larger than the hour you have to interview, something I support you do—as long as you set the right expectations, otherwise you will have a frozen person in front of you—, and you enabled the candidate to own the code in her machine, the bonus point is them continuing the project on their own. Until recently I sometimes suggested they do it. Nowadays I don’t even suggest that. If they have a way to reach me, they can do it if they want. I certainly take it as the strongest sign of interest in the part of the candidate if, hours or even a couple of days later, they get in touch with the completed project.