A personal take on interviewing programmers

A few months ago some members of the team I lead at Toptal’s core team started interviewing programmers.

The fundamental questions that popped up were simple and deep: What should I look for in the interviewee? How do I know I should pass or fail the person?

I really like the priorities my own former supervisor, Timo Roessner, advocates: to look for—in decreasing order of importance—a team player, a good communicator, and a technically excellent person.

The person should ideally be very good at all 3 requisites. The main point, however, is deemphasizing technical wizardry in favor of interpersonal skills.

How to assess team-playing chops in a 90-minute interview with a coding challenge

Team-playing chops and communication chops seem intermingled, and in fact I believe one potentializes the other, but they can be assessed quite separately in software engineers.

As you facilitate a coding challenge or nearly pair-program with the interviewee, try to make it as close to the interviewee’s normal workflow as you can. Don’t use CodePens, have the interviewee use the editor or IDE of her preference. Apart from the technical things you need to evaluate her on, encourage her to use her favorite tools and procedures.

What you need to evaluate the candidate on is up to the needs of your company or team, but in general I prefer to give the candidate a wide-open small project instead of some algorithmic or procedural task. The reason is you will get a chance to role-play being the customer or Product Manager, and you can talk to the interviewee as such and see if she is able to conduct a dialog like this.

Calibrate, as you go along and get a sense of the candidate, how high-level or low-level it’s better to be. That is, is this person more concerned about the coding nitty-gritty or about what the entire project should do? Both are fine and necessary programmer inclinations, but the quicker you spot the candidate’s preferences, the quicker you can develop a rapport.

As the coding challenge starts, pay attention to the questions and shared thoughts. It should be clear to you, given some experience, if the candidate has mileage or a willingness to openly express thoughts. If they don’t say anything at all, probably this person will behave the same way at work.

Avoid, or delay, “helping” the candidate. Generally, in a coding challenge lasting 1 hour, I may interfere 1 to 3 times. But when you do, try to make it very meaningful and make it so that the proposed change of course really takes things in a different direction.

For example you propose the programmer develop a simple game. She starts by crafting the algorithm that will define if the game is over. As the interview goes past half its allotted time, and if the candidate has shown a good direction regarding the solution, propose she create the UI (if she is a frontend developer), or suggest creating a way to store a leaderboard (if she is a backend developer).

The way the candidate reacts to such proposals can be telling of a team player.

Also, very often candidates will do or say things that you don’t know anything about. Tell them you don’t know anything about that and see how they react. Do they explain it to you? Are they surprised? Do they convey a sense that they respect you less for that? But don’t pretend you don’t know something you know—you don’t need any trickery to interview people well.

How to assess communication chops

Assessing communication skills is a bit easier than assessing team-player inclinations. Start with the basics: can you even understand what the interviewee is saying? Often that is not the case. If and when you don’t, tell them that in a gentle way. This happens particularly often in international interviewing when spoken languages are not native to you or the candidate.

The opposite can happen: the candidate does not understand you. You should, of course, try your best to communicate clearly, and if the candidate still has a hard time getting what you are saying or asking, notice if they have the energy to ask you to clarify. If so, that is a good sign.

Being able to listen deeply is what differentiates an ok communicator from a good or great one. Does the candidate flow from what you are saying? In other words, can you notice them using and reshaping pieces of your discourse? Repeating things back to you transformed? Those are good signs. If they just respond things that seem unrelated to what you have been saying, then clearly this is a minus. More often things fall around the middle, and with time you will develop a feel for how good the candidate seems to be, plus the courage to use your intuition when the decision is not clear-cut.

How to assess programming chops

This is the easiest part if you yourself are a programmer, but it’s still challenging sometimes. Granted, it’s easy to reach a conclusion in extreme cases: if the candidate did really poorly or just brilliantly in the coding part.

When things fall more in the middle, which is by far the most typical, your skill can make a huge difference in the decision to say Yes or No to a candidate.

I always like it when a candidate writes, in human language, something like a list of things that need to happen in the program. For example, if the proposed task is to build a search user interface: to jot down that the screen has an input and a submit button, the user will type a string in that input, then press the submit button, and so on. This helps the candidate “think like a computer”, it allows her to expose her thought process to the interviewer and, if her assumptions are off-base, to know early. The worst that can happen is to end up with a little algorithm of what to code.

TDD, or anything similar to that, is a huge positive differential. It is a kind of planning ahead.

I must say it’s very rare that anyone uses TDD in interviews I run, and I’ve run a few hundred of them at the very least. It has never ever happened that a candidate wrote down point by point what should happen in the application.

Finally, good code architecture early on is one of the strongest signs that the candidate is good. If they refactor continuously, improve variable names often, and order things in their codebase without losing their train of thought, you are looking at a hire. And that’s why giving people much harder tasks than they will face in their work is typically not a good idea: you need to see how they can make code understandable to fellow programmers.

So if they start one file and fill it up with disorganized, commented out code and huge blocks of attempts, unless you asked them to do something too hard, in general consider this as a potential flag. But don’t get stuck to this: it’s not uncommon for great developers to seem quite sloppy the first 15 minutes in, and 10 minutes before the hour, to start deleting code and apply a final layer of improvements that leaves the code looking quite wonderful.

If you give the candidate a project that is larger than the hour you have to interview, something I support you do—as long as you set the right expectations, otherwise you will have a frozen person in front of you—, and you enabled the candidate to own the code in her machine, the bonus point is them continuing the project on their own. Until recently I sometimes suggested they do it. Nowadays I don’t even suggest that. If they have a way to reach me, they can do it if they want. I certainly take it as the strongest sign of interest in the part of the candidate if, hours or even a couple of days later, they get in touch with the completed project.

Participating in Daily Standups

Purpose of Daily Standups

You should leave a daily standup:

  1. More energized than you entered it.
  2. Aligned to be in contact after the daily with anyone if your or their main task needs discussion.
  3. Remembering everyone’s update.

Things outside of Daily Standups that can ruin Daily Standups

  1. If team communicates 1-to-1 a lot and not on shared channels.
  2. If people are tackling too many things at the same time.
  3. All basic management deficiencies.

How to prepare for a Daily Standup

Half an hour before the Daily Standup, stop for 5 minutes and:

  1. Decide what is the one initiative you will mention.
  2. Write it down in your own words.
  3. Edit that down to 280 characters.
  4. Include expected time to finish task at the end.
  5. Say it aloud.
  6. Improve the wording or delivery until you have a clear and memorable update.

How to decide what to mention in a Daily Standup

By decreasing order:

  1. Task you are doing that is most problematic.
  2. Task you are doing that you know someone can help you with.
  3. Task that is going well.

How to listen well in Daily Standups

  1. Keep a log of Daily Standups and make notes of every update.
  2. Come back to that log half an hour after the daily and read it again.

Trusting the Scrum Master in Daily Standups

If each team member delivers their update clearly, nicely and concisely, the Scrum Master will have time to:

  1. Ask about important items that were not selected by the team.
  2. Raise questions.
  3. Ask about extraordinary additions to people’s update.
  4. Align people for pairing on issues raised.
  5. Use the extra time for having a bit of fun and enjoying the team before it gets back to work.

My Year in: Software Development

The year of 2018 has been a wild ride for me. It started at EmCasa.com, the startup I described a year ago. We were in the process of putting out a successful MVP, engaging clients, becoming accelerated by Harvard University and closing out an excellent seed round with important investors. All this in just a few months.

Very quickly my job morphed from do-it-all (think product and CTO duties, plus co-founder responsibilities) to team-building. This was a highlight as I was lucky to bring in great people like Gabriela Seabra, Nathan Queija and Rodrigo Nonose, each respectively owning mobile, web frontend and backend. On the design side, Plau.co did great work.

For the first time I felt fully ready to create a digital company in style. The Elixir-React stack worked beautifully, we rolled out features fast and the development team became a cohesive unit very quickly.

Towards the middle of the year, as the company reached 14 people, it had become clear to me that there wasn’t a good cultural fit among the founders. It was one of those very painful decisions but ultimately not difficult to make when visions don’t intersect well enough: we parted ways amicably at the end of July and I went looking for a new job.

That month was tense for me. I knew I had a deadline to basically stop getting paid, my wife was 6 months pregnant with twins and I had nothing lined up as far as work.

I am part of Toptal’s network of freelancers and that was one of the main ways I tried to find the next thing. I found something that really interested me: they had posted an opening to join their Core Team. I applied, passed, and joined the team as a freelancer in August. 

The first few weeks were hard in a sense: I and two other developers coming from Toptal’s talent network had a kind of secondary status, without access to many tools. Luckily our teammates were extremely helpful and welcoming. In a few weeks we became official team members and got access to what the other engineers see.

My team works on Toptal’s public pages, from home to skill pages, basically everything a logged out user can access in the website. 

It’s been a tremendous experience for me so far, as it’s the largest organization I’ve been a part of in technology. Toptal is in fact the largest fully distributed company in the world today. As such, I work with people from all over the world, and that’s the part I like best. To do it from home is the cherry on the cake as my children are very young and being close to the family is priceless at this moment.

At my team I’ve experienced the most well-developed set of processes I’ve ever come across. The firm has reached a level of maturity that allows for very well written task tickets, a proper retrospective at the end of each sprint, interesting strategic discussions among executives, and work that has large-scale, immediate impact. On the other hand, I am very far from top strategic decisions and information, and this feels weird to me having run my own company for so many years.

To a very large extent, this was the year when, for me, technical concerns became secondary to human concerns. Sure, if you’re starting a product alongside just one other person, what tech stack exactly to choose is of paramount importance. In a team of 200 that’s much less important, as each person can focus on a narrower part of the work, and the human interactions can catalyze or hinder progress.

Thus, I gave myself the chance of observing all the teams in the organization: dialogs, processes and rituals. Whatever was visible to me, I tried to learn from. It wasn’t long before I got the opportunity to onboard newcomers, help interview candidates, conduct daily standups and interface with other companies. That’s not to say I didn’t study programming this year: I continued my deep-dive into the Elixir codebase, studied GraphQL, some more Elm, algorithms, some Python and a little bit of Natural Language Processing.

It’s much easier to paste some code or links to a framework and illustrate what goes on as a programmer. Human interactions seem to me much harder to communicate. A book that does a great job of explaining management applied to software is The Manager’s Path, by Camille Fournier. I read it this year and it very neatly sums up many things we learn the hard way, plus many others I simply did not know.

So this year was one in which I started out as CTO and ended-up as an individual contributor. But each experience has taught me a lot about management and leadership, which will be my way forward in technology.


Naive Investor, Episode 400

Today Naive Investor has hit episode 400.

If anything, a good excuse to post about it here.

Initially I went for posting every single episode in this blog, but pretty quickly the blog would become just a mirror of the YouTube channel.

A lot of learning has indeed taken place. That was the main goal. Along the way, people have sent me corrections, suggestions and encouraging messages.

At year end I will publish the update to the little (but growing) investing partnership I’ve been running rather informally for now, and which incorporates the knowledge gathered from the Naive Investor videos.

If you don’t yet subscribe to the channel, please consider doing so: https://www.youtube.com/c/NaiveInvestor

And here’s the 400th episode:

On Learning How to Learn

These past few weeks I took Coursera’s Learning How to Learn, thanks to Toptal’s advancement program.

My main takeaways from this course were the Memory Palace technique and the Hard Start-Jump to Easy technique.

The course is about much more than just those two tips, but I should admit that to a large extent I already knew and practiced them: chunking, diffuse and focused modes of studying, using visual analogies for memorization, the importance of sleep in learning, how apparently unrelated interests help propel deep learning, motivation-procrastination-blocks, focus on process rather than final product. If anything, it was nice to get something of a more scientific basis for things that we learn in life by trial-and-error or by imitation.

The Memory Palace Technique is useful for when we need to memorize unrelated items, for example a shopping list. It’s a seemingly bizarre strategy: you mentally place each item in, say, rooms in your home. Cheese goes on top of the fridge, detergent goes inside the oven, toilet paper on your bed, and somehow, they say, your visual brain helps you recall the items much better.

I was not aware of this technique, and so far I used it for small lists of things, and they seemed to work. We’ll see for challenging, longer lists. And also it remains to be seen in what situations this is better than just writing it down.

The Hard Start-Jump to Easy technique is a test-taking approach I was also unaware of. My best strategy when taking exams and tests was to find the easiest questions and tackle those first, leaving the hard ones for later. The course offers another idea: you start by taking a hard question. Once you feel you got stuck in it and can’t progress, you leave it unfinished and then move to the easy questions. The rationale is this warms up your brain, or sets it in focus, so you gain some performance. 

I seldom take tests these days but I can suppose this is applicable to our own day: start by tackling a hard task, use it to jumpstart our focus, and then move on to easy tasks. Maybe some fine-tuning on this idea can improve performance. I do know that I start my day, whenever possible, by studying, usually things with very long-term goals.

I wish the course went in deeper with practical examples instead of overvarnishing each concept. Still it was enjoyable and it seems likely that it’s going to add to my repertoire for learning.

A Personal Review of Task Trackers

tl;dr: My favorite is Pivotal Tracker.

Last night a friend of mine asked me about task trackers. This text is my response.

Trackers I know off the top of my head:

  • Pivotal
  • Asana
  • Jira
  • sprint.ly
  • clubhouse.io
  • Trello
  • YouTrack
  • Github Projects

Trackers I’ve used for at least a couple of months:

  • Pivotal
  • Jira
  • Trello

Trello has very low support for any kind of “point system”.

The “point system”, which consists of assigning “points” to estimate task complexity or time to finish, is a little contentious. It’s difficult to agree upon the criteria for one “point”. I learned from the folks at CodeMiner 42 to give one point to each task estimated to take from 1 minute to one morning (half a day). Simple as that. The idea is just to try and have all tasks simplified into one point. After a while the results start to show the overall performance, with almost no time wasted estimating tasks.

About 2 years ago I used Trello with a kind of plugin for points. It was very rudimentary and did not generate reports later, so it was not very useful. I am not aware of improvements to this in Trello.

Jira is very complete, and can even be installed in your server. The version I used the longest was installed that way. Consequently, it was and looked very out-of-date. You can choose many variations of Kanban and others, but the interface is bloated and visually confusing. I was unhappy with Jira.

Pivotal Tracker, as I said, is the one I like the most. I really like the fact that it’s just two lanes: Backlog, for tasks that are estimated and prioritized, and Icebox, where you throw ideas to be thought about later. The app is very powerful, and with Epics and Tags you can really organize and grow a board without going crazy.

It’s very easy to change the status of a task. Instead of dragging-and-dropping to lanes, you click Start-Finish-Deliver-Accept or Reject. It’s a simple improvement but on the aggregate it’s significant.

I’ve heard good things about clubhouse.io, apparently a newcomer to this competitive problem space, but I haven’t tried it yet.

I’ve played with Github Project, but at least at the time it missed many basic project management tools.

For the last couple of weeks I’ve been using YouTrack. It resembles Jira but seems more well finished, and I haven’t seen the analytics part in detail yet.

So this is a summed up version of my feelings for task trackers.

Even if you work alone, a task tracker is an amazing stress reducer. As the team grows, it becomes more and more fundamental. If you don’t use one, I encourage you to try as many of the above as you can and see which you prefer.

A [WIP] checklist for interviewing (or getting onboarded) for new jobs or projects

After a year at EmCasa.com, I’m getting ready for a new job or project (let me know of a good one by the way: gustavo@poe.ma). At this point it’s clear that there are questions I should ask the people hiring me, probably before agreeing on the engagement.

Here’s a list of them:

  • What do you expect me to accomplish in 1 week?
  • What do you expect me to accomplish in 1 month?
  • What do you expect me to accomplish in 3 months (if the above two are met with responses)?
  • Who’s the team I’ll initially be working with, and what does each of the people do?
  • Is there a designated project manager in place?
  • Do you use methodologies such as Scrum, Agile?
  • Do you have daily standups, weeklies or recurring meetings?
  • Do you have a project board? Can I look at it?
  • Do you use Slack or some similar tool?

It’s ongoing at this point. I’m also agnostic about the order. Of course, in a conversation, it’s hard to stick to a script and that’s not the main idea anyway. But asking a few of these, I feel, will increase the likelihood of initiating better work relationships.

These blog posts also have good points-of-view:

View at Medium.com

My Current Process for Selecting Developers for Interviews

This year my work has increasingly involved recruiting developers for EmCasa. As I envision the need to involve our team in this crucial part of my job, and to hand over hiring responsibilities in the future, I thought I’d like to put my current process into words to share it and hopefully receive constructive criticism.

The complete hiring process involves more steps than the initial screening itself. Here I will focus on the step going from receiving applications to deciding who are the ones to set up interviews with.

The setting in which the selection process takes place has always been, for me, one in which this is one of many tasks running in parallel: currently I’m also continually reviewing (code and QA) our web frontend client, our mobile clients, our backend, the visual design of the applications, hiring designers, developing macro strategies with my co-founders, building a culture from scratch and the occasional meeting with people from other companies and investors.

Another factor to take into consideration this time around is the relative abundance of candidates, with perhaps 120-150 total this year.

Right now we are 4 developers at EmCasa, working remotely, using Elixir on the backend and React / React Native. We have been emphasizing Brazilian developers since we’re in Brazil.

Oh, and do I need to tell you we are hiring?

Given this scenario, I’ve been using a few heuristics and biases.

1. Good Github page

If the candidate has very few commits over the last 365 days, I move on to the next candidate. A couple of commits a week on average is a bare minimum. It’s ok if it’s clustered recently, it may mean that the person is looking for work.

If no pinned repository uses the technology we’re hiring for, I move on.

If the candidate’s repositories don’t have READMEs, I generally move on.

2. At least one strong public repository

Having found repos using the technologies we’re hiring for, I try to find one that is particularly strong: one where I can understand what it does, how to install and run, is at least a degree above trivial and a glance at the file structure and a few files show care on their part.

I should say that 80-90% of the candidates I review don’t get past this point.

Having installed and run the application, if it does what it needs to do, I take a more detailed look at the code. Perhaps 10 minutes typically.

3. Ability to communicate and think outside code

Having reached this step, and this means only 5-10% of the candidates, I look for evidence of good ability to communicate and think in the broadest terms possible.

This is the moment where I’ll look for the link to their website, blog, Linkedin, and other social media, in decreasing degrees of importance. If I see signs of sloppy writing or thinking, I move on to another candidate.

As you may have noticed, this process is heuristic-heavy and geared toward my specific situation at this moment. Some blind-spots are inevitable, such as the percentage of great developers that don’t use Github all that much. I try to avoid that by taking a quick glance at other possible indications that this is the case. If I can’t see any code, I will move on eventually. If I notice some outlier information, I will bypass my own criteria to some extent, which happens in less than 5% of the cases.

On average the screening of one candidate takes 5 to 10 minutes, but seems to require about as much to clear the mind. For the ones that do well, I will spend between 1 and 4 hours reviewing.

After these screening steps, I send the candidates an email for a quick hangout, which is the beginning of the next step in the hiring process.

My Year in: Software Development

2017 ends for me, at least in terms of software development, on a much higher note than it started.

From January till October I worked on a financial startup through Toptal. I was part of a team of between 5 and 15 people over the period, and my duties were far more restricted than what I’m used to: mostly markup and styling.

I did the work inside Ember but never warmed to the framework. I’m sure it was not because of technical matters per se, rather mostly because I perceived the framework as not picking up traction, therefore not deserving of much investment. It was hard to flow with it since auto-reloads were quite slow and the style of getting and setting values always felt strange to me.

There’s a lot I learned in this project in terms of organizing teams and the main lesson was: if you hire remote team members, embrace that fact full-heartedly. This is not to say that you either do it remotely or locally, but if you have even one team member that is working remotely, you should do everything as if the whole team was remote, otherwise the interactions will break down fast. I’d go further and say that dev teams should work as if remotely even if the team is local, persisting all relevant decisions in writing, avoiding one-to-one communication in favor of groups to save on recommunication and following the basic protocols of remote collaboration.

All the while, I kept spending some time on Elixir, and was always happy doing so even though the language has all but become hugely popular thus far.

Another significant change in my process was my move from Atom to Vim. Yes, the entry barrier is significant, and a bit more so since I use the Colemak keyboard layout, but I am very glad I did it. After a while you’re much more productive in Vim, and it’s a more direct connection between thought and writing. It took about 3 months for me to feel as productive in Vim as I had been in Atom.

Sometime in the middle of the year, a couple of fellows got in touch about starting a company in the real estate sector. After much talk we decided to join forces and we started EmCasa.com.

At EmCasa.com I was free to choose the stack for a greenfield project, and so I chose Postgres, Elixir/Phoenix and React for the web client. Postgres is the dependable relational database, with (I trust) more native support for geolocation, so I didn’t hesitate there. Elixir and Phoenix wasn’t much more of a difficult decision either: I trusted the language entirely, felt the tools are at a very reasonable point, enjoy using it a lot, and thought we could attract good developers to use it. So far it’s worked out that way, and as I get ready to hand over most of the Elixir side to a new team member, we talked to a number of pretty great developers that are fond of the language and excellent at it.

Finally, the React decision was a bit more difficult, and even though I consider myself a frontend developer, has been more fraught with difficulties. It’s hard to say I’d choose React again. I probably would, but I really don’t know if we’d been doing better with Vue.js or maybe even Elm.

EmCasa needs a lot of good SEO, so it needs server-side-rendering and very controllable html headers. It took me a while to sift through a few of the available possibilities, that being the main con of polylyths. Finally I made progress using Next.js, which has been a pleasure to use. I ended up throwing out Redux and I haven’t regretted it. I think Next.js has a good path ahead of itself.

We have a long list of ideas to implement at EmCasa, our backlog is public, and so are our backend and our frontend. If you’re reading this and would like to chat with us about joining the team, check out www.emcasa.com/jobs or write me at gustavo.saiani [at] emcasa.com.