Overview and Evaluation Methods of Live Coding Interviews

Live coding is an interview format adopted by tech companies like Google and Mercari.

Traditionally, it was common to write code on a whiteboard in an office while interacting with the interviewer. With the spread of remote work, it's now increasingly common to conduct coding and system design online via video calls.

In some cases, a learning period (around one month) may be provided before the selection process.

Reference: Image of Live Coding The following video published by Google serves as a good reference for the archetype of live coding. How to: Work at Google — Example Coding/Engineering Interview

Advantages and Disadvantages of Live Coding

While this format has clear advantages, there are also operational challenges.

Advantages

• Process Evaluation: Beyond the completeness of the code or design, it allows evaluation of the approach and thought process in tackling a problem.
• Communication Skills Assessment: Through interaction with the interviewer, it's possible to assess practical aptitudes such as problem identification methods, technical level, and how to proceed to a solution.
• Shorter Selection Period: Compared to pre-assignment formats, the candidate's time commitment is shorter, potentially shortening the overall selection process.

Disadvantages and Countermeasures

The biggest challenge is that the evaluation can vary significantly depending on the interviewer's skill. If the evaluation criteria are ambiguous, the candidate's intentions might not be correctly understood, leading to an inaccurate assessment. Furthermore, it's difficult to log the thought process, which tends to make evaluations subjective.

To address this, advanced companies implement thorough interviewer training.

Case Study: Interviewer Training at Mercari

To ensure the quality of live coding, Mercari undertakes the following initiatives:

• Introduction of Structured Interviews: Based on Google's methodology (re:Work), question content and Rubric (evaluation criteria) are clarified.
• Role-Playing: Internal engineers act as candidates, conducting simulated interviews including feedback, lasting 120 minutes.
• Reverse Shadowing: Main and sub-interviewers share a script for practical training.

Details: Improving the Engineer Hiring Process | Mercari Engineering

Evaluation Approaches

Evaluation methods are mainly categorized into two types: "Algorithm Format" and "System Design Format".

1. Algorithm Format

This format assesses understanding of computational complexity and data structures.

• Example Questions:
  • How would you implement this without using a hash map?
  • How can the computational complexity be further reduced?
• Evaluation Perspective:
  • Does the candidate recognize and understand that there are diverse means for implementation?

2. System Design Format

This format tests broader knowledge and design capabilities than the algorithm format. Since the correct answer changes depending on the prerequisites, flexible thinking is required.

• Example Questions:
  • If the number of users increases from 10,000 to 1,000,000, what should the design be to prevent failures?
  • If the number of overseas users outside Japan increases, what kind of design would be appropriate?
• Evaluation Perspective:
  • Can the candidate consider availability and scalability from multiple angles?
  • Does the candidate have a variety of design patterns and can they design considering trade-offs?

Important: HireRoo's Solution With HireRoo, the entire live coding process is logged, enabling objective evaluation while preventing subjectivity.