Optimizing Your Hiring Funnel

Interviewing engineers can be daunting, as it involves making critical decisions about a candidate's suitability for a role within a short period. In addition, most interviewers are skeptical about the reliability of their assessments, and it's common for them to lack confidence in their ability to make a hiring decision based on their interviews. However, there are ways to enhance the hiring process to minimize errors, promote fairness, and enable continuous improvement. To achieve this, we focus on determining the highest level at which a candidate can excel and make leveling decisions before final hiring decisions. By the end of this article, my hope for you all is to be enabled to create and implement the following principles for interview pipelines:

• Develop a detailed understanding of the candidate's strengths at a granular level.
• Emphasize the signals that distinguish the most senior candidates from others.
• Use a uniform vocabulary when describing a candidate's strengths.
• Establish a framework for integrating information and creating a candidate profile.

Define Clear Job Requirements

Establishing precise and unambiguous job requirements, such as technical skills, experience, and other relevant qualifications, is crucial before commencing interviews. These expectations should align with your internal career ladder, the position level based on current requirements and budget, and any additional qualifications pertinent to the role.

This can be particularly challenging for roles like engineering managers, product managers, or architects, which can vary significantly from company to company. However, it is often the primary reason for interview failures and requires reinforcing expectations during candidate debriefs to ensure that interviewers are on the same page.

In my experience, agreeing on the requisite skills for a job can be more challenging than anticipated and may require considerable time with interviewers to reach a consensus. This can be especially true for roles such as engineering management, DevOps, and data science, where there may be variations in the necessary programming experience.

1. Start with a comprehensive job description: Before the interview process begins, create a job description that outlines the specific responsibilities, technical and non-technical skills, and experience required for the role. The job description should be clear and detailed, outlining the critical requirements for the position.
2. Identify essential technical skills: Once you have created the job description, identify the crucial technical skills required for the role. This could include programming languages, software development methodologies, databases, and operating systems.
3. Prioritize non-technical skills: Besides technical skills, non-technical skills are required for the role. This could include communication, collaboration, problem-solving, and adaptability.

Create a rubric

Developing a rubric that assesses the candidate's performance on each characteristic and priority is essential. A good rubric should have clear criteria and explicit scores for each level. If creating a useful rubric proves difficult for a particular attribute, consider finding an alternative test that is easier to evaluate. Above all else, avoid using boolean-based rubrics. Specific tests have a binary outcome, such as whether a candidate has experience managing someone. These tests are inefficient as they don't provide much additional information beyond a simple "yes" or "no." Additionally, boolean questions can often be answered through a resume or pre-interview screening.

1. Identify essential skills and attributes for the role:
   1. Review the job description and identify the essential technical and soft skills needed for the role.
   2. Consider factors such as years of experience, programming languages, project management, communication, teamwork, problem-solving, and adaptability.
   3. Consult with stakeholders such as senior developers, project managers, or HR to understand the role's requirements.
2. Develop criteria and rating scales for each skill/attribute:
   1. Once you have identified the essential skills and attributes, develop specific criteria.
   2. Create rating scales that will enable the interviewer to grade candidates objectively based on their level of proficiency.
3. Create interview questions aligned with the criteria:
   1. Draft interview questions that align with the skills and attributes identified in step one.
   2. Ensure the questions are open-ended and specific, requiring more than a simple yes or no answer.
4. Train interviewers and use the rubric to evaluate candidates consistently:
   1. Train interviewers on the rubric and the interview questions to ensure everyone evaluates candidates similarly.
   2. Encourage interviewers to take detailed notes during the interview and rate the candidate objectively based on the established criteria.
   3. After each interview, compare notes and ratings to ensure that evaluations are consistent and fair, and use the rubric to make informed hiring decisions.

Here's a minimalist baseline to get started:

Keep in mind that the interviewers are not basing their decisions on the candidate's actual performance but on how that performance measures up to their expectations. Of course, such a subjective evaluation process could be better, and we recognize that it's different from how candidates would want to be evaluated. This goes back to not having binary systems for evaluation where interviewers determine whether a candidate "passed" or "failed." Instead, we provide them with a set of attribute rubrics to consult when summarizing the candidate's performance, as follows:

Architecture: Software Design

Involve multiple interviewers

Engaging several interviewers from different teams or departments can comprehensively assess a candidate's abilities and suitability within the organization. The advantages of incorporating diversity and various perspectives during an interview process for software engineers are manifold: A more inclusive evaluation process: When you have a diverse group of interviewers, you are likelier to have a fair and inclusive evaluation process. Interviewers may notice different strengths and weaknesses in a candidate, which can help mitigate any unconscious bias. Improved decision-making: Diversity in an interview panel can lead to more informed and well-rounded decision-making. Different perspectives can bring new insights and ideas, leading to a better evaluation of the candidate's abilities. Enhanced creativity and innovation: Having diverse interviewers can also promote creativity and innovation within the interview process. By incorporating different perspectives and approaches, you can identify unique solutions to problems or identify candidates who can bring new ideas and perspectives to your organization. A more representative workforce: By incorporating diversity in the interview process, you may also attract a more diverse pool of candidates. This can lead to a more representative workforce, bringing a range of benefits to your organization, including improved performance, increased innovation, and enhanced employee satisfaction.

Level with data and iterate with purpose

After establishing a solid foundation, the crucial step in creating a sustainable and effective hiring process is incorporating metrics at each stage (sourcing, phone screening, take-home tests, on-sites, offers, etc.) and tracking these over time. For example, if your referral-to-sourced/direct ratio decreases, it may signal a problem, precisely a morale issue in your existing team. Similarly, a decrease in your acceptance rate may indicate that your offers could be more competitive or that your best interviewer has experienced burnout and is repelling potential candidates.

Continuously monitor these metrics and seek candidate feedback to ensure your process remains robust and effective. I recommend establishing analytics as the final step in building a superior interviewing strategy. While traditional optimization techniques dictate measuring first and then optimizing, I propose the opposite approach.

Summary

To summarize, with this information, you can create a comprehensive guide to optimizing your hiring funnel when interviewing engineers. To achieve this, you must define precise job requirements and develop a rubric that assesses a candidate's performance on each skill and attribute. In addition, creating a job description, identifying essential technical skills, prioritizing non-technical skills, and establishing a framework for integrating information and creating a candidate profile. I hope this is a start for you and your team to develop a framework for integrating data and creating a candidate profile!