AI job interviews: HackerRank tests candidates’ thinking assessment methods.

HackerRank Introduces AI-Driven Interview Tool

HackerRank has unveiled a new tool named Chakra, an artificial intelligence agent designed to conduct job interviews with candidates. This innovative tool not only assesses candidates’ final answers but also evaluates their thought processes in arriving at those answers.

After approximately six months of testing, HackerRank officially launched Chakra for its clients on Monday.

Tool Trials with Companies

The startup reports that its AI interview agent conducted over 500,000 interviews during the testing phase. Companies such as Snowflake, Snorkel, and Capgemini participated in the trials. HackerRank also utilized the tool internally for testing and evaluation purposes.

Assessing Thought Processes

Artificial intelligence has already made its mark in recruitment interviews. Companies now employ voice agents and other automated tools to screen applicants and streamline the hiring process, according to a report from TechCrunch reviewed by Al Arabiya Business.

Job seekers also benefit from AI tools that assist them in navigating interviews, sometimes without the employers’ knowledge. However, HackerRank believes that Chakra can transform not only how interviews are conducted but also what companies can measure in candidates.

In addition to determining whether a candidate arrives at the correct answer, the company aims to evaluate traditionally hard-to-measure indicators such as critical thinking and decision-making abilities. They also seek to assess what they term “AI fluency,” which refers to a candidate’s ability to articulate a problem to the AI, evaluate its responses, and guide it toward the appropriate solution.

Redefining the Interview Process

Vivek Ravisankar, co-founder and CEO of HackerRank, stated that traditional evaluation methods focused primarily on assessing outputs. He added that AI has changed the equation, as nearly anyone can produce outputs or final products. Consequently, the question for employers becomes: Can we understand the thought processes and judgments that led to this work?

Simulating Real Job Tasks

Chakra’s interview format closely resembles actual job performance rather than a traditional programming test. Candidates receive a task related to a real code repository and work on it within a workspace that includes an AI assistant.

As candidates work, Chakra leverages the context to ask follow-up questions. For instance, the tool may inquire why a candidate chose a specific approach over another or how the solution would change if a new constraint were added to the task. This approach ensures that evaluation extends beyond the final code result to include the decisions made by the candidate and their thought processes during problem-solving.

Ravisankar emphasized that Chakra not only alters the interview format but also reshapes the hiring process itself. He explained that what previously required three separate stages—an interview with a recruiter, a take-home test, and a follow-up interview with an engineer—can now be integrated into a single interview using Chakra.

Concerns About Cheating

Allowing candidates to use AI during interviews may seem to facilitate cheating. However, HackerRank asserts that its findings indicate otherwise. Ravisankar noted that indicators of suspicious activity were 70% to 80% lower in Chakra interviews compared to similar traditional HackerRank tests. He pointed out that these rates vary based on factors such as the candidate’s geographic location and experience level.

According to him, permitting candidates to use AI officially reduces the incentive to secretly employ external tools that could provide answers during the interview. Rather than attempting to prevent AI usage, HackerRank focuses on allowing it and then assessing candidates’ ability to utilize it effectively.

Shifting the Business Model

HackerRank launched at the TechCrunch Disrupt event in 2012, primarily building its business around coding tests and challenges that help companies evaluate and hire developers based on their technical skills. The Y Combinator-backed startup currently serves over 3,000 corporate clients, including Amazon, Nvidia, Clay, and Replit, alongside a community of over 30 million developers worldwide.

However, Chakra represents a shift toward a different model than the one HackerRank has relied on for years. The company’s traditional product heavily emphasized testing developers’ ability to solve coding problems accurately. Ravisankar believes that the rise of AI has rendered this model less effective in measuring engineers’ true capabilities.

He likened the company’s transformation to Apple’s transition from the iPod to the iPhone, noting that while the old product still holds value, the new product reflects the direction the market is heading. Ravisankar stated that Chakra will be the focal point and the means by which HackerRank will advance.

AI and Hiring Decisions

Granting AI a larger role in evaluating candidates raises various questions, particularly regarding how much companies should delegate hiring decisions to algorithms. Ravisankar clarified that Chakra is designed to assess candidates and assign scores, not to make final hiring decisions, which remain in human hands.

He believes AI can handle the more structured parts of the interview by consistently applying the criteria set by the company across all candidates. In contrast, human interviewers can focus more on determining whether they genuinely want to work with the candidate and addressing their questions about the company, team, and position.

AI Bias Concerns

Ravisankar stated that AI can be significantly less biased than humans if calibrated correctly. He bases this on the ability to direct the system to apply the same evaluation criteria to all candidates, rather than being influenced by factors such as personal background or educational level.

However, applying the same standards consistently does not necessarily mean that the AI system is free from bias. Automated hiring tools may inherit biases present in the data, models, or evaluation criteria on which they are built, and in some cases, they can amplify these biases.

For this reason, regulatory bodies in several markets have begun scrutinizing the use of AI in hiring decisions.

Increasing Regulatory Oversight

The use of AI in recruitment is already facing heightened regulatory scrutiny. For instance, New York City requires employers using certain types of automated hiring decision tools to subject them to independent bias audits and notify candidates before using these tools in their evaluations.

Ravisankar acknowledged that hiring is a field subject to regulations and legal rules, explaining that compliance with these regulatory requirements was one of the aspects HackerRank had to consider while developing Chakra.

As AI transitions from a supportive role to that of an interviewer and evaluator, job interviews may become one of the most revealing areas of how technology can redefine the measurement of human skills, rather than merely automating traditional processes.

To follow the news in Arabic