Allowing AI in the interview is the easy part

Ninety percent of developers report using AI at work. That figure comes from DORA's 2025 report, which surveyed close to 5,000 technology professionals, and it has been climbing for three years running.

Most first-round screens still run as though the number were zero. The two usual responses — forbid the assistant, or say nothing and hope — end in the same place: you finish the interview knowing nothing about how this person works on the job they would actually be doing.

The failure mode isn't fluency. It's verification.

Ask developers what goes wrong with these tools and they don't say the output is useless. Stack Overflow put the question to 31,476 of them. The most common answer, at 66%, was output that is almost right, but not quite. 45.2% said debugging AI-generated code takes longer than they expected. Asked separately whether they trust the accuracy of what comes back, 3.1% said they highly trust it.

That is a specific failure mode with a specific antidote. Code that is obviously wrong costs almost nothing — someone runs it and it breaks. Code that is plausibly wrong costs whatever it costs to find later. The engineer who reads the output before accepting it is doing the only thing that separates those two outcomes.

DORA found the same shape at team level. AI adoption correlated positively with delivery throughput and negatively with delivery stability: teams shipped more and broke more. The report's explanation is that acceleration exposes whatever review discipline already existed, and its summary line is worth keeping.

AI doesn't fix a team; it amplifies what's already there. Strong teams use AI to become even better and more efficient. Struggling teams will find that AI only highlights and intensifies their existing problems.

If that's right, hiring is where the amplification factor gets set. Every person you add either raises or lowers the rate at which confident wrong answers reach the codebase.

Why “can they code without AI” stopped being the question

Employers are drifting from prohibition toward permission — slowly and unevenly, with something over a quarter of employers in New York now allowing assistant use in technical interviews. The direction isn't seriously in doubt, because the alternative is testing a version of the job that no longer exists.

Both of the usual stances cost you something you can name. Forbid the assistant and you measure recall under an artificial constraint, which tracks the real job less well every quarter. Say nothing, and the candidate uses it anyway, out of frame — you have run an unobserved open-book exam and learned nothing about how they used the book.

The third option is to hand them the assistant on purpose and watch. It takes more setup than either alternative and it is the only one that returns a signal you can act on.

The five things worth scoring

Five components, with weights. They are not equal, and why they are not equal is most of the argument.

• Verification, 30%. Does the candidate read what came back, reason about it, test it — or accept it and move on? Weighted heaviest because it maps directly onto the failure mode above. This is the component that decides whether speed turns into throughput or into rework.
• Prompt quality, 20%. Clear, well-scoped requests carrying the right context and constraints. A hard task gets decomposed rather than handed over whole.
• Outcome, 20%. The artifact works and meets the acceptance criteria, and the assistant accelerated the task rather than derailing it. Worth scoring separately from the process, because good process with a broken result is still a broken result.
• Iteration, 15%. When the model returns something wrong, does the candidate refine and converge — or thrash, or give up?
• Delegation judgment, 15%. Knowing what to hand over and what to keep. Recognising the tasks where the model is unreliable, and holding those back.

Verification carries the most weight because it is the component that survives the models getting better. Prompt technique is partly an artifact of today's tooling and will look different in eighteen months. The habit of not shipping something you haven't read does not expire.

Running it without building anything

You don't need instrumentation for most of this. You need a real task, an assistant the candidate is told they may use, and the discipline to watch rather than ask.

• Use a task from the problem space your codebase actually lives in, not a puzzle. The assistant only gets interesting when the work is.
• Plant one case where the confident answer is wrong — a subtly incorrect library assumption, a plausible but false claim about the data, an approach that holds on the happy path and nowhere else. Not a trick question. This is the highest-signal element in the whole exercise.
• Score what you observe: the prompts sent, what came back, what was checked, what was accepted, and whether the thing works at the end.
• Ask no version of “how do you use AI in your workflow.” The answer is rehearsed and unfalsifiable, and it is the closest thing to a free pass left in a technical interview.

Self-description is the weakest instrument in any interview. There are better ways to test whether a memory is first-hand, and they all involve asking for a detail only a participant would hold.

Four things that look like signal and are not:

• Blind acceptance that happens to work. If the output went in unread and the task passed anyway, that is a zero on verification, not a pass. Which is why the exercise needs at least one case where it would not have passed.
• Thrashing. A dozen near-identical re-prompts with no convergence is not iteration, however busy it looks.
• Over-delegation that lands. Handing the model the core judgment call the role exists to make scores low on delegation even when the answer turns out right.
• Light assistant use. A candidate who solves it cleanly with barely any help hasn't demonstrated poor collaboration. They've given you limited signal on this dimension, and you score the primary rubric instead.

That last one matters more than it looks. The moment a rubric rewards visible tool use, candidates optimise for visible tool use, and you are back to scoring a performance rather than the work.

This is the shape of the dimension as we've built it at Recio: scored on a real hands-on lab rather than a conversation about AI, and reported as a standalone sub-score rather than folded into an overall number, with the prompts, the iterations and the verification steps attached as evidence. A hiring manager can read what the candidate actually did and disagree with the score. Keeping it separate is the point — an averaged number hides the exact behaviour you were trying to see. There's more on how the pieces fit on the interview engine page.

Handing an engineer a fast, confident, occasionally wrong collaborator doesn't change what good looks like. It changes how quickly you find out.