Logo Indocyber
Search
ISO
ISO New

Result 0 items

How to Assess Developers in the AI Coding Era, Beyond Algorithm Tests

Mohammad Geralldine NurhadiBy Mohammad Geralldine Nurhadi
Share On :

Many companies still assess developers with algorithm puzzles. Yet AI coding tools can solve puzzles like that in seconds. A test result only shows whether a candidate can write code. It does not show whether the candidate can check code written by AI.

 

This article explains how to assess developers in the era of AI coding tools: what to evaluate, three tasks for a developer interview, and a scoring rubric your IT team can try right away.

 

Why Algorithm Tests Are No Longer Enough

 

Algorithm puzzles used to be useful, because writing code from scratch was hard and slow. Now AI writes most of the first draft. The Stack Overflow Developer Survey 2025 found that 84% of respondents use or plan to use AI tools, up from 76% a year earlier.

 

Even so, trust is still low. In the same survey, 46% of developers said they do not trust the accuracy of AI tools, while 33% said they do. The survey measures developer perception worldwide, not conditions in Indonesia.

 

Feeling faster is not always being faster. The METR randomized controlled trial in 2025 involved 16 experienced open-source developers and 246 tasks. Before starting, they expected AI to cut their working time by 24%. The measured result went the other way: tasks done with AI took 19% longer.

 

This result should be read with care. The sample was small, it covered one type of work, and the tools were early-2025 tools. In February 2026, METR wrote that its follow-up study was unreliable, because many developers refused to work without AI. METR also judged that developers are probably better helped now than in early 2025.

 

The conclusion is simple. AI is widely used, but its output cannot always be trusted. So the most valuable skill is checking code, not writing it from scratch, and the interview should measure that.

 

Four Things to Assess

 

The table below compares what the old tests measure with what needs to be measured now.

 

AreaOld testWhat to assess now
Understanding the problemSolving a problem that is already clearTurning a vague requirement into a clear task
Code qualityCode runs on the sample inputFinding mistakes in AI-written code
TestingBarely measuredWriting tests that prove the code is correct, including edge cases
Using AISeen as cheating or irrelevantGiving clear instructions and knowing when not to use the result

 

The biggest gap is in code quality and testing. A developer who only copies AI output can pass an algorithm test, but will struggle to prove that the code is correct.

 

Read also: Why Companies Are Now Looking for AI Engineering Skills, Not Just Programming Skills

 

Three Interview Tasks

 

These three tasks fit in one technical interview of about 90 minutes. The candidate may use AI coding tools, as in daily work. What you observe is how the candidate works with the tool.

 

  1. Code review of AI-written code. Give a piece of AI-generated code with two or three problems, for example unvalidated input or a missed edge case. Ask the candidate to find and explain them.

  2. Fix and test. Ask the candidate to fix one problem, then write a test that fails before the fix and passes after it.

  3. Explain the decisions. Ask which parts of the AI code the candidate accepted, changed, or rejected, and why.

 

Watch the thinking process, not only the final result. A candidate who can explain why they rejected an AI suggestion shows stronger understanding than one who accepted everything.

 

Include one security problem in the first task. Veracode tested more than 100 large language models on 80 programming tasks and reported in July 2025 that the generated code contained security flaws in 45% of cases. That figure comes from Veracode's test tasks, not from all AI code in production. The message still holds: code that runs is not necessarily safe.

 

A Rubric for Assessing Programmer Skills

 

Score each skill from 1 to 4, then note the evidence you saw during the interview:

 

  • Finding problems: did the candidate find all the planted problems, including one that was not obvious?

  • Testing: does the test really check the expected result, instead of just repeating the code?

  • Using AI: are the instructions clear, and is the result checked before it is used?

  • Explaining: can the candidate explain the reasons for a decision to a non-technical colleague?

 

Set a minimum score for each skill before the interview starts. That way, the assessment does not depend on the interviewer's personal impression.

 

Read also: AI Speeds Up Coding, but Does It Improve Software Delivery?

 

Limits of Using the Rubric

 

This rubric does not replace other assessments:

 

  • Basics such as data structures and problem solving still matter, because a developer who does not understand them will struggle to review AI code.

  • For junior roles, the code review task should be simpler and come with step-by-step hints.

  • The 90-minute task is only a small sample of how the candidate works. Project history and references still need to be checked.

  • After a few weeks of real work, compare the interview impression with the quality of pull requests and review results, then improve the rubric.

 

When Project Needs Move Faster Than Hiring

 

The assessment above can be used by your own IT team. When project needs arrive faster than your internal selection process, Indocyber can help through its IT Professional Services. This service meets IT talent needs with technical expertise that fits the project, including programmers who can make use of AI coding tools. The expertise covers Software Developer, Quality Assurance, as well as System Analyst and Business Analyst.

 

[ Discuss your project's IT talent needs with our team -> IT Professional Services Indocyber ]

 

 

supergrafis
supergrafis

Stay Connected with the Digital Transformation Updates

Indocyber

INDOCYBER GLOBAL TEKNOLOGI

An integrated information & technology system provider, dedicated to delivering highly effective solutions in the field of Information Systems.

FOLLOW US

CONTACT US

  • email icon
  • phone icon
Copyright © 2026 by PT Indocyber Global Teknologi