AI assistants may often hinder developers rather than aid them when it comes to complex projects.
Scientists confirm: This is the most effective way to get your cat’s attention, according to new research
Elderly Couple Refuses Reserved Seats—Viral Train Standoff Sparks Fiery Debate on Courtesy
Do AI-powered programming assistants truly expedite software development? This challenging question was investigated by Model Evaluation and Threat Research (METR), a nonprofit research institute that has previously collaborated with OpenAI and Anthropic.
AI May Extend the Time Needed to Solve Certain Problems, Says METR
To clarify this controversial topic, METR enlisted 16 developers who have been contributing to open-source projects for several years but had only “moderate experience” with AI tools. As part of the study, whose findings were released on July 10, 2025, each developer compiled a list of 246 tasks deemed useful for their repository, ranging from bug fixes to adding features—tasks that are typically part of their regular work, METR emphasized.
These tasks, which could be completed in two hours, were then randomly assigned: some were to be done with the help of an AI assistant, others without. When using artificial intelligence, the developers employed Cursor Pro, powered by either Claude 3.5 Sonnet or Claude 3.7 Sonnet—described by the research institute as “the most advanced models available at the time of the study.”
Surprisingly, the study revealed a “gap between perception and reality.” When developers were permitted to use AI-enhanced tools, they took 19% more time to solve problems, a result that puzzled the researchers. They had expected AI to speed up the process by 24%, and even after experiencing the slowdown, developers continued to believe that AI had accelerated their work by 20%, METR noted.
Why Did AI Slow Down the Developers?
To explain the discrepancy between the developers’ expectations and the observed outcomes, METR identified several contributing factors:
- Overblown Expectations: Developers believed that AI would significantly speed up the implementation time.
- High Expertise: Participants knew their projects so well that the AI added little value. “Developers slow down more on unfamiliar problems,” METR clarified.
Why You Should Never Reheat These Foods in the Microwave – The Hidden Dangers Experts Warn About
I tried the top 5 guard dogs—here’s what makes these breeds the ultimate protectors
- Lack of Context: The AI struggled to grasp the context in which it operated.
- Insufficient Reliability: Only 44% of the AI-generated suggestions were approved by the developers. Participants spent a lot of time correcting or revising the code that had been generated.
- Complex Environments: In repositories that were on average a decade old and contained over a million lines of code, the limitations of AI quickly became apparent.
“There may be a learning effect for AI tools like Cursor that only becomes apparent after several hundred hours of usage. Our participants have not reached this point yet,” METR nevertheless tempered.
Similar Posts
- Study Reveals: How Developers Will Use AI in 2025!
- Anthropic Unveils Claude 4.5 Sonnet: “World’s Best Coding Model!”
- Meet Claude 4 Opus and Sonnet: Anthropic Unveils Its Latest AI Innovations
- OpenAI Unveils Codex: Revolutionizing Automated Coding with ChatGPT AI Agent
- AI Not the End for Jobs: Insights and Tips from a Freelance Web Developer
Jordan Park writes in-depth reviews and editorial opinion pieces for Touch Reviews. With a background in UI/UX design, Jordan offers a unique perspective on device usability and user experience across smartphones, tablets, and mobile software.