Devs gaining little (if anything) from AI coding assistants

A recent study by Uplevel has found that developers using AI coding assistants, such as GitHub Copilot, are not experiencing significant productivity gains. The study analyzed the performance of around 800 developers over six months and reported that the use of Copilot resulted in a 41% increase in bugs without notable improvements in key programming metrics like pull request cycle time and throughput. While some developers claim increased productivity, the study suggests that many are spending more time reviewing code generated by AI rather than writing it. Additionally, the study indicated that AI tools have not alleviated developer burnout, as those using Copilot did not see a reduction in after-hours work compared to those who did not use the tool. Mixed results were reported across different companies; while some, like Innovative Solutions, noted significant productivity improvements, others, such as Gehtsoft USA, found AI-generated code challenging to debug and often preferred rewriting code from scratch. The findings highlight the need for tempered expectations regarding AI coding assistants, emphasizing that while they can enhance certain aspects of coding, they are not a replacement for human developers.

- A study found no significant productivity gains from AI coding assistants like GitHub Copilot.

- The use of AI tools has led to an increase in bugs in the code produced.

- Developers are spending more time reviewing AI-generated code rather than writing new code.

- Mixed results exist among companies regarding the effectiveness of AI coding assistants.

- Expectations for AI tools should be moderated, as they are not a substitute for human expertise in software development.