Discussion about this post

User's avatar
Sergio Visinoni's avatar

Interesting article, as usual.

I'm just unconvinced by a fundamental aspect of it:

> Our analysis focused on self-reported time savings, where developers estimated the number of hours per week they saved through the use of AI coding assistants.

You could argue that it's "old" in "AI terms" (I wouldn't agree); you can also argue with the exact findings, but one thing seemed to be pretty clear from the 2025 METR study on AI in OSS: humans are particularly bad and unreliable when self-reporting productivity gains.

https://metr.org/blog/2025-07-10-early-2025-ai-experienced-os-dev-study/

So, why are you still reporting that as a reliable indicator for the study?

Also, this conclusion

> This suggests that the plateau in time-savings may not be a failure of the tool or the user, but a sign that the bottleneck has shifted from individual code production to team-level coordination.

What evidence is there that the bottleneck has indeed shifted, meaning that there has ever been "individual code production"? Has the bottleneck really shifted, or was it always team-level coordination (and decision-making, and figuring out what's the right thing to do, and dealing with changing priorities, etc.)?

There is an expressed (implicit) belief that might be the case, but what's the evidence to support this statement?

1 more comment...

No posts

Ready for more?