Global tech research company Gartner estimates that by 2028, 75 percent of enterprise software engineers will use AI code assistants, up from less than 10 percent in early 2023.

As of the third quarter of 2023, 63 percent of organizations were piloting, deploying or had already deployed AI code assistants, said the survey of 598 software engineering leaders at large enterprises.

However, Philip Walsh, Gartner senior principal analyst, warns that there can be a mismatch between IT leadership's expectations and software teams' experience when it comes to productivity uptick.

He said vendors selling AI-assisted coding tools were claiming they could increase coder productivity by as much as 50 percent, while a third of CIOs (34 percent) and technology leaders thought AI code generation might be a "game changer" for their software development efforts.

"That's some really high expectations from the productivity gains from these AI code assistants," he said.

While the popularity of AI coding tools will undoubtedly increase, development teams might need to manage the expectations of their senior managers ... 'They're not going to hear it from the vendors'

But looking more closely at the vendors' claims reveals that the benefits of AI-powered coding tools can be confined to quite narrow tasks. For example, one study relied on an A-B style experiment where a team writing a web server in JavaScript was pitted against another that also employed AI coding tools. Writing a boilerplate for Python was another common comparison task, he said.

But these tasks might not be representative of the tool's abilities because of the abundance of training data online demonstrating how coders had already tackled the problems.

At the same time, coding itself did not make up the majority of effort in the whole software development lifecycle, Walsh said.

"There are a whole variety of tasks [involved in software development] from planning, design, research, and actual code generation and development … then a lot of testing and verification, and then deployment, configuration, and monitoring. Even if you're getting 50 percent faster task completion [on coding], that's only going to be 50 percent of 20 percent. So that means only 10 percent greater improvement in overall cycle time," he said.

While the popularity of AI coding tools will undoubtedly increase, development teams might need to manage the expectations of their senior managers.

"They're not going to hear it from the vendors," Walsh said. "Hopefully, their developers and the engineering leaders are going to tell them and they're going to listen. What we do not recommend is some kind of top-down productivity mandate. That doesn't work."

He said development teams need the freedom to figure out the best use cases. They need to work in a learning culture of experimentation, with the freedom to fail to get the most from the new tools.

"CIOs need to create that culture and listen to their people, but also create that space for experimentation and failure," he said. ®

https://www.theregister.com//2024/04/13/gartner_ai_enterprise_code/