There are few areas where AI has seen more robust deployment than the field of software development. From "vibe" coding to GitHub Copilot to startups building quick-and-dirty applications with support from LLMs, AI is already deeply integrated.
However, those claiming we're mere months away from AI agents replacing most programmers should adjust their expectations because models aren't good enough at the debugging part, and debugging occupies most of a developer's time. That's the suggestion of Microsoft Research, which built a new tool called debug-gym to test and improve how AI models can debug software.
Debug-gym (available on GitHub and detailed in a blog post) is an environment that allows AI models to try and debug any existing code repository with access to debugging tools that aren't historically part of the process for these models. Microsoft found that without this approach, models are quite notably bad at debugging tasks. With the approach, they're better but still a far cry from what an experienced human developer can do.
Steve Jennings / Stringer/Getty Images
Sometimes, it's OK to be lazy with your AI prompts thanks to models becoming smarter, the Stanford professor and former Google Brain scientist Andrew Ng has said.
Standard prompting advice is to give large language models, or LLMs, a lot of context and specific instructions for a query.
But Ng said in an X post that "lazy prompting" — when users put information into a model with little context or without explicit instructions — could, in some cases, be a better strategy.
"We add details to the prompt only when they are needed," he said Thursday.
He pointed to developers using LLMs to debug code as an example.
"When debugging code, many developers copy-paste error messages — sometimes pages of them — into an LLM without further instructions," Ng wrote. "Most LLMs are smart enough to figure out that you want them to help understand and propose fixes, so you don't need to explicitly tell them."
The idea is that the LLM can then detect a buggy implementation and quickly provide a good solution without much context.
Developers of foundational models are scrambling to make LLMs more inferential, which means the models go beyond producing output and begin to "reason" and gauge the intent of the prompt.
Ng said that lazy prompting is an "advanced" technique that works best when the LLM has enough preexisting context — and an ability to infer intent. It would only work when users can "iterate quickly using an LLM's web or app interface," he added.
It's not useful if the LLM needs a lot of context to provide a detailed response or if the model is unable to detect hidden bugs or errors, he said.
AI is fast transforming how people code and interact with software. Vibe coding — when people give natural language instructions to AI to write code — has recently swept Silicon Valley and beyond.
Last week, Ng launched a "Vibe Coding 101" short course for newbies who want to learn how to use generative AI tools to write and manage code.
Login