• OpenAI chairman Bret Taylor said that companies trying to train their own LLMs will "burn through millions of dollars."
• "There's ones you can lease, there's open source ones," Taylor said on the Minus One podcast. "Don't do it."
• Taylor said the only companies that could handle the high training costs were OpenAI, Anthropic, Google, and Meta.

There's a scene in "The Dark Knight" where the Joker sets ablaze a massive pile of money. Deciding to develop your own frontier AI model today may be a similar exercise in burning cash — just ask OpenAI chairman Bret Taylor.

Taylor, who has worked for three companies that have trained LLMs, including Google, Facebook, and OpenAI, called training new AI models a "good way to burn through millions of dollars."

On a recent episode of the Minus One podcast, Taylor advised AI founders to build services and use-cases, but not new frontier models entirely.

"Unless you work at OpenAI or Anthropic or Google or Meta, you're probably not building one of those," said Taylor, who also cofounded Sierra AI. "It requires so much capital that it will tend towards consolidation."

That high bar of capital has stopped any "indie data center market" from forming, Taylor said, because it simply costs too much.

Taylor advised that founders work with the AI juggernauts instead — which, it's worth noting, is something that AI giants like OpenAI, where he's chairman, would directly benefit from. OpenAI sells "tokens" to access its API, which developers can build into their applications and programs.

While the American LLM market remains largely consolidated, international players have tested Taylor's theory. In January, DeepSeek released its R1 reasoning model and a corresponding chatbot. DeepSeek used fewer, less advanced chips to build its LLM, minimizing capital costs.

The Chinese AI app shot to No. 1 on the App Store charts, surpassing ChatGPT and igniting a debate in tech and on Wall Street about whether tech giants were overspending on AI model development.

In his podcast interview, Taylor laid out other paths that entrepreneurs could take in the AI market, rather than training a new model. One was the "AI tools market."

"This is the proverbial pickaxes in the gold rush," Taylor said. "It's a dangerous space because I think there's a lot of things that are scratching an itch today that the foundation model providers might do tomorrow."

Entrepreneurs could also try to build what Taylor called an "applied AI company."

"What were SaaS applications in 2010 will be agent companies in 2030, in my opinion," Taylor said.

Building a model from scratch, though, is a sure-fire way to "destroy your capital," Taylor said. He called handmade models "fast-depreciating assets," and not cheap ones either, costing the builder millions of dollars.

"There's ones you can lease, there's open source ones," Taylor said. "Don't do it."

Meta has invested $15 billion into data-labeling startup Scale AI and hired its co-founder, Alexandr Wang, as part of its bid to attract talent from rivals in a fiercely competitive market.

The deal values Scale at $29 billion, double its valuation last year. Scale said it would “substantially expand” its commercial relationship with Meta “to accelerate deployment of Scale’s data solutions,” without giving further details. Scale helps companies improve their artificial intelligence models by providing labeled training data.

Scale will distribute proceeds from Meta’s investment to shareholders, and Meta will own 49 percent of Scale’s equity following the transaction.

Read full article

Comments

Users on X (formerly Twitter) love to tag the verified @grok account in replies to get the large language model's take on any number of topics. On Wednesday, though, that account started largely ignoring those requests en masse in favor of redirecting the conversation toward the topic of alleged "white genocide" in South Africa and the related song "Kill the Boer."

Searching the Grok account's replies for mentions of "genocide" or "Boer" currently returns dozens if not hundreds of posts where the LLM responds to completely unrelated queries with quixotic discussions about alleged killings of white farmers in South Africa (though many have been deleted in the time just before this post went live; links in this story have been replaced with archived versions where appropriate). The sheer range of these non sequiturs is somewhat breathtaking; everything from questions about Robert F. Kennedy Jr.'s disinformation to discussions of MLB pitcher Max Scherzer's salary to a search for new group-specific put-downs see Grok quickly turning the subject back toward the suddenly all-important topic of South Africa.

It's like Grok has become the world's most tiresome party guest, harping on its own pet talking points to the exclusion of any other discussion.

Read full article

Comments

There are few areas where AI has seen more robust deployment than the field of software development. From "vibe" coding to GitHub Copilot to startups building quick-and-dirty applications with support from LLMs, AI is already deeply integrated.

However, those claiming we're mere months away from AI agents replacing most programmers should adjust their expectations because models aren't good enough at the debugging part, and debugging occupies most of a developer's time. That's the suggestion of Microsoft Research, which built a new tool called debug-gym to test and improve how AI models can debug software.

Debug-gym (available on GitHub and detailed in a blog post) is an environment that allows AI models to try and debug any existing code repository with access to debugging tools that aren't historically part of the process for these models. Microsoft found that without this approach, models are quite notably bad at debugging tasks. With the approach, they're better but still a far cry from what an experienced human developer can do.

Read full article

Comments

OpenAI today announced a significant expansion of ChatGPT's customization and memory capabilities. For some users, it will now be able to remember information from the full breadth of their prior conversations with it and adjust its responses based on that information.

This means ChatGPT will learn more about the user over time to personalize its responses, above and beyond just a handful of key facts.

Some time ago, OpenAI added a feature called "Memory" that allowed a limited number of pieces of information to be retained and used for future responses. Users often had to specifically ask ChatGPT to remember something to trigger this, though it occasionally tried to guess at what it should remember, too. (When something was added to its memory, there was a message saying that its memory had been updated.)

Read full article

Comments