• Top OpenAI execs keep talking about the company's never-ending demand for GPUs.
• GPUs have become one of the key metrics in the AI race.
• OpenAI CEO Sam Altman already wants the company to 100x the 1 million GPUs it will have by the end of 2025.

OpenAI's C-suite can't stop talking about the company's insatiable demand for computing power.

"Every time we get more GPUs, they immediately get used," OpenAI CPO Kevin Weil recently told XPrize founder Peter Diamandis during an interview on Diamandis' "Moonshot" podcast.

Weil is just the latest OpenAI exec to sound off on the topic. OpenAI CEO Sam Altman said last month that the company will bring on more than 1 million GPUs by the end of the year. For comparison, Elon Musk's xAI disclosed that it used a supercluster of over 200,000 GPUs called Colossus to help train Grok4.

"very proud of the team but now they better get to work figuring out how to 100x that lol," Altman wrote on X in July.

Two days later, Musk's, Altman's former ally turned rival, said he wants xAI to have 50 million equivalents of Nvidia's H100 chip in the next five years.

The competition is for good reason. Jonathan Cohen, VP of Applied Research, recently said GPUs are like "currency" for AI researchers. Priscilla Chan, Mark Zuckerberg's wife and a cofounder of the couple's philanthropic organization, said the Chan Zuckerberg Initiative uses GPUs as a recruitment tool.

Weil said the necessity is really quite simple: "The more GPUs we get, the more AI we'll all use." He compared how adding bandwidth made the explosion of video possible.

"It's like the internet. Every bit that we lower latency, increase bandwidth on the internet, people do more things," he said. "Video used to be impossible. Now, video is everyday, because the capabilities are there the network can handle it."

The desire for more computing power led OpenAI to launch Stargate, CFO CFO Sarah Friar recently said. A $500 billion project, Stargate is a joint venture between OpenAI, Oracle, and SoftBank. During its unveiling at the White House in January, Altman said the project will allow the US to reach AGI, artificial general intelligence.

"It is voracious right now for GPUs and for compute," Friar told CNBC last week. "The biggest thing we face is being constantly under compute. That's why we launched Stargate. That's why we're doing the bigger builds."

On the product side alone, Weil said there are a number of areas where more GPUs can be plugged in.

"Whether it's we can take them on the product side, and use it to lower latency, or speed up token generation, or launch new products, take a product that's only available to pro users and bring it to plus users or free users, or it just means that we can run more experiments," he said.

At the same time, OpenAI has to balance researchers' requests.

"On the research side, there's basically infinite demand for GPUs within these walls, and that's why we're doing so much to build capacity," Weil said.

• Mark Zuckerberg's Meta is betting on GPUs and compute power to help attract top talent.
• His philanthropic organization, the Chan Zuckerberg Initiative, is doing the same.
• His wife, Priscilla Chan, talked about CZI's recruitment efforts on a recent podcast episode.

Compute power is a big draw for top talent, but not just in the world of AI.

Priscilla Chan, Mark Zuckerberg's wife and the cofounder of the couple's philanthropic organization, the Chan Zuckerberg Initiative, spoke about the appeal of massive GPU clusters for biology researchers during a recent episode of Ashlee Vance's "Core Memory" podcast.

"The other thing researchers really care about is access to GPUs," she said. "You're not going to make the most of someone if you don't actually have the GPUs for them to work from."

Chan said, "We have that at CZI," adding that the organization has roughly 1,000 GPUs in its cluster, with plans to keep growing.

In short, Chan said the pitch is: "Come work with us because we're going to have the computing power to support the research that you want to do."

Another important factor is compensation, which she said is "obviously important," though she added that "we cannot compete with tech companies on this."

CZI has in recent years narrowed its mission to focus on its "next phase" with a "bolder, clearer identity as a science-first philanthropy." The change marks a strategic shift, as the organization previously also supported education and other causes.

"While CZI remains committed to our work in education and our local communities, we recognize that science is where our biggest investments and bets have been and will be made moving forward," Chan, a pediatrician by training, wrote in a memo to staff last year.

Zuckerberg made a similar point about the importance of GPUs in recruiting on a recent episode of The Information's TITV show. Meta is spending billions to build an AI division it calls Superintelligence Labs.

"Historically, when I was recruiting people to different parts of the company, people are like, 'Okay, what's my scope going to be?'" the Meta CEO said. "Here, people say, 'I want the fewest number of people reporting to me and the most GPUs.'"

Meta, of course, has significantly more GPUs than CZI. Zuckerberg has said the company will have 1.3 million GPUs for AI by the end of 2025.

"Having basically the most compute per researcher is definitely a strategic advantage, not just for doing the work but for attracting the best people," he said.

Nvidia is recommending a mitigation for customers of one of its GPU product lines that will degrade performance by up to 10 percent in a bid to protect users from exploits that could let hackers sabotage work projects and possibly cause other compromises.

The move comes in response to an attack a team of academic researchers demonstrated against Nvidia’s RTX A6000, a widely used GPU for high-performance computing that’s available from many cloud services. A vulnerability the researchers discovered opens the GPU to Rowhammer, a class of attack that exploits physical weakness in DRAM chip modules that store data.

Rowhammer allows hackers to change or corrupt data stored in memory by rapidly and repeatedly accessing—or hammering—a physical row of memory cells. By repeatedly hammering carefully chosen rows, the attack induces bit flips in nearby rows, meaning a digital zero is converted to a one or vice versa. Until now, Rowhammer attacks have been demonstrated only against memory chips for CPUs, used for general computing tasks.

Read full article

Comments

On Wednesday, the Trump administration announced plans to rescind and replace a Biden-era rule regulating the export of high-end AI accelerator chips worldwide, Bloomberg and Reuters reported.

A Department of Commerce spokeswoman told Reuters that officials found the previous framework "overly complex, overly bureaucratic, and would stymie American innovation" and pledged to create "a much simpler rule that unleashes American innovation and ensures American AI dominance."

The Biden administration issued the Framework for Artificial Intelligence Diffusion in January during its final week in office. The regulation represented the last salvo of a four-year effort to control global access to so-called "advanced" AI chips (such as GPUs made by Nvidia), with a focus on restricting China's ability to obtain tech that could enhance its military capabilities.

Read full article

Comments