Andrew E. Freedman, Tom’s Hardware:
OpenAI CEO Sam Altman has data centers full of memory, but seemingly it’s the 32GB of RAM in your desktop that’s the problem. In an interview over beers with the YouTube channel Cheeky Pint, Altman said that local inference, and, by extension, people having DRAM in computers at home, is “quite terrible.”
When discussing local inference on home computers with open-source models, Altman discussed the assumption that people will be able to run local AI on their own computers, the ones they may already own or want to build. Then he explained why that’s a bad idea (at the 29-minute mark):
“From a global allocation perspective, actually, it’s quite terrible to have everybody have a bunch of the world’s limited supply of DRAM sitting in their own computer and being used 5% of the time,” Altman said. “You want maximum utilization in the cloud if you’re trying to deal with this very supply-constrained environment. And somehow many people’s intuition is the opposite of that.”
Of course a CEO who rents access to cloud-based LLMs is going to crap on local LLMs. Imagine the McDonald’s CEO saying “it’s quite terrible to have everyone have a kitchen being used 5% of the time.” It’s quite obviously an idiotic take, and Altman is being roundly mocked for it by every halfwit with a social media presence.
Oliver Haslam, Tom’s Hardware:
Elon Musk mocks OpenAI CEO Sam Altman’s claim that PCs having RAM is terrible — ‘The people deserve to have their RAM too!’ […]
Mocking the point, Musk shared a meme on X that referenced a movie in which a German WWII officer appears to be hunting for people hiding sticks of DRAM.
(The movie is Inglourious Basterds. It’s a widely used meme. I’m not sure why Haslam doesn’t name it.)
I hate myself for saying this (and leaving aside how that scene proceeds), but it’s a genuinely funny rejoinder from Musk, whom I’ve always found to be exceptionally mirthless.
I can’t believe I’m agreeing with the least liked man in America as he dunks on the least cool man in America.
I feel sick.