Matthew Gribben, on the r/macpro sub-Reddit, describes his experience installing Ubuntu and a local LLM on a 2013 “trash can” Mac Pro with 64GB RAM and two D700 GPUs with 12GB total memory:

[…] I discovered that while its old southern islands based GPUs weren’t supported in ROCm, they were now supported under Vulkan — thanks to new drivers and a new Linux kernel.

That means it can run basically any model that llama cpp can throw at its 12gb of VRAM. Not a lot but it’s enough for some small models.

Gribben describes the results as “[n]ot exactly lightening fast but totally usable, especially for planning tasks where you can just set it and forget it.”

In his installation guide he writes:

The 2013 Mac Pro is still a strange machine: thermally dense, beautifully overbuilt, and awkwardly dependent on two workstation GPUs that most modern ML stacks have forgotten.

He also notes:

The point of the D700 is not that it wins benchmarks. It is that a sunk-cost workstation can still be a reliable local inference endpoint when the model is sized correctly and the Vulkan path is configured well.

There’s life still in the thermally challenged Mac Pro. If you happen to have one sitting idle, this seems like a fun project. (You can find the D700 configuration on eBay for around $300, though I wouldn’t recommend buying one just to run local LLMs—even at 2×–3× the price, an $800 Mac mini is still a better deal.)