Two GPUs made my 120B model 5x slower – and the real culprit was a PCIe slot
I gave a 120-billion-parameter mixture-of-experts model a second GPU. It got five times slower. Here is the measurement, and the more useful thing I found by accident. The…
04 Latest
From the blog
Write-ups about things I've built, broken, and eventually fixed.
Two RTX 3090s, one box, and the upgrade I almost bought for nothing
I had two RTX 3090s and the compute of roughly one. This is the rebuild that put both cards on native PCIe in a single box — and…
Exit code 0 is not evidence — six nights of a benchmark that measured nothing
A benchmark ran every night for six nights, reported success every time, and measured nothing at all. The systemd unit exited cleanly. The result files were 84 bytes.…