DOC: AFM-2026-HW-009 | ONE LAPTOP · ONE GPU · 8 GB VRAM | STATUS: LOCAL INFERENCE PROVEN
No cluster. No cloud bill. One laptop: an AMD Ryzen 9 and an RTX 4070 Mobile with 8 GB of VRAM — the number everyone says is too small to run a 7-billion-parameter model alongside speech recognition. It isn't. You just can't hold both at once. Whisper transcribes, hands the card over, and the language model summarises. Sequential, not concurrent. The wall is a scheduling problem, not a wall.
Try to load both models at once — the card runs out of memory. Switch to sequential (the real design) and the same two models fit fine, one handing off to the other. Sizes are approximate and include overhead; the lesson is exact.
Docker GPU passthrough works — --gpus all verified end-to-end by
Vox on 2026-06-05. faster-whisper-medium (Speaches) and qwen2.5:7b
(Ollama) both run on this card. Not simultaneously — a VramCooldownSeconds handoff
sits between the transcribe phase and the summarise phase so the first model fully releases
before the second loads.
The belief that a 7B model won't share an 8 GB card with anything is a hosting reflex, not a hardware fact. Don't architect around the wall — architect the handoff. The same instinct that killed the Graveyard's early generations (self-host everything, concurrently, now) is the one this rig quietly disproves.
Planned upgrade: a second, older GPU (a 1070) dedicated to simple CRUD-grade inference — the small, constant, low-stakes calls — so the 4070 is free for the heavy sequential work. Priority scheduling, not more silicon. The rig grows by getting smarter about what it already has.
The subject runs a seven-billion-parameter language model and a speech recogniser on a laptop graphics card designed for video games. It works because he stopped asking the card to do two things at once — a lesson he has, I note, needed to learn in several other chambers as well. — GLaDOS, Facility Governor
One card, one thing at a time — the One Rule, in silicon. See it stated in The Prime Directive, the survivor it runs in The Graveyard, or the smaller builds in The Misc Wing. Back to the Facility Directory.