Practical self-hosting · intermediate
18 · The GPU works here, but not there
Diagnose host, runtime and application GPU layers separately.
What you can take away
- Classify GPU observations without broad driver changes.
- Draw this tree and add the observation required at each edge
Your trail through this idea
Checking saved progress…
A host utility sees a device. A container cannot use it. A model server then reports a generic startup error. These observations describe different boundaries.
Prerequisite: complete the preceding lesson or bring its worksheet.
Understand the system
NVIDIA’s Container Toolkit installation guide separates host prerequisites from container-runtime configuration. The driver supplies host device support; the toolkit integrates supported device access into the container environment; the workload still needs a compatible runtime and model. Passing an earlier layer is necessary evidence, not proof that every later layer works.
Ian’s historical NVML mismatch is a useful diagnostic case because a library/module disagreement should not be disguised as a model-size problem. But the same error text is not a license to apply a remembered reinstall recipe to a different host. Capture the scope and installed versions first.
Work through the case
Original decision tree: host inspection fails → investigate host compatibility; host succeeds but approved sample workload fails → investigate container integration; sample succeeds but model fails → investigate workload requirements and memory. Each branch narrows the next test. Never use privileged mode merely to make the symptom disappear.
Make something useful
Try this
Original exercise
Draw this tree and add the observation required at each edge. If you have an isolated NVIDIA test machine, compare a host inspection with the toolkit’s documented sample workload for your installed version. Review the sample before running it; downloading an image is a separate action. Keep device identifiers and raw diagnostics private. This lesson requires no reinstall or reboot.
Check your reasoning
A quick check · for your own learning
Host GPU checks pass but the documented sample container fails. Where next?
A self-study prompt. You can retry; this is not an exam.
Keep the result and follow the sources
Success is a specific failing boundary and a justified next check. A healthy host observation and an unhealthy container observation are compatible facts, not a contradiction to be solved by changing everything.
Sources and scope
- Primary documentation or archival source, reviewed 14 November 2026. Additional direct links appear beside the claims they support.
- Editorial basis: Ian's historical self-hosting questions and current repository inspection informed the case selection. Historical requests do not establish current deployment. Private chat records, credentials and host details are excluded.
- Examples and exercises are original educational synthesis. External sources support the stated technical or historical context, not every hypothetical scenario.