Skip to content
Skip to this exploration
All explorations

Practical self-hosting · intermediate

18 · The GPU works here, but not there

Diagnose host, runtime and application GPU layers separately.

10 min 5 sectionsv1

What you can take away

  • Classify GPU observations without broad driver changes.
  • Draw this tree and add the observation required at each edge

Your trail through this idea

Checking saved progress…

A host utility sees a device. A container cannot use it. A model server then reports a generic startup error. These observations describe different boundaries.

Prerequisite: complete the preceding lesson or bring its worksheet.

Mark sections complete when you choose. Reading alone does not mark them.

Understand the system

NVIDIA’s Container Toolkit installation guide separates host prerequisites from container-runtime configuration. The driver supplies host device support; the toolkit integrates supported device access into the container environment; the workload still needs a compatible runtime and model. Passing an earlier layer is necessary evidence, not proof that every later layer works.

Ian’s historical NVML mismatch is a useful diagnostic case because a library/module disagreement should not be disguised as a model-size problem. But the same error text is not a license to apply a remembered reinstall recipe to a different host. Capture the scope and installed versions first.

Checking saved progress…

Work through the case

Original decision tree: host inspection fails → investigate host compatibility; host succeeds but approved sample workload fails → investigate container integration; sample succeeds but model fails → investigate workload requirements and memory. Each branch narrows the next test. Never use privileged mode merely to make the symptom disappear.

Checking saved progress…

Make something useful

Try this

Original exercise

Draw this tree and add the observation required at each edge. If you have an isolated NVIDIA test machine, compare a host inspection with the toolkit’s documented sample workload for your installed version. Review the sample before running it; downloading an image is a separate action. Keep device identifiers and raw diagnostics private. This lesson requires no reinstall or reboot.

Checking saved progress…

Check your reasoning

A quick check · for your own learning

Host GPU checks pass but the documented sample container fails. Where next?

Choose one answer

A self-study prompt. You can retry; this is not an exam.

Checking saved progress…

Keep the result and follow the sources

Success is a specific failing boundary and a justified next check. A healthy host observation and an unhealthy container observation are compatible facts, not a contradiction to be solved by changing everything.

Sources and scope

  • Primary documentation or archival source, reviewed 14 November 2026. Additional direct links appear beside the claims they support.
  • Editorial basis: Ian's historical self-hosting questions and current repository inspection informed the case selection. Historical requests do not establish current deployment. Private chat records, credentials and host details are excluded.
  • Examples and exercises are original educational synthesis. External sources support the stated technical or historical context, not every hypothetical scenario.
Checking saved progress…

These checks are for reflection and practice. Your answers and reflection notes are not sent anywhere.

Back to all explorations