Skip to content

fix(sandbox/bootstrap): GPU Landlock baseline paths and CDI spec missing diagnosis#710

Open
pimlock wants to merge 6 commits intomainfrom
fix/cdi-gpu-tweaks
Open

fix(sandbox/bootstrap): GPU Landlock baseline paths and CDI spec missing diagnosis#710
pimlock wants to merge 6 commits intomainfrom
fix/cdi-gpu-tweaks

Conversation

@pimlock
Copy link
Copy Markdown
Collaborator

@pimlock pimlock commented Mar 31, 2026

Summary

Fixes GPU sandbox failures and general Landlock gaps exposed after the per-path enforcement fix (#677) landed.

Related Issue

Related to #398, follow-on to #503 and #677

Background

When the CDI injection PR (#503) was tested, it appeared to work because of an unrelated Landlock bug (#677, fixed Mar 30): a missing path (/app) in the baseline caused PathFd::new() to abort the entire Landlock ruleset under BestEffort mode, silently disabling all filesystem restrictions. Once #677 fixed that, Landlock was properly enforced — exposing multiple missing paths.

Changes

Proxy baseline (PROXY_BASELINE_READ_ONLY)

Added /proc and /dev/urandom — virtually every process needs these (already in restrictive_default_policy() but missing from the enrichment baseline).

GPU baseline (GPU_BASELINE_READ_WRITE_FIXED)

  • Device nodes: /dev/nvidiactl, /dev/nvidia-uvm, /dev/nvidia-uvm-tools, /dev/nvidia-modeset + dynamically enumerated /dev/nvidiaX — NVML/CUDA open these with O_RDWR.
  • /proc (read-write): CUDA writes to /proc/<pid>/task/<tid>/comm during cuInit() to set thread names. Without write access, cuInit() returns error 304 (cudaErrorOperatingSystem). Must use /proc rather than /proc/self/task because Landlock rules bind to inodes — /proc/self/task in the pre_exec hook resolves to the shell's PID, but child processes have different procfs inodes.

GPU baseline (GPU_BASELINE_READ_ONLY_FIXED)

  • /run/nvidia-persistenced: NVML tries to connect to the persistenced socket; if blocked by Landlock it gets EACCES (not ECONNREFUSED) and aborts. Only read/traversal access needed.

CDI specs missing diagnosis

  • Added failure pattern to detect when Docker has CDI configured but no spec files exist, surfacing recovery steps.

Testing

  • mise run pre-commit passes
  • nvidia-smi verified working in GPU sandbox
  • PyTorch CUDA compute (1024x1024 matmul) verified working from both SSH and sandbox connect interactive shell
  • UUID generation (/dev/urandom) verified working
  • E2E tests (GPU not available in CI)

Checklist

  • Follows Conventional Commits
  • No AI attribution in commits

pimlock added 2 commits March 31, 2026 11:38
…k baseline

Landlock READ_FILE/WRITE_FILE restricts open(2) on character device files
even when DAC permissions would otherwise allow it. GPU sandboxes need
/dev/nvidiactl, /dev/nvidia-uvm, /dev/nvidia-uvm-tools, /dev/nvidia-modeset,
and per-GPU /dev/nvidiaX nodes in the policy to allow NVML initialization.

Additionally, CDI bind-mounts /run/nvidia-persistenced/socket into the
container. NVML tries to connect to this socket at init time; if the
directory is not in the landlock policy, it receives EACCES (not
ECONNREFUSED), which causes NVML to abort with NVML_ERROR_INSUFFICIENT_PERMISSIONS
even though nvidia-persistenced is optional.

Both classes of paths are auto-added to the baseline when /dev/nvidiactl is
present. Per-GPU device nodes are enumerated at runtime to handle multi-GPU
configurations.
…eline

NVML only needs to traverse the directory and connect to the Unix socket —
it does not create or modify files there.  Read-only (traversal) access is
sufficient; read-write was unnecessarily broad.
@pimlock pimlock self-assigned this Mar 31, 2026
When Docker has CDI configured but no CDI spec files exist on the host,
container startup fails with an opaque error. Add a failure pattern to
detect this and surface actionable recovery steps pointing to
`nvidia-ctk cdi generate`.
@pimlock pimlock changed the title fix(sandbox): add GPU device and persistenced paths to Landlock baseline fix(sandbox/bootstrap): GPU Landlock baseline paths and CDI spec missing diagnosis Mar 31, 2026
pimlock added 2 commits March 31, 2026 12:59
…U and proxy mode

After the Landlock per-path fix (#677), missing paths no longer silently
disable the entire ruleset.  This exposed two gaps:

Proxy baseline:
- /proc and /dev/urandom are needed by virtually every process (already
  in restrictive_default_policy but missing from the enrichment baseline).

GPU baseline:
- /proc must be read-write because CUDA writes to
  /proc/<pid>/task/<tid>/comm during cuInit() to set thread names.
  Using /proc (not /proc/self/task) because Landlock rules bind to inodes
  and child processes have different procfs inodes than the parent shell.

Also skips chown for virtual filesystem paths (/proc, /sys) in
prepare_filesystem since they are kernel-managed and may contain
symlinks (e.g. /proc/self) that trigger the symlink safety check.
@pimlock pimlock marked this pull request as ready for review April 1, 2026 01:56
@pimlock pimlock requested a review from a team as a code owner April 1, 2026 01:56
…are_filesystem

chown on /proc and /sys succeeds silently (kernel ignores it for virtual
filesystems), so the special-case skip added in the previous commit is
not needed.
@pimlock pimlock requested a review from johntmyers April 1, 2026 07:22
Comment on lines +893 to +894
"/proc",
"/dev/urandom",
Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/// user working directory and temporary files.
const PROXY_BASELINE_READ_WRITE: &[&str] = &["/sandbox", "/tmp"];

/// Fixed read-only paths required when a GPU is present.
Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This whole section is similar to the baseline stuff, but activated when the GPU passthrough is on.

This could benefit from the mechanism we will have for providers bringing in their policies, though this is different as it applies sandbox wide and is not limited by binaries.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant