Skip to content

Expand nvidia-smi symlink resolution to additional host paths#2611

Open
vishalanarase wants to merge 1 commit into
NVIDIA:mainfrom
vishalanarase:chore/expand-host-nvidia-smi-search-paths
Open

Expand nvidia-smi symlink resolution to additional host paths#2611
vishalanarase wants to merge 1 commit into
NVIDIA:mainfrom
vishalanarase:chore/expand-host-nvidia-smi-search-paths

Conversation

@vishalanarase

@vishalanarase vishalanarase commented Jul 8, 2026

Copy link
Copy Markdown

Description

Fixes #2506.

PR #2464 updated host nvidia-smi validation to resolve symlinks under /usr/bin/nvidia-smi using pathrs.OpenInRoot. Review feedback noted that other common locations are also used in the wild and should be searched in a follow-up:

  • /usr/sbin/nvidia-smi — Talos and similar distros
  • /usr/lib/wsl/lib/nvidia-smi — WSL

This change expands resolveHostNvidiaSMI to search multiple host paths (aligned with getNvidiaSMIPath() in find.go and the DRA driver prestart script):

/opt/bin/nvidia-smi
/usr/bin/nvidia-smi
/usr/sbin/nvidia-smi
/bin/nvidia-smi
/sbin/nvidia-smi
/usr/lib/wsl/lib/nvidia-smi

Each path is resolved via pathrs.OpenInRoot, so symlink handling remains scoped to the mounted host root. The function now returns the resolved path, and validateHostDriver / runValidation use that path in chroot instead of relying on PATH lookup.

Checklist

  • No secrets, sensitive information, or unrelated changes
  • Lint checks passing (make lint)
  • Generated assets in-sync (make validate-generated-assets)
  • Go mod artifacts in-sync (make validate-modules)
  • Test cases are added for new code paths

Testing

Unit tests added/extended in cmd/nvidia-validator/main_test.go (TestResolveHostNvidiaSMI), covering:

  • nvidia-smi present at each search path (/opt/bin, /bin, /usr/bin, /usr/sbin, WSL)
  • Symlink resolution for /usr/bin, /usr/sbin, and WSL paths
  • Search order preference when multiple paths exist
  • Error when nvidia-smi is missing or resolves outside the host root

go test ./cmd/nvidia-validator -run TestResolveHostNvidiaSMI -v

Note: pathrs-lite is Linux-only (//go:build linux), so tests must be run on Linux or via Docker:

❯ docker run --rm \
  -v "$(pwd)":/src -w /src \
  golang:1.26.3 \
  go test ./cmd/nvidia-validator -run TestResolveHostNvidiaSMI -v
=== RUN   TestResolveHostNvidiaSMI
=== RUN   TestResolveHostNvidiaSMI/nvidia-smi_exists_in_/usr/bin
=== RUN   TestResolveHostNvidiaSMI/nvidia-smi_exists_through_absolute_/usr/bin_symlink
=== RUN   TestResolveHostNvidiaSMI/nvidia-smi_exists_through_relative_/usr/bin_symlink
=== RUN   TestResolveHostNvidiaSMI/nvidia-smi_exists_in_/opt/bin
=== RUN   TestResolveHostNvidiaSMI/nvidia-smi_exists_in_/bin
=== RUN   TestResolveHostNvidiaSMI/nvidia-smi_exists_in_/usr/sbin
=== RUN   TestResolveHostNvidiaSMI/nvidia-smi_exists_through_absolute_/usr/sbin_symlink
=== RUN   TestResolveHostNvidiaSMI/nvidia-smi_exists_in_WSL_path
=== RUN   TestResolveHostNvidiaSMI/nvidia-smi_exists_through_absolute_WSL_path_symlink
=== RUN   TestResolveHostNvidiaSMI/earlier_search_path_is_preferred_when_multiple_exist
=== RUN   TestResolveHostNvidiaSMI/parent_dir_is_symlink_to_path_not_within_root
=== RUN   TestResolveHostNvidiaSMI/nvidia-smi_does_not_exist
--- PASS: TestResolveHostNvidiaSMI (0.00s)
    --- PASS: TestResolveHostNvidiaSMI/nvidia-smi_exists_in_/usr/bin (0.00s)
    --- PASS: TestResolveHostNvidiaSMI/nvidia-smi_exists_through_absolute_/usr/bin_symlink (0.00s)
    --- PASS: TestResolveHostNvidiaSMI/nvidia-smi_exists_through_relative_/usr/bin_symlink (0.00s)
    --- PASS: TestResolveHostNvidiaSMI/nvidia-smi_exists_in_/opt/bin (0.00s)
    --- PASS: TestResolveHostNvidiaSMI/nvidia-smi_exists_in_/bin (0.00s)
    --- PASS: TestResolveHostNvidiaSMI/nvidia-smi_exists_in_/usr/sbin (0.00s)
    --- PASS: TestResolveHostNvidiaSMI/nvidia-smi_exists_through_absolute_/usr/sbin_symlink (0.00s)
    --- PASS: TestResolveHostNvidiaSMI/nvidia-smi_exists_in_WSL_path (0.00s)
    --- PASS: TestResolveHostNvidiaSMI/nvidia-smi_exists_through_absolute_WSL_path_symlink (0.00s)
    --- PASS: TestResolveHostNvidiaSMI/earlier_search_path_is_preferred_when_multiple_exist (0.00s)
    --- PASS: TestResolveHostNvidiaSMI/parent_dir_is_symlink_to_path_not_within_root (0.00s)
    --- PASS: TestResolveHostNvidiaSMI/nvidia-smi_does_not_exist (0.00s)
PASS
ok      github.com/NVIDIA/gpu-operator/cmd/nvidia-validator     0.012s

Signed-off-by: Vishal Anarase <iamvishalanarase@gmail.com>
@copy-pr-bot

copy-pr-bot Bot commented Jul 8, 2026

Copy link
Copy Markdown

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Update Symlink evaluation for NVIDIA-SMI

1 participant