Skip to content

Harden the vulnerability refresh pipeline: bounded parallelism, Trivy timeout default, stale Running runs #170

Description

@LarsLaskowski

Builds on the vulnerability rework in PR #164. Read CLAUDE.md and .github/instructions/csharp.instructions.md first — the code-style rules there (regions, XML docs on everything, == false instead of !, aligned wrapped parameters, MSTest assertion messages, reihitsu-format ./ before building) are binding.

Background

src/DockerUpdateGuard/Images/VulnerabilityEnrichmentService.cs processes every relevant image version strictly sequentially. A Trivy scan of a large image can take minutes; with many images one refresh cycle can exceed the configured interval (Scanning:VulnerabilityRefreshIntervalMinutes, default 180). Additionally, the default Vulnerabilities:RequestTimeoutSeconds of 30 (see Configuration/VulnerabilityOptions.cs and appsettings.json) is too tight for first-time Trivy scans of large images. Finally, if the process crashes mid-run, the ScanRun row stays in status Running forever (PR #164 fixed the normal completion path, but not crashes).

Task — three independent sub-items

1. Bounded parallelism for provider calls

  • Add int MaxParallelScans { get; set; } = 1; to Configuration/VulnerabilityOptions.cs (XML-documented). Validate 1..8 in Configuration/DockerUpdateGuardOptionsValidator.ValidateVulnerabilityOptions using the existing ValidateRange helper, and add the key with value 1 to appsettings.json under Vulnerabilities. Default 1 keeps current behavior — parallelism is opt-in.
  • Critical constraint: DockerUpdateGuardDbContext is not thread-safe. Only the provider call (IVulnerabilityProvider.GetVulnerabilitiesAsync) may run in parallel; all DbContext work (loading active findings, upserting, SaveChangesAsync) must stay on one logical flow. Recommended structure: process images in batches of MaxParallelScans — for each batch, start all provider calls (Task.WhenAll over GetVulnerabilitiesAsync), then apply the results sequentially through the existing UpsertVulnerabilityFindingsAsync/status logic and the per-image SaveChangesAsync. Do not introduce parallel DbContext access.
  • The per-image status aggregation (finalStatus, statusMessages) must produce the same results as sequential processing (same Partial/Failed semantics, order of messages may differ; Distinct() is already applied).
  • Update existing tests and add one: with MaxParallelScans = 2 and 3 images, a substitute provider that tracks concurrent invocations (see Tests/Helper/ConcurrencyTrackingHttpMessageHandler.cs for the counting pattern) must observe at most 2 concurrent calls, and all findings must be persisted correctly.

2. Raise the Trivy/scan timeout default

  • Change VulnerabilityOptions.RequestTimeoutSeconds default from 30 to 120 and update appsettings.json accordingly. The validator already allows up to 300 — unchanged. Update Tests that assert the --timeout 30s Trivy argument (TrivyVulnerabilityProviderTests builds providers with explicit RequestTimeoutSeconds = 30, so most tests are unaffected; verify).
  • Mention the new default in DOCKER.md/README.md wherever RequestTimeoutSeconds is documented (search both files for the key).

3. Mark stale Running vulnerability runs as failed

  • On startup and on each cleanup cycle, Images/ScanCleanupBackgroundService.cs should mark ScanRun rows that are still Running but demonstrably dead as Failed: condition Status == ScanRunStatus.Running && StartedAtUtc < utcNow - staleThreshold, with staleThreshold derived as 2 × VulnerabilityRefreshIntervalMinutes for vulnerability runs — but implement it generically for all run types using each type's interval option where available, or a single conservative threshold of 24 hours if that is simpler; state your choice in the PR. Set CompletedAtUtc = utcNow and ErrorMessage = "Marked as failed by cleanup: the run never completed (process restart or crash)".
  • Log the number of repaired runs via a new LoggerMessage in the cleanup service's logging class (follow the existing EventId scheme there).
  • Test: seed a Running run older than the threshold and a fresh Running run; after the cleanup executes (see Tests/Helper/TestScanCleanupBackgroundService.cs for how the cleanup is driven in tests), only the stale one is Failed with message and completion timestamp.

Acceptance criteria

  • MaxParallelScans option exists, is validated (1..8), defaults to 1, and demonstrably bounds provider-call concurrency without parallel DbContext access.
  • Timeout default is 120 s in code, appsettings.json, and docs.
  • Stale Running runs are repaired by cleanup; fresh runs untouched; behavior covered by tests.
  • Follow {Class}{Scenario}{ExpectedResult} test naming with assertion messages; reihitsu-format ./ clean; dotnet build DockerUpdateGuard.slnx -c Release clean; all tests pass.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions