Pester 6.0.0-rc pilot: compatibility signal across 15 real-world v5 repos

[nohwnd]

Why

To gauge how feasible the v5→v6 migration is and get early CI feedback before 6.0.0 final, we opened [DO NOT MERGE] draft PRs pinning ~15 real-world Pester-v5 consumer repos to 6.0.0-rc1/rc2. Each is throwaway (suggested not to merge); the goal was to surface breaks against real suites. This is the aggregated signal.

Overall: 🟢 feasible

Nearly every break is a documented change with a mechanical fix. The only undocumented behavior change found (Should -HaveCount counting dictionary entries) was caught and reverted pre-release. No undocumented break survives. One repo (maester) had a CI cancellation that is not a Pester regression (could not reproduce on rc2).

Pilot PRs

Repo	PR	Signal
dsccommunity/SqlServerDsc	#2464	🟢 green on v6 (after fixes)
dsccommunity/ComputerManagementDsc	#467	🟢 green on v6 (after fixes)
dsccommunity/ActiveDirectoryDsc	#751	🟢 unit green (HQRM red = PSSA, not Pester)
dsccommunity/NetworkingDsc	#545	🟢 unit green
dataplat/dbachecks	#1038	🟢 Linux/Package; Windows pending
Badgerati/Pode	#1732	🟡 documented breaks only
dataplat/dbatools	#10405	🟡 documented breaks only
chocolatey/choco	#3917	🟢 / awaiting approval
chocolatey/cChoco	#194	awaiting CI
Windos/BurntToast	#293	awaiting approval
techthoughts2/Catesta	#108	awaiting approval
PowerShell/PSScriptAnalyzer	#2191	awaiting approval (CLA ok)
maester365/maester	#1884	⚪ inconclusive — not Pester (see below)
igoravl/TfsCmdlets	#281	⚪ not Pester (missing fork secrets)
gaelcolas/Sampler	#577	⚪ Azure didn't trigger
PowerShell/PowerShellEditorServices	—	skipped (no live Pester CI path)

Breaking-change tally (consumer test changes needed)

Change	Documented	Hits	Fix
Mock -ParameterFilter no-match throws (no fall-through to real cmd)	yes	41	add a default Mock, or forward: Mock X { & (Get-Command X -CommandType Cmdlet) @PesterBoundParameters }
Should -HaveCount counts dict entries	no → reverted	40	n/a (reverted in #2783)
Assert-MockCalled removed	yes	8	→ Should -Invoke
Assert-VerifiableMock removed	yes	3	→ Should -InvokeVerifiable
Duplicate AfterAll throws	yes	2	merge blocks
-ForEach $null/@() throws	yes	2	-AllowNullOrEmptyForEach / guard

Lessons

• Mock fall-through removal is the dominant break (41 hits). It's the only change that needs case-by-case work; everything else is mechanical search/replace. Worth highlighting prominently in the migration guide.
• The mechanical breaks (Assert-MockCalled, Assert-VerifiableMock, -ForEach @(), duplicate AfterAll) were trivial to fix and behaved exactly as documented — no surprises.
• One undocumented behavior change slipped in: Should -HaveCount started counting a piped dictionary's entries instead of treating it as one object. The pilot caught it (top breakage bucket) and it was reverted before release.
• maester (CC using Measure-Script #1884): rc1 CI jobs were cancelled at ~3h vs <2min on v5, which looked like a perf regression. Reproduced locally on rc2: full suite finishes in minutes, the 408-file PSScriptAnalyzer.Tests.ps1 runs 661 tests in ~6s with no super-linear discovery cost. Not a Pester regression — likely an env hang (Graph/coverage) or a kill of a queued run. Needs a maintainer re-run on rc2.

Bottom line

Migration is feasible. Breaks are documented and mostly mechanical; the single doc-gap was found and reverted. Suggested follow-ups: emphasize Mock fall-through removal in migration docs, and re-run maester on rc2 to confirm.

[kborowinski]

Do you have any timeframe to implement #2793? As for now I can't run my tests suite in parallel because they require the parameters to be passed by the container. Just asking.

[nohwnd]

Added to 6.0.0 milestone, tasking copilot with it. So either in the initial release, or soon after.

[nohwnd]

Progress update (end of day)

Since filing, two repos we patched with the documented fix recipes have re-run on rc2 — both went green, and Sampler's CI finally triggered. Only changed rows shown:

Repo	This morning	Now	What changed
Badgerati/Pode #1732	🟡 documented breaks only	✅ all green	Applied fix (Assert-MockCalled→Should -Invoke ×151, -ForEach guard ×5). Every build + build-preview leg (mac/ubuntu/win) + snyk pass.
dataplat/dbatools #10405	🟡 documented breaks only	✅ effectively green	Applied fix (Assert-MockCalled→Should -Invoke ×128, New-Object forwarding mock ×9). SINGLE×3 / MULTI / COPY / HADR / RESTART all pass; the one remaining red is Install-DbaInstance failing on WinRM "server name cannot be resolved" — remote-host infra, not Pester.
gaelcolas/Sampler #577	⚪ Azure didn't trigger	🟡 docs-break + PR artifact	CI ran. Most reds = our minimal pilot PR adds no CHANGELOG entry (Should have CHANGELOG.md in the diff) — a PR artifact, not Pester. One genuine Pester red: Get-SamplerAbsolutePath Mock -ParameterFilter no-match throws in Set-SamplerTaskVariable.Tests.ps1 — the documented fall-through removal. Build + coverage pass.

Notes

• The documented fix recipes work end-to-end in production CI. Pode and dbatools were the two largest 🟡 consumers; both are green after mechanical migration plus the New-Object/Mock forwarding-default pattern. Nothing in product code changed.
• Sampler is a nice data point for the dominant break: rc2's diagnostic for the fall-through removal is clear and actionable — "No mock for command 'Get-SamplerAbsolutePath' matched the call … there is no default mock to fall back to. Add a default mock (e.g. Mock Get-SamplerAbsolutePath { ... }) or adjust an existing -ParameterFilter."
• maester CC using Measure-Script #1884 is unchanged — still the original rc1 run cancelled at ~3h (same run/job IDs); needs a maintainer re-run on rc2 to confirm the local result (suite finishes in minutes; not a Pester regression).
• Everything else holds: choco green; dbachecks Linux/Package green with the Windows legs starved for a runner (24h queue timeout, infra); BurntToast / Catesta / PSScriptAnalyzer still gated on first-time-contributor approval.

Net: still zero surviving undocumented breaks, and the two biggest documented-break consumers are now green.

[nohwnd]

@kborowinski rc3 is out with the fix for data passing.

[kborowinski]

@kborowinski rc3 is out with the fix for data passing.

I'll test it tomorrow at work. Thanks a lot!

[kborowinski]

@nohwnd I have an edge case that fails when running in parallel. One of my modules that I use for infrastructure testing KBInfrastructureTesting requires specific version of Pester in the manifest:

RequiredModules = @(
    @{ModuleName = 'KBActiveDirectory'; ModuleVersion = '7.0.0'},
    @{ModuleName = 'KBSystemFunctions'; ModuleVersion = '7.0.0'},
    @{ModuleName = 'KBSystemManagement'; ModuleVersion = '7.0.0'},
    @{ModuleName = 'KBServiceManagement'; ModuleVersion = '7.0.0'},
    @{ModuleName = 'KBDomainComputerManagement'; ModuleVersion = '7.0.0'},
    @{ModuleName = 'Pester'; ModuleVersion = '5.7.1'},
    @{ModuleName = 'PowerShellHumanizer'; ModuleVersion = '13.2.1'},
    'SQLServer',
    'Microsoft.PowerShell.ThreadJob'
)

In one of my tests I'm testing module import, and this fails now:

Context 'Module Import' {
    It '[<_.Name>] Module imports successfully' {
        {Import-Module -ModuleInfo $_ -Force -Verbose:$false -ErrorAction Stop} |
            Should -Not -Throw
    }
}

The error:

[Bukem@VOYAGER]≥# gmo KBInfrastructureTesting -ListAvailable | Test-KBModule -Tag Manifest
Running tests from 1 files in parallel.
[-] Manifest Specification In Module: "KBInfrastructureTesting".Module Integrity.Manifest Property.Module Import.[KBInfrastructureTesting] Module imports successfully 2.87s
 Expected no exception to be thrown, but an exception "The required module 'Pester' was not loaded because no valid module file was found in any module directory." was thrown from D:\Users\Bukem\Documents\PowerShell\.projects\Modules\KBDevelopmentTools\Integration\01-Manifest.Tests.ps1:236 char:22
     + …            {Import-Module -ModuleInfo $_ -Force -Verbose:$false -Erro …
     +               ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~.
Tests completed in 4.42s
Tests Passed: 21, Failed: 1, Skipped: 0, Inconclusive: 0, NotRun: 0
Exception: Pester run failed, because 1 test failed

The fix is easy. I could move the Pester module specification from RequiredModules to ExternalModuleDependecies and then check for the required version in the root module. But I just wanted to let you know anyways.

[kborowinski]

I've deleted comment with other edge case because that was an actual bug in my module that Run.Parallel exposed 😅

[nohwnd]

Status snapshot — consolidating rc2 results before moving the fleet to rc4

No CI movement since the last update (the runs below are the same as before), so this is a consolidation rather than new reds/greens. Two housekeeping changes: chocolatey/choco (closed by us) and chocolatey/cChoco (closed by the maintainer) are out — neither runs Pester tests, so there's nothing to validate there. Next step is bumping every still-open PR from rc2 → rc4 to re-validate on the latest prerelease.

Full fleet snapshot (on rc2)

Repo	PR	Signal	Detail
dsccommunity/SqlServerDsc	#2464	✅ fully green	Unit + full integration matrix (SQL2017–2025 × WIN2022/2025) + AppVeyor + HQRM all pass
dsccommunity/ComputerManagementDsc	#467	✅ fully green	Unit + Integration + HQRM all pass
dsccommunity/ActiveDirectoryDsc	#751	✅ unit green	HQRM red = custom PSSA rule (Measure-Keyword), not Pester
dsccommunity/NetworkingDsc	#545	✅ unit green	Unit pass WS2022+WS2025; HQRM red = PSSA/CHANGELOG QA, not Pester
Badgerati/Pode	#1732	✅ all green	After fix (Assert-MockCalled→Should -Invoke, -ForEach guard)
dataplat/dbatools	#10405	✅ Pester green	After fix (Assert-MockCalled→Should -Invoke ×128, New-Object forwarding mock ×9). Lone red = Install-DbaInstance WinRM "server name cannot be resolved" = infra
dataplat/dbachecks	#1038	✅ / infra	Linux + Package green; Windows ×2 = job exceeded 24h awaiting a runner (queue starvation), not a test hang
gaelcolas/Sampler	#577	🟡 docs-break	1 documented Mock -ParameterFilter fall-through (Get-SamplerAbsolutePath); rest = our PR adds no CHANGELOG (Should have CHANGELOG.md in the diff artifact)
maester365/maester	#1884	⚪ inconclusive	rc1 build-validation cancelled at ~3h; not reproduced on rc2 locally (suite finishes in minutes, PSScriptAnalyzer.Tests = 661 tests/~6s). Not a Pester regression; still pinned rc1
igoravl/TfsCmdlets	#281	⚪ not Pester	Build fail = missing fork secrets (TFS creds); still rc1
PowerShell/PSScriptAnalyzer	#2191	⏳ gated	Tests await first-time-contributor approval (CLA signed)
Windos/BurntToast	#293	⏳ gated	Tests await first-time-contributor approval
techthoughts2/Catesta	#108	⏳ gated	Tests await first-time-contributor approval
chocolatey/choco	#3917	⚪ closed	Closed; no Pester test CI
chocolatey/cChoco	#194	⚪ closed	Closed by maintainer; no Pester test CI

Where the signal stands

• Still zero surviving undocumented breaks. Every red is either a documented change (all with known fix recipes), a non-Pester toolchain nit (PSSA/HQRM, CHANGELOG artifact), or infra (WinRM, runner starvation, missing secrets).
• The two biggest documented-break consumers (Pode, dbatools) are green after mechanical fixes.
• Next: re-validating the open PRs on 6.0.0-rc4; will report any deltas the fresh runs surface.

[nohwnd]

rc4 re-audit — correcting a dbachecks misclassification, and re-checking every "green"

Two things prompted this. First, I moved the fleet to 6.0.0-rc4 and re-ran everything. Second — and more importantly — I had marked dataplat/dbachecks green, and that was wrong. Its build check was green while the tests underneath were actually failing. That made me distrust every other "green", so I re-audited the whole fleet by reading the actual per-file / per-container Pester results, not the check badge.

What went wrong with dbachecks (and why it slipped past me)

dbachecks' Linux check was green, but two containers were failing: tests/Unit.Tests.ps1 and tests/Project.Tests.ps1. Both fail with the documented breaking change:

Value cannot be null or empty array. If this is expected, use -AllowNullOrEmptyForEach on this It, or set the Run.FailOnNullOrEmptyForEach configuration option to $false … (Parameter 'ForEach')

• Unit.Tests.ps1 — nested -ForEach over parsed check tags/describes that can legitimately be empty for some groups.
• Project.Tests.ps1 — Context -ForEach $scripts, where $scripts comes from a git log --grep/git diff that returns nothing on a fresh PR checkout → empty -ForEach → throws.

The reason it showed green: a container/discovery-level failure reports Result = Failed with FailedCount = 0 (zero individual test failures), and the Sampler build gate asserts on failed test count, not failed container count. So the container throw never failed the build. I'd trusted the badge. Lesson logged: for every repo I now read FailedContainersCount and the per-file Result rows, not just pass/fail.

Reproduced locally on rc4; classified 🟡 documented and fixable with -AllowNullOrEmptyForEach / Run.FailOnNullOrEmptyForEach = $false.

Severity legend

• 🟢 green — Pester tests genuinely pass.
• 🟡 documented — a real Pester 6 behaviour change, but it's in the rc release notes and has a known fix recipe.
• 🟡 not-Pester — fails only on non-Pester artifacts: missing CHANGELOG.md in the PR diff, PSScriptAnalyzer/HQRM quality rules, WinRM/infra, missing fork secrets. Would fail (or does) regardless of Pester version.
• ⚪ no-signal — tests never ran (fork-PR runs await maintainer approval, or a branch filter excludes the PR branch).
• 🔴 undocumented — an unlisted regression. Count across the fleet: 0.

Corrected fleet snapshot (rc4)

Repo	PR	Signal	Detail (verified from actual test output)
dsccommunity/SqlServerDsc	#2464	🟢 green	Unit 5923/0, HQRM 17803/0, full integration matrix (SQL2017–2025 × WIN2022/2025) + AppVeyor all pass. Unit log verified — no masked container. Strongest result in the fleet.
dsccommunity/ComputerManagementDsc	#467	🟢 green	Unit 713/0, HQRM 4938/0, Integration pass. No masked container.
Badgerati/Pode	#1732	🟢 green	5322/0 + 1944/0 + 85/0 across suites on macOS/ubuntu/windows. No container failures.
dataplat/dbatools	#10405	🟡 not-Pester	7/8 AppVeyor jobs green; lone fail = Install-DbaInstance WinRM "server name cannot be resolved" across all SQL versions = infra (no remote host on a fork). AppVeyor reports it honestly.
dsccommunity/ActiveDirectoryDsc	#751	🟡 not-Pester	Unit 1292/0 (verified clean). HQRM red = 3× "custom DSC Resource Kit PSSA rules" (Measure-Keyword, Measure-ParameterBlockParameterAttribute) — code-quality, not Pester.
dsccommunity/NetworkingDsc	#545	🟡 not-Pester	Unit 1331/0 on WS2022+WS2025 (verified clean). HQRM red = "CHANGELOG.md in the diff" artifact + 2 PSSA rules. Not Pester.
dataplat/dbachecks	#1038	🟡 documented	Correction (was ✅/infra). Unit.Tests.ps1 + Project.Tests.ps1 fail on the documented empty--ForEach throw; masked green by the build gate (see above). Fix = -AllowNullOrEmptyForEach / Run.FailOnNullOrEmptyForEach=$false. Bumping to rc4 + applying the opt-out.
gaelcolas/Sampler	#577	🟡 documented	Documented Mock -ParameterFilter fall-through (Set-SamplerTaskVariable→Get-SamplerAbsolutePath), confirmed at rc2. On rc4 the fork branch doesn't auto-trigger its Azure pipeline, so no fresh run.
igoravl/TfsCmdlets	#281	🟡 not-Pester	Build red = 139 tests / 33 containers all throw "Missing credentials … TFSCMDLETS_ACCESS_TOKEN / _COLLECTION_URL" from _TestSetup.ps1 BeforeAll. Missing fork secrets, not Pester. (Pester 6 correctly surfaces the BeforeAll throw as failed.)
maester365/maester	#1884	⚪ no-signal	Workflows action_required — fork PR awaiting approval. (Earlier rc1 ~3h cancel was not reproduced locally on rc2: full suite finishes in minutes, PSScriptAnalyzer.Tests = 661 tests/~6s — not a Pester perf regression.)
PowerShell/PSScriptAnalyzer	#2191	⚪ no-signal	Workflow action_required (jobs: []) — fork PR awaiting approval. CLA signed.
Windos/BurntToast	#293	⚪ no-signal	Workflow action_required (jobs: []) — fork PR awaiting approval.
techthoughts2/Catesta	#108	⚪ no-signal	Workflows action_required — fork PR awaiting approval.
chocolatey/choco	#3917	⛔ excluded	Closed; no Pester test CI.
chocolatey/cChoco	#194	⛔ excluded	Closed by maintainer; no Pester test CI.

Where the signal stands after the re-audit

• Still zero undocumented breaks. Every red resolves to a documented change (dbachecks -ForEach, Sampler mock fall-through — both with fix recipes) or a non-Pester artifact (HQRM/PSSA, missing CHANGELOG, WinRM infra, missing fork secrets).
• Corrected count: 🟢 3 · 🟡 6 (2 documented, 4 not-Pester) · ⚪ 4 no-signal · ⛔ 2 closed.
• The one process fix on my side: never trust a green Sampler-style build gate — read FailedContainersCount and the per-file Result, because container/discovery failures (like an empty -ForEach) don't move FailedCount and slip through as green. That's what bit dbachecks.

[nohwnd]

Pilot wrap-up — Pester 6.0.0 is released 🎉

6.0.0 is live on the PowerShell Gallery (https://github.com/pester/Pester/releases/tag/6.0.0), so I've closed out the compatibility pilot. Every pilot PR has now been either updated to the final 6.0.0 or closed with a thank-you note.

Updated to 6.0.0 — 6 PRs (green / almost-green)

Re-pinned from the release candidate to the released 6.0.0, dropped the [DO NOT MERGE] title, and reworded the body. The Pester-relevant results are green; the tests changes needed for v6 are already in each PR.

Repo	PR	Result on 6.0.0
dsccommunity/SqlServerDsc	#2464	🟢 unit + HQRM + integration matrix green
dsccommunity/ComputerManagementDsc	#467	🟢 unit + HQRM + integration green
Badgerati/Pode	#1732	🟢 full suite green (macOS/Ubuntu/Windows)
dataplat/dbatools	#10405	🟢 Pester green (AppVeyor + S3 + build); lone red = Install-DbaInstance WinRM infra
dsccommunity/ActiveDirectoryDsc	#751	🟡 unit green; HQRM red = custom PSSA rules (not Pester)
dsccommunity/NetworkingDsc	#545	🟡 unit green; HQRM red = CHANGELOG/PSSA gates (not Pester)

Closed — 7 PRs

Each closed with a note explaining the reason (and, where relevant, the documented fix recipe + link to https://pester.dev/docs/migrations/v5-to-v6).

Repo	PR	Why closed
dataplat/dbachecks	#1038	🟡 documented empty--ForEach throw — fix: -AllowNullOrEmptyForEach / Run.FailOnNullOrEmptyForEach=$false
gaelcolas/Sampler	#577	🟡 documented Mock -ParameterFilter fall-through — fix: add a default Mock
igoravl/TfsCmdlets	#281	⚪ no signal — all tests error on missing fork secrets (infra, not Pester)
maester365/maester	#1884	⚪ no signal — fork workflows action_required (never ran)
PowerShell/PSScriptAnalyzer	#2191	⚪ no signal — CI Tests action_required
Windos/BurntToast	#293	⚪ no signal — BurntToast CI action_required
techthoughts2/Catesta	#108	⚪ no signal — Catesta-* action_required

Already closed — 2 PRs

chocolatey/choco #3917 and chocolatey/cChoco #194 — no Pester test CI to validate.

Net

Migration held up in production CI: every surviving red is either a documented v6 change with a known fix recipe, or non-Pester (infra / a repo's own quality gate / a fork run awaiting approval). Zero undocumented breaks. Huge thanks to every maintainer who let us pilot the release candidates against their suite — the real-world signal is a big part of why 6.0.0 shipped in good shape.

Pilot complete. 🎉

— Jakub & Copilot

[nohwnd]

Patched where we could, so repos can update directly to 6.0.0.