fix(validator): treat "**" as recursive anywhere in a flow pattern#116
fix(validator): treat "**" as recursive anywhere in a flow pattern#116jsonITP wants to merge 2 commits into
Conversation
matchPattern only treated "**" as recursive when the pattern was literally "**" or started with "**/". Patterns with a directory prefix — e.g. "tests/**/*.yaml" — fell through to filepath.Glob, which does not understand "**" and matches it like a single "*", so each extra "**" matched one more directory level instead of any depth. Route any pattern containing "**" through collectRecursive, and split the pattern on the first "**" into an optional prefix (walked from) and an optional suffix (matched per-segment against each file's trailing path).
|
Thanks @jsonITP — great catch, and a really well-written PR. The silent depth-limited matching was a nasty one; flows quietly not running is the worst kind of bug for a test runner, and the repro table made it easy to verify. The zero-directory case ( I've verified the fix and the existing suite passes. We'll merge this. Two small edge cases we'll handle in a follow-up (or feel free to push to this branch if you'd like):
And yes — we'd gladly take the extra tests you offered; the two cases above would be great additions to that list. |
Follow-up to the review on devicelab-dev#116: 1. A pattern whose prefix directory doesn't exist (e.g. "subflows/**/*.yaml" with no subflows/) now returns an empty match instead of an lstat error from filepath.Walk. 2. A shell wildcard in the prefix (e.g. "flows-*/**/*.yaml") is expanded through filepath.Glob before walking, so each matching directory is traversed. Refactors collectRecursive to split into expandPrefixRoots (resolves the prefix into concrete walk roots) and walkRecursive (per-root file walk). Results are deduped so an ambiguous glob can't return the same file twice. Adds five tests covering the shapes discussed on the PR: - tests/**/*.yaml at four depths - flows/** as a prefix-only pattern - a/**/c/*.yaml with a multi-segment suffix - subflows/**/*.yaml with a missing prefix dir (silent no-match) - flows-*/**/*.yaml with a wildcard prefix
|
Thanks for the review @omnarayan, pushed a follow-up commit that handles both edge cases plus the promised tests:
Refactor: split Added 5 tests in
Ready for another look whenever. |
Codecov Report❌ Patch coverage is
📢 Thoughts on this report? Let us know! |
Problem
matchPatterninpkg/validator/validator.goonly treats**as recursive when the pattern is literally**or starts with**/. Anything else (tests/**/*.yaml,auth/**,a/b/**/c.yaml) falls through tofilepath.Glob, which does not understand**and matches it like a single*. As a result each additional**matches exactly one more directory level, and flows nested deeper than the number of**tokens are silently skipped.Repro against
mainon a fixture with files at depths 2, 3 and 4 undertests/:config.yamltests/**/*.yamltests/**/**/*.yamltests/**/**/**/*.yamlWith this fix,
tests/**/*.yamlon its own picks up files at every depth (53 in the same fixture), matching how the same glob behaves in bashglobstar,doublestar,minimatchand every other tool authors have muscle-memory for.Fix
matchPattern: route any pattern containing**throughcollectRecursive, not just patterns that start with**or**/.collectRecursive: split the pattern on the first**into an optional prefix and an optional suffix. The prefix (before**) is joined ontodirto form the walk root. The suffix (after**/) is matched per-segment against the trailing path segments of each file, usingfilepath.Matchon each segment. This makes patterns likea/b/**/c/*.yamlbehave as expected.No new dependencies.
Backward compatibility
Patterns without
**are unchanged (they still go throughfilepath.Glob). The two existing recursive shapes (**and**/<name>) still resolve to the same set of files. Verified by running the existingpkg/validatortest suite, which passes as-is:TestValidate_RecursivePattern(pattern**)TestValidate_RecursivePatternWithSuffix(pattern**/login*.yaml)TestValidate_ConfigYamlFlowPatterns,TestValidate_NestedSubdirPattern,TestValidate_PatternMatchesDirectory(no**, unchanged code path)Test coverage
Follow-up commit adds five tests plus the two prefix edge cases the reviewer flagged (missing prefix dir, wildcard in prefix). See the follow-up comment for the shape list.