Trigger conforma e2e tests on cli pull requests

[cuipinghuo]

Add a PipelineRun that triggers the conforma/e2e-tests pipeline when a PR targets the main branch, using the Tekton git resolver.

Pin the pipeline ref to a specific commit SHA and configure Renovate
to auto-update it by adding a customManagers regex for git-refs
annotations in Tekton YAML files.

Ref: https://redhat.atlassian.net/browse/KONFLUX-14184

[coderabbitai]

Warning

Review limit reached

@cuipinghuo, we couldn't start this review because you've reached your PR review rate limit.

More reviews will be available in 8 minutes and 19 seconds. Learn how PR review limits work.

Your organization has used up its prepaid credits, and credit purchases are no longer available. Enable the review add-on in the billing tab to keep reviews running — you're only billed for reviews past your plan's rate limits ($0.25/file).

⌛ How to resolve this issue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based credits.

🚦 How do rate limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan refill rate.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, the refill rate gradually slows as usage increases. The highest same-day bursts are limited more strictly.

Please see our Fair Usage Limits Policy for further information.

ℹ️ Review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Enterprise

Run ID: 699bb859-78e3-418e-86c7-c909fa1fb5e2

📥 Commits

Reviewing files that changed from the base of the PR and between cafa8ce and 89b63ca.

📒 Files selected for processing (2)

• .tekton/cli-e2e-pull-request.yaml
• renovate.json

📝 Walkthrough

Walkthrough

This PR introduces a new Tekton PipelineRun manifest that automatically triggers E2E tests on pull requests targeting the main branch. The manifest configures pipeline parameters for the git repository, container registry, and AWS credentials, and resolves the pipeline definition from the external conforma/e2e-tests repository. A Renovate custom manager configuration is added to keep the git references in that manifest up-to-date.

Changes

E2E Test Automation Setup

Layer / File(s)	Summary
PipelineRun manifest for E2E test automation .tekton/cli-e2e-pull-request.yaml	New Tekton PipelineRun named cli-e2e-on-pull-request in the rhtap-contract-tenant namespace. Metadata declares AppStudio/Tekton labels and Pipelines-as-Code annotations with a CEL trigger condition to run only on pull requests targeting main, cancel in-progress runs, and retain a maximum of 2 runs. The spec wires parameters for the conforma/e2e-tests git repository at a fixed revision, OCI container registry (quay.io/conforma/e2e-tests), and AWS credentials/deprovisioning secrets. A pipelineRef resolver points to .tekton/pipelines/conforma-e2e/pipeline.yaml in the conforma/e2e-tests repository, and taskRunTemplate assigns konflux-integration-runner as the service account for task execution.
Renovate custom manager for git reference updates renovate.json	Updated renovate.json with a customManagers configuration targeting .tekton/*.yaml files. The manager uses a regex pattern to capture depName, currentValue (branch), and currentDigest from # renovate: annotations in manifest files, with datasourceTemplate configured as git-refs to enable automated tracking and updates of git repository references in the PipelineRun manifest.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~8 minutes

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title clearly summarizes the main change: adding a PipelineRun to trigger conforma e2e tests on CLI pull requests targeting the main branch.
Description check	✅ Passed	The description is directly related to the changeset, explaining the PipelineRun addition, commit SHA pinning, and Renovate configuration for managing git-refs.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[fullsend-ai-review]

🤖 Review · Started 7:23 PM UTC
Commit: 47d3320 · View workflow run →

[coderabbitai]

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In @.tekton/cli-e2e-pull-request.yaml:
- Line 4: The manifest contains a duplicate YAML mapping key "metadata.name"
which makes the resource invalid; remove the redundant "metadata.name" entry so
only a single metadata.name is present in the same mapping (locate the duplicate
"metadata.name" keys in the resource block and delete the second occurrence or
merge its value into the first), ensuring the final YAML has exactly one
metadata.name definition for this resource.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Enterprise

Run ID: 30a1eea7-c2c1-4ca8-9a5e-d4852e442f0c

📥 Commits

Reviewing files that changed from the base of the PR and between c6df9ad and 0de5cb1.

📒 Files selected for processing (1)

• .tekton/cli-e2e-pull-request.yaml

[fullsend-ai-review]

🤖 Finished Review · ✅ Success · Started 7:33 PM UTC · Completed 7:40 PM UTC
Commit: 47d3320 · View workflow run →

[fullsend-ai-review]

Looks good to me

Previous run

Review

Findings

Low

• [missing-authorization] — PR references only an external Jira ticket (KONFLUX-14184) with no linked GitHub issue. Many organizations use Jira as primary tracker; the Jira ticket serves as the authorization artifact. Noted for traceability.

• [pattern-inconsistency] .tekton/cli-e2e-pull-request.yaml:1 — The new file omits the creationTimestamp: field in the metadata section, while both existing pipeline files (cli-main-pull-request.yaml, cli-main-push.yaml) include it. Minor consistency gap with no functional impact.

Previous run (2)

Review

Findings

Medium

• [logic-error] .tekton/cli-e2e-pull-request.yaml:30 — The parameter aws-credentials-secret is defined twice with the same value mapt-kind-secret (lines 29-32). This is a duplicate entry that is almost certainly a copy-paste error. In Tekton PipelineRun params, the second entry silently overwrites the first. Since both have the same value the pipeline will still function, but this likely indicates the second entry should have been a different parameter name.
  Remediation: Remove the duplicate aws-credentials-secret entry, or rename the second one to the intended parameter name if this was a copy-paste mistake.

Low

• [edge-case] renovate.json:9 — The Renovate regex matchStrings pattern requires the value: line to immediately follow the renovate comment line (with only whitespace between). If content is inserted between them, Renovate will silently stop matching and the SHA will not be auto-updated. This is a fragile contract that is easy to break during future edits.

• [missing-conventional-field] .tekton/cli-e2e-pull-request.yaml:3 — The existing .tekton files both include a creationTimestamp: field (set to null) in metadata. The new file omits it, breaking the established pattern in this directory.

Info

• [permissions/service-account] .tekton/cli-e2e-pull-request.yaml:44 — New PipelineRun introduces a different service account (konflux-integration-runner) compared to the existing build pipelines which use build-pipeline-cli-main. This expands the set of secrets reachable from PR-triggered pipelines.

• [annotation-value-inconsistency] .tekton/cli-e2e-pull-request.yaml:10 — The pipelinesascode.tekton.dev/max-keep-runs annotation is set to "2" while both existing .tekton files use "3". This may be intentional for an e2e test pipeline.

• [authorization] .tekton/cli-e2e-pull-request.yaml — No linked GitHub issue; authorization traces to Jira KONFLUX-14184 only. Acceptable per project conventions but noted for traceability.

• [architectural-coherence] renovate.json — The Renovate customManagers addition is well-scoped and consistent with the existing extension pattern.

Previous run (3)

Review

Findings

Low

• [pattern-inconsistency] .tekton/cli-e2e-pull-request.yaml:10 — max-keep-runs value ("2") differs from established pattern in existing Tekton files (cli-main-pull-request.yaml, cli-main-push.yaml) which use "3". May be intentional for a different pipeline type but deviates from codebase convention.
  Remediation: Change to "3" to align with existing pipelines, unless there is a specific operational reason for keeping runs limited to 2.

• [permission-expansion] .tekton/cli-e2e-pull-request.yaml:11 — New PipelineRun triggers on pull_request events targeting main. Uses Tekton git resolver to fetch pipeline from external repo (conforma/e2e-tests) at a pinned commit SHA. Pipeline reference is pinned to a specific commit rather than a mutable branch. Since this is the same organization, the risk is low.

Info

• [architectural-coherence] .tekton/cli-e2e-pull-request.yaml — The new pipeline file follows the established naming convention and structure of existing Tekton files. Uses standard patterns: PipelineRun kind, pipelinesascode annotations, AppStudio/RHTAP labels. Architecturally coherent with existing CI/CD infrastructure.

• [secret-exposure] .tekton/cli-e2e-pull-request.yaml:28 — The PipelineRun references Kubernetes secrets by name (konflux-test-infra, mapt-kind-secret, mapt-kind-secret-1). These are secret name references, not secret values, which is the correct pattern consistent with Tekton best practices.

• [permission-expansion] .tekton/cli-e2e-pull-request.yaml:43 — The new PipelineRun uses service account konflux-integration-runner which differs from existing pipelines that use build-pipeline-cli-main. This is a standard Konflux integration runner account expected for integration test pipelines.

Previous run (4)

Review

Findings

Low

• [naming-convention] .tekton/cli-e2e-pull-request.yaml:15 — PipelineRun label uses pipelines.appstudio.openshift.io/type: test, while existing CLI pipelines use type: build. This is likely intentional — the existing pipelines are build pipelines, while this new one triggers e2e tests. The type label correctly classifies the pipeline's purpose.

• [annotation-pattern] .tekton/cli-e2e-pull-request.yaml:10 — max-keep-runs is set to "2", while existing pipelines use "3". Minor inconsistency; may be intentional since e2e test runs are less critical to retain than build runs.

• [permission-expansion] .tekton/cli-e2e-pull-request.yaml:33 — The new PipelineRun uses the Tekton git resolver with a pinned commit SHA to reference an external pipeline from conforma/e2e-tests. The existing files use bundle resolvers with OCI digest-pinned task references, which provide stronger immutability guarantees (OCI digests are immutable; git commit SHAs are technically mutable via force-push). The risk is modest and the pattern is common in the Tekton ecosystem.

Info

• [secret-exposure] .tekton/cli-e2e-pull-request.yaml:29 — References Kubernetes secrets by name (konflux-test-infra, mapt-kind-secret, mapt-kind-secret-1). These are secret name references, not secret values — no credentials are exposed.

Previous run (5)

Review

Findings

Low

• [Indentation & YAML Formatting] .tekton/cli-e2e-pull-request.yaml:20 — The new file uses 4-space indentation for list items under spec.params, whereas the existing Tekton files use 2-space indentation with list markers at the same level as the mapping key. Both are valid YAML but the convention in this repo is the latter.
  Remediation: Adjust indentation to match the established style in existing .tekton/ files.

• [authorization / service account] .tekton/cli-e2e-pull-request.yaml:44 — The new pipeline uses service account konflux-integration-runner, which differs from the existing pipelines that use build-pipeline-cli-main. Using a different service account for a test pipeline vs a build pipeline is reasonable (separation of concerns), but verify that konflux-integration-runner follows least-privilege and is scoped appropriately for e2e test workloads only.

• [supply chain / pipeline integrity] .tekton/cli-e2e-pull-request.yaml:33 — The pipeline definition is fetched from an external Git repository (conforma/e2e-tests) via the Tekton git resolver, pinned to a specific commit SHA. The Renovate custom manager will automatically propose updates to track the latest main branch commit. Ensure that Renovate PRs updating this SHA receive the same review scrutiny as any other code change.

Info

• [Annotation Values] .tekton/cli-e2e-pull-request.yaml:9 — max-keep-runs is set to "2" whereas the existing pull request pipeline uses "3". This may be intentional to reduce resource usage for e2e test runs that provision AWS infrastructure.

• [secrets handling] .tekton/cli-e2e-pull-request.yaml:29 — The pipeline references Kubernetes secrets konflux-test-infra and mapt-kind-secret by name (expected Tekton pattern). Pipelines-as-Code enforces that only approved pipeline definitions from the default branch are executed, preventing PR authors from modifying the pipeline to exfiltrate secrets.

• [authorization] No linked GitHub issue, but the PR references Jira ticket KONFLUX-14184, consistent with project conventions. CI infrastructure change carries implicit authorization.

• [scope-creep] renovate.json evolves from pure delegation to containing local overrides with the new customManagers section. Natural evolution for this use case.

• [architectural-coherence] The new file introduces e2e as a component identifier (vs main in existing files) with no push counterpart. This is expected since e2e tests are only meaningful as pre-merge PR validation.

Previous run (6)

Review

Findings

Medium

• [logic-error] .tekton/cli-e2e-pull-request.yaml:30 — The parameter aws-credentials-secret is declared twice with identical name and value (mapt-kind-secret). Tekton requires unique parameter names in a PipelineRun; duplicate entries will cause the PipelineRun to fail validation at submission time. This is a copy-paste error.
  Remediation: Remove one of the two aws-credentials-secret entries.

• [unpinned-pipeline-ref] .tekton/cli-e2e-pull-request.yaml:36 — The pipelineRef uses the Tekton git resolver to fetch a pipeline from conforma/e2e-tests.git at revision main without a commit-SHA pin. A compromise of the main branch (or a force-push) would silently change what pipeline definition executes in this tenant. The existing .tekton files pin every task reference to an immutable bundle digest. See also: [secret-exposure] finding.
  Remediation: Pin the git resolver revision parameter to a specific commit SHA instead of the mutable branch name main.

• [secret-exposure] .tekton/cli-e2e-pull-request.yaml:28 — Secret references (konflux-test-infra and mapt-kind-secret) are passed as parameters to a pipeline fetched from an unpinned mutable git ref. A compromised upstream pipeline definition could exfiltrate these secrets. See also: [unpinned-pipeline-ref] finding.
  Remediation: Pin the pipeline ref revision to a specific commit SHA.

Low

• [service-account-divergence] .tekton/cli-e2e-pull-request.yaml:42 — Uses serviceAccountName konflux-integration-runner instead of the existing build-pipeline-cli-main. This is expected since the new file is a test pipeline (type: test) rather than a build pipeline.

Info

• [sub-agent-failure] The style-conventions and intent-coherence sub-agents were unavailable (model not deployed). These are non-critical review dimensions for this change.

[fullsend-ai-review]

See the review comment for full details.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

Flag	Coverage Δ
acceptance	53.43% <ø> (-0.01%)	⬇️
generative	16.79% <ø> (ø)
integration	27.66% <ø> (ø)
unit	69.13% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[joejstuart]

/retest

[cuipinghuo]

/retest

[cuipinghuo]

/retest

[cuipinghuo]

/retest

[cuipinghuo]

/retest cli-e2e-on-pull-request

[cuipinghuo]

/retest

[fullsend-ai-review]

🤖 Review · ⚠️ Cancelled · Started 2:28 PM UTC · Ended 2:30 PM UTC
Commit: 47d3320 · View workflow run →

[fullsend-ai-review]

🤖 Finished Review · ✅ Success · Started 2:33 PM UTC · Completed 2:44 PM UTC
Commit: 47d3320 · View workflow run →

[cuipinghuo]

/retest cli-e2e-on-pull-request

[fullsend-ai-review]

🤖 Finished Review · ✅ Success · Started 5:20 PM UTC · Completed 5:29 PM UTC
Commit: 47d3320 · View workflow run →

[fullsend-ai-review]

🤖 Finished Review · ✅ Success · Started 5:42 PM UTC · Completed 5:53 PM UTC
Commit: 47d3320 · View workflow run →

[fullsend-ai-review]

🤖 Finished Review · ✅ Success · Started 3:02 AM UTC · Completed 3:12 AM UTC
Commit: 47d3320 · View workflow run →

[fullsend-ai-review]

See the review comment for full details.

[fullsend-ai-review]

🤖 Finished Review · ✅ Success · Started 3:30 AM UTC · Completed 3:41 AM UTC
Commit: 47d3320 · View workflow run →

[fullsend-ai-review]

🤖 Finished Review · ✅ Success · Started 3:54 AM UTC · Completed 4:02 AM UTC
Commit: 47d3320 · View workflow run →