fix: prerender validation endpoints#2793
Conversation
…dpoint with Myers word diff Implements LLMO-5857 Task 3 — comparison API for prerender validation. Fetches S3-prerendered HTML and live Lambda HTML, runs Myers word-level diff via jsdiff diffWords, and returns structured comparison metrics including pageStatus (SUCCESS / FAILURE_BLOCKED / FAILURE_CONTENT), word counts, and diffMatchPct. Adds @adobe/spacecat-shared-html-analyzer and diff as runtime dependencies. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…tils Remove defensive null/undefined guards on extractWordCount results (r?.word_count ?? 0) that created uncovered branch paths. extractWordCount never returns null in Node.js, so direct property access (r.word_count) is safe. Add route to INTERNAL_ROUTES in facs-capabilities.js and required-capabilities.js for S2S routing enforcement. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Add PATCH /sites/:siteId/opportunities/:opportunityId/prerender-validation that merges prerender validation lifecycle state into the opportunity's data server-side, touching only data.prerenderValidation so other data fields written by the prerender audit are preserved. - new patchPrerenderValidation controller method + status enum (in_progress | completed_success | completed_fail | error) - route registration + required/facs capability entries (opportunity:write) - tests with 100% coverage; fix a pre-existing leak where a patchOpportunity test reassigned mockOpptyEntity.save without restoring it - spec in docs/specs/ Used by the prerender validation tool to report run status back to SpaceCat. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
POST /prerender-validation/compare had no real caller — tokowaka's actual comparison logic (processSite in s3-lambda-compare.js) does the S3-vs-live diff entirely locally, duplicating the same libraries this endpoint used, rather than calling it. The route was marked "internal audit-worker only" but audit-worker's prerender-validation handler is also unwired/unused in the shipped design, so this endpoint had zero callers. Removes the controller, utility, and their tests; unwires the route from index.js/routes/index.js/required-capabilities.js/facs-capabilities.js; drops the now-unused diff and @adobe/spacecat-shared-html-analyzer deps and regenerates the lockfile; updates the stale route-list test. Keeps PATCH /sites/:siteId/opportunities/:opportunityId/prerender-validation (status reporting), which tokowaka does actually call.
…y as internal The PATCH .../prerender-validation status endpoint is only ever called by tokowaka using an admin/S2S token, which bypasses FACS entirely (FACS applies to external customer callers only, per facs-wrapper.js). The llmo/can_configure and aso/can_edit product grants added for this route were therefore unnecessary -- no human/browser-session caller uses this endpoint today. Removes both product grants and instead lists the route under INTERNAL_ROUTES, satisfying the invariant that every declared route must be classified as either belonging to a product or explicitly internal.
The prerender-validation PATCH route was added to src/routes/index.js but never added to the hardcoded expectedDynamicRouteKeys list in the route-segregation test, causing a 450 vs 449 member mismatch.
|
This PR will trigger a minor release when merged. |
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
There was a problem hiding this comment.
Hey @jindaliiita,
Verdict: Request changes - one material gap to close before merge.
Complexity: HIGH - large diff (package-lock metadata); API surface + FACS signal.
Changes: Adds a dedicated PATCH endpoint for updating prerender validation status on opportunities via server-side merge, avoiding read-modify-write on the generic PATCH (8 files).
Note: Recommend a human read before merge - possible divergence from the documented design (heuristic, flagged by the conventions reviewer).
Must fix before merge
- [Important] No type/format validation on
startedAt/completedAtallows storing arbitrary values -src/controllers/opportunities.js:410(details inline)
Non-blocking (2): minor issues and suggestions
- suggestion: Add OpenAPI spec entry for the new endpoint per the repo's "OpenAPI First" / "Specification Sync" must-follow patterns in CLAUDE.md -
src/routes/index.js:334 - suggestion: Add a test case verifying that omitting
completedAtfrom the body preserves an existingcompletedAtvalue (the key partial-update semantic) -test/controllers/opportunities.test.js
Skill: pr-review | Model: us.anthropic.claude-opus-4-6-v1[1m] | Duration: 3m 31s | Cost: $7.98 | Commit: a519066ba7ce172832ff2c03d6acd1941cab5208
If this code review was useful, please react with 👍. Otherwise, react with 👎.
… or null Prevents non-string/invalid-date values from being persisted into prerenderValidation, since downstream consumers (UI via OpportunityDto) expect date strings. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
There was a problem hiding this comment.
Hey @jindaliiita,
Verdict: Request changes - two documented must-follow conventions remain unmet.
Complexity: HIGH - large diff (package-lock metadata); API surface + FACS signal.
Changes: Adds timestamp validation and a reason field to the prerender-validation lifecycle endpoint, addressing the prior review's blocking finding (8 files).
Note: Recommend a human read before merge - possible divergence from the documented design (heuristic, flagged by the conventions reviewer).
Previously flagged, now resolved
- Timestamp validation for
startedAt/completedAtnow enforced viaisValidTimestampguard - Test coverage for partial-update semantics added (merge behavior, reason clearing)
Must fix before merge
- [Important] Missing OpenAPI specification - CLAUDE.md "Must-Follow Patterns" requires "OpenAPI First: Define API contract in
docs/openapi/before implementation" and "Specification Sync: Keep OpenAPI specs and implementation in sync". The newPATCH .../prerender-validationendpoint has no entry indocs/openapi/. The design spec indocs/specs/is valuable context but does not satisfy this requirement. - [Important] Missing integration tests - CLAUDE.md states "New or modified endpoints must include integration tests in
test/it/". The endpoint has thorough unit tests but no integration test wiring (seed IDs, seed data, shared test factory, wiring file per the Adding a New Endpoint checklist step 10).
Non-blocking (4): minor issues and suggestions
- suggestion: Add type validation for
reason(accept only string or null) to prevent storing non-string values that could break downstream rendering -src/controllers/opportunities.js:410 - suggestion: Tighten timestamp validation with an ISO 8601 regex before
Date.parse-Date.parseaccepts values like"Tuesday"or"1"which are not ISO 8601 -src/controllers/opportunities.js:405 - nit: Design spec should document the
reasonfield added in this commit -docs/specs/2026-07-03-prerender-validation-status-endpoint.md - nit: Add a completedAt-with-number-type test for symmetry with the existing startedAt number test -
test/controllers/opportunities.test.js
Skill: pr-review | Model: us.anthropic.claude-opus-4-6-v1[1m] | Duration: 1m 31s | Cost: $4.14 | Commit: 871a8774e797741d843688a9482f56a371899619
If this code review was useful, please react with 👍. Otherwise, react with 👎.
…validation Adds the missing OpenAPI contract for the prerender-validation lifecycle endpoint (PrerenderValidationUpdate schema, path entry, wiring into api.yaml), per the "OpenAPI First" must-follow pattern. Also updates the design spec to document the reason field and the startedAt/completedAt/reason validation added in the prior commit. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…son type - Gate startedAt/completedAt with a full ISO 8601 date-time regex before Date.parse, since Date.parse alone accepts non-ISO strings like "Tuesday" or "1". - Validate reason is a string or null before persisting, preventing non-string values (numbers, objects, arrays) from reaching downstream consumers that expect a string. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…nder-validation Adds IT coverage for the prerender-validation endpoint to the existing opportunities shared test factory: status merge, reason set-and-clear across runs, invalid status/startedAt, 403 on denied site, 404 on missing opportunity. No new seed IDs/data needed — mirrors the sibling PATCH block's pattern of creating a test-scoped opportunity via POST. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
There was a problem hiding this comment.
Hey @jindaliiita,
Verdict: Approve - all prior blocking findings addressed; remaining items are non-blocking hardening suggestions.
Complexity: HIGH - large diff (package-lock metadata); API surface + FACS signal.
Changes: Adds OpenAPI spec, tightens timestamp validation with ISO 8601 regex, validates reason type, and adds integration tests for the prerender-validation endpoint (8 files).
Note: Recommend a human read before merge - this change adds a new shared contract (OpenAPI spec in docs/openapi/). The bot review is a complement to, not a replacement for, a human read here.
Previously flagged, now resolved
- Timestamp validation for
startedAt/completedAtnow enforced via ISO 8601 regex + Date.parse - OpenAPI specification added for the new endpoint in
docs/openapi/ - Integration tests added covering happy path, validation errors, access control (403), and not-found (404)
reasonfield type validation added (string or null)- Design spec updated to document the
reasonfield and its always-reset semantics
Non-blocking (3): minor issues and suggestions
- suggestion: Add a length cap on
reason(e.g., 1024 chars) to prevent a buggy S2S caller from persisting multi-MB strings -src/controllers/opportunities.js:410 - suggestion: Add a unit test documenting that 4+ fractional-second digits are intentionally rejected (e.g.,
2026-07-12T11:31:12.1234Zreturns 400), so the 3-digit cap is a tested contract rather than an implicit side effect -test/controllers/opportunities.test.js - nit: Integration test suite does not exercise the
errorstatus enum value; one additional test case would close the gap -test/it/shared/tests/opportunities.js
Skill: pr-review | Model: us.anthropic.claude-opus-4-6-v1[1m] | Duration: 1m 1s | Cost: $6.84 | Commit: 8345f23c17fe115cb5cd84ec6fac67787cba2881
If this code review was useful, please react with 👍. Otherwise, react with 👎.
…point Forwards to the internal llmo-prerender-api service's POST /api/compare/run using the caller's own Authorization header, so a comparison run can be triggered directly through spacecat-api-service instead of hitting the internal service's own host. Response (status + body) is passed through as-is. Wired into route-segregation, required-capabilities (opportunity:write), and facs-capabilities (internal/S2S bypass) the same way as the existing PATCH .../prerender-validation endpoint, plus an OpenAPI spec.
…tion-endpoints # Conflicts: # docs/index.html # package-lock.json
…rvice The internal prerender-validation service now authorizes this service-to-service call by source IP allowlist instead of a caller-supplied Bearer token, so this endpoint no longer needs to read or forward the Authorization header.
…tion-endpoints # Conflicts: # docs/index.html # package-lock.json
…equestId tracking runPrerenderValidation now gates locally on the opportunity's existing prerenderValidation status (409 if already completed_success or a non-stale in_progress run) before calling tokowaka, avoiding a wasted call. maxPages/enableAiAnalysis/checkAuditAge are no longer caller-configurable — always sent as 100/false/true. Added request/ response/error logging and a 5s timeout. patchPrerenderValidation now accepts and carries forward a requestId field so the tokowaka job id is visible on the opportunity once a run starts.
There was a problem hiding this comment.
Hey @jindaliiita,
Verdict: Request changes - input validation gap and stale documentation need fixing before merge.
Complexity: HIGH - large diff (package-lock metadata); API surface + FACS signal.
Changes: Adds a POST endpoint for triggering prerender-validation comparison runs via an internal service proxy, alongside the previously-approved PATCH lifecycle endpoint (13 files).
Note: Recommend a human read before merge - this change adds a new shared contract (OpenAPI spec in docs/openapi/). The bot review is a complement to, not a replacement for, a human read here.
Previously flagged, now resolved
- Timestamp validation for
startedAt/completedAtenforced via ISO 8601 regex + Date.parse - OpenAPI specification added for both endpoints
- Integration tests added for PATCH endpoint
reasonfield type validation added (string or null)- Auth forwarding removed in favor of IP-based allowlisting (commit bced77d)
Must fix before merge
- [Important] Stale OpenAPI description claims caller auth is forwarded to internal service - code no longer does this (IP allowlist since bced77d) -
docs/openapi/site-opportunities.yaml:~248(see below) - [Important]
customUrlsfrom user input forwarded to internal service without type, length, or scheme validation -src/controllers/opportunities.jsinrunPrerenderValidation(see below) - [Important] Error message in 502 response leaks internal hostname via
e.message-src/controllers/opportunities.jscatch block (see below)
Details
1. Stale OpenAPI description (docs/openapi/site-opportunities.yaml)
The POST .../prerender-validation/run description says: "using the caller's own Authorization header - this endpoint does not hold its own credential for the internal service." This was true before commit bced77d removed auth forwarding. The code now explicitly does NOT forward any Authorization header (IP-based source allowlisting instead). Consumers reading the API docs will assume their token reaches the internal service, which is both incorrect and a security concern from their perspective.
Fix: Update the description to: "Authentication with the internal service is handled via IP-based source allowlisting - the caller's Authorization header is NOT forwarded."
2. customUrls unvalidated (src/controllers/opportunities.js)
The customUrls field from context.data is spread directly into the upstream request body with zero runtime validation. The OpenAPI schema declares type: array, items: { type: string, format: uri } but no runtime enforcement exists. A caller can pass non-array values, thousands of entries, or non-URL strings (including file://, data: URIs) that get forwarded to the internal comparison service.
Fix: Add validation before forwarding:
if (customUrls !== undefined) {
if (!Array.isArray(customUrls) || customUrls.some((u) => typeof u !== 'string')) {
return badRequest('customUrls must be an array of strings');
}
if (customUrls.length > 100) {
return badRequest('customUrls cannot exceed 100 entries');
}
}3. Error message leaks internal hostname (src/controllers/opportunities.js)
The catch block returns message: e.message in the 502 response. A failed fetch() message typically contains the target URL (e.g., "request to https://sj1010010249075.corp.adobe.com/... failed, reason: ENOTFOUND..."), leaking the internal corporate hostname to the authenticated external caller.
Fix: Return a generic message. The server-side log.error already captures the full error:
return createResponse(
{ error: 'prerenderValidationServiceUnreachable', message: 'Internal prerender service is temporarily unavailable' },
502,
);Non-blocking (4): minor issues and suggestions
- suggestion: For non-2xx upstream responses, whitelist known fields (
error,reason,requestId) before passing through to caller to prevent leaking internal debug info -src/controllers/opportunities.js - nit: FACS comment says "internal tokowaka-only" but any S2S consumer with
opportunity:writecan call these endpoints; update comment to reflect this -src/routes/facs-capabilities.js:259 - suggestion: Document the
completed_successterminal-state behavior (callers must PATCH to reset before re-triggering a run) in the OpenAPI description or design spec -docs/openapi/site-opportunities.yaml - suggestion: The JSDoc on
runPrerenderValidationalso mentions "caller's own Authorization header is forwarded as-is" - update to match the code -src/controllers/opportunities.js
Skill: pr-review | Model: us.anthropic.claude-opus-4-6-v1[1m] | Duration: 3m 25s | Cost: $9.09 | Commit: 8225dce733d6444abda83ba7d90a77ea965994f6
If this code review was useful, please react with 👍. Otherwise, react with 👎.
Please ensure your pull request adheres to the following guidelines:
describe here the problem you're solving.
If the PR is changing the API specification:
yet. Ideally, return a 501 status code with a message explaining the feature is not implemented yet.
If the PR is changing the API implementation or an entity exposed through the API:
If the PR is introducing a new audit type:
Related Issues
Thanks for contributing!