feat(agent-bff): expose collection list and count with flat mapping

[nbouliol]

What

Slice 3 of the Forest BFF: expose POST /v1/:collection/list and /count as flat REST over the agent.

• list → { data: [{ ...directFields, id, __forest: { collection, primaryKey } }], meta: { countStatus: "not_requested" } }. primaryKey is the opaque packed id unpacked into a typed { field: value } map using the read-model's key metadata (composite ids supported; a composite id without key metadata is surfaced as a mapping error, never guessed).
• count → { count, countStatus }. Deactivation is derived from the raw agent payload (meta.count === "deactivated"), not the lossy Number() helper. Active empty collection → 0/available; no usable signal → mapping_error (never a silent 0).
• Nested-relation guard (PRD-669) wired on list projection/filter/sort and count filter → 422 relation_field_not_supported.
• Collection resolution via the read-model allow-list.
• Errors mapped through PRD-670 (mapAgentError for agent failures; local envelope for BFF-origin errors, incl. a new 503 schema_unavailable).
• All agent calls go through the exported HttpRequester, so the BFF controls the query params — notably the resolved timezone (spec: never lean on agent-client's Europe/Paris default).

Stacking / dependencies

This work depends on two PRs still in review, whose branches are merged into this one. Until they land on main, this PR's diff temporarily includes their code:

• feat(agent-bff): fetch and cache agent schema, capabilities and allow-list #1732 — PRD-668 (schema/capabilities/allow-list read-model)
• feat(agent-bff): map agent errors to the BFF error contract #1738 — PRD-670 (agent error → BFF error contract)

PRD-669 (#1736) is already merged to main; its content here is identical. Rebase onto main once #1732 and #1738 land, then this diff narrows to the data/ additions + read-model/cli-core/bff-local-errors changes.

Open item — 403 not satisfiable in Slice 3

The read-model exposes a single allow-list (the exposed collections), so it cannot distinguish an unknown collection from a known-but-disallowed one. Every absent name maps to 404 unknown_collection; collection_not_allowed (403) has no local trigger. Documented with a TODO in data-routes-middleware.ts and escalated on the ticket.

Tests

Focused tests with a stubbed agent client + read-model: flat mapping + __forest + meta.countStatus; packed/composite id roundtrip; count deactivated vs active-zero vs mapping-error fallback; guard 422 across all four surfaces; 404 resolution; 503 schema_unavailable; agent failure via the error contract; timezone propagated to the agent query. Full package: build + lint + 499 tests green.

Fixes PRD-671

🤖 Generated with Claude Code

Note

Expose collection list and count endpoints in agent-bff with flat mapping

• Adds POST /agent/v1/:collection/list and POST /agent/v1/:collection/count routes in data-routes-middleware.ts, proxying requests to the configured agent with validation and response shaping.
• Introduces a ReadModel built from the Forest schema that exposes allowed collections, relations, actions, and primary keys; cached via SchemaCache and CapabilitiesCache with TTL and single-flight refresh.
• Adds mapListResponse and mapCountResponse in response-mappers.ts to produce flat records with __forest metadata (including structured primaryKey) and deactivation-aware count status.
• Adds assertNoRelationFieldPaths guard that rejects nested relation field paths (containing :) with a 422 error across projection, filter, and sort inputs.
• Wires real data routes into cli-core.ts, falling back to a stub middleware with a warning when agent URL or API key config is missing.
• Risk: pagination now enforces that offset must be a multiple of limit; requests with arbitrary offsets are rejected with 400.

^{Macroscope summarized e86119b.}

[linear-code]

PRD-671

[qltysh]

3 new issues

Tool	Category	Rule	Count
qlty	Structure	Function with many returns (count = 5): mapAgentError	3

[qltysh]

Coverage Impact

⬆️ Merging this pull request will increase total coverage on main by 0.02%.

Modified Files with Diff Coverage (20)

Rating	File	% Diff	Uncovered Line #s
	packages/agent-bff/src/http/bff-http-error.ts	100.0%
	packages/agent-bff/src/validation/relation-field-guard.ts	100.0%
	packages/agent-bff/src/cli-core.ts	100.0%
	packages/agent-bff/src/read-model/create-read-model.ts	100.0%
	packages/agent-bff/src/read-model/forest-schema-client.ts	100.0%
	packages/agent-bff/src/read-model/read-model.ts	100.0%
	packages/agent-bff/src/http/agent-error-mapper.ts	98.3%	97
	packages/agent-bff/src/data/agent-data-client.ts	33.3%	22-28
	packages/agent-bff/src/read-model/schema-cache.ts	100.0%
	packages/agent-bff/src/read-model/agent-capabilities-fetcher.ts	100.0%
	packages/agent-bff/src/read-model/action-endpoint-resolver.ts	100.0%
	packages/agent-bff/src/data/pack-id.ts	100.0%
	packages/agent-bff/src/data/response-mappers.ts	100.0%
	packages/agent-bff/src/data/agent-query.ts	97.2%	71
	packages/agent-bff/src/read-model/capabilities-cache.ts	100.0%
	packages/agent-bff/src/read-model/read-model-store.ts	100.0%
	packages/agent-bff/src/read-model/errors.ts	100.0%
	packages/agent-bff/src/adapters/console-metrics.ts	100.0%
	packages/agent-bff/src/data/data-routes-middleware.ts	90.7%	34, 48-50, 97
	packages/agent-bff/src/http/bff-local-errors.ts	100.0%
	Total	97.4%

🤖 Increase coverage with AI coding...

In the `feature/prd-671-expose-collection-list-and-count-with-flat-mapping` branch, add test coverage for this new code:

- `packages/agent-bff/src/data/agent-data-client.ts` -- Line 22-28
- `packages/agent-bff/src/data/agent-query.ts` -- Line 71
- `packages/agent-bff/src/data/data-routes-middleware.ts` -- Lines 34, 48-50, and 97
- `packages/agent-bff/src/http/agent-error-mapper.ts` -- Line 97

🚦 See full report on Qlty Cloud »

🛟 Help

• Diff Coverage: Coverage for added or modified lines of code (excludes deleted files). Learn more.

• Total Coverage: Coverage for the whole repository, calculated as the sum of all File Coverage. Learn more.

• File Coverage: Covered Lines divided by Covered Lines plus Missed Lines. (Excludes non-executable lines including blank lines and comments.)

  • Indirect Changes: Changes to File Coverage for files that were not modified in this PR. Learn more.

[macroscopeapp]

🟡 Medium http/agent-error-mapper.ts:85

When a JSON:API error from the agent has a name not present in AGENT_ERROR_TYPE_MAP, mapJsonApiError returns a 500 mapping_error instead of preserving the agent's original status and type. For example, a custom agent error subclass named CustomNotFoundError that carries status: 404 gets remapped to a 500 internal BFF failure, turning a legitimate 4xx client error into a server error.

This happens in the branch where agentError.name !== undefined and AGENT_ERROR_TYPE_MAP[agentError.name] returns undefined. Consider falling back to fallbackTypeByStatus(status) (the same path used when name is absent) rather than calling mappingError(), so unmapped error names still preserve their original HTTP status and a sensible error type.

  if (agentError.name !== undefined) {
-    const type = AGENT_ERROR_TYPE_MAP[agentError.name];
-
-    if (type === undefined) {
-      logger('Error', 'Unmapped agent error name', { name: agentError.name, status });
-
-      return mappingError(`Unmapped agent error name: ${agentError.name}`);
-    }
-
-    return new BffHttpError(status, type, message, agentError.data);
+    const type = AGENT_ERROR_TYPE_MAP[agentError.name] ?? fallbackTypeByStatus(status);
+
+    return new BffHttpError(status, type, message, agentError.data);

Also found in 1 other location(s)

packages/agent-bff/src/read-model/action-endpoint-resolver.ts:39

resolve() swallows SchemaUnavailableError from getReadModel() and returns undefined as if the action mapping were simply missing. On a cold schema cache while Forest is unavailable, callers cannot distinguish "schema unavailable" from "unknown action" and will mis-handle the request (typically turning a transient 503 condition into a false miss/404).

🚀 Reply "fix it for me" or copy this AI Prompt for your agent:

In file @packages/agent-bff/src/http/agent-error-mapper.ts around lines 85-95:

When a JSON:API error from the agent has a `name` not present in `AGENT_ERROR_TYPE_MAP`, `mapJsonApiError` returns a 500 `mapping_error` instead of preserving the agent's original status and type. For example, a custom agent error subclass named `CustomNotFoundError` that carries `status: 404` gets remapped to a 500 internal BFF failure, turning a legitimate 4xx client error into a server error.

This happens in the branch where `agentError.name !== undefined` and `AGENT_ERROR_TYPE_MAP[agentError.name]` returns `undefined`. Consider falling back to `fallbackTypeByStatus(status)` (the same path used when `name` is absent) rather than calling `mappingError()`, so unmapped error names still preserve their original HTTP status and a sensible error type.

Also found in 1 other location(s):
- packages/agent-bff/src/read-model/action-endpoint-resolver.ts:39 -- `resolve()` swallows `SchemaUnavailableError` from `getReadModel()` and returns `undefined` as if the action mapping were simply missing. On a cold schema cache while Forest is unavailable, callers cannot distinguish "schema unavailable" from "unknown action" and will mis-handle the request (typically turning a transient `503` condition into a false miss/`404`).

[macroscopeapp]

🟡 Medium read-model/read-model.ts:24

toRelationTarget returns an empty string as the target collection for relations whose field.reference is a bare collection name (e.g. 'orders') with no trailing .<key> segment. split('.').slice(0, -1).join('.') strips the only token, so getRelationTarget() records target: '' for a valid HasMany/BelongsToMany relation, and callers that rely on the resolved target mis-handle or reject the relation. The code assumes every reference is ${foreignCollection}.${key}, but schemas also use bare collection names. Consider falling back to the full field.reference when no . is present.

-    return { type, polymorphic: false, target: field.reference.split('.').slice(0, -1).join('.') };
+    const target = field.reference.includes('.')
+      ? field.reference.split('.').slice(0, -1).join('.')
+      : field.reference;
+    return { type, polymorphic: false, target };

🚀 Reply "fix it for me" or copy this AI Prompt for your agent:

In file @packages/agent-bff/src/read-model/read-model.ts around line 24:

`toRelationTarget` returns an empty string as the target collection for relations whose `field.reference` is a bare collection name (e.g. `'orders'`) with no trailing `.<key>` segment. `split('.').slice(0, -1).join('.')` strips the only token, so `getRelationTarget()` records `target: ''` for a valid `HasMany`/`BelongsToMany` relation, and callers that rely on the resolved target mis-handle or reject the relation. The code assumes every `reference` is `${foreignCollection}.${key}`, but schemas also use bare collection names. Consider falling back to the full `field.reference` when no `.` is present.

[macroscopeapp]

🟠 High src/cli-core.ts:191

In OAuth mode, requests to /agent/v1/:collection/list|count fail because buildDataRoutesMiddleware is mounted for every authenticated /agent request but only the API-key branch populates ctx.state.agentToken. When the OAuth path runs, createAuthModeMiddleware sets only ctx.state.principal, leaving ctx.state.agentToken undefined, so the data routes call the agent with Authorization: Bearer undefined. Either pass the OAuth bearer token into ctx.state.agentToken for the data routes or gate buildDataRoutesMiddleware so it only runs in API-key mode.

🚀 Reply "fix it for me" or copy this AI Prompt for your agent:

In file @packages/agent-bff/src/cli-core.ts around line 191:

In OAuth mode, requests to `/agent/v1/:collection/list|count` fail because `buildDataRoutesMiddleware` is mounted for every authenticated `/agent` request but only the API-key branch populates `ctx.state.agentToken`. When the OAuth path runs, `createAuthModeMiddleware` sets only `ctx.state.principal`, leaving `ctx.state.agentToken` undefined, so the data routes call the agent with `Authorization: Bearer undefined`. Either pass the OAuth bearer token into `ctx.state.agentToken` for the data routes or gate `buildDataRoutesMiddleware` so it only runs in API-key mode.

[nbouliol]

Addressed review feedback in d885e3b:

• Function size / duplication — dataRoutesMiddleware split into handleList / handleCount + a callAgent(fn) wrapper (the middleware is now routing + resolution + dispatch; the duplicated try/catch → mapAgentError is gone).
• Dead CountStatus export — removed (ListResponse/CountResponse keep their inline unions).
• Dead ConditionTreeBranch.aggregator — removed (only conditions is read).
• Magic 'Number' — named NUMBER_COLUMN_TYPE constant in pack-id.ts. Did not narrow PrimaryKeyField.type to a literal union: the value is sourced from ForestSchemaField.type: string and the domain ForestServerColumnType is a non-literal composite (array/object forms), so a union would only relocate the risk into an unchecked as cast at construction.
• Schema-first count-deactivation — confirmed payload-only is the only viable derivation (neither the Forest schema nor capabilities() exposes a count flag; the agent decides countability at request time in access/count.ts). Documented as a comment on mapCountResponse; no code change.

build + lint + 499 tests green.

[nbouliol]

Macroscope review triaged in e86119b.

Fixed (this PR's files):

• 🟠 OAuth Bearer undefined — data routes now fail closed with 401 unauthorized when ctx.state.agentToken is absent (only the API-key path mints it today), instead of calling the agent with Bearer undefined. Minting an agent token from the OAuth principal is OAuth-foundation work — TODO(PRD-637) added.
• 🟡 decodeURIComponent → 500 — malformed percent-encoded collection names now return 400 invalid_request (kept in the BFF error envelope, not the ad-hoc body from the suggestion).
• 🟡 unpackPrimaryKey NaN — a non-numeric segment for a Number key throws a mapping error (mirrors the agent's IdUtils.unpackId) instead of emitting { id: NaN }.
• qlty handleList params — back to 3 via a dedicated ListHandlerDeps (count still carries no unused fields).

Not changing (dependency-PR files, not owned here):

• 🟡 agent-error-mapper.ts unmapped name → mapping_error: lives in feat(agent-bff): map agent errors to the BFF error contract #1738 (PRD-670), and it's intentional per that spec ("never silently collapsed to a generic type" — unmapped names are a stop condition). The suggested status-fallback contradicts it.
• 🟡 read-model.ts toRelationTarget empty-string on a bare reference & action-endpoint-resolver.ts swallowing SchemaUnavailableError: both are feat(agent-bff): fetch and cache agent schema, capabilities and allow-list #1732 (PRD-668) logic, unused by list/count here. Flagging to that PR.
• qlty mapAgentError many-returns: feat(agent-bff): map agent errors to the BFF error contract #1738 file.

build + lint + 502 tests green.

[macroscopeapp]

🟡 Medium data/data-routes-middleware.ts:128

A request with projection set to a string (e.g. { "projection": "id" }) or sort set to a string causes the list endpoint to throw a plain TypeError and return a 500 instead of a 400 invalid_request error. The body is cast to ListRequestBody & CountRequestBody without validating that projection and sort are arrays, so non-array values reach buildListAgentQuery() / collectListFieldPaths() and fail when calling array methods like .join/.forEach/.map on a string. Consider validating the shape of body before dispatching to handleList / handleCount so malformed input returns invalidRequest().

🚀 Reply "fix it for me" or copy this AI Prompt for your agent:

In file @packages/agent-bff/src/data/data-routes-middleware.ts around line 128:

A request with `projection` set to a string (e.g. `{ "projection": "id" }`) or `sort` set to a string causes the list endpoint to throw a plain `TypeError` and return a 500 instead of a 400 `invalid_request` error. The body is cast to `ListRequestBody & CountRequestBody` without validating that `projection` and `sort` are arrays, so non-array values reach `buildListAgentQuery()` / `collectListFieldPaths()` and fail when calling array methods like `.join`/`.forEach`/`.map` on a string. Consider validating the shape of `body` before dispatching to `handleList` / `handleCount` so malformed input returns `invalidRequest()`.