Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .changeset/eql-v3-ffi-0-28-concrete-types.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
'@cipherstash/stack': minor
---

Upgrade `@cipherstash/protect-ffi` to 0.28.0 and update EQL v3 concrete Postgres domain names to match the SQL fixture (`integer*`, `smallint*`, `bool`, `real*`, and `double*`). The public factories remain semantic (`Integer`, `Smallint`, `Boolean`, `Real`, `Double`) while their concrete domains change, so this is a minor release.
5 changes: 5 additions & 0 deletions .changeset/eql-v3-public-domains.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
'@cipherstash/stack': patch
---

Re-vendor the EQL v3 SQL bundle and align the v3 DSL to it: encrypted type domains now live in the `public` schema (`public.text`, `public.integer`, …) rather than `eql_v3`, and the boolean domain is `public.boolean` (was `eql_v3.bool`). The `eql_v3` schema now holds only the operator-backing functions, and the index-term constructors (`hmac_256`, `ore_block_256`, `bloom_filter`) moved to `eql_v3_internal`. This keeps the SDK's emitted domain names byte-matched to the installed bundle so `CREATE TABLE`/cast resolution succeeds.
6 changes: 3 additions & 3 deletions .changeset/eql-v3-typed-client.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,19 +11,19 @@ Every method derives its types from the concrete `table` / `column` builder
arguments:

- `encrypt` / `encryptQuery` pin the plaintext to the column's domain type
(`text → string`, `int8 → bigint`, `timestamp → Date`, …).
(`text → string`, `timestamp → Date`, …).
- `encryptQuery` constrains `queryType` to the column's capabilities and rejects
storage-only columns at compile time.
- `encryptModel` / `bulkEncryptModels` validate schema-column fields against their
inferred plaintext type (passthrough fields are untouched) and return a precise
encrypted model.
- `decryptModel` / `bulkDecryptModels` return the precise plaintext model,
reconstructing `Date` / `bigint` values from the encrypt-config `cast_as`.
reconstructing `Date` values from the encrypt-config `cast_as`.

Because the typed methods bind to the concrete branded v3 classes, a hand-rolled
structural table/column is rejected — closing the soundness gap where a non-branded
table could be encrypted at runtime while typed as plaintext.

Runtime behaviour is unchanged: the encrypt/query paths return the same operations
as the base client; only the model-decrypt paths add a per-column `Date` / `bigint`
as the base client; only the model-decrypt paths add a per-column `Date`
reconstruction step. The v2 client surface (`Encryption`) is untouched.
4 changes: 2 additions & 2 deletions .changeset/eql-v3-typed-schema.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,6 @@
'@cipherstash/stack': minor
---

Add EQL v3 schema builders for all generated SQL domains under `@cipherstash/stack/eql/v3`, exposed as the `types` namespace (one member per EQL v3 domain, e.g. `types.TextEq` / `types.Int4Ord` / `types.Timestamp`), including explicit query capability metadata (`getQueryCapabilities()` / `isQueryable()`) and v3 table support in model encryption helpers (`encryptModel` / `bulkEncryptModels`).
Add EQL v3 schema builders for supported generated SQL domains under `@cipherstash/stack/eql/v3`, exposed as the `types` namespace (one member per supported EQL v3 domain, e.g. `types.TextEq` / `types.IntegerOrd` / `types.Timestamp`), including explicit query capability metadata (`getQueryCapabilities()` / `isQueryable()`) and v3 table support in model encryption helpers (`encryptModel` / `bulkEncryptModels`).

Also widen the accepted plaintext input type for `encrypt` / `encryptQuery` to include `Date` and `bigint` (via the new `Plaintext` type), so v3 `date` / `timestamp` / `int8` domains can be encrypted and queried with their natural JavaScript values.
Also widen the accepted plaintext input type for `encrypt` / `encryptQuery` to include `Date` (via the new `Plaintext` type), so v3 `date` / `timestamp` domains can be encrypted and queried with their natural JavaScript values.
5 changes: 5 additions & 0 deletions .changeset/skip-v3-bigint.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
'@cipherstash/stack': patch
---

Omit EQL v3 bigint factories from the public DSL and test matrix until native bigint round-tripping is supported.
43 changes: 43 additions & 0 deletions .github/actions/require-cs-secrets/action.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
name: Require CipherStash secrets
description: >-
Fail the job when any CS_* credential is missing. The live test suites gate
themselves on these vars (vitest `describe.skip`, Deno `test.ignore`), so a
rotated / cleared / fork-PR-absent secret would make them silently skip and
let CI go green with ZERO live coverage. This turns that silent skip into a
loud failure. Requires the repo to be checked out first (local action path).
inputs:
workspace-crn:
description: secrets.CS_WORKSPACE_CRN
required: true
client-id:
description: secrets.CS_CLIENT_ID
required: true
client-key:
description: secrets.CS_CLIENT_KEY
required: true
client-access-key:
description: secrets.CS_CLIENT_ACCESS_KEY
required: true

runs:
using: composite
steps:
- name: Assert CS_* secrets are present
shell: bash
env:
CS_WORKSPACE_CRN: ${{ inputs.workspace-crn }}
CS_CLIENT_ID: ${{ inputs.client-id }}
CS_CLIENT_KEY: ${{ inputs.client-key }}
CS_CLIENT_ACCESS_KEY: ${{ inputs.client-access-key }}
run: |
missing=0
for v in CS_WORKSPACE_CRN CS_CLIENT_ID CS_CLIENT_KEY CS_CLIENT_ACCESS_KEY; do
if [ -z "${!v}" ]; then
echo "::error::Required secret $v is not set on this runner — live test suites would silently skip."
missing=1
fi
done
if [ "$missing" -ne 0 ]; then
exit 1
fi
10 changes: 10 additions & 0 deletions .github/workflows/prisma-example-readme-e2e.yml
Original file line number Diff line number Diff line change
Expand Up @@ -65,6 +65,16 @@ jobs:
- name: Install dependencies
run: pnpm install --frozen-lockfile

# A missing / rotated / fork-PR-absent secret makes the walkthrough skip
# its live steps silently, hiding regressions behind a green job. Fail loud.
- name: Require CipherStash secrets
uses: ./.github/actions/require-cs-secrets
with:
workspace-crn: ${{ secrets.CS_WORKSPACE_CRN }}
client-id: ${{ secrets.CS_CLIENT_ID }}
client-key: ${{ secrets.CS_CLIENT_KEY }}
client-access-key: ${{ secrets.CS_CLIENT_ACCESS_KEY }}

# Build via turbo so `^build` on `@cipherstash/prisma-next` and
# its `@cipherstash/stack` peer is honoured. The test's
# `pnpm install` subprocess inside `examples/prisma/` is a no-op
Expand Down
11 changes: 11 additions & 0 deletions .github/workflows/prisma-next-e2e.yml
Original file line number Diff line number Diff line change
Expand Up @@ -67,6 +67,17 @@ jobs:
- name: Install dependencies
run: pnpm install --frozen-lockfile

# The global-setup hook hard-errors without CS_WORKSPACE_CRN, but a
# missing sibling secret could still degrade coverage silently — assert
# all four up front so a rotated / fork-PR-absent secret fails loudly.
- name: Require CipherStash secrets
uses: ./.github/actions/require-cs-secrets
with:
workspace-crn: ${{ secrets.CS_WORKSPACE_CRN }}
client-id: ${{ secrets.CS_CLIENT_ID }}
client-key: ${{ secrets.CS_CLIENT_KEY }}
client-access-key: ${{ secrets.CS_CLIENT_ACCESS_KEY }}

# Write the CS_* credentials and the harness DATABASE_URL into the
# example app's .env so the runtime + the `prisma-next migration
# apply` invocation in global-setup both pick them up. The harness
Expand Down
44 changes: 31 additions & 13 deletions .github/workflows/tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -59,6 +59,16 @@ jobs:
- name: Install dependencies
run: pnpm install --frozen-lockfile

# Fail loudly if any live-test credential is missing, so the v3 live
# matrix (and every other `describeLive` suite) can't silently skip.
- name: Require CipherStash secrets
uses: ./.github/actions/require-cs-secrets
with:
workspace-crn: ${{ secrets.CS_WORKSPACE_CRN }}
client-id: ${{ secrets.CS_CLIENT_ID }}
client-key: ${{ secrets.CS_CLIENT_KEY }}
client-access-key: ${{ secrets.CS_CLIENT_ACCESS_KEY }}

- name: Type tests (stack)
run: pnpm --filter @cipherstash/stack run test:types

Expand Down Expand Up @@ -152,6 +162,16 @@ jobs:
- name: Install dependencies
run: pnpm install --frozen-lockfile

# Auth-dependent `e2e/` suites skip themselves without these vars — a
# silent skip would hide regressions behind a green job. Fail loudly.
- name: Require CipherStash secrets
uses: ./.github/actions/require-cs-secrets
with:
workspace-crn: ${{ secrets.CS_WORKSPACE_CRN }}
client-id: ${{ secrets.CS_CLIENT_ID }}
client-key: ${{ secrets.CS_CLIENT_KEY }}
client-access-key: ${{ secrets.CS_CLIENT_ACCESS_KEY }}

# Run the standalone `e2e/` workspace via turbo so the `^build`
# dep on the `test:e2e` task builds cli + wizard first. CLI's own
# E2E (`packages/cli/tests/e2e/**`) is covered by the `run-tests`
Expand Down Expand Up @@ -219,19 +239,17 @@ jobs:
- name: Build stack
run: pnpm exec turbo run build --filter @cipherstash/stack

# roundtrip.test.ts skips itself (Deno.test.ignore) when any of
# the four CS_* env vars is missing — that's the right behaviour
# for local runs, but in CI a silent skip would mean a rotated /
# cleared secret hides a real WASM regression behind a green job.
# Fail loudly instead.
- name: Assert CS_* secrets are present
run: |
for v in CS_WORKSPACE_CRN CS_CLIENT_ID CS_CLIENT_KEY CS_CLIENT_ACCESS_KEY; do
if [ -z "${!v}" ]; then
echo "::error::Required secret $v is not set on this runner — the WASM smoke test would silently skip."
exit 1
fi
done
# roundtrip.test.ts skips itself (Deno.test.ignore) when any of the four
# CS_* env vars is missing — fine locally, but in CI a silent skip would
# let a rotated / cleared secret hide a real WASM regression behind a
# green job. Fail loudly instead.
- name: Require CipherStash secrets
uses: ./.github/actions/require-cs-secrets
with:
workspace-crn: ${{ secrets.CS_WORKSPACE_CRN }}
client-id: ${{ secrets.CS_CLIENT_ID }}
client-key: ${{ secrets.CS_CLIENT_KEY }}
client-access-key: ${{ secrets.CS_CLIENT_ACCESS_KEY }}

- name: Run Deno WASM smoke test
working-directory: e2e/wasm
Expand Down
116 changes: 116 additions & 0 deletions docs/eql-v3-ord-term-ordering-defect.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,116 @@
# EQL v3 defect: `ORDER BY eql_v3.ord_term(col)` does not sort in ORE order

**Status:** Root-caused with live evidence. The fix does **not** belong in our
SDK — it is an upstream limitation of the vendored EQL v3 bundle. Our tests/docs
are corrected to stop relying on the broken construct; the bundle issue is
recorded here for upstream (`cipherstash/encrypt-query-language`).

## Summary

For an encrypted ordering column (any `*_ord` / `text_search` domain), the range
comparison functions `eql_v3.gte`/`lte`/`lt`/`gt` sort **correctly** (true ORE
order), but `ORDER BY eql_v3.ord_term(col)` sorts **incorrectly** — deterministic
but not the ORE/lexical order.

Reproduced against live Postgres (role `is_superuser = off`) with emails
`ada@example.com`, `grace@example.com`, `alan@example.net`, `zora@example.org`:

| Construct | Result | Correct? |
|---|---|---|
| pairwise `eql_v3.lt(col, col)` (the ORE comparator) | `ada < alan < grace < zora` | ✅ ORE/lexical |
| `ORDER BY eql_v3.ord_term(col)` | `zora, alan, ada, grace` | ❌ record_ops |
| `ORDER BY col` (domain default) | `grace, ada, alan, zora` (structural) | ❌ jsonb_ops |
| `ORDER BY col USING <` / `USING OPERATOR(public.<)` | error: *operator < is not a valid ordering operator* | ❌ no orderable family |

So on a non-superuser install there is **no** `ORDER BY` construct that yields
ORE order. Only the boolean comparison operators (used in `WHERE` predicates) are
ORE-correct.

## Root cause

File: `packages/stack/__tests__/fixtures/eql-v3/cipherstash-encrypt-v3.sql`
(the vendored bundle — do not hand-edit; it is regenerated on re-vendor).

1. `eql_v3.ord_term(a public.text_search)` (and every `*_ord` domain, e.g.
`public.integer_ord` at line 18117) returns the **composite** type
`eql_v3_internal.ore_block_256` — line **38141-38144**:
```sql
CREATE FUNCTION eql_v3.ord_term(a public.text_search)
RETURNS eql_v3_internal.ore_block_256 ...
```
`ore_block_256` is `CREATE TYPE ... AS (terms eql_v3_internal.ore_block_256_term[])`
at line **193**.

2. The ORE-correct default btree opclass for that composite type,
`eql_v3_internal.ore_block_256_operator_class` (backed by
`eql_v3_internal.compare_ore_block_256_terms`), is created inside a
superuser-gated `DO` block — lines **3117-3138**. Creating an operator
class requires superuser, so on managed/non-superuser Postgres it is
**skipped** on `insufficient_privilege` (lines 3134-3136):
```
EQL: skipped operator class eql_v3_internal.ore_block_256_operator_class
(requires superuser); ORE ordered indexes on ore_block_256 unavailable ...
```
Live check confirms: `SELECT ... FROM pg_opclass ... WHERE typname='ore_block_256'`
returns `[]`.

3. With the opclass absent, `ORDER BY eql_v3.ord_term(col)` still "works"
because PostgreSQL supplies a built-in **record comparison** for any
composite type. That compares the `ore_block_256_term[]` fields by their raw
bytes — deterministic, but unrelated to ORE order. This is the silent-wrong
footgun: it does not error, it just sorts wrong.

4. Meanwhile `eql_v3.gte/lte/lt/gt` are ORE-correct **regardless of superuser**
because they do not go through index/sort machinery. `eql_v3.lt(text_search)`
(line 38206) is `SELECT eql_v3.ord_term(a) < eql_v3.ord_term(b)`, and the
`public.<` operator on `ore_block_256` is wired directly to
`eql_v3_internal.compare_ore_block_256_terms`. The domain-level `<` operator
(`CREATE OPERATOR <`, line 38566, `FUNCTION = eql_v3.lt`) exists, but it is
**not** a member of any btree operator family, so `ORDER BY col USING <`
raises "operator < is not a valid ordering operator".

The bundle comment (lines 3104-3114) treats the missing opclass as an accepted
managed-Postgres limitation ("ORE ordered scans ... unavailable"). The real
defect is that `ORDER BY ord_term(col)` **silently returns wrong results** in
that state instead of failing, and there is no non-superuser ordering path.

## Minimal repro

```sql
-- role without superuser; EQL v3 bundle installed
CREATE TABLE t (email public.text_search NOT NULL, label text);
-- insert encrypted 'ada@example.com','grace@example.com',
-- 'alan@example.net','zora@example.org' (labels ada/grace/alan/zora)

SELECT current_setting('is_superuser'); -- off
SELECT count(*) FROM pg_opclass c JOIN pg_type ty
ON ty.oid=c.opcintype WHERE ty.typname='ore_block_256'; -- 0

SELECT label FROM t ORDER BY eql_v3.ord_term(email); -- zora,alan,ada,grace (WRONG)
-- vs the ORE comparator, which is correct:
SELECT x.label, y.label, eql_v3.lt(x.email, y.email) FROM t x CROSS JOIN t y; -- ada<alan<grace<zora
```

## Recommended upstream fix (encrypt-query-language)

Provide a non-superuser ordering path so `ORDER BY` is correct everywhere, e.g.
either:
- expose an `eql_v3.order_by(col)` returning a scalar with a **native** btree
opclass (as EQL v2 does — `eql_v2.order_by()`), or
- make `ord_term`'s return type orderable without a custom opclass, or
- at minimum, make the composite `ore_block_256` have **no** usable default
comparison so `ORDER BY ord_term(col)` errors loudly instead of sorting wrong.

## What we changed in this repo

- `packages/stack/__tests__/schema-v3-pg.test.ts`: corrected the misleading
comment on the range test and added an explicit ORE-total-order proof using
pairwise `eql_v3.lt` (the only ORE-correct path), so the guarantee is asserted
positively rather than merely avoided.
- No change to the vendored `.sql` (it is regenerated on re-vendor).
- Docs that prescribe `orderBy: (col) => eql_v3.ord_term(col)` for a future v3
Drizzle/Prisma adapter (e.g.
`docs/superpowers/2026-06-12-eql-v3-drizzle-adapter-walkthrough.md`,
`docs/superpowers/plans/2026-07-06-eql-v3-drizzle-concrete-types.md`) would
ship this defect if implemented as written. Any v3 `orderBy` must not lower to
a bare `ORDER BY ord_term(...)` until the upstream fix lands.
23 changes: 22 additions & 1 deletion e2e/wasm/deno.json
Original file line number Diff line number Diff line change
Expand Up @@ -3,13 +3,34 @@
"//2": "stack is imported via a file URL because the wasm-inline subpath isn't on a published version yet — once stack ships with /wasm-inline this can switch to a plain npm: specifier. protect-ffi and auth resolve via npm: against the workspace's installed versions (Deno's nodeModulesDir: auto lets it use what pnpm fetched).",
"//3": "No --allow-ffi grant. If protect-ffi ever silently fell back to a native binding under Deno, the test would fail on missing FFI permission — this is the WASM guarantee.",
"nodeModulesDir": "auto",
"//4": "Deno 2.9+ enforces a 24h minimumDependencyAge cooldown by default. Mirror pnpm-workspace.yaml's supply-chain policy (minimumReleaseAge: 7d + minimumReleaseAgeExclude) so the two toolchains agree: keep the 7-day cooldown for third-party deps, but exempt the first-party CipherStash packages this test imports — they ship on their own cadence and are bumped in lockstep. Without the exclude, a just-published protect-ffi/auth (e.g. 0.28.0) fails resolution until it ages past the window.",
"//5": "Deno's exclude does not support globs, so the per-platform native binary sub-packages (optional deps published in lockstep at the same version) are enumerated explicitly — mirrors pnpm-workspace.yaml's protect-ffi-* / auth-* wildcards.",
"minimumDependencyAge": {
"age": "P7D",
"exclude": [
"npm:@cipherstash/protect-ffi",
"npm:@cipherstash/protect-ffi-darwin-arm64",
"npm:@cipherstash/protect-ffi-darwin-x64",
"npm:@cipherstash/protect-ffi-linux-arm64-gnu",
"npm:@cipherstash/protect-ffi-linux-x64-gnu",
"npm:@cipherstash/protect-ffi-linux-x64-musl",
"npm:@cipherstash/protect-ffi-win32-x64-msvc",
"npm:@cipherstash/auth",
"npm:@cipherstash/auth-darwin-arm64",
"npm:@cipherstash/auth-darwin-x64",
"npm:@cipherstash/auth-linux-arm64-gnu",
"npm:@cipherstash/auth-linux-x64-gnu",
"npm:@cipherstash/auth-linux-x64-musl",
"npm:@cipherstash/auth-win32-x64-msvc"
]
},
"tasks": {
"//task": "--no-check because TypeScript can't infer the column-on-table intersection types when stack is loaded via file URL (the dist .d.ts strips brand info). This test exists to verify RUNTIME WASM round-trips — type narrowing is covered by the package's own vitest suite.",
"test": "deno test --no-check --allow-env --allow-net --allow-read --allow-sys --no-prompt"
},
"imports": {
"@cipherstash/stack/wasm-inline": "../../packages/stack/dist/wasm-inline.js",
"@cipherstash/protect-ffi/wasm-inline": "npm:@cipherstash/protect-ffi@0.26.0/wasm-inline",
"@cipherstash/protect-ffi/wasm-inline": "npm:@cipherstash/protect-ffi@0.28.0/wasm-inline",
"@cipherstash/auth/wasm-inline": "npm:@cipherstash/auth@0.40.0/wasm-inline"
}
}
2 changes: 1 addition & 1 deletion packages/stack/__tests__/cjs-require.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -93,7 +93,7 @@ describe('CJS consumers can require the built bundles', () => {
`if (typeof v3.encryptedTable !== 'function') { throw new Error('missing v3 CJS export: encryptedTable') }`,
`if (typeof v3.buildEncryptConfig !== 'function') { throw new Error('missing v3 CJS export: buildEncryptConfig') }`,
`if (typeof v3.types !== 'object' || v3.types === null) { throw new Error('missing v3 CJS export: types namespace') }`,
`const requiredTypes = ['TextSearch', 'TextEq', 'Int4Ord', 'Bool', 'Timestamp']`,
`const requiredTypes = ['TextSearch', 'TextEq', 'IntegerOrd', 'Boolean', 'Timestamp']`,
`const missing = requiredTypes.filter((k) => typeof v3.types[k] !== 'function')`,
`if (missing.length > 0) { throw new Error('missing v3 types.* CJS members: ' + missing.join(', ')) }`,
].join('\n')
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,7 @@ const users = encryptedTable('users', {
})

const usersV3 = encryptedTableV3('users_v3', {
score: types.Int4Ord('score'),
score: types.IntegerOrd('score'),
})

// biome-ignore lint/suspicious/noExplicitAny: test helper reads the Result union
Expand Down
Loading
Loading