Add EQL v3 Drizzle support

[tobyhede]

Summary

Adds the EQL v3 Drizzle integration (@cipherstash/stack/eql/v3/drizzle) plus hardened live test coverage, on top of the v3 typed client / schema DSL.

• A Drizzle-native types namespace whose columns are the semantic eql_v3.<domain> Postgres types.
• createEncryptionOperatorsV3 — capability-checked operators (eq/ne/gt/gte/lt/lte/between/notBetween/contains/inArray/notInArray/asc/desc/and/or/not/isNull/isNotNull/exists/notExists) that emit the two-argument eql_v3 term-function SQL. The concrete column type drives which operators are legal (e.g. contains needs a match index; asc/between need order).
• extractEncryptionSchemaV3 rebuilds the encrypt schema for EncryptionV3.
• Live test coverage — a type-driven 35-domain matrix drives live encrypt/decrypt + SQL query proofs. Recent additions strengthen assertion quality: range exclusion (not just inclusion), type-boundary values through real domain columns, NULL persistence across capability tiers, and lock-context querying through the operator path — plus a CI guard that fails loudly if the live suites silently skip.

The existing v2 @cipherstash/stack/drizzle integration is unchanged. v3 free-text search is token containment, so contains replaces SQL like/ilike (deliberately not exposed).

Usage

import { drizzle } from "drizzle-orm/postgres-js"
import { integer, pgTable } from "drizzle-orm/pg-core"
import postgres from "postgres"
import { EncryptionV3 } from "@cipherstash/stack/v3"
import {
  createEncryptionOperatorsV3,
  extractEncryptionSchemaV3,
  types,
} from "@cipherstash/stack/eql/v3/drizzle"

const users = pgTable("users", {
  id: integer("id").primaryKey().generatedAlwaysAsIdentity(),
  email: types.TextSearch("email"),   // free-text search
  nickname: types.TextEq("nickname"), // equality
  age: types.IntegerOrd("age"),       // order + range
})

const client = await EncryptionV3({ schemas: [extractEncryptionSchemaV3(users)] })
const e = createEncryptionOperatorsV3(client)
const db = drizzle({ client: postgres(process.env.DATABASE_URL!) })

// Filter operators are async (they encrypt the query term); asc/desc are sync.
const rows = await db
  .select()
  .from(users)
  .where(await e.contains(users.email, "example.com"))
  .orderBy(e.asc(users.age))

Test Plan

• cd packages/stack && pnpm biome check src/eql/v3/drizzle __tests__/{drizzle-v3,v3-matrix}
• cd packages/stack && pnpm vitest run __tests__/{drizzle-v3,v3-matrix}
• cd packages/stack && pnpm test:types && pnpm build
• Live suites (Postgres + eql_v3) run when DATABASE_URL + CS_* are set and self-install the eql_v3 extension at beforeAll. The Drizzle lock-context suite additionally needs USER_JWT and soft-skips without it.

Notes

• Live tests are credential-gated; a new REQUIRE_LIVE / REQUIRE_LIVE_PG guard turns a silent skip into a CI failure so the live matrix can't quietly go green.
• The two-argument eql_v3.* function forms are interim (tracked by CIP-3402 / CIP-3423).

Summary by CodeRabbit

• New Features

  • Added support for a new encrypted database integration with richer query capabilities, including equality, ranges, pattern matching, sorting, and membership checks.
  • Introduced a new set of column types for working with encrypted data while keeping query behavior aligned with the underlying field type.
  • Existing encrypted database integration remains available unchanged.

• Bug Fixes

  • Improved handling of null and undefined values when reading and writing encrypted JSON-backed fields.

[coderdan]

Looks great. A couple of items that should be addressed before merge. Also, the docs are almost non-existent. Should there be a section in the README and some typedoc?

[coderdan]

Code review — EQL v3 Drizzle integration + bigint domains

Reviewed at extra-high effort (multi-angle finder pass → verify → sweep) against dbe1ec0. The change is well-tested (deterministic + live-pg matrix), which refuted several plausible bugs (see bottom). 12 findings survived, correctness first.

Correctness / robustness

1. bigint domains are type-blessed but have no runtime guard → opaque low-level error — packages/stack/src/eql/v3/columns.ts:166 (also types.ts:128, Plaintext in types.ts:96)
types.Bigint/BigintEq/… are exported publicly and PlaintextForColumn resolves them to bigint, but the only value validator (assertValidNumericValue, helpers/validation.ts:43) checks typeof === 'number', so a bigint passes every guard to protect-ffi 0.27, which cannot marshal a JS BigInt across the Neon boundary.
Failure: client.encrypt(123n, { table, column }) on a types.BigintEq column type-checks and passes unit CI (FFI mocked), then fails at runtime with a cryptic Neon/serialization error instead of a clear "bigint not yet supported" SDK error. On the drizzle operator path the throw isn't wrapped in EncryptionOperatorError. The gate is prose comments only — an explicit runtime guard on cast_as === 'bigint' (or not exporting the factories until the pin bumps) would keep the type/runtime contract from diverging silently.

2. decryptModel/bulkDecryptModels key the reconstructor by table object identity → a valid, correctly-typed call can return a failure — packages/stack/src/encryption/v3.ts:217 (Map built at :190)
EncryptedTable is structurally typed (tableName: string, brand is only the phantom _columnType), so two separately-constructed but identical tables are the same type yet different objects. The old code built rowReconstructor(table) from whatever table was passed; the new code does reconstructors.get(table) and returns unknownTableFailure on a miss.
Failure: const c = await EncryptionV3({schemas:[usersA]}) then c.decryptModel(row, usersB) where usersB = encryptedTable('users', {…same…}) (a re-import, HMR reload, or table built in another module). Compiles (structural match), but at runtime returns DecryptionError: "…a table this client was not initialized with…" instead of decrypting — a regression vs. prior behavior. Keying by tableName would match the semantic identity the FFI/build() already use.

3. Drizzle customType declares plaintext data types but toDriver only JSON-serializes (no encryption) — packages/stack/src/eql/v3/drizzle/codec.ts:10 (type declared in column.ts:88)
The column is typed data: PlaintextForColumn<C> (Date/bigint/number/string) with toDriver(value) = JSON.stringify(value). The intended flow inserts pre-encrypted envelopes (live tests cast through as never), but nothing in the types enforces that.
Failure: For types.Bigint, the type-valid db.insert(t).values({ n: 1n }) calls JSON.stringify(1n) → throws TypeError: Do not know how to serialize a BigInt synchronously. For date/string, the same type-valid plaintext insert bypasses encryption and JSON-stringifies plaintext into the eql_v3 column (a plaintext-exposure footgun — some domain CHECK constraints may reject the non-envelope shape, but the type actively invites the write). Consider typing data as the encrypted envelope, or documenting/guarding the insert path.

4. Drizzle types mirror can't reach EncryptedTextSearchColumn.freeTextSearch(opts) → tokenization silently locked to defaults — packages/stack/src/eql/v3/drizzle/types.ts:11
Each factory is (name) => makeEqlV3Column(factory(name)), returning a Drizzle column — there's no way to call the chainable .freeTextSearch(opts) tuner. The canonical flow (extractEncryptionSchemaV3(table) → EncryptionV3({schemas}), per the live tests) bakes the recovered builder's build().indexes.match into the real encrypt config.
Failure: A drizzle user who needs custom tokenizer/k/m (short-substring or CJK free-text) has no drizzle-path way to set it; every text_search column gets defaultMatchOpts() ngram settings, and contains() behaves against the default tokenization with no error or warning. The JSDoc claims to "mirror" the v3 types namespace but omits this capability.

5. bigint decrypt is unsound once the encrypt gate lifts — packages/stack/src/eql/v3/columns.ts:681 (reconstructor at encryption/v3.ts:147)
PlaintextFromKind types bigint columns as bigint, and rowReconstructor intentionally passes them through (only DATE_LIKE_CASTS are rebuilt). The FFI cannot return a JS bigint, so a decrypted bigint column would hold a number/string while typed bigint. Currently masked because encrypt throws first (finding 1).
Failure: After the FFI pin bumps, decryptModel returns the FFI's serialized form for a bigint column but the type asserts bigint; downstream value + 1n throws Cannot mix BigInt and other types. Worth a reconstruction step (or a test) landing alongside the pin bump.

Efficiency

6. inArray/notInArray re-resolve the column context and call build() once per array element — packages/stack/src/eql/v3/drizzle/operators.ts:269
inArrayOp maps every value through equality(...), and each equality runs resolveContext → getEqlV3Column + builder.build().indexes (:132). The context is identical for all N values.
Failure: inArray(col, [...1000 ids]) triggers 1000 getEqlV3Column calls and 1000 build() invocations (for a text_search column each build() deep-clones the match block). Resolve context once, then reuse it per operand (keep the concurrency-4 encryption). A WeakMap<builder, indexes> memo on build() would also fix per-operator rebuilds.

7. between/notBetween encrypt min then max sequentially — packages/stack/src/eql/v3/drizzle/operators.ts:237
const encMin = await encryptOperand(...) then const encMax = await encryptOperand(...) — the two operands are independent.
Failure: Every range predicate pays 2× encrypt latency (round-trips to the crypto backend). const [encMin, encMax] = await Promise.all([...]) halves it.

Cleanup / conventions

8. EncryptionOperatorError is redeclared, forking a class that already exists and is exported by v2 — packages/stack/src/eql/v3/drizzle/operators.ts:29
v2 (src/drizzle/operators.ts:82, re-exported from src/drizzle/index.ts) defines an identical-shaped EncryptionOperatorError. Two distinct runtime class identities: a consumer's catch (e) { if (e instanceof EncryptionOperatorError) } against one import silently fails to match the other, and the shared context shape must be edited in two places. Lift to one shared module.

9. A Linear issue ID appears in a source comment (and in the changeset) — packages/stack/src/eql/v3/columns.ts:165 (also .changeset/eql-v3-bigint-domains.md lines 6 and 10)
Internal issue trackers shouldn't leak into this public repo's source or its published CHANGELOG. Drop the IDs or reword. (IDs omitted here for the same reason.)

10. Drizzle table-name extraction via Symbol.for('drizzle:Name') is hand-inlined across ~5 sites — packages/stack/src/eql/v3/drizzle/schema-extraction.ts:11 (again at operators.ts:115; v2 has getDrizzleTableName)
The Drizzle-internal drizzle:Name contract is hard-coded in multiple places with no single source of truth; a Drizzle upgrade that changes it breaks all of them independently. Share one getDrizzleTableName(table) helper.

11. EQL_V3_DOMAINS and buildersByDomain run the same probe iteration twice and can drift — packages/stack/src/eql/v3/drizzle/column.ts:10
Both do Object.values(v3Types).map(f => f('__probe__').getEqlType()), constructing throwaway probe columns per domain. EQL_V3_DOMAINS is exactly buildersByDomain's key set — derive it: new Set(buildersByDomain.keys()).

12. Dead EQL_V3_COLUMN_LEGACY_PARAM (_eqlv3Column) redundancy — packages/stack/src/eql/v3/drizzle/column.ts:30
_eqlv3Column is never read outside this file; writeBuilder always sets it together with the Symbol, and readBuilder returns the Symbol first via ??, so the legacy branch is unreachable (the Symbol carrier is confirmed to survive pgTable via config.customTypeParams). The const, type member, extra write, and ?? branch can be deleted; "legacy" also implies a nonexistent external contract.

---

Cleared during verification (not bugs): eql_v3.eq/neq on ORE-only *_ord columns (live-pg matrix proves exact/complement selection); applyOperationOptions discarding withLock.audit()'s return (audit() mutates this and returns this); resolveMatchOpts vs. the old inline merge (behavior-identical, now safer — it clones for v2 too); EncryptedTextSearchColumn.build() super.build()+spread (byte-identical index block); V3DecryptedModel = V3ModelInput alias (textually identical to the prior mapped type); DATE_LIKE_CASTS filter (same set as === 'date' || === 'timestamp'); new Date(value) reconstruction (correct instant for the UTC samples; unchanged by this PR); mapWithConcurrency (no race, no unhandled-rejection leak); stash recovery before/after pgTable; and the batch-encrypt-query.ts change (type-only, runtime unchanged).

_{🤖 Generated with Claude Code}