Skip to content

feat: enable remote customer-cloud storage bindings (ALIEN-291)#156

Draft
lilienblum wants to merge 31 commits into
mainfrom
lilienblum/alien-291-remote-bindings
Draft

feat: enable remote customer-cloud storage bindings (ALIEN-291)#156
lilienblum wants to merge 31 commits into
mainfrom
lilienblum/alien-291-remote-bindings

Conversation

@lilienblum

@lilienblum lilienblum commented Jul 20, 2026

Copy link
Copy Markdown
Contributor

Linear: ALIEN-291 — BYOB v0: provision customer-cloud storage and access it remotely

Summary

  • Add POST /v1/bindings/resolve for authoritative, resource-scoped remote Storage bindings.
  • Expose a dedicated Rust/N-API/TypeScript RemoteStorage surface with only get, put, head, list, and delete.
  • Rediscover manager assignment and refresh short-lived credentials with per-resource single-flight refresh, bounded backoff, expiry-safe cache fallback, and fail-closed expiry handling.
  • Reconcile cloud grants on setup, update, import, and teardown for AWS S3, GCP Cloud Storage, and Azure Blob Storage.
  • Add a public trusted-backend BYOB Storage example plus compiled live-cloud E2E coverage.
  • Add a reusable hosted TypeScript smoke runner that exercises put, get, head, list, and delete, including post-delete absence verification.
  • Regenerate the manager OpenAPI 3.1/3.0 documents and Rust SDK inputs.
  • Keep stack-level wildcard management grants owned by remote stack management, while per-resource helpers apply only exact scopes; reject exact management scopes that would put bootstrap prerequisites behind the manager they create.

Security and ownership

  • Resolution requires exact deployment/resource authorization, the current release, operational deployment state, a Running/Frozen remote-enabled Storage resource, and setup-owned infrastructure.
  • Existing/external buckets and containers are rejected at deployment preflight, binding publication, grant derivation, and manager resolution.
  • AWS returns STS session credentials constrained by an exact bucket session policy; GCP uses a Credential Access Boundary for one bucket; Azure returns a user-delegation SAS for one container.
  • Refreshable source credentials never cross the API, credential-bearing values are not cached or logged, and every unsupported or mismatched state fails closed.
  • The dedicated storage/remote-data-write permission set contains only the object and multipart operations required by this API.
  • Copy and signed URLs remain outside the initial API.

Breaking migration

  • Remove the unscoped /v1/resolve-credentials route.
  • Replace BindingsProvider::for_remote_deployment with Bindings::for_remote_deployment.
  • Decide the release/version treatment before marking this PR ready.

Verification

  • Exact-head preflight coverage: 223 library tests and 9 infrastructure-mutation integration tests passed.
  • Exact-head infrastructure coverage with all features: 490 library tests, 25 importer tests, and 5 doc tests passed (28 non-runnable controller examples ignored).
  • Manager library: 175 tests passed; focused binding routes: 12 tests passed.
  • Exact-head manager integration: 25 credential/binding tests and 20 stack-import/replay tests passed; exact command capability test passed.
  • Remote Rust binding suite: 15 tests passed, including retry/backoff and manager reassignment.
  • Infrastructure remote-storage suite: 12 tests passed; external-binding suite: 6 tests passed; focused bootstrap dependency suite: 5 tests passed.
  • Provider coverage passed for AWS STS/materialization, GCP attenuation, Azure SAS, remote stack management, imports, and Azure runtime permissions.
  • GCP deleted-role recovery passed 7 focused fail-closed tests; GCP client coverage passed 23 tests; permission operation coverage passed 2 tests.
  • N-API release addon built; bindings TypeScript suite: 60 tests passed; package build and TypeScript tests passed.
  • Hosted remote-storage smoke runner: 6 tests passed; package TypeScript check, build, and Biome passed.
  • BYOB example check/build and Biome passed.
  • Manager OpenAPI passes the open-source Speakeasy CLI lint with zero errors; generated Rust manager SDK cargo check passed.
  • cargo fmt --all -- --check, git diff --check, secret scanning, and OSS-boundary scanning passed.
  • Cloud slot recovery safely undeletes recoverable GCP custom roles; the workflow moved to clean slot 10 after the previous role ID entered GCP's longer reserved-tombstone window.
  • AWS/GCP/Azure comprehensive Rust and all three provider cleanups passed at 75e408e: run 29813653350. The later 04e19ad6 commit changes only smoke-runner deletion verification.
  • Exact-head CI Fast passed at e7739d70 in run 29821403874; its paired unlabeled PR E2E gate passed with cloud jobs intentionally skipped in run 29821403903.
  • Two independent thermo-nuclear/security reviews found no remaining source-code authorization, remote-credential, hosted-fixture, or OSS-boundary gaps. Generated public SDK drift remains the blocker listed below.

Draft validation

  • Regenerate both the manager TypeScript SDK (/v1/bindings/resolve) and Platform TypeScript SDK (strict {commandId} | {project} token request) with the repository's pinned open-source Speakeasy CLI workflows. The installed CLI currently reports Not authenticated; generated files were intentionally not hand-edited.
  • Run the final TypeScript live-cloud matrix after SDK generation fixes the final branch head.
  • Confirm the breaking release/version plan.

@lilienblum lilienblum changed the title feat: add remote binding resolver feat: enable remote customer-cloud storage bindings Jul 20, 2026
@lilienblum

Copy link
Copy Markdown
Contributor Author

CI Fast reached and passed lint, TypeScript typecheck, the real bindings addon build, package-layout validation, and the full TypeScript unit suite. It then stopped on the existing alien-cloudformation::generator::aws_open_search_renders_next_gen_collection_with_data_access snapshot-order drift. The same test and field-order-only diff fail on current main in run https://github.com/alienplatform/alien/actions/runs/29750083342. This branch does not touch OpenSearch generation or that snapshot, so I have not added unrelated snapshot churn to this draft.

ALIEN-291-specific Rust, Node addon, TypeScript, IAM dataset, and generated AWS/GCP/Azure artifact checks are listed in the PR body and pass locally.

@lilienblum
lilienblum force-pushed the lilienblum/alien-291-remote-bindings branch from 3f13df3 to 75e408e Compare July 21, 2026 08:15
@lilienblum
lilienblum deployed to e2e-tests July 21, 2026 08:23 — with GitHub Actions Active
…remote-bindings

# Conflicts:
#	crates/alien-bindings/src/bindings.rs
#	crates/alien-manager/src/auth/authz.rs
#	crates/alien-manager/src/providers/oss_authz.rs
#	crates/alien-manager/src/routes/commands.rs
#	packages/bindings/PACKAGE_LAYOUT.md
#	packages/bindings/README.md
#	packages/bindings/src/__tests__/loader.test.ts
#	packages/bindings/src/factories.ts
…remote-bindings

# Conflicts:
#	crates/alien-manager/src/providers/oss_authz.rs
@lilienblum lilienblum changed the title feat: enable remote customer-cloud storage bindings feat: enable remote customer-cloud storage bindings (ALIEN-291) Jul 21, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant