feat: Railway integration — service logs + resource metrics

[Makisuo]

Summary

Adds a Railway integration: an org pastes a Railway workspace/account API token, picks the project environments to ingest, and Maple pulls their runtime service logs (GraphQL environmentLogs subscription over WebSocket) and resource metrics (CPU, memory, network, disk via the metrics query, 5-minute default) — converted to OTLP and pushed through the org's own ingest endpoint, so the data is billed, org-scoped, and warehouse-routed exactly like customer OTLP. Railway has no log-drain feature, so pull-based is the only integration shape possible.

How it works

• Storage (packages/db/src/schema/railway.ts): railway_connections (one AES-256-GCM-encrypted token per org, revocation surfacing) + railway_targets (one per project/environment; logs/metrics toggles, cached service list, health + monotonic watermark columns). Migration 0012.
• Domain contracts (packages/domain/src/http/railway.ts, railway-internal.ts): new railway MapleApi group (status/connect/discover/target CRUD) and the internal wire schemas for the connector.
• API (apps/api): RailwayIntegrationService (token validation via me with a projects-query fallback for workspace tokens, discovery, CRUD, TTL-refreshed service cache, watermark/health recording) + railway.http.ts (admin-gated mutations) + railway-internal.http.ts (SD_INTERNAL_TOKEN-authenticated work list carrying decrypted tokens + org maple_pk_* ingest keys, mirroring scraper-internal).
• Connector (apps/railway-connector, new standalone Bun/Effect service modeled on apps/scraper): reconcile loop with fingerprint-keyed fibers; per-environment graphql-transport-ws log subscription (hand-rolled ~200-line client, bounded drop-oldest buffering, exponential-backoff reconnect, watermark dedupe at the reconnect boundary); per-service metrics polling with rate-limit-aware delays (Railway budgets by the customer's plan — the poller honors X-RateLimit-Remaining/Retry-After); OTLP/JSON conversion matching the ingest gateway's Rust deserializer contract. Deployed like the scraper (Dockerfile + railway.json, /health, single replica). No changes to apps/ingest or apps/alerting.
• UI (apps/web): Railway card in the integrations hub — paste-token connect, project → environment board with per-environment ingest/logs/metrics toggles, per-target health, and a "Reconnect needed" banner on token revocation.
• Docs: customer guide (apps/landing/.../integrations/railway.md) + internal architecture notes (docs/railway-integration.md).

Testing

• bun typecheck: 30/30 tasks green (includes cargo check for the untouched ingest gateway).
• bun run test (turbo → vitest): 22/22 packages green; 754 api tests including 14 new ones (connect/encrypt round-trip, workspace-token fallback, discovery annotation, target CRUD + duplicate rejection, internal marshaling, watermark monotonicity, unauthorized-report → reconnect flag, disconnect cascade) and 29 new connector tests (log/metric converters, severity + unit scaling, watermark filtering, rate-budget delay math, WS protocol client against a fake socket, scheduler fingerprints, chunked result flushing).
• Not covered here: a live end-to-end run against a real Railway token (the environmentLogs/metrics GraphQL shapes were written against Railway's public schema as used by its official Prometheus-exporter template and community log-egress tools; the decoders are tolerant and target health surfaces any drift). Recommend one staging pass with a real token before announcing.

Notes for reviewers

• v1 boundaries (deliberate): workspace/account tokens only, runtime logs only (no build/deploy or HTTP edge logs), one connection per org, single-replica connector (no lease — two replicas would double-ingest).
• Metric naming uses a railway.* namespace (railway.cpu.usage, railway.memory.usage in bytes, …) landing in the existing metrics_gauge datasource — no Tinybird changes.
• Deploying needs one new Railway service in Maple's own project pointing at apps/railway-connector/railway.json with MAPLE_API_URL, SD_INTERNAL_TOKEN, MAPLE_INGEST_URL.

🤖 Generated with Claude Code

https://claude.ai/code/session_01EaKNwwLJc9Dm6YLdqh1Hyv

---

Generated by Claude Code

---

[pullfrog]

Run failed. View the logs →

  ^{｜ ⚠️ this action is pinned to a commit SHA, which freezes the cleanup step — switch to @v0 or keep the SHA fresh with Dependabot ｜ Rerun failed job ➔ ｜ View workflow run ｜ via Pullfrog ｜ Using Claude Opus ｜ 𝕏}

[github-actions]

Ingest Rust Test + Benchmark Results

Commit: d5500fb649a6a6c29b7ba2406b1e2cc72c3a675b

Load Benchmark — tinybird mode, median of 3 run(s) vs main

Metric	main (median)	PR (median)	Delta
Requests/sec	887.77	771.62	-13.1% worse
Rows/sec	8877.74	7716.20	-13.1% worse
p50 latency	39.81 ms	53.67 ms	+34.8% worse
p95 latency	209.30 ms	222.65 ms	+6.4% worse
p99 latency	306.04 ms	307.47 ms	+0.5% worse
Export catch-up	0.026 s	0.026 s	-0.5% better
Max RSS	100.71 MiB	103.61 MiB	+2.9% worse
Failures	0	0	same

Same code path on both sides (same LOAD_TEST_INGEST_MODE), so the delta column is meaningful. Numbers come from ubuntu-latest, which is noisy — treat single-digit-percent deltas as noise.

PR load benchmark JSON (per-iteration)

[
  {
    "ingest_mode": "tinybird",
    "requests": 2000,
    "successes": 2000,
    "failures": 0,
    "rows_sent": 20000,
    "rows_exported": 20000,
    "imports": 25,
    "duration_seconds": 1.857356016,
    "export_catchup_seconds": 0.026765631,
    "request_rps": 1076.7994841975412,
    "row_rps": 10767.994841975411,
    "p50_ms": 38.807,
    "p95_ms": 153.118,
    "p99_ms": 196.424,
    "max_rss_mb": 105.08984375,
    "max_cpu_percent": 33.9,
    "avg_cpu_percent": 27.2
  },
  {
    "ingest_mode": "tinybird",
    "requests": 2000,
    "successes": 2000,
    "failures": 0,
    "rows_sent": 20000,
    "rows_exported": 20000,
    "imports": 27,
    "duration_seconds": 2.98928384,
    "export_catchup_seconds": 0.02587949,
    "request_rps": 669.0565724263909,
    "row_rps": 6690.565724263909,
    "p50_ms": 55.274,
    "p95_ms": 247.909,
    "p99_ms": 442.985,
    "max_rss_mb": 103.609375,
    "max_cpu_percent": 32.1,
    "avg_cpu_percent": 21.45714285714286
  },
  {
    "ingest_mode": "tinybird",
    "requests": 2000,
    "successes": 2000,
    "failures": 0,
    "rows_sent": 20000,
    "rows_exported": 20000,
    "imports": 26,
    "duration_seconds": 2.591949204,
    "export_catchup_seconds": 0.0259818,
    "request_rps": 771.6200598813896,
    "row_rps": 7716.200598813895,
    "p50_ms": 53.669,
    "p95_ms": 222.649,
    "p99_ms": 307.474,
    "max_rss_mb": 101.421875,
    "max_cpu_percent": 22.1,
    "avg_cpu_percent": 15.9
  }
]

main load benchmark JSON (per-iteration)

[
  {
    "ingest_mode": "tinybird",
    "requests": 2000,
    "successes": 2000,
    "failures": 0,
    "rows_sent": 20000,
    "rows_exported": 20000,
    "imports": 27,
    "duration_seconds": 2.17485476,
    "export_catchup_seconds": 0.026111194,
    "request_rps": 919.6016381342173,
    "row_rps": 9196.016381342173,
    "p50_ms": 39.815,
    "p95_ms": 209.301,
    "p99_ms": 306.036,
    "max_rss_mb": 100.71484375,
    "max_cpu_percent": 23.5,
    "avg_cpu_percent": 17.54
  },
  {
    "ingest_mode": "tinybird",
    "requests": 2000,
    "successes": 2000,
    "failures": 0,
    "rows_sent": 20000,
    "rows_exported": 20000,
    "imports": 27,
    "duration_seconds": 2.32557981,
    "export_catchup_seconds": 0.025561285,
    "request_rps": 860.0005862623997,
    "row_rps": 8600.005862623997,
    "p50_ms": 40.208,
    "p95_ms": 234.837,
    "p99_ms": 333.31,
    "max_rss_mb": 100.57421875,
    "max_cpu_percent": 26.8,
    "avg_cpu_percent": 20.080000000000002
  },
  {
    "ingest_mode": "tinybird",
    "requests": 2000,
    "successes": 2000,
    "failures": 0,
    "rows_sent": 20000,
    "rows_exported": 20000,
    "imports": 25,
    "duration_seconds": 2.252825807,
    "export_catchup_seconds": 0.026180502,
    "request_rps": 887.7739209953928,
    "row_rps": 8877.739209953928,
    "p50_ms": 37.373,
    "p95_ms": 179.993,
    "p99_ms": 292.378,
    "max_rss_mb": 101.06640625,
    "max_cpu_percent": 20.1,
    "avg_cpu_percent": 14.720000000000002
  }
]

WAL-acked microbench (cargo bench --bench ingest_bench)

   Compiling maple-ingest v0.1.0 (/home/runner/work/maple/maple/apps/ingest)
    Finished `bench` profile [optimized] target(s) in 42.06s
     Running benches/ingest_bench.rs (target/release/deps/ingest_bench-581d2100de893627)
Gnuplot not found, using plotters backend
test ingest_accept/logs_10_rows_wal_ack ... bench:     1028278 ns/iter (+/- 360567)
test ingest_accept/traces_10_spans_wal_ack ... bench:     2695546 ns/iter (+/- 1434988)

cargo test

test telemetry::tests::metrics_summary_data_points_are_dropped ... ok
test telemetry::tests::metrics_emit_exactly_the_jsonpaths_declared_in_datasources_ts ... ok
test telemetry::tests::pipeline_can_start_for_clickhouse_only_without_tinybird_credentials ... ok
test telemetry::tests::clickhouse_export_drops_passworded_non_https_endpoint_without_sending ... ok
test telemetry::tests::pipeline_e2e_exports_gzip_ndjson_to_fake_tinybird ... ok
test telemetry::tests::migrate_legacy_shard_relocates_frames_into_lanes ... ok
test telemetry::tests::pipeline_e2e_exports_traces_to_fake_tinybird ... ok
test telemetry::tests::sampling_keeps_errors_even_when_ratio_low ... ok
test telemetry::tests::scraper_contract::scraper_otlp_json_decodes_with_gateway_serde_and_encodes_to_rows ... ok
test telemetry::tests::signal_tag_round_trips_all_variants ... ok
test telemetry::tests::pipeline_e2e_exports_metrics_to_fake_tinybird ... ok
test telemetry::tests::telemetry_signal_as_str_is_canonical_lowercase ... ok
test telemetry::tests::timestamp_has_nano_precision ... ok
test telemetry::tests::timestamps_match_clickhouse_datetime64_nine_format ... ok
test telemetry::tests::trace_encoder_matches_tinybird_row_shape ... ok
test telemetry::tests::traces_emit_exactly_the_jsonpaths_declared_in_datasources_ts ... ok
test telemetry::tests::clickhouse_breaker_sheds_after_threshold_failures ... ok
test telemetry::tests::wal_round_trips_frame ... ok
test telemetry::tests::slow_clickhouse_lane_does_not_block_cosharded_tinybird_org ... ok
test telemetry::tests::pipeline_exports_ready_org_to_clickhouse_without_tinybird_calls ... ok
test telemetry::tests::wal_partial_drain_advances_cursor_without_truncating ... ok
test telemetry::tests::wal_truncates_after_full_drain_allowing_further_appends ... ok

test result: ok. 36 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 1.75s

     Running unittests src/bin/load_test.rs (target/debug/deps/load_test-661a0aa1eb3f6d6d)

running 0 tests

test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

     Running unittests src/main.rs (target/debug/deps/maple_ingest-c33bf80c577edb95)

running 38 tests
test autumn::tests::allowed_only_no_balance_field ... ok
test autumn::tests::flat_hardcap_depleted_blocks ... ok
test autumn::tests::flat_hardcap_with_remaining_allows ... ok
test autumn::tests::flat_overage_allows ... ok
test autumn::tests::flat_sub_one_gb_remaining_still_allows ... ok
test autumn::tests::flat_unlimited_allows ... ok
test autumn::tests::nested_balance_object_depleted_blocks ... ok
test autumn::tests::nested_overage_allows ... ok
test autumn::tests::nested_balance_object_with_remaining_allows ... ok
test autumn::tests::null_balance_no_subscription_blocks ... ok
test tests::api_error_from_pipeline_maps_variants_to_status ... ok
test autumn::tests::unrecognized_shape_returns_none ... ok
test tests::api_error_kind_maps_status_to_stable_label ... ok
test tests::clickhouse_target_resolver_decrypts_current_schema_password ... ok
test tests::clickhouse_target_resolver_rejects_password_over_http ... ok
test tests::clickhouse_destination_uses_native_pipeline_even_in_forward_mode ... ok
test tests::clickhouse_destination_is_terminal_in_dual_mode ... ok
test tests::cloudflare_log_record_maps_body_severity_and_attributes ... ok
test tests::cloudflare_ndjson_payload_parses_multiple_records ... ok
test tests::cloudflare_timestamps_support_rfc3339_unix_and_unix_nano ... ok
test tests::clickhouse_target_resolver_requires_current_schema ... ok
test tests::cloudflare_validation_payload_is_detected ... ok
test tests::decrypt_aes256_gcm_matches_node_crypto_fixture ... ok
test tests::enrichment_overwrites_tenant_fields ... ok
test tests::hash_is_deterministic ... ok
test tests::extract_ingest_key_returns_sentinel_literal_unchanged ... ok
test tests::rejection_span_status_is_error_only_for_5xx ... ok
test tests::resolve_ingest_key_keeps_stale_schema_on_managed_native_path ... ok
test tests::resolve_connector_refreshes_routing_before_auth_cache_expires ... ok
test tests::resolve_ingest_key_returns_none_when_hash_missing ... ok
test tests::resolve_ingest_key_refreshes_routing_before_auth_cache_expires ... ok
test tests::resolve_ingest_key_returns_self_managed_false_when_no_settings_row ... ok
test tests::resolve_ingest_key_returns_self_managed_true_when_active_settings_row ... ok
test tests::sentinel_token_matches_only_exact_literal ... ok
test tests::tinybird_destination_keeps_forward_mode_on_forward_path ... ok
test autumn::tests::fails_open_on_transport_error ... ok
test tests::resolve_ingest_key_serves_last_known_routing_when_refresh_fails ... ok
test tests::forward_mode_switches_ready_org_to_clickhouse_without_forwarding_again ... ok

test result: ok. 38 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.29s

   Doc-tests maple_ingest

running 0 tests

test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s