Skip to content

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

472 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

HTTP Benchmarks

Side-by-side benchmarks. Same API.
Real databases. No toy endpoints.

Grafana Node.js Bun Deno Go Python TypeScript PostgreSQL MongoDB Redis Cassandra Express Fastify NestJS Hono Elysia Oak Chi Gin Fiber FastAPI

Results (200 Concurrency, 5s per endpoint) 📊

Benchmark Results

Stack Map 📦

Inside every container the server listens on the canonical port 8080 (baked as PORT in the Dockerfile, PLAN §6 rule 1); the benchmark client maps a dynamic host port. The port below is each server's assigned 2LRFF host port (dev_port in its bench.json, PLAN §6), used by its .env.example and as the code default in the Go/Rust/Kotlin/Zig servers; Node/Deno just dev binds the canonical 8080 unless PORT is set (e.g. via cp .env.example .env).

Folder Runtime Framework Dev port
benchmark Go 1.27rc1 - -
servers/go-stdlib Go 1.27rc1 net/http (stdlib) 21001
servers/go-chi Go 1.27rc1 Chi 5.3.0 21002
servers/go-gin Go 1.27rc1 Gin 1.12.0 21003
servers/go-fiber Go 1.27rc1 Fiber 3.4.0 21004
servers/go-echo Go 1.27rc1 Echo 4.13.4 21005
servers/ts-express Node 26.4.0 Express 5.2.1 22001
servers/ts-nestjs Node 26.4.0 NestJS 11.1.27 22002
servers/ts-fastify Node 26.4.0 Fastify 5.9.0 22003
servers/ts-honojs Node 26.4.0 Hono 4.12.27 22005
servers/ts-bun-honojs Bun 1.3.14 Hono 4.12.27 22105
servers/ts-bun-elysia Bun 1.3.14 Elysia 1.4.29 22106
servers/ts-deno-oak Deno 2.9.1 Oak 17.2.0 22204
servers/ts-deno-honojs Deno 2.9.1 Hono 4.12.27 22205
servers/py-fastapi Python 3.14.6 FastAPI >=0.128 23001
servers/py-django Python 3.14.6 Django >=6.0 23002
servers/py-flask Python 3.14.6 Flask 3.x 23003
servers/rs-axum Rust 1.96.1 Axum 0.8.9 24001
servers/rs-actix Rust 1.96.1 Actix Web 4.14.0 24002
servers/kt-ktor Kotlin 2.3.21 Ktor 3.5.1 25001
servers/kt-spring-boot Kotlin 2.3.21 Spring Boot 4.1.0 25002
servers/zig Zig 0.16 http.zig 26001

Pinned dependencies 📌

Deliberate prerelease pins (PLAN §10), exempt from the blanket just update bump; each tracks the newest release of its dist-tag channel via scripts/update.mts, never the stable latest line:

Package Pinned at Channel Why
typescript 7.0.1-rc rc TypeScript 7 native tsc (latest is still 6.x)
drizzle-orm 1.0.0-rc.4 rc drizzle 1.0 RC (latest is still 0.45.x)

Held back: bson is pinned to 7.2.0 for the whole workspace via pnpm-workspace.yaml overrides. bson 7.3.x calls v8.startupSnapshot.isBuildingSnapshot() at import time, which Bun 1.3.14 ships as a throwing stub (NotImplementedError), crashing the server on boot. Since PR #26 mongodb is a single shared dependency (@bench/shared), and pnpm resolves one bson version for every consumer (overrides are graph-path scoped, not per-member), so the Node servers pin to 7.2.0 too — pnpm cannot scope an override to just the Bun members. 7.2.0 sits inside mongodb's own ^7.2.0 range; drop the override once Bun implements the stub.

NestJS dev mode tradeoff: under TS7 nest start --watch is gone, so ts-nestjs's dev compiles with tsc then runs node --watch dist/main.js. That restarts the server on rebuild but does not watch-recompile sources — edit, re-run tsc (or just dev ts-nestjs) to pick up changes.

Quick Start 🚀

just benchmark                # Run benchmark (interactive mode)
just benchmark --servers=a,b  # Run benchmark for specific servers only
just dev ts-bun-honojs        # Start dev server (ts-bun-honojs, ts-express, go-chi, etc.)

First pnpm install over a checkout that predates the pnpm workspace can want to purge the old top-level node_modules. pnpm-workspace.yaml sets confirmModulesPurge: false, so the install proceeds non-interactively (no CI=true or TTY needed) instead of aborting with ERR_PNPM_ABORTED_REMOVE_MODULES_DIR_NO_TTY.

Configuration ⚙️

Variable Default Description
ENV dev dev enables logger, prod disables it
HOST 0.0.0.0 IP or localhost (mapped to 0.0.0.0)
PORT See Stack Map Server port

Benchmark config is JSON-only and lives at config/config.json.

API Surface 🌐

Base Routes

Method Route Response
GET / { "hello": "world"}
GET /health OK (text/plain)

Params Routes (/params/*)

Method Route Description
GET /params/search Query q (trim, default none) and limit (safe int, default 10)
GET /params/url/:val Returns { "dynamic": "<val>" }
GET /params/header Reads X-Custom-Header (trim, default none)
POST /params/body Validates JSON object (no array/null), returns { "body": <parsed> }
GET /params/cookie Reads cookie foo (trim, default none), sets cookie bar
POST /params/form Supports urlencoded/multipart, returns { "name": "<trim>", "age": <int> }
POST /params/file Multipart file (max 1MB, text/plain only), returns { "filename", "size", "content" }

Database Routes (/db/{database}/*)

Supported databases: postgres, mongodb, redis, cassandra

Method Route Description
GET /db/{database}/health Database health check
POST /db/{database}/users Create user { "name", "email", "favoriteNumber?" } (201)
GET /db/{database}/users/{id} Get user by ID (200 or 404)
PATCH /db/{database}/users/{id} Update user fields (200 or 404)
DELETE /db/{database}/users/{id} Delete user by ID (200 or 404)
DELETE /db/{database}/users Delete all users (200)
DELETE /db/{database}/reset Reset database (200)

Error Responses ⚠️

All errors return JSON { "error": "<message>" }.

Status Messages
400 invalid JSON body, invalid form data, invalid multipart form data, file not found in form data
404 not found
413 file size exceeds limit
415 only text/plain files are allowed, file does not look like plain text
500 internal error

Databases 🗄️

All servers connect to all 4 databases with the same user schema.

Database ID Type
PostgreSQL UUID v7
MongoDB ObjectId (native)
Redis UUID v7
Cassandra UUID v7

User schema: id, name, email, favoriteNumber (optional)

Grafana 📊

Metrics are exported to a dedicated metrics PostgreSQL (separate from the benchmarked postgres) and visualized in Grafana during benchmarks. Run history is durable: the metrics volume survives stack restarts, so runs can be compared across weeks.

Service URL Username Password
Grafana http://localhost:20090 admin 123456
metrics-postgres postgres://localhost:20091 benchmark benchmark

Exported Metrics

Aggregate tables carry exact numbers computed from the full in-memory result set before any sampling; request_events is sampled raw drilldown only. Canonical queries live in infra/grafana/queries/.

Table Contents Key columns
runs one row per run: sample rate + write accounting run_id, started_at, finished_at
endpoint_stats exact per-endpoint aggregates (rps, avg/p50/p95/p99/p99.9, open-mode fields) run_id, server, endpoint, method, source
sequence_stats exact per-sequence aggregates (full-sequence durations) run_id, server, sequence_id, database
resource_samples container memory/CPU min/avg/max per server run run_id, server, source, database (DB only)
request_events sampled raw request/sequence events with real timestamps run_id, server, endpoint, source, database

Development 🛠️

Per-Target Commands

All verification commands accept an optional target (default: all).

just typecheck go-chi        # Type check only go-chi
just fmt ts-bun-honojs       # Format only ts-bun-honojs
just lint py-fastapi         # Lint only py-fastapi
just verify ts-express       # Full verification for ts-express

Valid targets: ts-express, ts-nestjs, ts-fastify, ts-deno-oak, ts-bun-honojs, ts-bun-elysia, go-chi, go-gin, go-fiber, py-fastapi, zig, benchmark, root (the servers/*/bench.json name fields, plus benchmark and root)

Full Command Reference

just benchmark           # Run benchmark (interactive mode)
just install             # Install all dependencies
just typecheck           # Type/compile check all projects
just fmt                 # Format all code
just lint                # Lint all code
just verify              # Non-mutating gate: typecheck -> format-check -> lint
just images              # Build all Docker images
just clean               # Remove build artifacts and node_modules
just remove-images       # Remove Docker images
just grafana-up          # Start Grafana/metrics-postgres stack
just grafana-down        # Stop Grafana/metrics-postgres stack (volumes are kept)
just db-up               # Start database stack
just db-down             # Stop database stack

About

⚡️ A side-by-side HTTP performance comparison focused on identical behavior, real database flows, and metrics that enable fair cross-runtime evaluation.

Resources

Stars

Watchers

Forks

Releases

Packages

Contributors

Languages