Task/perf benchmark methodology#59
Merged
Merged
Conversation
…y page, fix dead CTA The home-page performance section previously showed unsubstantiated figures and compared against unrelated stacks, and its 'View Technical Methodology' CTA was a dead <button> pointing nowhere. - Add benchmarks/ harness: raw FastAPI baseline vs FastAPI Startkit serving identical TechEmpower-style endpoints, driven by ApacheBench under keep-alive, reporting peak throughput across trials. - Run it and publish real measured numbers (Apple M3 Pro, single worker): Startkit is within +/-3% of raw FastAPI, i.e. no measurable overhead. - Add docs/benchmarks.md methodology page documenting test types, method, load-testing hygiene, limitations, and reproduction steps. - Rewrite PerformanceBenchmarks.vue to show the honest overhead delta and wire the CTA to /docs/benchmarks. - Add the page to the sidebar nav.
Drop the unsubstantiated 99.9% Uptime SLA stat from the hero section and the Observability Stack module from the core modules grid. Observability is documented in the Logging docs, so it should not be advertised as a standalone home-page claim. Adjust the surrounding grid layouts.
Keep the third core-module slot but repurpose it from the observability stack (documented in the Logging docs) to an AI & Agents module, describing LLM providers, tool orchestration, and streaming on the async core via the container/provider pattern.
Drop the remaining <20ms P99 Latency stat; with no substantiated metrics left, remove the hero stats block entirely.
Delete the local benchmark harness (benchmarks/) and rewire the docs to
the external github.com/fastapi-startkit/web-frameworks harness, which is
now the single source of truth for benchmarking.
- Replace the old local-harness figures (parity / no measurable overhead)
with the real web-frameworks relative-delta results for the shared
GET /, GET /user/{id}, and POST /user routes, attributed to the source.
- Update methodology to match how web-frameworks measures (Docker
python:3.14-slim, uvicorn --workers=nproc, oha, keepalive, c64/c256) and
document the FastAPI 0.139 include_router overhead behind POST /user.
- Point the Reproduce section at web-frameworks; remove the cd benchmarks
/ bench.py steps and all dangling harness references.
- Update PerformanceBenchmarks.vue so the home-page card matches the page
(per-route deltas, honest framing, web-frameworks attribution).
- Rename the third core module to 'AI & MCP' and note it is built on top
of LangChain (MCP servers, LLM providers, streaming) instead of the
generic 'AI & Agents' framing.
- Replace the removed hero SLA stat with a '100% Test Coverage' stat.
- Soften unsubstantiated performance superlatives in the hero, footer,
and core-modules copy ('fastest possible', 'high-performance',
'engineered for reliability/speed') to defensible wording.
Contributor
Author
|
Superseded by curated PR #58, which brings over the meaningful leftover changes without reverting main-ahead docs (agent.md/fastapi.md/getting-started.md). Closing to prevent those reverts. |
# Conflicts: # .vitepress/theme/components/HeroSection.vue # .vitepress/theme/components/SiteFooter.vue
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.