ModelCard Forge

v0.1.0 — turn real experiment metrics, dataset manifests, confusion matrices, and version evidence into auditable model cards rather than filling in a blank template.

• Every rendered conclusion identifies its input path and SHA-256 digest.
• Missing evidence is rendered as unknown / unverified, never invented.
• One command emits Markdown, HTML, and a Hugging Face README fragment offline.

python -m pip install -e '.[dev]'
modelcard-forge generate examples/complete -o output

The resulting output/model-card.md contains metrics such as accuracy: 0.92 and their source digest. See examples/complete for the real input and examples/generated/complete for generated output.

What it does

ModelCard Forge validates evidence, inventories unknown fields, derives metric and slice summaries, records scope and limitations, and produces Markdown, HTML, and a Hugging Face-ready Markdown fragment. It is a local CLI; it does not upload data.

Non-goals

It does not train models, calculate fairness claims without supplied slice evidence, or replace human safety review.

Architecture

The pure domain layer interprets evidence, adapters read files, and the service layer orchestrates rendering. Details: docs/ARCHITECTURE.md.

Installation and usage

Requires Python 3.12–3.14. Run modelcard-forge generate INPUT_DIR -o OUTPUT_DIR. INPUT_DIR may contain metrics.json, dataset_manifest.json, confusion_matrix.csv, and version.json; all are optional but absence is visible in output.

make demo
make verify
make package
make release-check

Sample data

Both synthetic samples are committed under examples/: complete has all expected evidence and missing-bias intentionally omits slice/bias evidence. They contain no model weights or personal data and are licensed under MIT with this repository.

Testing

make verify runs Ruff, mypy, unit/integration/E2E tests, coverage, and a build. docs/BENCHMARK.md records the sample-scale measurement method.

Privacy and security

Input files stay local. The renderer redacts values under keys such as token, password, and secret before they can reach report metadata. Review docs/PRIVACY_AND_SECURITY.md before handling sensitive data.

Roadmap and contribution

Next: schema adapters for common ML evaluation tools and signed evidence bundles. See CONTRIBUTING.md and docs/RELEASE_CHECKLIST.md.

Related projects and differentiation

A public-repository sample scan found no active project with the same name and highly isomorphic workflow. TensorFlow Model Card Toolkit is the closest adjacent tool; ModelCard Forge focuses on an offline evidence ledger that makes every conclusion point to a path and digest. See docs/COMPETITOR_SCAN.md.

FAQ

Can it make a fairness conclusion? Only from supplied slice evidence; otherwise the report says the assessment is unverified. Can I use it with Hugging Face? Yes: copy the generated huggingface-readme.md into a repository README.