Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
32 changes: 0 additions & 32 deletions .github/workflows/security.yml
Original file line number Diff line number Diff line change
Expand Up @@ -100,35 +100,3 @@ jobs:

- name: Typecheck MCP
run: bun run typecheck

web-security:
runs-on: ubuntu-24.04
timeout-minutes: 30
defaults:
run:
working-directory: web
steps:
- name: Checkout
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0
with:
persist-credentials: false

- uses: jdx/mise-action@5228313ee0372e111a38da051671ca30fc5a96db
with:
version: 2026.7.1
install: true
cache: true
- name: Install web dependencies
run: mise run setup

- name: Audit web dependencies
run: bun audit

- name: Typecheck web app
run: bun run typecheck

- name: Lint web app
run: bun run lint

- name: Build web app
run: mise x -- bun run build
761 changes: 0 additions & 761 deletions bun.lock

Large diffs are not rendered by default.

7 changes: 7 additions & 0 deletions docs/CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,13 @@ All notable changes to this project will be documented in this file.
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).

## [Unreleased]

### Removed
- Remove the standalone DocPull website, its Next.js workspace, and web-only
CI/release gates. GitHub, PyPI, and the MCP Registry remain the canonical
public product surfaces.

## [6.3.0] - 2026-07-16

### Added
Expand Down
2 changes: 1 addition & 1 deletion docs/release-claims.json
Original file line number Diff line number Diff line change
Expand Up @@ -110,7 +110,7 @@
]
},
{
"area": "Web/docs copy",
"area": "Docs/public contract",
"claim": "Release-facing documentation is checked against the public surface registry and claim-audit manifest.",
"references": [
{"type": "code", "path": "src/docpull/surface.py"},
Expand Down
12 changes: 6 additions & 6 deletions docs/release-post-v6.3.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# DocPull 6.3: faster local evidence workflows and a complete product surface
# DocPull 6.3: faster local evidence workflows and a complete workflow surface

DocPull 6.3 preserves the 6.2 evidence/workflow contracts while reducing the
cost of using them. Public Python exports now load lazily, identical concurrent
Expand All @@ -19,11 +19,11 @@ Every input receives exactly one coverage result. Missing evidence is a
`coverage_gap`, never a negative ownership or independence claim, and all
candidates remain observations until a downstream human approves them.

The repository now includes a production-ready DocPull site with pricing,
privacy, terms, machine-readable `llms.txt`, metadata, robots, sitemap, icons,
and reusable launch assets. The complete Python, MCP, and web surface is covered
by locked dependencies, automatic CI/security triggers, lint, type checks,
audits, tests, and reproducible release builds.
The standalone product website that initially shipped with 6.3 was subsequently
decommissioned. GitHub, PyPI, and the MCP Registry are the canonical public
surfaces; the Python and MCP packages remain covered by locked dependencies,
automatic CI/security triggers, lint, type checks, audits, tests, and
reproducible release builds.

No migration is required for existing packs or consumers. CLI, Python SDK,
MCP, JSON Schema, `intelligence.bundle.v1`, change-event, and
Expand Down
26 changes: 11 additions & 15 deletions docs/v6-directory-submission-plan.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,21 +20,19 @@ Ready:
- GitHub README has package links, MCP install instructions, and clear install
CTAs.

Fixed in this launch-prep pass, deploy before submission:
Canonical owned surfaces:

- `/pricing`
- `/privacy`
- `/terms`
- `/llms.txt`
- GitHub repository and release pages.
- PyPI package page.
- Official MCP Registry metadata.
- No standalone DocPull website is operated or required.

Still recommended before Product Hunt:

- Record a 45-60 second terminal demo from `pip install docpull` to `docpull
sync`, `docpull diff`, and `docpull export context-pack --target cursor`.
- Refresh launch screenshots from the deployed v6 site after the policy pages
are live.
- Decide whether Product Hunt should launch from the website or GitHub URL. Use
the website unless the launch story is explicitly "open-source repo first."
- Use terminal, GitHub, and package-install screenshots for launch materials.
- Use the GitHub repository URL for Product Hunt and other directory listings.

## Positioning

Expand Down Expand Up @@ -81,12 +79,10 @@ Third-party API stance:

Do this first because every directory links back here.

1. Deploy the web changes for `/pricing`, `/privacy`, `/terms`, and
`/llms.txt`.
2. Confirm the deployed routes return 200.
3. Publish or tag the v6 GitHub release if not already done.
4. Confirm PyPI shows `6.0.0`.
5. Pin the v6 X thread after posting.
1. Confirm the GitHub repository and latest release are public.
2. Confirm PyPI shows the current supported version.
3. Confirm the official MCP Registry metadata is published.
4. Pin the v6 X thread after posting.

### Batch 1: Developer and MCP surfaces

Expand Down
11 changes: 7 additions & 4 deletions docs/v6-directory-todo.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,11 +7,13 @@
- MCP.Directory: watch email/listing status because the form cleared after submit but did not show a reliable success banner.
- HN: add the posted Show HN item URL to `docs/v6-directory-tracker.csv`.

## Site Readiness
## Owned-Surface Readiness

- Deploy the restored web routes for `/privacy`, `/terms`, `/pricing`, and `/llms.txt`; they were implemented locally on 2026-07-06 but still need live 200 checks.
- Keep GitHub, PyPI, and official MCP Registry metadata current. DocPull does
not operate a standalone website.
- Add launch screenshots and a 45-60 second demo video before Product Hunt, DevHunt, Fazier, Microlaunch, or Uneed.
- Product Hunt: schedule only after the launch assets are ready and the founder account is warmed up; do not ask for upvotes, ask for feedback.
- Product Hunt: link to GitHub, schedule only after the launch assets are ready
and the founder account is warmed up; do not ask for upvotes, ask for feedback.

## Package / Registry Work

Expand All @@ -24,7 +26,8 @@

- AgentLocker: create/verify an account, then submit through `/agent/submit`.
- OpenTools: sign in and look for an authenticated registry submission path; no public endpoint was found.
- DevHunt, Fazier, Microlaunch, Uneed: retry after policy/pricing pages and launch assets are deployed.
- DevHunt, Fazier, Microlaunch, Uneed: use the GitHub URL and retry after launch
assets are ready.

## Content / Community

Expand Down
12 changes: 6 additions & 6 deletions docs/v6-directory-tracker.csv
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
Directory,Tier,URL,Category,Priority,Status,Submission Date,Live URL,Backlink Verified,Positioning Variant,Tags,Account Needed,Notes
Product Hunt,Flagship,https://www.producthunt.com/launch,Launch,High,Blocked - launch assets/deployed pages,,,,Startup,Engineering & Development; AI Agents; LLMs,Yes,"Schedule after /privacy, /terms, /pricing, screenshots, and a 45-60s demo are live; deployed policy/pricing pages returned 404 on 2026-07-04. Local web routes for /privacy, /terms, /pricing, and /llms.txt were restored on 2026-07-06; deploy and verify live 200 responses next."
Product Hunt,Flagship,https://www.producthunt.com/launch,Launch,High,Blocked - launch assets,,,,Startup,Engineering & Development; AI Agents; LLMs,Yes,"Use the GitHub repository URL; schedule after terminal/package screenshots and a 45-60s demo are ready. DocPull intentionally has no standalone website."
Hacker News Show HN,Community,https://news.ycombinator.com/submit,Developer community,High,Submitted,2026-07-04,,,Dev,Show HN; Python; open source; AI agents,Yes,Posted by founder from zacharyr0th; add item URL when available.
DevHunt,Launch,https://devhunt.org/,Developer tools,High,Ready after deploy,,,,Dev,Developer Tools; Open Source; AI; CLI; Python; MCP,Yes,Best non-PH launch fit.
Official MCP Registry,MCP,https://modelcontextprotocol.io/registry/about,MCP registry,High,Published,2026-07-04,https://registry.modelcontextprotocol.io/v0.1/servers?search=io.github.raintree-technology/docpull,Yes,MCP,MCP; public web context; developer tools; web extraction,Yes,"Published active/latest registry entry for io.github.raintree-technology/docpull version 6.0.1 at 2026-07-04T22:00:42Z after PyPI 6.0.1 marker release and GitHub OIDC workflow publish; reverified 2026-07-07."
DevHunt,Launch,https://devhunt.org/,Developer tools,High,Ready after assets,,,,Dev,Developer Tools; Open Source; AI; CLI; Python; MCP,Yes,Best non-PH launch fit; use the GitHub repository URL.
Official MCP Registry,MCP,https://modelcontextprotocol.io/registry/about,MCP registry,High,Published,2026-07-04,https://registry.modelcontextprotocol.io/v0.1/servers?search=io.github.raintree-technology/docpull,Yes,MCP,MCP; public web context; developer tools; web extraction,Yes,"Published active/latest registry entry for io.github.raintree-technology/docpull version 6.3.0 via the GitHub OIDC workflow on 2026-07-16."
Glama MCP,MCP,https://glama.ai/mcp/servers,MCP registry,High,Published,2026-07-04,https://glama.ai/mcp/servers/raintree-technology/docpull,Yes,MCP,MCP server; web scraping; documentation; RAG,Yes,"Live listing and score badge reverified on 2026-07-07; punkpeye/awesome-mcp-servers #9276 includes the required Glama badge and check-submission passes."
Smithery,MCP,https://smithery.ai/docs/build/publish,MCP registry,High,Blocked - packaging mismatch,2026-07-04,,,MCP,MCP; local server; developer tools,Yes,Current Smithery publish docs support public HTTPS servers or local MCPB bundles; DocPull is a PyPI stdio server. Needs MCPB bundle or remote Streamable HTTP server before submission.
mcp.so,MCP,https://mcp.so/,MCP directory,High,Submitted,2026-07-04,https://github.com/chatmcp/mcpso/issues/3016,,MCP,MCP server; web scraping; documentation; developer tools,Yes,"Submitted via GitHub issue from zacharyr0th with install command, stdio config, PyPI link, and 17k+ downloads; issue still open with no maintainer comments on 2026-07-07, and common mcp.so listing paths still show Project not found content."
Expand Down Expand Up @@ -31,9 +31,9 @@ awesome-ai-devtools,Awesome list,https://github.com/jamesmurdza/awesome-ai-devto
Awesome-LLMOps,Awesome list,https://github.com/tensorchord/Awesome-LLMOps,LLMOps,Medium,Prep after tutorial,,,,AI,LLMOps; RAG; data management; search,Yes,Submit after a short RAG/context-dependency tutorial exists.
Awesome-RAG,Awesome list,https://github.com/Danielskry/Awesome-RAG,RAG,Medium,Prep after tutorial,,,,AI,RAG; context packs; data ingestion; citations,Yes,Better with an example RAG pack or tutorial link.
awesome-harness-engineering,Awesome list,https://github.com/ai-boost/awesome-harness-engineering,Agent harness,Medium,Prep after article,,,,AI,agent harness; context delivery; MCP; verification,Yes,Likely stronger as a technical article/resource submission than a bare repo.
Uneed,Launch,https://www.uneed.best/submit-a-tool,Product directory,Medium,Blocked - account/CAPTCHA/payment path,,,,Startup,Developer Tools; AI; Open Source,Yes,Static page exposes Turnstile/Supabase account flow; retry after deployed policy/pricing pages and manual account/CAPTCHA step.
Microlaunch,Launch,https://microlaunch.net/,Product directory,Medium,Blocked - account/launch prep,,,,Startup,Developer Tools; AI Tools; Productivity,Yes,Account-based launch surface; retry after deployed policy/pricing pages and launch assets.
Fazier,Launch,https://fazier.com/,Product directory,Medium,Blocked - sign-in/launch prep,,,,Startup,Developer Tools; AI; Open Source,Yes,Sign-in/join gated launch surface; retry after deployed policy/pricing pages and launch assets.
Uneed,Launch,https://www.uneed.best/submit-a-tool,Product directory,Medium,Blocked - account/CAPTCHA/payment path,,,,Startup,Developer Tools; AI; Open Source,Yes,Static page exposes Turnstile/Supabase account flow; use the GitHub URL after the manual account/CAPTCHA step.
Microlaunch,Launch,https://microlaunch.net/,Product directory,Medium,Blocked - account/launch prep,,,,Startup,Developer Tools; AI Tools; Productivity,Yes,Account-based launch surface; retry with the GitHub URL after launch assets are ready.
Fazier,Launch,https://fazier.com/,Product directory,Medium,Blocked - sign-in/launch prep,,,,Startup,Developer Tools; AI; Open Source,Yes,Sign-in/join gated launch surface; retry with the GitHub URL after launch assets are ready.
AlternativeTo,SEO,https://alternativeto.net/,Software alternatives,Medium,Defer,,,,SaaS,web scraping; developer tools; open source,Yes,Better after comparison pages are more complete.
SaaSHub,SEO,https://www.saashub.com/submit/list,SaaS directory,Low,Defer,,,,SaaS,developer tools; web scraping; AI tools,Yes,Only after alternatives/comparison pages are ready.
Future Tools,AI directory,https://futuretools.io/submit-a-tool,AI directory,Low,Defer,,,,AI,AI agents; developer tools; research,Yes,"Generic AI audience, use after developer surfaces."
Expand Down
11 changes: 5 additions & 6 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -8,19 +8,18 @@
"test": "uv run --locked --all-extras python -m pytest -q && bun run --cwd mcp test",
"test:unit": "uv run --locked --all-extras python -m pytest -q",
"test:e2e": "uv run --locked python scripts/real_feature_smoke.py --json",
"lint": "uv run --locked --extra dev python scripts/sync_release_metadata.py --check && uv run --locked --extra dev python scripts/generate_contract_schemas.py --check && uv run --locked --extra dev python -m ruff check . && uv run --locked --extra dev python -m ruff format --check . && bun run --cwd web lint",
"lint:fix": "uv run --locked --extra dev python -m ruff check . --fix && uv run --locked --extra dev python -m ruff format . && bun run --cwd web lint:fix",
"lint": "uv run --locked --extra dev python scripts/sync_release_metadata.py --check && uv run --locked --extra dev python scripts/generate_contract_schemas.py --check && uv run --locked --extra dev python -m ruff check . && uv run --locked --extra dev python -m ruff format --check .",
"lint:fix": "uv run --locked --extra dev python -m ruff check . --fix && uv run --locked --extra dev python -m ruff format .",
"format": "uv run --locked --extra dev python -m ruff format .",
"typecheck": "uv run --locked --extra dev python -m mypy src && bun run --cwd mcp typecheck && bun run --cwd web typecheck",
"typecheck": "uv run --locked --extra dev python -m mypy src && bun run --cwd mcp typecheck",
"validate": "bun run lint && bun run typecheck && bun run test",
"validate:full": "bun run validate && bun run build && bun run --cwd web build"
"validate:full": "bun run validate && bun run build"
},
"overrides": {
"postcss": "^8.5.10"
},
"workspaces": [
"mcp",
"web"
"mcp"
],
"engines": {
"node": "24.x"
Expand Down
32 changes: 4 additions & 28 deletions scripts/release_a_plus_check.py
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@
"Auth",
"Monitor",
"Security/static quality",
"Web/docs copy",
"Docs/public contract",
"Release hygiene",
)

Expand All @@ -56,11 +56,10 @@
"pip_audit",
"gitleaks",
"mcp_bun_audit",
"web_bun_audit",
"package_build",
"twine_check",
),
"Web/docs copy": ("claim_audit", "web_typecheck", "web_lint", "web_build"),
"Docs/public contract": ("claim_audit",),
"Release hygiene": ("git_status_clean", "release_metadata"),
}

Expand Down Expand Up @@ -140,7 +139,7 @@ def run_scorecard(args: argparse.Namespace) -> dict[str, Any]:
gates.append(GateResult(name=name, status="skip", note="--skip-commands was supplied"))

gate_by_name = {gate.name: gate for gate in gates}
areas = _grade_areas(gate_by_name, smoke_payload, area_gates=_active_area_gates(repo))
areas = _grade_areas(gate_by_name, smoke_payload, area_gates=AREA_GATES)
all_a_plus = all(area["grade"] == "A+" for area in areas)
payload = {
"schema_version": SCORECARD_SCHEMA_VERSION,
Expand Down Expand Up @@ -237,16 +236,6 @@ def _run_command_gates(
"required",
),
]
if (repo / "web" / "package.json").exists():
commands.extend(
[
("web_bun_install", ["bun", "install", "--frozen-lockfile"], repo / "web", 300, "supporting"),
("web_bun_audit", ["bun", "audit"], repo / "web", 180, "required"),
("web_typecheck", ["bun", "run", "typecheck"], repo / "web", 240, "required"),
("web_lint", ["bun", "run", "lint"], repo / "web", 240, "required"),
("web_build", ["bun", "run", "build"], repo / "web", 300, "required"),
]
)
if (repo / "mcp" / "package.json").exists():
commands.extend(
[
Expand Down Expand Up @@ -508,17 +497,6 @@ def _grade_areas(
return areas


def _active_area_gates(repo: Path) -> dict[str, tuple[str, ...]]:
"""Return gates for product surfaces that exist in this checkout."""
active = dict(AREA_GATES)
if not (repo / "web" / "package.json").is_file():
active.pop("Web/docs copy")
active["Security/static quality"] = tuple(
name for name in active["Security/static quality"] if name != "web_bun_audit"
)
return active


def _missing_smoke_markers(area: str, smoke_payload: dict[str, Any] | None) -> list[str]:
markers = A_PLUS_SMOKE_MARKERS.get(area, ())
if not markers:
Expand Down Expand Up @@ -595,8 +573,6 @@ def _planned_command_names(repo: Path) -> list[str]:
"twine_check",
"gitleaks",
]
if (repo / "web" / "package.json").exists():
names.extend(["web_bun_install", "web_bun_audit", "web_typecheck", "web_lint", "web_build"])
if (repo / "mcp" / "package.json").exists():
names.extend(["mcp_bun_install", "mcp_bun_audit", "mcp_bun_test", "mcp_typecheck"])
return names
Expand All @@ -623,7 +599,7 @@ def main(argv: list[str] | None = None) -> int:
args = create_parser().parse_args(argv)
if args.plan_only:
repo = Path(args.repo).resolve()
areas = list(_active_area_gates(repo))
areas = list(AREA_GATES)
commands = _planned_command_names(repo)
smoke_command = (
"scripts/real_feature_smoke.py --json --full-mcp --strict-ci --auth-matrix --monitor-soak-minutes"
Expand Down
8 changes: 3 additions & 5 deletions tests/test_ci_policy.py
Original file line number Diff line number Diff line change
Expand Up @@ -215,16 +215,14 @@ def test_security_and_publish_bandit_scan_scripts() -> None:
assert "python -m bandit -q -c pyproject.toml -r src scripts" in publish


def test_security_workflow_uses_locked_dependencies_and_checks_web() -> None:
def test_security_workflow_uses_locked_dependencies_without_a_web_app_job() -> None:
security = (WORKFLOW_DIR / "security.yml").read_text()
release_requirements = (REPO_ROOT / "requirements-release.txt").read_text()

assert "uv sync --locked --extra dev" in security
assert "setuptools==83.0.0" in release_requirements
assert "working-directory: web" in security
assert "Audit web dependencies" in security
assert "Lint web app" in security
assert "Build web app" in security
assert "working-directory: web" not in security
assert "web-security:" not in security


def test_workflows_use_module_entrypoints_for_python_tooling() -> None:
Expand Down
Loading
Loading