Skip to content

perf: do per-run work once in update instead of once per plugin - #2

Merged
Alishahzad1903 merged 1 commit into
mainfrom
alifsayalee/faster-update
Jul 30, 2026
Merged

perf: do per-run work once in update instead of once per plugin#2
Alishahzad1903 merged 1 commit into
mainfrom
alifsayalee/faster-update

Conversation

@alifsayalee

Copy link
Copy Markdown
Member

Fixes apimatic/contextmatic-crawler#31.

The problem

updateAll re-installs each recorded plugin by calling the full install path. That meant per plugin:

  1. re-fetch marketplace.json
  2. git clone the entire marketplace repo into a temp dir, sparse-check out one folder, then delete it
  3. claude plugin marketplace list + claude plugin marketplace update — two CLI invocations and another network fetch
  4. actually install the plugin

Steps 1–3 are identical for every plugin from the same repository. Only step 4 is genuinely per-plugin.

The compounding part the issue doesn't name: the clone in step 2 is proportional to the whole marketplace (1,850 files, 29 plugins, 1,368 skills), not to the one plugin you want. So the real cost is plugins installed × marketplace size — per-plugin time gets worse as the marketplace grows even if you install nothing new. That's consistent with the reporter measuring 9.6s/plugin on macOS against the original 6.5s: same code, bigger repo.

The fix

A per-run session (src/session.js) holding the three shared things: the registry, an open repo handle, and the Claude marketplace registration. All keyed by repo@ref, because manifest entries each record their own and an update may legitimately span more than one marketplace.

  • updateAll creates one session and passes it to every plugin.
  • A lone install gets a throwaway session, so that path is behaviourally unchanged.
  • openRepo clones once and checks plugin folders out on demand via sparse-checkout add, replacing one clone per plugin.
  • The API fallback now reuses a single tree response — which also cuts requests against the unauthenticated 60/hour limit.
  • The Claude registration is memoized per session; the promise is cached, so a failure is shared rather than retried N times for a deterministic error.

Measured

4 plugins, sandboxed Cursor + VS Code, same machine and state directory, main vs this branch:

Wall clock Per plugin
main 23.6s 5.9s
this branch 11.5s 2.9s

That's the registry + clone savings only — these targets don't touch Claude Code. Runs including Claude save two CLI invocations and a fetch per plugin on top.

Tests

144 pass (136 existing, unchanged, + 8 new). New coverage:

  • one repo handle fetches the API tree once and serves every plugin from it
  • checking the same plugin out twice doesn't re-download
  • a session reads a registry once however many plugins ask
  • separate marketplaces keep separate registries
  • session cleanup disposes everything the run opened
  • the Claude marketplace is registered once per session — and still per-call without one
  • end-to-end: update reads the registry once for a two-plugin run

Review notes

  • deps.materialize is preserved as the injection seam and still runs per plugin, so existing tests and any caller relying on it behave as before. Only the real path uses the shared clone.
  • openRepo now honours deps.env for the git lookup. That's what makes the API path testable on a machine that has git installed, and it matches how deps.env is already used elsewhere.
  • resolvePlugin takes an optional pre-loaded catalog. undefined means "not supplied"; null is a valid answer meaning the repo has no registry, so it deliberately does not trigger a second fetch.
  • Sparse checkout uses add, not set. A --sparse clone starts with only root files, so for the first path the two are equivalent — but add lets later plugins join the same working tree instead of replacing it.
  • I did not add parallelism. It's the obvious next step and pool() already exists, but concurrent claude plugin install calls mutate Claude's shared config, and concurrent VS Code installs all rewrite the same settings.json — that's a lost-update race. Worth doing deliberately in its own change rather than smuggled in here.

There is still a floor: claude plugin install must run once per plugin. That part is irreducible.

🤖 Generated with Claude Code

updateAll re-installed each recorded plugin through the full install path, so
for every plugin it re-read the marketplace registry, re-cloned the whole
marketplace repository, and re-registered the marketplace with Claude Code.
That work is identical for every plugin from the same repo, so a run cost
O(plugins installed x marketplace size) - and got slower as the marketplace
grew, even with nothing new installed.

Introduce a per-run session holding the registry, an open repo handle, and the
Claude registration, keyed by repo@ref because manifest entries carry their
own. updateAll creates one and passes it to every plugin; a lone install gets
a throwaway session, so that path is unchanged.

openRepo clones (or fetches the API tree) once and checks plugin folders out
of it on demand, replacing one clone per plugin. The API path now reuses a
single tree response, which also cuts unauthenticated requests against the
60/hour limit.

Measured on 4 plugins into a sandboxed Cursor + VS Code: 23.6s -> 11.5s. The
Claude Code path saves two CLI invocations and a fetch per plugin on top.

Refs apimatic/contextmatic-crawler#31

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@alifsayalee
alifsayalee force-pushed the alifsayalee/faster-update branch from e746ecd to 0a683fd Compare July 30, 2026 10:05
@Alishahzad1903
Alishahzad1903 merged commit 331bbc0 into main Jul 30, 2026
11 checks passed
@github-actions

Copy link
Copy Markdown

🎉 This PR is included in version 0.4.2 🎉

The release is available on:

Your semantic-release bot 📦🚀

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants