feat(storage)!: per-keypair Tigris buckets, one per repo#5
Open
Xe wants to merge 4 commits into
Open
Conversation
Repositories were served from one shared bucket chrooted by a raw, variable-depth path. Every git request now resolves through a pluggable repofs.Resolver hook keyed on a validated {orgID}/{repoName} path (the .git suffix is stripped).
The production resolver (internal/tigrisfs) treats the HTTP Basic-auth pair as a Tigris keypair, builds and caches one storage.Client per keypair, and gives each repository its own bucket (objgit-<base36 sha256 of orgID/name>), created on the push path only.
Smart-HTTP now uses ServeMux wildcards (paths are fixed-depth), threads the credential into resolution, and logs every authorization decision; auth.Operation/Decision gain String() and s3fs.S3Client is exported so the resolver can cache the hardened client.
BREAKING CHANGE: repository URLs must be {orgID}/{repoName}; repositories moved from one shared bucket to a bucket per repo, so repositories created under the old layout no longer resolve.
Assisted-by: Claude Opus 4.8 via Claude Code
Signed-off-by: Xe Iaso <xe@tigrisdata.com>
aws-sdk-go-v2 (s3 >= v1.73) defaults RequestChecksumCalculation to "when supported", sending PutObject bodies with a trailing CRC32 via aws-chunked content encoding. Some S3-compatible endpoints mishandle that framing and store an empty or corrupt object. Force the legacy "when required" behavior in Harden so bodies are sent plain. Assisted-by: Claude Opus 4.8 via Claude Code Signed-off-by: Xe Iaso <xe@tigrisdata.com>
MkdirAll wrote a zero-byte marker at the exact key (e.g. "refs/heads"), which lists as a regular file. go-git's reference walker then read it as a ref and failed the receive-pack advertisement with "ref file is empty" — so a first push to a fresh per-repo bucket never completed.
Write the marker with a trailing separator ("refs/heads/") so it lists as a CommonPrefix (directory); listChildren already skips the trailing-slash self-marker. Also key the marker through fs3.key and short-circuit the bucket root.
Adds regression coverage that git.Init at the bucket root advertises references cleanly, and that MkdirAll yields a directory entry.
Assisted-by: Claude Opus 4.8 via Claude Code
Signed-off-by: Xe Iaso <xe@tigrisdata.com>
Captures the design for per-repo {orgID}/{repoName} filesystem dispatch and the per-keypair bucket-per-repo Tigris resolver.
Assisted-by: Claude Opus 4.8 via Claude Code
Signed-off-by: Xe Iaso <xe@tigrisdata.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Repositories were served from one shared bucket chrooted by a raw, variable-depth path. This PR replaces that with a per-repo storage model:
internal/repofs): every git request resolves to abilly.FilesystemviaResolver.Resolve(ctx, ref, cred, create), keyed on a validated{orgID}/{repoName}path (.gitstripped).BucketResolver(chroot one bucket) stays for tests.internal/tigrisfs): treats the HTTP Basic-auth pair as a Tigris keypair, builds and caches onestorage.Clientper keypair, and gives each repository its own bucket (objgit-<base36 sha256 of orgID/name>), created on the push path only.ServeMuxwildcards (paths are fixed-depth), threads the credential into resolution, and logs every authorization decision (auth.Operation/DecisiongainedString());s3fs.S3Clientis exported so the resolver can cache the hardened client.Fixes found bringing it up against real Tigris
fix(s3fs)directory markers:MkdirAllwrote a zero-byte marker at the exact key (refs/heads), which lists as a file; go-git's reference walker read it as a ref and failed the receive-pack advertisement withref file is empty. Markers now end in a trailing slash (refs/heads/) so they list as directories. This was the actual push-blocking bug.fix(s3fs)checksums: opt out of aws-sdk-go-v2's defaultRequestChecksumCalculation=when_supported(aws-chunked + trailing CRC32) inHarden, which some S3-compatible endpoints mishandle. Independent safeguard.Breaking changes
{orgID}/{repoName}.-bucketis now system-only (SSH host key).Testing
go build ./...,go vet ./...,go test ./...all green.repofs.Parse,tigrisfs(bucket naming, create-gating, credential→401), ands3fsinit-at-bucket-root advertising refs +MkdirAlldirectory markers.git pushto a fresh{org}/{repo}.gitcreates the bucket and agit cloneround-trips the content.Assisted-by: Claude Opus 4.8 via Claude Code