Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
68 commits
Select commit Hold shift + click to select a range
953f79d
TDD: add tests for terra
bwalsh Jul 8, 2026
c4580fc
Support DRS URI pointers in LFS cache paths
bwalsh Jul 8, 2026
ba1a2ea
Adds Terra remote model, type
bwalsh Jul 8, 2026
f4ce335
Updated git drs ping for Terra
bwalsh Jul 8, 2026
3537355
adds terra ping
bwalsh Jul 9, 2026
0e652b8
Merge branch 'development' into feature/terra
matthewpeterkort Jul 10, 2026
31ec98c
Adds DRS uri <--> OID clarification
bwalsh Jul 13, 2026
481c89f
clarifies pointer file usage
bwalsh Jul 13, 2026
bc81881
adds sequence diagram
bwalsh Jul 13, 2026
10d7c2c
typo
bwalsh Jul 14, 2026
52a70a3
Merge branch 'development' into feature/terra
matthewpeterkort Jul 15, 2026
23c9b96
add-ref contacts the configured source authority
bwalsh Jul 15, 2026
b535e9d
Document multi-remote add-ref resolution
bwalsh Jul 15, 2026
e6bf3e9
documents Terra creds
bwalsh Jul 15, 2026
5fd8c6f
adds loadGitConfigOverrides
bwalsh Jul 15, 2026
b1478d8
adds service-info to ping
bwalsh Jul 15, 2026
e109fe8
adds anvil poc
bwalsh Jul 20, 2026
0392d64
Implement AnVIL/Terra reference workflow
bwalsh Jul 20, 2026
1d833f8
Fix unknown flag error in git drs command
bwalsh Jul 20, 2026
c3fe7f3
fix drs uri validation
bwalsh Jul 20, 2026
4563ac8
fix clean-filter bug
bwalsh Jul 20, 2026
5bb0921
Keep hydrated DRS URI references clean.
bwalsh Jul 21, 2026
19475be
adds presentation
bwalsh Jul 21, 2026
e1ff2ee
adds anvil manifest
bwalsh Jul 21, 2026
d61dcde
Add example AnVIL manifest workflow slide
bwalsh Jul 21, 2026
1b7b7b0
adds docs/index.html
bwalsh Jul 21, 2026
87c4cd8
docs: propose a unified remote CLI and multi-DRS architecture
bwalsh Jul 21, 2026
57f85fe
ignore working dirs
bwalsh Jul 21, 2026
7cb1642
clarified --auth and --credential
bwalsh Jul 21, 2026
a24b78a
Restrict override reads to repo-local config
bwalsh Jul 21, 2026
1259a5a
Remove Terra auth and mode keys with the remote
bwalsh Jul 21, 2026
5f7aba1
Match the DRS access header schema
bwalsh Jul 21, 2026
2c63f35
fix: Destination validation can escape through symlinks
bwalsh Jul 21, 2026
7f76d07
Fix Terra automatic smudge downloads
bwalsh Jul 21, 2026
18ec625
Clarify read-only Terra push behavior
bwalsh Jul 21, 2026
acb8601
docs DRS compact identifiers and auth discovery
bwalsh Jul 21, 2026
9553fec
Make pulled files read-only for read-only remotes
bwalsh Jul 21, 2026
d4b7bb7
Document rollback for accidental read-only DRS edits
bwalsh Jul 22, 2026
1a2850c
Refresh AnVIL POC gaps and roadmap
bwalsh Jul 22, 2026
fdd0065
Publish end-user docs and presentation with GitHub Pages
bwalsh Jul 22, 2026
68c083e
Define DRS preset catalog ownership and safe overrides
bwalsh Jul 22, 2026
521d994
Implement unified remote CLI and built-in DRS presets
bwalsh Jul 22, 2026
d598b8d
Update user docs for unified remote presets
bwalsh Jul 22, 2026
16d5e91
listed preset includes auth type
bwalsh Jul 27, 2026
55ac90a
Complete generic remote runtime boundary
bwalsh Jul 27, 2026
c8ebbdd
Clarify read-only Terra push error
bwalsh Jul 27, 2026
001ac73
Support inline and non-first DRS access methods
bwalsh Jul 28, 2026
6221437
Resolve unified Gen3 credential sources at runtime
bwalsh Jul 28, 2026
6b899fa
Resolve scoped Gen3 buckets at runtime
bwalsh Jul 28, 2026
76098af
Preserve source routing in add-ref pointers
bwalsh Jul 28, 2026
48c83ab
Validate AnVIL checksums before caching downloads
bwalsh Jul 28, 2026
be4a0c6
Hydrate DRS OIDs through the URI downloader
bwalsh Jul 28, 2026
a9c249e
Skip DRS URI references during push sync
bwalsh Jul 29, 2026
3018371
Reject unresolved providers during remote add
bwalsh Jul 29, 2026
3f294a0
Validate generic Gen3 authentication modes
bwalsh Jul 29, 2026
191c38e
Track add-ref destinations from repository root
bwalsh Jul 29, 2026
dca5d11
Fix read-only gitattributes marker
bwalsh Jul 29, 2026
7252e01
Persist local DRS metadata for manifest references
bwalsh Jul 29, 2026
5d3f2c9
Update documentation for unified remote add.
bwalsh Jul 29, 2026
427288b
Fix AnVIL HTTPS access selection
bwalsh Jul 29, 2026
ff9da80
Fix AnVIL HTTPS access selection
bwalsh Jul 29, 2026
8966490
1 of 2 update presentation
bwalsh Jul 29, 2026
a1cf36f
2 of 2 update presentation
bwalsh Jul 29, 2026
512dd9b
correct formatting
bwalsh Jul 29, 2026
0308579
more presentation fixes
bwalsh Jul 29, 2026
d1c5ccc
yet more presentation fixes
bwalsh Jul 29, 2026
ea11cc1
mkdocs options
bwalsh Jul 29, 2026
5ea5a46
mkdocs options take 2
bwalsh Jul 29, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
88 changes: 88 additions & 0 deletions .github/workflows/pages.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,88 @@
name: Publish documentation

on:
push:
branches: [main]
paths:
- docs/**
- mkdocs.yml
- .github/workflows/pages.yaml
workflow_dispatch:

permissions:
contents: read
pages: write
id-token: write

concurrency:
group: pages
cancel-in-progress: true

jobs:
build:
runs-on: ubuntu-latest
steps:
- name: Check out repository
uses: actions/checkout@v4

- name: Configure GitHub Pages
uses: actions/configure-pages@v5

- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: "3.13"

- name: Install documentation builder
run: python -m pip install mkdocs-material==9.6.14

- name: Select end-user documentation
run: |
mkdir -p .pages-docs
cp docs/index.md .pages-docs/
cp docs/quickstart.md .pages-docs/
cp docs/getting-started.md .pages-docs/
cp docs/commands.md .pages-docs/
cp docs/pointer-files.md .pages-docs/
cp docs/adding-s3-files.md .pages-docs/
cp docs/remove-files.md .pages-docs/
cp docs/troubleshooting.md .pages-docs/
cp docs/*.png .pages-docs/

- name: Build documentation and presentation
run: |
mkdir -p .pages-build
npx --yes @marp-team/marp-cli@4.2.3 \
docs/anvil-terra-poc-presentation.md \
--html \
--template bespoke \
--output .pages-build/anvil-terra-poc-presentation.html
mkdocs build --strict
# Keep the standalone Bespoke document byte-for-byte intact. Passing
# it through MkDocs is unnecessary for this standalone entry point.
cp .pages-build/anvil-terra-poc-presentation.html \
_site/anvil-terra-poc-presentation.html

- name: Verify presentation artifact
run: |
# A Bespoke deck needs its embedded browser runtime for slide paging.
grep -q '<script' _site/anvil-terra-poc-presentation.html
grep -q 'bespoke' _site/anvil-terra-poc-presentation.html
# Guard code-block contrast against Marp's highlighting stylesheet.
grep -q 'pre code.hljs' _site/anvil-terra-poc-presentation.html

- name: Upload GitHub Pages artifact
uses: actions/upload-pages-artifact@v3
with:
path: _site

deploy:
environment:
name: github-pages
url: ${{ steps.deployment.outputs.page_url }}
runs-on: ubuntu-latest
needs: build
steps:
- name: Deploy GitHub Pages
id: deployment
uses: actions/deploy-pages@v4
6 changes: 6 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -18,3 +18,9 @@ coverage/unit/
admin/
# Ignore any file named .env anywhere in the repository
.env
# ignore working dirs, files
attic/
patch.txt
.pages-docs/
_site/

15 changes: 11 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,9 @@ At a high level:

```bash
git drs install
git drs remote add gen3 production HTAN_INT/BForePC --cred /path/to/credentials.json
git drs remote add production https://example-gen3.org --provider gen3 \
--scope HTAN_INT/BForePC --auth provider-helper:gen3-profile \
--credential file:/path/to/credentials.json
git drs track "*.bam"
git add .gitattributes
git add sample.bam
Expand All @@ -63,12 +65,14 @@ The cleaned CLI intentionally removed legacy commands:
- `git drs download`
- `git drs pull` is hydration-only
- `git drs ls-files` is the local file inventory command
- `git drs remote add gen3` takes scope as `organization/project`
- the unified `git drs remote add` takes scope through
`--scope <organization/project>`

Example:

```bash
git drs remote add gen3 production HTAN_INT/BForePC --cred /path/to/credentials.json
git drs remote add production calypr --scope HTAN_INT/BForePC \
--credential file:/path/to/credentials.json
```

Current command split:
Expand All @@ -91,7 +95,8 @@ Push and pull depend on server-side bucket mapping for the requested scope. That
| --- | --- |
| `git drs install` | Install global `git-drs` filter config |
| `git drs init` | Explicitly initialize or repair repository-local `git-drs` state |
| `git drs remote add gen3 [remote] <org/project>` | Add or refresh a Gen3/Syfon remote |
| `git drs remote add <endpoint-or-alias>` | Add a DRS remote (operational: Calypr/Gen3 and Terra; catalog-only: Synapse and CGC) |
| `git drs preset list` | List the non-secret presets embedded in this release |
| `git drs remote list` | List configured remotes |
| `git drs remote remove <name>` | Remove a configured DRS remote |
| `git drs remote set <name>` | Set the default remote |
Expand All @@ -108,10 +113,12 @@ Push and pull depend on server-side bucket mapping for the requested scope. That

## Documentation

- [Documentation Home](docs/index.md)
- [Getting Started](docs/getting-started.md)
- [Commands Reference](docs/commands.md)
- [Troubleshooting](docs/troubleshooting.md)
- [Developer Guide](docs/developer-guide.md)
- [Publishing documentation with GitHub Pages](docs/github-pages.md)
- [GA4GH DRS Scalability Gaps](docs/ga4gh-drs-scalability-gaps.md)

## Requirements
Expand Down
Loading
Loading