From c2c2c13a1f2764156054df11a9fbb0130b9bc0b3 Mon Sep 17 00:00:00 2001 From: Stephen Aylward Date: Thu, 9 Jul 2026 11:17:56 -0400 Subject: [PATCH 1/5] ENH: Specify Python 3.11 for nightly health / remote runner Also, small tweak to data/readme.md --- .github/workflows/nightly-health.yml | 2 +- .gitignore | 2 + data/README.md | 366 ++++++++++++++++++++++++--- 3 files changed, 333 insertions(+), 37 deletions(-) diff --git a/.github/workflows/nightly-health.yml b/.github/workflows/nightly-health.yml index 31e9bb8..6b0a604 100644 --- a/.github/workflows/nightly-health.yml +++ b/.github/workflows/nightly-health.yml @@ -58,7 +58,7 @@ jobs: # Python 3.11 is required because the [physicsnemo] extra pulls in # nvidia-physicsnemo, which requires Python >= 3.11. run: | - & py -m venv "$env:RUNNER_TEMP\physiotwin4d-venv" + & py -3.11 -m venv "$env:RUNNER_TEMP\physiotwin4d-venv" echo "$env:RUNNER_TEMP\physiotwin4d-venv\Scripts" >> $env:GITHUB_PATH - name: Cache uv packages diff --git a/.gitignore b/.gitignore index 1e9db68..3922777 100644 --- a/.gitignore +++ b/.gitignore @@ -13,6 +13,8 @@ build __pycache__ _version.py htmlcov +.cursor +.vscode # AI files .claude diff --git a/data/README.md b/data/README.md index dcfa81e..ab7fbc8 100644 --- a/data/README.md +++ b/data/README.md @@ -1,44 +1,85 @@ # PhysioTwin4D Data Directory -This directory holds the example datasets used by PhysioTwin4D's tutorials, -experiments, and tests. Each subdirectory is one dataset and has its own -`README.md` with the download URL(s), directory layout, data sizes, and the -exact `physiotwin4d-download-data` command for that dataset. This file is -just the map — see the subdirectory README before downloading anything. +This directory contains sample datasets used for experiments, testing, and development of the PhysioTwin4D library. Each subdirectory contains a specific medical imaging dataset. -## Datasets +## Directory Structure -| Directory | Data Type | Provided By | Auto-Download | -| ----------------------------------- | ------------------------------------------------------- | --------------------------------------- | ------------- | -| [`Slicer-Heart-CT/`](Slicer-Heart-CT/README.md) | 4D gated cardiac CT (`.seq.nrrd`) | Jolley Lab (CHOP) / SlicerHeart on GitHub | Yes | -| [`DirLab-4DCT/`](DirLab-4DCT/README.md) | 4D lung CT respiratory benchmark + landmarks (`.mhd/.raw`) | DIR-Lab (Emory / MD Anderson) | No — manual, may require registration | -| [`KCL-Heart-Model/`](KCL-Heart-Model/README.md) | Statistical heart shape model meshes (`.vtk`) | King's College London, via Zenodo | Yes | -| [`CHOP-Valve4D/`](CHOP-Valve4D/README.md) | FEBio valve FE model, converted to VTK/ITK + Simpleware segmentation | Jolley Lab (CHOP), converted by PhysioTwin4D | Yes | +``` +data/ +├── Slicer-Heart-CT/ # 4D cardiac CT with gated cardiac phases (AUTO-DOWNLOAD) +├── DirLab-4DCT/ # 4D lung CT benchmark dataset (MANUAL) +├── KCL-Heart-Model/ # Statistical shape model of the heart (MANUAL) +├── CHOP-Valve4D/ # 4D valve models (MANUAL) +``` + +## Data Download Methods + +### Automatic Download (Only Slicer-Heart-CT) + +Only the **Slicer-Heart-CT** dataset can be automatically downloaded by running the appropriate script. + +### Manual Download (All Others) + +The following datasets must be **manually downloaded and preprocessed** by the user: + +- **DirLab-4DCT**: Respiratory motion benchmark data +- **KCL-Heart-Model**: Statistical cardiac shape models +- **CHOP-Valve4D**: Time-varying valve reconstructions + +See individual dataset sections below for download instructions and preprocessing requirements. + +--- + +## Slicer-Heart-CT AUTO-DOWNLOAD + +### Description + +4D cardiac CT dataset with temporal gating showing complete cardiac cycle motion. Pediatric cardiac CT with truncal valve visualization. + +### Specifications + +- **Format**: `.seq.nrrd` (4D NRRD sequence file) +- **Phases**: 21 temporal cardiac phases +- **Size**: ~1.2 GB +- **Content**: Contrast-enhanced cardiac CT +- **Anatomy**: Heart, great vessels, thoracic structures -[`data/test/`](test/README.md) is not a dataset — it is a cache -automatically managed by the pytest infrastructure (mainly from -`Slicer-Heart-CT`) and is not used by the workflows, tutorials, or CLIs. +### Acknowledgement -## Downloading Data +Data provided by Jolley Lab at CHOP (Children's Hospital of Philadelphia): -Datasets marked **Yes** above can be fetched with the -`physiotwin4d-download-data` CLI (or the corresponding -`DataDownloadTools.Download*` method), for example: +- [https://www.linkedin.com/company/jolleylab](https://www.linkedin.com/company/jolleylab) +- [https://github.com/Slicer-Heart-CT/Slicer-Heart-CT](https://github.com/Slicer-Heart-CT/Slicer-Heart-CT) + +### Downloading the Data + +**Automatic download** (recommended): + +```python +from physiotwin4d import DataDownloadTools + +DataDownloadTools.DownloadSlicerHeartCTData("data/Slicer-Heart-CT") +assert DataDownloadTools.VerifySlicerHeartCTData("data/Slicer-Heart-CT") +``` + +**Manual download** (alternative): ```bash -physiotwin4d-download-data Slicer-Heart-CT --directory data/Slicer-Heart-CT -physiotwin4d-download-data KCL-Heart-Model --directory data/KCL-Heart-Model -physiotwin4d-download-data CHOP-Valve4D --directory data/CHOP-Valve4D +# Direct download link: +wget https://github.com/Slicer-Heart-CT/Slicer-Heart-CT/releases/download/TestingData/TruncalValve_4DCT.seq.nrrd -P data/Slicer-Heart-CT/ ``` -Datasets marked **No** must be downloaded by hand following the -instructions in that dataset's own README (`DirLab-4DCT/README.md`). +### Usage + +- Primary dataset for tutorials +- Primary dataset for `Heart-GatedCT_To_USD` experiments +- Used in test suite (`tests/test_download_heart_data.py`) +- Example data for cardiac motion visualization in Omniverse ### Verification Helpers -Each dataset also has a `DataDownloadTools.Verify*Data()` helper that checks -whether the expected files are already present, so scripts can skip -downloading when the data is cached: +PhysioTwin4D exposes a small public utility for checking optional dataset +layouts: ```python from physiotwin4d import DataDownloadTools @@ -49,13 +90,266 @@ DataDownloadTools.VerifyKCLHeartModelData("data/KCL-Heart-Model") DataDownloadTools.VerifyCHOPValve4DData("data/CHOP-Valve4D") ``` -## Guidelines +--- + +## DirLab-4DCT MANUAL DOWNLOAD + +### Description + +Benchmark dataset for 4D CT respiratory motion analysis. Contains 10 cases of lung CT scans at different respiratory phases with annotated landmark points for registration validation. + +### Specifications + +- **Format**: `.mhd/.raw` (MetaImage format) +- **Cases**: 10 patient cases (Case 1-10) +- **Phases**: 10 respiratory phases per case (T00-T90) +- **Content**: Non-contrast lung CT +- **Anatomy**: Lungs, airways, thoracic structures +- **Landmarks**: 300+ annotated points per case for validation + +### Acknowledgement + +Data provided by the DIR-Lab at MD Anderson Cancer Center: + +- **Project**: COPDGene 4D-CT dataset +- **Publication**: Castillo et al., "A reference dataset for deformable image registration spatial accuracy evaluation using the COPDGene study archive" +- **Website**: [https://med.emory.edu/departments/radiation-oncology/research-laboratories/deformable-image-registration/index.html](https://med.emory.edu/departments/radiation-oncology/research-laboratories/deformable-image-registration/index.html) + +### Downloading the Data + +**MANUAL DOWNLOAD REQUIRED** + +Users must manually download and preprocess this dataset. Follow these steps: + +**Step 1: Manual Download** + +```python +# Using provided utilities in experiment scripts. +# See: experiments/Lung-GatedCT_To_USD/0-register_dirlab_4dct.py +# The script includes download utilities but requires manual execution. +``` + +**Step 2: User Preprocessing** +Users are responsible for: + +- Downloading data from DIR-Lab website +- Extracting and organizing files in the proper directory structure +- Running preprocessing scripts if needed + +### Directory Structure + +``` +DirLab-4DCT/ +├── Case1Pack/ +│ ├── Images/ # T00-T50 phase images +│ ├── ExtremePhases/ # T00 and T50 (max inhale/exhale) +│ └── Sampled4D/ # Sampled time points +├── Case1Pack_T00.mhd # Extracted phase files +├── Case1Pack_T10.mhd +... +├── Case10Pack/ +└── dirlab_password.txt # Password for access (if needed) +``` + +### Usage + +- Primary dataset for `Lung-GatedCT_To_USD` experiments +- Registration algorithm validation +- Respiratory motion analysis +- Benchmark for deformable registration accuracy + +--- + +## KCL-Heart-Model MANUAL DOWNLOAD + +### Description + +Statistical shape model (SSM) of the human heart derived from cardiac imaging data. Includes principal component analysis (PCA) modes of shape variation. + +### Specifications + +- **Format**: `.vtk`, `.vtp` (VTK PolyData formats) +- **Content**: + - Average heart surface and mesh + - Individual heart models +- **Components**: Full heart mesh with chambers and vessels + +### Files + +- `average_mesh.vtk` - Mean heart volume mesh (UnstructuredGrid) +- input_meshes/01.vtk + +### Acknowledgement + +Data from King's College London (KCL): + +- **Repository**: Cardiac imaging research group +- **License**: Check `citation.txt` for proper attribution + +### Downloading the Data + +**MANUAL DOWNLOAD REQUIRED** + +Users must manually obtain and place this data: + +1. Obtain data from published research repositories or contact authors +2. Place files in `data/KCL-Heart-Model/` directory +3. Required files: `average_mesh.vtk` + +Check the included PDFs (if available) for source information and proper citation. + +### Usage + +- **Statistical shape model creation** (`experiments/Heart-Create_Statistical_Model/`) ⭐ **Primary use case** +- **Model-to-patient registration** (`experiments/Heart-Statistical_Model_To_Patient/`) +- VTK to USD conversion experiments (`experiments/Convert_VTK_To_USD/`) +- Shape-based cardiac analysis +- Atlas-based segmentation initialization +- Population-based statistical analysis + +--- + +## CHOP-Valve4D MANUAL DOWNLOAD + +### Description + +Time-varying 4D valve reconstruction models showing valve motion over the cardiac cycle. These datasets represent dynamic valve geometries reconstructed from medical imaging data. + +### Specifications + +- **Format**: `.vtk` (VTK PolyData files) +- **Content**: Time series of valve surface meshes +- **Valves**: Alterra, TPV25, and other valve types +- **Phases**: Multiple time points per cardiac cycle (200+ frames) +- **Resolution**: High-resolution surface meshes with anatomical features + +### Directory Structure + +``` +CHOP-Valve4D/ +├── Alterra/ +│ ├── frame_0000.vtk +│ ├── frame_0001.vtk +│ └── ... (232 frames) +├── TPV25/ +│ ├── frame_0000.vtk +│ ├── frame_0001.vtk +│ └── ... (265 frames) +``` + +### Acknowledgement + +Data provided by Jolley Lab at CHOP (Children's Hospital of Philadelphia): + +- [https://www.linkedin.com/company/jolleylab](https://www.linkedin.com/company/jolleylab) + +### Downloading the Data + +**MANUAL DOWNLOAD REQUIRED** + +**Availability**: This dataset will soon be publicly available for download from the **FEBio website** under the **Creative Commons Attribution (CC-BY) license**. + +- **Source**: [https://febio.org/](https://febio.org/) (coming soon) +- **License**: CC-BY (Creative Commons Attribution) +- **Citation**: Please cite the Jolley Lab and FEBio when using this data + +**Setup Instructions**: + +1. Download valve reconstruction data from FEBio website when available +2. Place files in `data/CHOP-Valve4D/` with proper subdirectory structure +3. Ensure files are named sequentially for time-series processing (e.g., `frame_0000.vtk`, `frame_0001.vtk`, ...) +4. Organize by valve type in subdirectories (e.g., `Alterra/`, `TPV25/`) + +### Usage + +- Time-series VTK to USD conversion (`experiments/Convert_VTK_To_USD/`) +- 4D valve motion visualization in NVIDIA Omniverse +- Temporal cardiac mechanics analysis +- Valve dynamics studies and surgical planning + +### Related Resources + +- **FEBio**: Finite Element Biomechanics software suite ([https://febio.org/](https://febio.org/)) +- **Jolley Lab**: Cardiac imaging and computational modeling research + +--- + +## Data Usage Guidelines + +### For Testing + +- Tests automatically use cached data when available +- Download occurs only if data is missing +- Tests use subsets (e.g., first 2 time points) for speed + +### For Experiments + +- Full datasets used for complete analysis +- Results saved to respective `experiments/*/results/` directories +- Original data remains unmodified + +### For Development + +- Use small subsets for rapid iteration +- Full datasets for validation and benchmarking +- Cache intermediate results to avoid reprocessing + +--- + +## Data Access and Licensing + +- **Slicer-Heart-CT** : Public release from GitHub (auto-download available) +- **DirLab-4DCT** : Public benchmark dataset (manual download required, may require registration) +- **KCL-Heart-Model** : Requires manual download from research repositories +- **CHOP-Valve4D** : Soon available from FEBio website under CC-BY license (manual download) + +**Important**: Always cite the original data sources in publications and respect any usage restrictions. + +### Summary of Download Methods + + +| Dataset | Auto-Download | Manual Required | License | Source | Used in Tests | +| --------------- | ------------- | --------------- | --------------- | ------------------- | ------------------------ | +| Slicer-Heart-CT | Yes | No | Public | GitHub | Yes | +| DirLab-4DCT | No | Yes | Public/Academic | DIR-Lab | No | +| KCL-Heart-Model | No | Yes | Check citation | Zenodo/KCL | Yes (skipped if missing) | +| CHOP-Valve4D | No | Yes | CC-BY | FEBio (coming soon) | No | + + +--- + +## References + +### Slicer-Heart-CT + +- Jolley Lab: [https://www.linkedin.com/company/jolleylab](https://www.linkedin.com/company/jolleylab) +- GitHub: [https://github.com/Slicer-Heart-CT/Slicer-Heart-CT](https://github.com/Slicer-Heart-CT/Slicer-Heart-CT) + +### DirLab-4DCT + +- Castillo et al., "A reference dataset for deformable image registration spatial accuracy evaluation using the COPDGene study archive" +- DIR-Lab: [https://med.emory.edu/departments/radiation-oncology/research-laboratories/deformable-image-registration/](https://med.emory.edu/departments/radiation-oncology/research-laboratories/deformable-image-registration/) + +### KCL-Heart-Model + +- Rodero et al. (2021), "Linking statistical shape models and simulated function in the healthy adult human heart", *PLOS Computational Biology* +- DOI: [10.1371/journal.pcbi.1008851](https://doi.org/10.1371/journal.pcbi.1008851) +- Zenodo: [https://zenodo.org/records/4590294](https://zenodo.org/records/4590294) + +### CHOP-Valve4D + +- Jolley Lab (CHOP): [https://www.linkedin.com/company/jolleylab](https://www.linkedin.com/company/jolleylab) +- FEBio Project: [https://febio.org/](https://febio.org/) (dataset coming soon) +- License: Creative Commons Attribution (CC-BY) +- Citation: Please acknowledge Jolley Lab at CHOP and the FEBio Project + +--- + +## Tips + +1. **Storage**: Ensure adequate disk space (~10-20GB for all datasets) +2. **Download Time**: Initial downloads can be slow; be patient +3. **Organization**: Keep data organized; don't modify original files +4. **Backups**: Consider backing up processed results separately +5. **Documentation**: Update this README when adding new datasets -- Keep original downloaded data unmodified; write experiment/tutorial - results under `experiments/*/results/` or `tutorials/*/results/`, not - back into `data/`. -- Always respect the license and citation requirements of each dataset — - see the "Citation" / "Acknowledgement" section in the relevant - subdirectory README. -- If you add a new dataset, add both a row above and a subdirectory - `README.md` following the existing datasets as a template. From a40c37170087e0f395a56406ece7a23e50a433e7 Mon Sep 17 00:00:00 2001 From: Stephen Aylward Date: Thu, 9 Jul 2026 12:00:19 -0400 Subject: [PATCH 2/5] ENH: Code rabbit --- .gitignore | 3 +- data/DirLab-4DCT/README.md | 2 +- data/README.md | 85 +++++++++++++++++++++----------------- 3 files changed, 51 insertions(+), 39 deletions(-) diff --git a/.gitignore b/.gitignore index 3922777..3e12bf9 100644 --- a/.gitignore +++ b/.gitignore @@ -14,7 +14,8 @@ __pycache__ _version.py htmlcov .cursor -.vscode +.vscode/* +!.vscode/settings.json # AI files .claude diff --git a/data/DirLab-4DCT/README.md b/data/DirLab-4DCT/README.md index be5199d..686b65d 100644 --- a/data/DirLab-4DCT/README.md +++ b/data/DirLab-4DCT/README.md @@ -68,7 +68,7 @@ points for registration validation. ### Dataset Details -- **Format**: `.mhd`/`.raw` (MetaImage format) +- **Format**: `.mhd` headers + `.img` raw volumes (MetaImage format) - **Cases**: 10 patient cases (Case 1-10) - **Phases**: 10 respiratory phases per case (T00-T90) - **Content**: Non-contrast lung CT diff --git a/data/README.md b/data/README.md index ab7fbc8..82b5050 100644 --- a/data/README.md +++ b/data/README.md @@ -4,27 +4,31 @@ This directory contains sample datasets used for experiments, testing, and devel ## Directory Structure -``` +```text data/ ├── Slicer-Heart-CT/ # 4D cardiac CT with gated cardiac phases (AUTO-DOWNLOAD) ├── DirLab-4DCT/ # 4D lung CT benchmark dataset (MANUAL) -├── KCL-Heart-Model/ # Statistical shape model of the heart (MANUAL) -├── CHOP-Valve4D/ # 4D valve models (MANUAL) +├── KCL-Heart-Model/ # Statistical shape model of the heart (AUTO-DOWNLOAD) +├── CHOP-Valve4D/ # 4D valve models (AUTO-DOWNLOAD) ``` ## Data Download Methods -### Automatic Download (Only Slicer-Heart-CT) +### Automatic Download (Slicer-Heart-CT, KCL-Heart-Model, CHOP-Valve4D) -Only the **Slicer-Heart-CT** dataset can be automatically downloaded by running the appropriate script. +These datasets can be downloaded automatically via `DataDownloadTools` or the +`physiotwin4d-download-data` CLI: -### Manual Download (All Others) +```bash +physiotwin4d-download-data Slicer-Heart-CT --directory data/Slicer-Heart-CT +physiotwin4d-download-data KCL-Heart-Model --directory data/KCL-Heart-Model +physiotwin4d-download-data CHOP-Valve4D --directory data/CHOP-Valve4D +``` -The following datasets must be **manually downloaded and preprocessed** by the user: +### Manual Download (DirLab-4DCT) -- **DirLab-4DCT**: Respiratory motion benchmark data -- **KCL-Heart-Model**: Statistical cardiac shape models -- **CHOP-Valve4D**: Time-varying valve reconstructions +- **DirLab-4DCT**: Respiratory motion benchmark data must be **manually + downloaded and preprocessed** by the user. See individual dataset sections below for download instructions and preprocessing requirements. @@ -100,7 +104,7 @@ Benchmark dataset for 4D CT respiratory motion analysis. Contains 10 cases of lu ### Specifications -- **Format**: `.mhd/.raw` (MetaImage format) +- **Format**: `.mhd` headers + `.img` raw volumes (MetaImage format) - **Cases**: 10 patient cases (Case 1-10) - **Phases**: 10 respiratory phases per case (T00-T90) - **Content**: Non-contrast lung CT @@ -138,7 +142,7 @@ Users are responsible for: ### Directory Structure -``` +```text DirLab-4DCT/ ├── Case1Pack/ │ ├── Images/ # T00-T50 phase images @@ -148,7 +152,6 @@ DirLab-4DCT/ ├── Case1Pack_T10.mhd ... ├── Case10Pack/ -└── dirlab_password.txt # Password for access (if needed) ``` ### Usage @@ -160,7 +163,7 @@ DirLab-4DCT/ --- -## KCL-Heart-Model MANUAL DOWNLOAD +## KCL-Heart-Model AUTO-DOWNLOAD ### Description @@ -188,15 +191,20 @@ Data from King's College London (KCL): ### Downloading the Data -**MANUAL DOWNLOAD REQUIRED** +**Automatic download** (recommended): -Users must manually obtain and place this data: +```bash +physiotwin4d-download-data KCL-Heart-Model --directory data/KCL-Heart-Model +``` + +```python +from physiotwin4d import DataDownloadTools -1. Obtain data from published research repositories or contact authors -2. Place files in `data/KCL-Heart-Model/` directory -3. Required files: `average_mesh.vtk` +DataDownloadTools.DownloadKCLHeartModelData("data/KCL-Heart-Model") +assert DataDownloadTools.VerifyKCLHeartModelData("data/KCL-Heart-Model") +``` -Check the included PDFs (if available) for source information and proper citation. +See `data/KCL-Heart-Model/README.md` for details on what is downloaded. ### Usage @@ -209,7 +217,7 @@ Check the included PDFs (if available) for source information and proper citatio --- -## CHOP-Valve4D MANUAL DOWNLOAD +## CHOP-Valve4D AUTO-DOWNLOAD ### Description @@ -225,7 +233,7 @@ Time-varying 4D valve reconstruction models showing valve motion over the cardia ### Directory Structure -``` +```text CHOP-Valve4D/ ├── Alterra/ │ ├── frame_0000.vtk @@ -245,20 +253,23 @@ Data provided by Jolley Lab at CHOP (Children's Hospital of Philadelphia): ### Downloading the Data -**MANUAL DOWNLOAD REQUIRED** +**Automatic download** (recommended): -**Availability**: This dataset will soon be publicly available for download from the **FEBio website** under the **Creative Commons Attribution (CC-BY) license**. +```bash +physiotwin4d-download-data CHOP-Valve4D --directory data/CHOP-Valve4D +``` -- **Source**: [https://febio.org/](https://febio.org/) (coming soon) -- **License**: CC-BY (Creative Commons Attribution) -- **Citation**: Please cite the Jolley Lab and FEBio when using this data +```python +from physiotwin4d import DataDownloadTools -**Setup Instructions**: +DataDownloadTools.DownloadCHOPValve4DData("data/CHOP-Valve4D") +assert DataDownloadTools.VerifyCHOPValve4DData("data/CHOP-Valve4D") +``` -1. Download valve reconstruction data from FEBio website when available -2. Place files in `data/CHOP-Valve4D/` with proper subdirectory structure -3. Ensure files are named sequentially for time-series processing (e.g., `frame_0000.vtk`, `frame_0001.vtk`, ...) -4. Organize by valve type in subdirectories (e.g., `Alterra/`, `TPV25/`) +This fetches the PhysioTwin4D convenience release (VTK/ITK conversion of the +original FEBio model). The original `.feb` source is available separately +from the FEBio website under CC-BY. See `data/CHOP-Valve4D/README.md` for +details and citation. ### Usage @@ -300,8 +311,8 @@ Data provided by Jolley Lab at CHOP (Children's Hospital of Philadelphia): - **Slicer-Heart-CT** : Public release from GitHub (auto-download available) - **DirLab-4DCT** : Public benchmark dataset (manual download required, may require registration) -- **KCL-Heart-Model** : Requires manual download from research repositories -- **CHOP-Valve4D** : Soon available from FEBio website under CC-BY license (manual download) +- **KCL-Heart-Model** : Public release from Zenodo (auto-download available) +- **CHOP-Valve4D** : PhysioTwin4D convenience release under CC-BY license (auto-download available) **Important**: Always cite the original data sources in publications and respect any usage restrictions. @@ -312,8 +323,8 @@ Data provided by Jolley Lab at CHOP (Children's Hospital of Philadelphia): | --------------- | ------------- | --------------- | --------------- | ------------------- | ------------------------ | | Slicer-Heart-CT | Yes | No | Public | GitHub | Yes | | DirLab-4DCT | No | Yes | Public/Academic | DIR-Lab | No | -| KCL-Heart-Model | No | Yes | Check citation | Zenodo/KCL | Yes (skipped if missing) | -| CHOP-Valve4D | No | Yes | CC-BY | FEBio (coming soon) | No | +| KCL-Heart-Model | Yes | No | Check citation | Zenodo/KCL | Yes (skipped if missing) | +| CHOP-Valve4D | Yes | No | CC-BY | GitHub release | No | --- @@ -339,7 +350,7 @@ Data provided by Jolley Lab at CHOP (Children's Hospital of Philadelphia): ### CHOP-Valve4D - Jolley Lab (CHOP): [https://www.linkedin.com/company/jolleylab](https://www.linkedin.com/company/jolleylab) -- FEBio Project: [https://febio.org/](https://febio.org/) (dataset coming soon) +- Original FEBio source model: [repo.febio.org/permalink/project/136](https://repo.febio.org/permalink/project/136) - License: Creative Commons Attribution (CC-BY) - Citation: Please acknowledge Jolley Lab at CHOP and the FEBio Project From ec329632a05030429a9a781d9f00d1bd028acec3 Mon Sep 17 00:00:00 2001 From: Stephen Aylward Date: Thu, 9 Jul 2026 12:22:23 -0400 Subject: [PATCH 3/5] ENH: Coderabbit - and unreachable code in vtk_to_usd/usd_utils --- data/README.md | 1 + src/physiotwin4d/vtk_to_usd/usd_utils.py | 12 ++---------- 2 files changed, 3 insertions(+), 10 deletions(-) diff --git a/data/README.md b/data/README.md index 82b5050..d35c308 100644 --- a/data/README.md +++ b/data/README.md @@ -10,6 +10,7 @@ data/ ├── DirLab-4DCT/ # 4D lung CT benchmark dataset (MANUAL) ├── KCL-Heart-Model/ # Statistical shape model of the heart (AUTO-DOWNLOAD) ├── CHOP-Valve4D/ # 4D valve models (AUTO-DOWNLOAD) +├── test/ # pytest-managed cache, not a manual/auto dataset ``` ## Data Download Methods diff --git a/src/physiotwin4d/vtk_to_usd/usd_utils.py b/src/physiotwin4d/vtk_to_usd/usd_utils.py index 7408bc1..7d06626 100644 --- a/src/physiotwin4d/vtk_to_usd/usd_utils.py +++ b/src/physiotwin4d/vtk_to_usd/usd_utils.py @@ -163,16 +163,11 @@ def numpy_to_vt_array(array: NDArray, data_type: DataType) -> Any: array_uc = array.astype(np.uint8) return Vt.UCharArray.FromNumpy(array_uc.flatten()) - elif data_type in [DataType.SHORT, DataType.USHORT]: + else: # DataType.SHORT, DataType.USHORT # Convert to int array_i = array.astype(np.int32) return Vt.IntArray.FromNumpy(array_i.flatten()) - else: - # Fallback to float - array_f = array.astype(np.float32) - return Vt.FloatArray.FromNumpy(array_f.flatten()) - def get_sdf_value_type(data_type: DataType, num_components: int) -> Sdf.ValueTypeName: """Get appropriate SDF value type for primvar creation. @@ -214,12 +209,9 @@ def get_sdf_value_type(data_type: DataType, num_components: int) -> Sdf.ValueTyp elif data_type in [DataType.UCHAR, DataType.CHAR]: return Sdf.ValueTypeNames.UCharArray - elif data_type in [DataType.SHORT, DataType.USHORT]: + else: # DataType.SHORT, DataType.USHORT return Sdf.ValueTypeNames.IntArray - else: - return Sdf.ValueTypeNames.FloatArray - def sanitize_primvar_name(name: str) -> str: """Sanitize a name to be USD-compliant. From 4fd8d4fddd16050bcf5e8f74e4441af68e501256 Mon Sep 17 00:00:00 2001 From: Stephen Aylward Date: Thu, 9 Jul 2026 13:09:43 -0400 Subject: [PATCH 4/5] BUG: Fix url to SlicerHeart --- data/README.md | 6 +++--- tests/README.md | 2 +- tutorials/tutorial_01_heart_gated_ct_to_usd.py | 2 +- 3 files changed, 5 insertions(+), 5 deletions(-) diff --git a/data/README.md b/data/README.md index d35c308..d73a03e 100644 --- a/data/README.md +++ b/data/README.md @@ -54,7 +54,7 @@ See individual dataset sections below for download instructions and preprocessin Data provided by Jolley Lab at CHOP (Children's Hospital of Philadelphia): - [https://www.linkedin.com/company/jolleylab](https://www.linkedin.com/company/jolleylab) -- [https://github.com/Slicer-Heart-CT/Slicer-Heart-CT](https://github.com/Slicer-Heart-CT/Slicer-Heart-CT) +- [https://github.com/SlicerHeart/SlicerHeart](https://github.com/SlicerHeart/SlicerHeart) ### Downloading the Data @@ -71,7 +71,7 @@ assert DataDownloadTools.VerifySlicerHeartCTData("data/Slicer-Heart-CT") ```bash # Direct download link: -wget https://github.com/Slicer-Heart-CT/Slicer-Heart-CT/releases/download/TestingData/TruncalValve_4DCT.seq.nrrd -P data/Slicer-Heart-CT/ +wget https://github.com/SlicerHeart/SlicerHeart/releases/download/TestingData/TruncalValve_4DCT.seq.nrrd -P data/Slicer-Heart-CT/ ``` ### Usage @@ -335,7 +335,7 @@ details and citation. ### Slicer-Heart-CT - Jolley Lab: [https://www.linkedin.com/company/jolleylab](https://www.linkedin.com/company/jolleylab) -- GitHub: [https://github.com/Slicer-Heart-CT/Slicer-Heart-CT](https://github.com/Slicer-Heart-CT/Slicer-Heart-CT) +- GitHub: [https://github.com/SlicerHeart/SlicerHeart](https://github.com/SlicerHeart/SlicerHeart) ### DirLab-4DCT diff --git a/tests/README.md b/tests/README.md index ad4adaa..0798ced 100644 --- a/tests/README.md +++ b/tests/README.md @@ -254,7 +254,7 @@ When creating new tests: ## Test Data ### Slicer-Heart-CT Dataset -- **Source**: [Slicer-Heart-CT GitHub](https://github.com/Slicer-Heart-CT/Slicer-Heart-CT) +- **Source**: [SlicerHeart GitHub](https://github.com/SlicerHeart/SlicerHeart) - **File**: `TruncalValve_4DCT.seq.nrrd` (~1.2 GB) - **Content**: 21-phase pediatric cardiac CT - **Usage**: Tests use first 2 time points for speed diff --git a/tutorials/tutorial_01_heart_gated_ct_to_usd.py b/tutorials/tutorial_01_heart_gated_ct_to_usd.py index eba2bd3..71bd7a9 100644 --- a/tutorials/tutorial_01_heart_gated_ct_to_usd.py +++ b/tutorials/tutorial_01_heart_gated_ct_to_usd.py @@ -60,7 +60,7 @@ Data Required ------------- See data/README.md for download instructions and dataset licensing. -Dataset: Slicer-Heart-CT - https://github.com/Slicer-Heart-CT/Slicer-Heart-CT +Dataset: Slicer-Heart-CT - https://github.com/SlicerHeart/SlicerHeart This script expects the data to already exist at ``data/Slicer-Heart-CT/TruncalValve_4DCT.seq.nrrd``. Run the repository data download notebook or download the file manually before running this tutorial. From 0a9216cb1e06ee7135c01fa29600eb75f0b6f1fd Mon Sep 17 00:00:00 2001 From: Stephen Aylward Date: Thu, 9 Jul 2026 15:31:12 -0400 Subject: [PATCH 5/5] ENH: Simplify top-level data/README, delegating details to subdirs --- data/README.md | 396 ++++--------------------------------------------- 1 file changed, 31 insertions(+), 365 deletions(-) diff --git a/data/README.md b/data/README.md index d73a03e..bc0b525 100644 --- a/data/README.md +++ b/data/README.md @@ -1,367 +1,33 @@ # PhysioTwin4D Data Directory -This directory contains sample datasets used for experiments, testing, and development of the PhysioTwin4D library. Each subdirectory contains a specific medical imaging dataset. - -## Directory Structure - -```text -data/ -├── Slicer-Heart-CT/ # 4D cardiac CT with gated cardiac phases (AUTO-DOWNLOAD) -├── DirLab-4DCT/ # 4D lung CT benchmark dataset (MANUAL) -├── KCL-Heart-Model/ # Statistical shape model of the heart (AUTO-DOWNLOAD) -├── CHOP-Valve4D/ # 4D valve models (AUTO-DOWNLOAD) -├── test/ # pytest-managed cache, not a manual/auto dataset -``` - -## Data Download Methods - -### Automatic Download (Slicer-Heart-CT, KCL-Heart-Model, CHOP-Valve4D) - -These datasets can be downloaded automatically via `DataDownloadTools` or the -`physiotwin4d-download-data` CLI: - -```bash -physiotwin4d-download-data Slicer-Heart-CT --directory data/Slicer-Heart-CT -physiotwin4d-download-data KCL-Heart-Model --directory data/KCL-Heart-Model -physiotwin4d-download-data CHOP-Valve4D --directory data/CHOP-Valve4D -``` - -### Manual Download (DirLab-4DCT) - -- **DirLab-4DCT**: Respiratory motion benchmark data must be **manually - downloaded and preprocessed** by the user. - -See individual dataset sections below for download instructions and preprocessing requirements. - ---- - -## Slicer-Heart-CT AUTO-DOWNLOAD - -### Description - -4D cardiac CT dataset with temporal gating showing complete cardiac cycle motion. Pediatric cardiac CT with truncal valve visualization. - -### Specifications - -- **Format**: `.seq.nrrd` (4D NRRD sequence file) -- **Phases**: 21 temporal cardiac phases -- **Size**: ~1.2 GB -- **Content**: Contrast-enhanced cardiac CT -- **Anatomy**: Heart, great vessels, thoracic structures - -### Acknowledgement - -Data provided by Jolley Lab at CHOP (Children's Hospital of Philadelphia): - -- [https://www.linkedin.com/company/jolleylab](https://www.linkedin.com/company/jolleylab) -- [https://github.com/SlicerHeart/SlicerHeart](https://github.com/SlicerHeart/SlicerHeart) - -### Downloading the Data - -**Automatic download** (recommended): - -```python -from physiotwin4d import DataDownloadTools - -DataDownloadTools.DownloadSlicerHeartCTData("data/Slicer-Heart-CT") -assert DataDownloadTools.VerifySlicerHeartCTData("data/Slicer-Heart-CT") -``` - -**Manual download** (alternative): - -```bash -# Direct download link: -wget https://github.com/SlicerHeart/SlicerHeart/releases/download/TestingData/TruncalValve_4DCT.seq.nrrd -P data/Slicer-Heart-CT/ -``` - -### Usage - -- Primary dataset for tutorials -- Primary dataset for `Heart-GatedCT_To_USD` experiments -- Used in test suite (`tests/test_download_heart_data.py`) -- Example data for cardiac motion visualization in Omniverse - -### Verification Helpers - -PhysioTwin4D exposes a small public utility for checking optional dataset -layouts: - -```python -from physiotwin4d import DataDownloadTools - -DataDownloadTools.VerifySlicerHeartCTData("data/Slicer-Heart-CT") -DataDownloadTools.VerifyDirLab4DCTData("data/DirLab-4DCT") -DataDownloadTools.VerifyKCLHeartModelData("data/KCL-Heart-Model") -DataDownloadTools.VerifyCHOPValve4DData("data/CHOP-Valve4D") -``` - ---- - -## DirLab-4DCT MANUAL DOWNLOAD - -### Description - -Benchmark dataset for 4D CT respiratory motion analysis. Contains 10 cases of lung CT scans at different respiratory phases with annotated landmark points for registration validation. - -### Specifications - -- **Format**: `.mhd` headers + `.img` raw volumes (MetaImage format) -- **Cases**: 10 patient cases (Case 1-10) -- **Phases**: 10 respiratory phases per case (T00-T90) -- **Content**: Non-contrast lung CT -- **Anatomy**: Lungs, airways, thoracic structures -- **Landmarks**: 300+ annotated points per case for validation - -### Acknowledgement - -Data provided by the DIR-Lab at MD Anderson Cancer Center: - -- **Project**: COPDGene 4D-CT dataset -- **Publication**: Castillo et al., "A reference dataset for deformable image registration spatial accuracy evaluation using the COPDGene study archive" -- **Website**: [https://med.emory.edu/departments/radiation-oncology/research-laboratories/deformable-image-registration/index.html](https://med.emory.edu/departments/radiation-oncology/research-laboratories/deformable-image-registration/index.html) - -### Downloading the Data - -**MANUAL DOWNLOAD REQUIRED** - -Users must manually download and preprocess this dataset. Follow these steps: - -**Step 1: Manual Download** - -```python -# Using provided utilities in experiment scripts. -# See: experiments/Lung-GatedCT_To_USD/0-register_dirlab_4dct.py -# The script includes download utilities but requires manual execution. -``` - -**Step 2: User Preprocessing** -Users are responsible for: - -- Downloading data from DIR-Lab website -- Extracting and organizing files in the proper directory structure -- Running preprocessing scripts if needed - -### Directory Structure - -```text -DirLab-4DCT/ -├── Case1Pack/ -│ ├── Images/ # T00-T50 phase images -│ ├── ExtremePhases/ # T00 and T50 (max inhale/exhale) -│ └── Sampled4D/ # Sampled time points -├── Case1Pack_T00.mhd # Extracted phase files -├── Case1Pack_T10.mhd -... -├── Case10Pack/ -``` - -### Usage - -- Primary dataset for `Lung-GatedCT_To_USD` experiments -- Registration algorithm validation -- Respiratory motion analysis -- Benchmark for deformable registration accuracy - ---- - -## KCL-Heart-Model AUTO-DOWNLOAD - -### Description - -Statistical shape model (SSM) of the human heart derived from cardiac imaging data. Includes principal component analysis (PCA) modes of shape variation. - -### Specifications - -- **Format**: `.vtk`, `.vtp` (VTK PolyData formats) -- **Content**: - - Average heart surface and mesh - - Individual heart models -- **Components**: Full heart mesh with chambers and vessels - -### Files - -- `average_mesh.vtk` - Mean heart volume mesh (UnstructuredGrid) -- input_meshes/01.vtk - -### Acknowledgement - -Data from King's College London (KCL): - -- **Repository**: Cardiac imaging research group -- **License**: Check `citation.txt` for proper attribution - -### Downloading the Data - -**Automatic download** (recommended): - -```bash -physiotwin4d-download-data KCL-Heart-Model --directory data/KCL-Heart-Model -``` - -```python -from physiotwin4d import DataDownloadTools - -DataDownloadTools.DownloadKCLHeartModelData("data/KCL-Heart-Model") -assert DataDownloadTools.VerifyKCLHeartModelData("data/KCL-Heart-Model") -``` - -See `data/KCL-Heart-Model/README.md` for details on what is downloaded. - -### Usage - -- **Statistical shape model creation** (`experiments/Heart-Create_Statistical_Model/`) ⭐ **Primary use case** -- **Model-to-patient registration** (`experiments/Heart-Statistical_Model_To_Patient/`) -- VTK to USD conversion experiments (`experiments/Convert_VTK_To_USD/`) -- Shape-based cardiac analysis -- Atlas-based segmentation initialization -- Population-based statistical analysis - ---- - -## CHOP-Valve4D AUTO-DOWNLOAD - -### Description - -Time-varying 4D valve reconstruction models showing valve motion over the cardiac cycle. These datasets represent dynamic valve geometries reconstructed from medical imaging data. - -### Specifications - -- **Format**: `.vtk` (VTK PolyData files) -- **Content**: Time series of valve surface meshes -- **Valves**: Alterra, TPV25, and other valve types -- **Phases**: Multiple time points per cardiac cycle (200+ frames) -- **Resolution**: High-resolution surface meshes with anatomical features - -### Directory Structure - -```text -CHOP-Valve4D/ -├── Alterra/ -│ ├── frame_0000.vtk -│ ├── frame_0001.vtk -│ └── ... (232 frames) -├── TPV25/ -│ ├── frame_0000.vtk -│ ├── frame_0001.vtk -│ └── ... (265 frames) -``` - -### Acknowledgement - -Data provided by Jolley Lab at CHOP (Children's Hospital of Philadelphia): - -- [https://www.linkedin.com/company/jolleylab](https://www.linkedin.com/company/jolleylab) - -### Downloading the Data - -**Automatic download** (recommended): - -```bash -physiotwin4d-download-data CHOP-Valve4D --directory data/CHOP-Valve4D -``` - -```python -from physiotwin4d import DataDownloadTools - -DataDownloadTools.DownloadCHOPValve4DData("data/CHOP-Valve4D") -assert DataDownloadTools.VerifyCHOPValve4DData("data/CHOP-Valve4D") -``` - -This fetches the PhysioTwin4D convenience release (VTK/ITK conversion of the -original FEBio model). The original `.feb` source is available separately -from the FEBio website under CC-BY. See `data/CHOP-Valve4D/README.md` for -details and citation. - -### Usage - -- Time-series VTK to USD conversion (`experiments/Convert_VTK_To_USD/`) -- 4D valve motion visualization in NVIDIA Omniverse -- Temporal cardiac mechanics analysis -- Valve dynamics studies and surgical planning - -### Related Resources - -- **FEBio**: Finite Element Biomechanics software suite ([https://febio.org/](https://febio.org/)) -- **Jolley Lab**: Cardiac imaging and computational modeling research - ---- - -## Data Usage Guidelines - -### For Testing - -- Tests automatically use cached data when available -- Download occurs only if data is missing -- Tests use subsets (e.g., first 2 time points) for speed - -### For Experiments - -- Full datasets used for complete analysis -- Results saved to respective `experiments/*/results/` directories -- Original data remains unmodified - -### For Development - -- Use small subsets for rapid iteration -- Full datasets for validation and benchmarking -- Cache intermediate results to avoid reprocessing - ---- - -## Data Access and Licensing - -- **Slicer-Heart-CT** : Public release from GitHub (auto-download available) -- **DirLab-4DCT** : Public benchmark dataset (manual download required, may require registration) -- **KCL-Heart-Model** : Public release from Zenodo (auto-download available) -- **CHOP-Valve4D** : PhysioTwin4D convenience release under CC-BY license (auto-download available) - -**Important**: Always cite the original data sources in publications and respect any usage restrictions. - -### Summary of Download Methods - - -| Dataset | Auto-Download | Manual Required | License | Source | Used in Tests | -| --------------- | ------------- | --------------- | --------------- | ------------------- | ------------------------ | -| Slicer-Heart-CT | Yes | No | Public | GitHub | Yes | -| DirLab-4DCT | No | Yes | Public/Academic | DIR-Lab | No | -| KCL-Heart-Model | Yes | No | Check citation | Zenodo/KCL | Yes (skipped if missing) | -| CHOP-Valve4D | Yes | No | CC-BY | GitHub release | No | - - ---- - -## References - -### Slicer-Heart-CT - -- Jolley Lab: [https://www.linkedin.com/company/jolleylab](https://www.linkedin.com/company/jolleylab) -- GitHub: [https://github.com/SlicerHeart/SlicerHeart](https://github.com/SlicerHeart/SlicerHeart) - -### DirLab-4DCT - -- Castillo et al., "A reference dataset for deformable image registration spatial accuracy evaluation using the COPDGene study archive" -- DIR-Lab: [https://med.emory.edu/departments/radiation-oncology/research-laboratories/deformable-image-registration/](https://med.emory.edu/departments/radiation-oncology/research-laboratories/deformable-image-registration/) - -### KCL-Heart-Model - -- Rodero et al. (2021), "Linking statistical shape models and simulated function in the healthy adult human heart", *PLOS Computational Biology* -- DOI: [10.1371/journal.pcbi.1008851](https://doi.org/10.1371/journal.pcbi.1008851) -- Zenodo: [https://zenodo.org/records/4590294](https://zenodo.org/records/4590294) - -### CHOP-Valve4D - -- Jolley Lab (CHOP): [https://www.linkedin.com/company/jolleylab](https://www.linkedin.com/company/jolleylab) -- Original FEBio source model: [repo.febio.org/permalink/project/136](https://repo.febio.org/permalink/project/136) -- License: Creative Commons Attribution (CC-BY) -- Citation: Please acknowledge Jolley Lab at CHOP and the FEBio Project - ---- - -## Tips - -1. **Storage**: Ensure adequate disk space (~10-20GB for all datasets) -2. **Download Time**: Initial downloads can be slow; be patient -3. **Organization**: Keep data organized; don't modify original files -4. **Backups**: Consider backing up processed results separately -5. **Documentation**: Update this README when adding new datasets - +This directory holds the sample datasets used for experiments, testing, and +development of the PhysioTwin4D library. Each subdirectory contains one +dataset and its own `README.md` with download instructions, specifications, +and citation — this file is just an index; treat the per-dataset READMEs as +the source of truth. + +## Datasets + +| Directory | Description | Provided By | Download | README | +| --- | --- | --- | --- | --- | +| `Slicer-Heart-CT/` | 4D cardiac CT with gated cardiac phases | Jolley Lab, Children's Hospital of Philadelphia (CHOP) | Automatic | [Slicer-Heart-CT/README.md](Slicer-Heart-CT/README.md) | +| `DirLab-4DCT/` | 4D lung CT respiratory motion benchmark | DIR-Lab, MD Anderson Cancer Center / Emory University | Manual | [DirLab-4DCT/README.md](DirLab-4DCT/README.md) | +| `KCL-Heart-Model/` | Statistical shape model of the heart | King's College London (KCL) | Automatic | [KCL-Heart-Model/README.md](KCL-Heart-Model/README.md) | +| `CHOP-Valve4D/` | 4D valve reconstruction models | Jolley Lab, CHOP (original FEBio model) | Automatic | [CHOP-Valve4D/README.md](CHOP-Valve4D/README.md) | +| `test/` | pytest-managed cache; not a downloadable dataset | — | N/A | [test/README.md](test/README.md) | + +## Automatic Download + +`Slicer-Heart-CT`, `KCL-Heart-Model`, and `CHOP-Valve4D` can be fetched with +the `physiotwin4d-download-data` CLI or `DataDownloadTools`; see each +dataset's README for the exact command. `DirLab-4DCT` has no automatic +downloader — DIR-Lab distributes each case individually and may require +registration, so it must be obtained manually; see +[DirLab-4DCT/README.md](DirLab-4DCT/README.md). + +## Notes + +- Always cite the original data source in publications — see each dataset's + README for the required citation. +- The full set of datasets is ~10-20 GB; ensure adequate disk space before + downloading everything.