test(rf3): shorten and expand rf3 integration test#363
Open
woodsh17 wants to merge 8 commits into
Open
Conversation
…(4 aa) Replace the protein-only test input with a minimal 4-residue peptide (AGAG) to cut the non-ligand fixtures from ~30 min to ~10 min on CPU. Ligand tests (test_fold_from_json_with_ligand, test_fold_from_cif_with_ligand, test_ground_truth_conformer_selection) are unchanged and still use 1CYO. - Add agag_from_json.json and agag.cif as committed test data - Update conftest.py fixtures: basic_folds_dir, annotate_b_factor_dir, early_stopping_dir, seed_dirs, template_selection_dir, skip_existing_dirs - Update test_basic_fold.py, test_options.py, test_cpu_gpu_parity.py to reference agag_from_json / agag output names - Update integration_baselines/README.md GPU baseline generation command Note: AGA (3 residues) crashes in MaskPolymerResiduesWithUnresolvedFrameAtoms; AGAG (4 residues) is the minimum working length. See summary.txt for details. The CPU/GPU parity test skips until a GPU baseline for agag_from_json is committed. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Retire the 91-residue 1CYO ligand inputs in favour of the shared AGAG peptide paired with three ligands, modeled on docs/examples/7o1r_from_json.json: MG (ccd_code), HEM (SDF path), and imidazole (smiles). This exercises all three ligand-loading modes on a fast CPU-friendly input. - Add agag_with_ligands.json, ligands/HEM.sdf, and agag_with_ligands_from_cif.cif (real rf3 fold output committed as the CIF-mode input). - Remove 1cyo.cif, 1cyo_with_ligand.json, 1cyo_from_json.json, and the orphaned integration_baselines/1cyo_from_json/ baseline. - Point basic_folds_dir and ground_truth_conformer_dir (chain C = HEM) at the new inputs; update the two ligand tests and the ground-truth-conformer test. SDF/SMILES ligands get generated residue names (L:0, L:1), so tests assert on the 4-chain count plus the surviving MG/L:0 names rather than a literal HEM. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Cover rf3 input modes beyond the single-chain / single-ligand basics, all with minimal CPU-fast inputs: - test_complexes.py: two protein chains (asserts iptm>0 and a non-null cross-chain PAE, vs the single-chain iptm=0.0 sentinel), protein+DNA complex, covalent bond via the JSON `bonds` field, multiple examples in one JSON file, and directory input. - test_error_handling.py: nonexistent input path and malformed JSON both exit non-zero (fast; they fail before model load). - conftest.py: add complex_folds_dir and dir_input_dir fixtures plus a residue_names_in_cif() helper. Protein inputs use GLKE, not AGAG: a sequence over only the A/C/G/T/U alphabet is inferred as nucleic acid, so AGAG cannot stand in for a peptide. DNA is specified with chain_type=POLYDEOXYRIBONUCLEOTIDE. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
A sequence over only the A/C/G/T/U alphabet is inferred as nucleic acid, so the "AGAG" inputs were folding as RNA, not a peptide. Replace every AGAG test input with GLKE (an unambiguous 4-residue protein) and rename agag* -> glke*. - Rename glke_from_json.json, glke_with_ligands.json (seq AGAG -> GLKE). - Regenerate glke.cif and glke_with_ligands_from_cif.cif from real folds; the ligand chain layout is unchanged (A=protein, B=MG, C=HEM, D=imidazole), so ground_truth_conformer_selection=[C] still targets HEM. - Update conftest.py, test_basic_fold.py, test_options.py, test_cpu_gpu_parity.py, and integration_baselines/README.md accordingly. The parity baseline path is now integration_baselines/glke_from_json/. Full integration suite: 18 passed, 1 skipped (parity, no GLKE GPU baseline yet). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The batched session fixtures share a single seeded RNG stream, so each example's stochastic outputs (notably has_clash) depend on batch order. Document this on basic_folds_dir and complex_folds_dir so reordering inputs prompts a re-check of the has_clash assertions. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Fixes the integration-test findings from the branch review: - Fix CWD-dependent SDF ligand path: basic_folds_dir now materializes glke_with_ligands.json with the HEM.sdf path rewritten to absolute (anchored at REPO_ROOT), so the fold no longer requires pytest to run from the repo root. Verified passing from a non-repo CWD. - Extract assert_valid_plddt / assert_chain_count helpers, removing the triplicated 4-chain assertion (and its copy-pasted message) and the repeated pLDDT range check. - Replace raw "MG"/"L:0" CIF substring checks with residue_names_in_cif(). - Drop batch-order-sensitive has_clash assertions from the two multi-polymer complex tests (two_protein_chains, protein_dna_complex); keep them on the single-polymer inputs where has_clash is trivially False. - Strengthen the two-chain interface test to verify a 2x2 chain_pair_pae. - Anchor the error-handling pytest.raises with match="rf3 fold failed". - Rename dir_input_dir -> directory_input_dir; fix "DNA duplex" wording (the input is single-stranded) and a README grammar nit. - Correct the GPU-baseline regeneration command in the parity docstring and integration_baselines/README.md to include early_stopping_plddt_threshold=0.0 so it matches the fixture's SPEED_FLAGS (apples-to-apples). Validated: 18 passed, 1 skipped (GPU parity baseline) via .venv-host. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…ance Switch the CPU/GPU parity test from the single-chain glke_from_json input to the two-chain two_protein_chains input and tighten the per-metric tolerance from +/-0.05 to +/-0.02. The two-chain input has a real interface, so iptm (~0.0025) and ranking_score (~0.019) are genuine computed values rather than the iptm=0.0 sentinel single-chain inputs hit. Commit the GPU-generated two_protein_chains_summary_confidences.json baseline (generated on the JHU GPU, slurm017) so test_confidence_metrics_match_gpu_baseline is now active instead of skipping. Verified on CPU: the test PASSES against this baseline within +/-0.02.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Makes the
rf3 foldCPU integration suite fast enough for CI and broadensits coverage, then re-activates the CPU/GPU parity check against a fresh
GPU-generated baseline.
minimal 4-residue peptide, cutting the suite from ~30 min to ~5.5 min on
CPU. The peptide is
GLKE.tests: two protein chains (real interface metrics), protein + ssDNA,
peptide + NAG with an explicit covalent bond, multiple examples in one
JSON, directory input, and clean failures on missing/malformed input.
single-chain
glke_from_json(whereiptmis pinned at the0.0sentinel) to the two-chain
two_protein_chains, whose real interfacemakes
iptmandranking_scoregenuine computed values.