Revive block export and add PIPS-IPM++ solving#862
Open
FabianHofmann wants to merge 9 commits into
Open
Conversation
Reimplement the block-structured PIPS export against the current flat/blockmap API and cover it with an output-verifying test. Replace the per-block Python loop in Constraints.set_blocks with vectorized term-dim reductions, making the block calculation O(entries) instead of O(N_blocks * entries).
Add to_pips_files (CSR + indicator-flag exporter built on the internal matrix), read_pips_files/read_pips_problem/read_pips_solution, and expose to_pips_files as a Model method alongside the legacy COO to_block_files. Wire an env-gated PIPS solver (PIPS_BINARY/PIPS_MPI_RANKS via mpirun) and add a HiGHS round-trip test validating the reconstructed LP.
Independently reassemble the exported per-block CSR files, keying each nonzero to its (constraint, variable) label via the on-disk *_labels files, and assert equality against m.matrices.A plus the rhs/sense carve. Shares no code with read_pips_files, so a writer bug cannot be masked by the reader.
Document the external MUMPS-backed PIPS build (verified apt line, mainline clone, gams/gmspips disable, CMake install), the export/solve flow (m.to_pips_files, mpirun -np <=N, read_pips_solution), the PIPS-free round-trip validation, and the env-gated pips solver. Link it into the Solving toctree.
Clean up directory problem files (to_pips_files writes a block dir) with shutil.rmtree instead of os.remove, so m.solve(solver_name="pips") completes its finally-block. Add test_pips_end_to_end_solve: runs the real mpirun PIPS solve and checks the objective against HiGHS, skipped unless PIPS_BINARY is set (CI-safe).
Add an early 'Using the pips solver' section showing m.solve(solver_name= "pips") plus the PIPS_BINARY/PIPS_MPI_RANKS env vars, cross-referencing the installation steps as a prerequisite. Fold in and drop the redundant later 'Solver integration' section.
for more information, see https://pre-commit.ci
Merging this PR will not alter performance
Comparing 
Footnotes
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This revives
to_block_filesunder current linopy conventions and builds out the missing consumer half — a full, tested export-and-solve path to the distributed block solver PIPS-IPM++.Note
The following content was generated by AI.
Changes proposed in this Pull Request
Block export
to_block_filesunder current linopy conventions; it stays as the COO debug/visualization writer.Constraints.set_blocksso block assignment no longer scales with the number of blocks (it previously looped per block).PIPS-IPM++ export / import
Model.to_pips_filesexports a block-structured model (assigned viaModel.blocks) into PIPS's doubly-bordered block-diagonal ("arrowhead") format, deriving theA/B/C/D/BL/DLsubmatrices from linopy's internal CSR (so duplicate terms are already summed) and encoding±infbounds as PIPS indicator flags.linopy.io.read_pips_filesreconstructs an equivalent flat model straight from the export, andread_pips_solutionreads the solver output back. The reconstruction makes the export verifiable with any solver and no PIPS build.Solver integration
pipssolver runs the export through PIPS undermpirunand maps the primal, duals and objective back onto the model (m.solve(solver_name="pips")). It stays absent fromavailable_solversunlessPIPS_BINARYis set, so ordinary installs are unaffected.shutil.rmtree), which the PIPS export produces.Tests
Model.matrices(matrix, rhs and sense), sharing no code with the reader.read_pips_files→ solve with HiGHS → equal objective).mpirunPIPS solve, skipped unlessPIPS_BINARYis set.Docs
doc/pips-installation.rst), added to the Solving section, leading with the one-linem.solve(solver_name="pips")usage.The external C++ callback driver that PIPS itself needs is kept under
dev-scripts/pips/(not shipped) and is documented in the guide. The full pipeline — export → MUMPS-backed PIPS solve → read-back — was validated end to end against HiGHS (primal, objective and duals). This path is experimental and not run in CI.Checklist
AGENTS.md).doc.doc/release_notes.rstof the upcoming release is included.