cuDSS implementations of Cholesky solver and refactorization by andrewxu319 · Pull Request #459 · ORNL/ReSolve

andrewxu319 · 2026-07-21T20:57:57Z

Description

Added cuDSS implementations of Cholesky solver and refactorization on CUDA. These are currently implemented in cuSolver, which is an older library. Updated unit tests for refactorization to test both the cuDSS and cuSolver implementations.

Proposed changes

Cholesky solver: rewrote with cuDSS, but retained cuSolver implementation. The user can pass the use_cudss parameter in the constructor to decide which version to use. It is true by default. The cuDSS implementation is around 4x the speed of the cuSolver implementation.

Refactorization: added LinSolverDirectCuDssRf class as an independent solver. It is very slightly slower than the cuSolver implementation. The interface is kept unchanged, but the following variables are actually unused:

Matrices L and U are allocated internally by cuDSS
Permutation vector Q is unused, only P. cuDSS only takes one permutation vector and computes the other
nboost / pivot_boost have no equivalents in cuDSS
Other changes:
Updated SystemSolver.cpp to accept cudssrf as a factorization method
Updated testRefactor.cpp and testSysRefactor.cpp to run tests on both implementations

Checklist

Further comments

shakedregev · 2026-07-22T13:14:58Z

Fixed the changelog "conflict". Reviewing and testing this now. Please pull before making new changes.

shakedregev · 2026-07-22T13:26:36Z

+    error_sum += cudssMatrixCreateDn(&rhs_descr, n, 1, n, rhs->getData(memory::DEVICE), CUDA_R_64F, CUDSS_LAYOUT_COL_MAJOR);
+    error_sum += cudssMatrixCreateDn(&x_descr, n, 1, n, x->getData(memory::DEVICE), CUDA_R_64F, CUDSS_LAYOUT_COL_MAJOR);


Nothing to fix here, but I noted that CUDA doesn't support row major. This could cause conversion issues with the code down the line.

shakedregev

Please make a new example which uses cudss. Ensure that the old examples with cuSolver still work with no changes. Also, unless you have tested and shown that transposing on the host is faster, I would transpose on the device. I expect it to be faster for large matrices. No need to reimplement everything, just call our existing functions.

…acro

andrewxu319 · 2026-07-22T15:20:48Z

Please make a new example which uses cudss. Ensure that the old examples with cuSolver still work with no changes. Also, unless you have tested and shown that transposing on the host is faster, I would transpose on the device. I expect it to be faster for large matrices. No need to reimplement everything, just call our existing functions.

What are you referring to by transpose?

andrewxu319 · 2026-07-22T17:55:05Z

Please make a new example which uses cudss. Ensure that the old examples with cuSolver still work with no changes. Also, unless you have tested and shown that transposing on the host is faster, I would transpose on the device. I expect it to be faster for large matrices. No need to reimplement everything, just call our existing functions.

Done. The new example cudssRefactor is a modified copy of sysRefactor.

shakedregev · 2026-07-23T14:40:41Z

+        cudssMatrixDestroy(descr_b_);
+        cudssMatrixDestroy(descr_x_);
+      }
+      else


what is this else for? it seems within the nested if, but then the commands are outside the outer if

If RESOLVE_USE_CUDSS is defined, there is an if/else that checks use_cudss_ and takes one branch. If RESOLVE_USE_CUDSS is not defined, there is no if/else, so the else is compiled out.

…ntations

…e other Cholesky classes

shakedregev · 2026-07-24T17:47:29Z

+#ifdef RESOLVE_USE_CUDSS
+    }
+    else if (refactorizationMethod_ == "cudssrf")
+    {
+      refactorizationSolver_ = new ReSolve::LinSolverDirectCuDssRf();
+#endif


I find this writing style very confusing, though I see you are copying it and see why it works. I really this brackets should be within #ifdef statements.

shakedregev · 2026-07-24T17:48:07Z

+#ifdef RESOLVE_USE_CUDSS
+    else if (refactorizationMethod_ == "cudssrf")
+    {
+      LinSolverDirectCuDssRf* Rf = dynamic_cast<LinSolverDirectCuDssRf*>(refactorizationSolver_);
+      Rf->setNumericalProperties(1e-14, 1e-1);
+
+      status += refactorizationSolver_->setup(A_, L_, U_, P_, Q_);
+
+      is_solve_on_device_ = false;
+    }
+#endif


This way looks much better and is easier to follow.

Co-authored-by: Shaked Regev <35384901+shakedregev@users.noreply.github.com>

shakedregev

Great work. Checked the tests and examples

andrewxu319 and others added 4 commits July 21, 2026 16:19

cuDSS implementation of Cholesky

490439d

cuDSS implementation of refactorization (rf)

2e9ea51

Update changelog and fix typo

ed65374

Apply pre-commmit fixes

7f33a4b

andrewxu319 requested a review from shakedregev July 21, 2026 20:58

Merge branch 'develop' into andrew/cudss

b729725

shakedregev reviewed Jul 22, 2026

View reviewed changes

Comment thread cmake/ReSolveFindCudaLibraries.cmake Outdated

shakedregev reviewed Jul 22, 2026

View reviewed changes

Comment thread resolve/LinSolverDirectCuDssRf.cpp Outdated

Apply suggestion from @shakedregev

fd1d825

shakedregev reviewed Jul 22, 2026

View reviewed changes

Comment thread resolve/LinSolverDirectCuDssRf.cpp Outdated

Apply suggestion from @shakedregev

c4c24bb

shakedregev reviewed Jul 22, 2026

View reviewed changes

shakedregev requested changes Jul 22, 2026

View reviewed changes

andrewxu319 and others added 2 commits July 22, 2026 14:44

Make cuDSS an optional dependency enabled via the RESOLVE_USE_CUDSS m…

d7336cd

…acro

Apply pre-commmit fixes

7dce176

andrewxu319 and others added 2 commits July 22, 2026 17:53

Add cudssRefactor example

1b93dab

Apply pre-commmit fixes

a39a689

andrewxu319 requested a review from shakedregev July 22, 2026 17:55

shakedregev reviewed Jul 23, 2026

View reviewed changes

andrewxu319 and others added 4 commits July 23, 2026 15:33

Make Cholesky unit tests test both the CuSolver and the cuDSS impleme…

b024bca

…ntations

Apply pre-commmit fixes

df242ed

Split CholeskySolverCuda into two versions and reverted changes to th…

f07044b

…e other Cholesky classes

Apply pre-commmit fixes

3380b90

shakedregev reviewed Jul 24, 2026

View reviewed changes

Comment thread resolve/hykkt/cholesky_cudss/CholeskySolverCuDss.cpp Outdated

shakedregev reviewed Jul 24, 2026

View reviewed changes

Comment thread tests/unit/hykkt/runHykktCholeskyCuDssTests.cpp Outdated

shakedregev and others added 3 commits July 24, 2026 13:51

Minor authorship changes

25679f5

Co-authored-by: Shaked Regev <35384901+shakedregev@users.noreply.github.com>

Make separate tests for cuDSS refactor

8c63206

Apply pre-commmit fixes

ca98f19

shakedregev approved these changes Jul 24, 2026

View reviewed changes

shakedregev reviewed Jul 24, 2026

View reviewed changes

Comment thread examples/cudssRefactor.cpp Outdated

Apply suggestion from @shakedregev

196d4df

shakedregev merged commit 6038206 into develop Jul 24, 2026
6 checks passed

shakedregev deleted the andrew/cudss branch July 24, 2026 19:06

		error_sum += cudssMatrixCreateDn(&rhs_descr, n, 1, n, rhs->getData(memory::DEVICE), CUDA_R_64F, CUDSS_LAYOUT_COL_MAJOR);
		error_sum += cudssMatrixCreateDn(&x_descr, n, 1, n, x->getData(memory::DEVICE), CUDA_R_64F, CUDSS_LAYOUT_COL_MAJOR);

Uh oh!

Conversation

andrewxu319 commented Jul 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Proposed changes

Checklist

Further comments

Uh oh!

shakedregev commented Jul 22, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

shakedregev Jul 22, 2026

Choose a reason for hiding this comment

Uh oh!

shakedregev left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

andrewxu319 commented Jul 22, 2026

Uh oh!

andrewxu319 commented Jul 22, 2026

Uh oh!

shakedregev Jul 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

andrewxu319 Jul 23, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

shakedregev Jul 24, 2026

Choose a reason for hiding this comment

Uh oh!

shakedregev Jul 24, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

shakedregev left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

andrewxu319 commented Jul 21, 2026 •

edited

Loading

shakedregev commented Jul 22, 2026 •

edited

Loading

shakedregev left a comment •

edited

Loading

shakedregev Jul 23, 2026 •

edited

Loading

shakedregev Jul 24, 2026 •

edited

Loading