Add invert_permute operator with SYCL implementation for XPU by aagalleg · Pull Request #71 · intel/torchlib-xpu

aagalleg · 2026-06-18T23:20:20Z

This PR introduces the invert_permute operator to fbgemm-xpu, enabling inverse permutations on Intel XPU devices.

Changes

Core Implementation

SYCL Kernel (invert_permute_kernel.cpp/h): High-performance SYCL implementation that computes inverse permutations on XPU devices
Operator Registration (ops_registry.cpp): Registers the operator with PyTorch's dispatch system using conditional schema registration to avoid conflicts
Python API (ops.py): Clean Python wrapper function with type hints for easy integration

Infrastructure

Build System (CMakeLists.txt): Added SYCL kernel to build configuration
Torch Library Utilities (torch_library.h): Helper utilities for schema existence checking and operator registration
Cleanup: Removed .gitkeep files from populated directories

Testing

Comprehensive Test Suite (test_invert_permute.py):
- Correctness validation for int32/int64 data types
- Edge cases (empty, single element, identity, reverse permutations)
- Input validation and error handling
- CPU-XPU consistency verification

cc: @manuelhsantana, @flezaalv

dvrogozh

@aagalleg, I missed the beginning of the PRs chain and left all the comments in #73. Comments are pretty generic, can you please, refer to them?

For the convenience, please, mark all follow up PRs as drafts - in this way we will clearly know what should be reviewed first.

dvrogozh

In addition to inline comments:

Repository has lint CI - please, address identified issues.
What we implement and merge must be tested under CI. And IMHO doing that by engineers who implemented the code is the best variant rather than offloading that to dedicated CI engineers. Thus, please, modify .github/workflows/ci.yml and add a test section for FBGEMM following the pattern outlined for torchcodec (see link below). The test job for FBGEMM will be much simpler. Target only bmg for now is fine. Note that fbgemm wheel is already getting build by CI.

torchlib-xpu/.github/workflows/ci.yml

Line 199 in d4230aa

test-torchcodec:

flezaalv · 2026-07-07T17:20:59Z

In addition to inline comments:

Repository has lint CI - please, address identified issues.

What we implement and merge must be tested under CI. And IMHO doing that by engineers who implemented the code is the best variant rather than offloading that to dedicated CI engineers. Thus, please, modify .github/workflows/ci.yml and add a test section for FBGEMM following the pattern outlined for torchcodec (see link below). The test job for FBGEMM will be much simpler. Target only bmg for now is fine. Note that fbgemm wheel is already getting build by CI.

torchlib-xpu/.github/workflows/ci.yml

Line 199 in d4230aa

test-torchcodec:

Hi @dvrogozh, regarding the CI, we added it here: #86. Please take a look and let me know if you have any comments.

aagalleg · 2026-07-09T22:28:02Z

In addition to inline comments:

Repository has lint CI - please, address identified issues.

What we implement and merge must be tested under CI. And IMHO doing that by engineers who implemented the code is the best variant rather than offloading that to dedicated CI engineers. Thus, please, modify .github/workflows/ci.yml and add a test section for FBGEMM following the pattern outlined for torchcodec (see link below). The test job for FBGEMM will be much simpler. Target only bmg for now is fine. Note that fbgemm wheel is already getting build by CI.

torchlib-xpu/.github/workflows/ci.yml

Line 199 in d4230aa

test-torchcodec:

1- Done.
2- Regarding testing. The custom test_invert_permute.py has been removed. Instead, we follow the same convention as torchcodec-xpu and created a patch to be applied to the upstream FBGEMM tests v1.7.0. The patch adapts fbgemm_gpu/test/sparse/misc_ops_test.py to run test invert_permute on XPU. See 0001-Add-XPU-support-to-fbgemm-tests.patch.
Once PR #86 is merged to main I'll add the test to the CI.

dvrogozh

@aagalleg, this looks exactly as what I expect to see from the enabling stand point. The only missing part is actual integration into CI run here: https://github.com/intel/torchlib-xpu/blob/main/.github/workflows/ci.yml. Do you work on that?

aagalleg · 2026-07-20T22:45:28Z

@aagalleg, this looks exactly as what I expect to see from the enabling stand point. The only missing part is actual integration into CI run here: https://github.com/intel/torchlib-xpu/blob/main/.github/workflows/ci.yml. Do you work on that?

@flezaalv was working on the CI integration in PR #86. He added a fbgemm-ci.yml file in the workflows directory, not directly to ci.yml.

flezaalv · 2026-07-21T17:39:57Z

@aagalleg, this looks exactly as what I expect to see from the enabling stand point. The only missing part is actual integration into CI run here: https://github.com/intel/torchlib-xpu/blob/main/.github/workflows/ci.yml. Do you work on that?

@flezaalv was working on the CI integration in PR #86. He added a fbgemm-ci.yml file in the workflows directory, not directly to ci.yml.

@dvrogozh I added a dedicated CI file because I think torchcodec-xpu and fbgemm-xpu packages will have different release times. Please let me know if you have any comments.

dvrogozh · 2026-07-21T19:01:54Z

@dvrogozh I added a dedicated CI file because I think torchcodec-xpu and fbgemm-xpu packages will have different release times. Please let me know if you have any comments.

We have a complexity here as besides torchcodec-xpu and fbgemm-xpu we also have an umbrella package called torchlib-xpu which depends on other 2. As a result we can't actually decouple releases of torchcodec-xpu and fbgemm-xpu right away. While we maintain packages in current design paradigm I prefer to have a single ci.yml file which handles everything. I have an idea how to reduce burden on the testing - I will post a PR shortly. Hold on.

aagalleg · 2026-07-28T14:37:50Z

@dvrogozh Patch apply and testing were added to CI.

dvrogozh

../../../../github/home/miniconda3/envs/test-env/lib/python3.10/site-packages/torch/xpu/__init__.py:251
  /github/home/miniconda3/envs/test-env/lib/python3.10/site-packages/torch/xpu/__init__.py:251: UserWarning: Can't initialize Level Zero Sysman
    return _enum_zes_device_infos(visible_mask)

../../../../github/home/miniconda3/envs/test-env/lib/python3.10/site-packages/torch/xpu/__init__.py:278
  /github/home/miniconda3/envs/test-env/lib/python3.10/site-packages/torch/xpu/__init__.py:278: UserWarning: XPU device count is zero! (Triggered internally at /__w/pytorch/pytorch/c10/xpu/XPUFunctions.cpp:117.)
    count = torch._C._xpu_getDeviceCount() if zes_count < 0 else zes_count

These warnings in the tests are related to XPU. I am especially worried about the 2nd one as it basically says that XPU is unavailable. With that I am confused considering that torch.xpu.is_available() was returning True.

dvrogozh · 2026-07-28T15:12:44Z

I am especially worried about the 2nd one as it basically says that XPU is unavailable. With that I am confused considering that torch.xpu.is_available() was returning True.

Same warning exists in TorchCodec CI as well: https://github.com/intel/torchlib-xpu/actions/runs/29928230040/job/88954373767.

dvrogozh · 2026-07-28T15:26:02Z

https://github.com/pytorch/pytorch/blob/18f0b90367333ef99381cb3cb2164085a4733f04/torch/xpu/__init__.py#L278

Ok, I think above 2 warnings are related. The XPU works for us because there is a fallback to detect XPU devices with SYCL runtime if detection with L0 has failed. I am not sure though why L0 path has failed - this seems to be an issue (not with torchlib).

Remove .gitkeep placeholders

- Add invert_permute kernel to CMake build - Implement invert_permute Python wrapper in ops.py - Register invert_permute operator with schema existence check - Add torch_library.h utility for schema validation

Add SYCL/XPU kernel implementation for invert_permute operation.

Add complete test coverage for invert_permute operator on XPU devices, covering correctness, validation, parity, and performance. Test coverage includes: - Correctness tests for int32/int64 with edge cases (empty, single element, identity, reverse, random permutations) - Input validation tests for invalid dimensions and dtypes - Meta function tests for torch.compile compatibility - PyTorch opcheck validation for operator conventions - Parametric tests with varying sizes (1 to 1M elements) - CPU-XPU parity tests to ensure consistent results - Performance benchmarks measuring execution time and bandwidth

Replace the custom standalone test_invert_permute.py with a git-am patch applied to upstream FBGEMM v1.7.0 misc_ops_test.py, following the torchcodec-xpu convention. The patch makes test_invert_permute run on XPU (permute.xpu(), gated on torch.xpu.is_available()) and skips the remaining operator tests that are not implemented on XPU.

Collapse the two type-specific kernel functors (InvertPermuteKernelInt32, InvertPermuteKernelInt64) into a single templated functor InvertPermuteKernel<index_t>, mirroring the reference CUDA kernel invert_permute_kernel<index_t>.

… order

…cture - Fix test patches to match FBGEMM v1.8.0 tests. - Move test patches from test/patches/ subdirectory to patches/ at the package root level for better organization. Remove now-unnecessary .gitkeep file and update patch with correct base commit reference.

Replace TODO placeholders with actual commands to apply the FBGEMM XPU test patch and execute pytest for misc_ops_test.py in the CI pipeline.

dvrogozh · 2026-07-29T18:21:55Z

Issue with CI runner is tracked and closed in #104.

dvrogozh

LGTM

aagalleg requested review from dvrogozh, eromomon and luis-real as code owners June 18, 2026 23:20

aagalleg mentioned this pull request Jun 18, 2026

Add permute_1D_sparse_data Operation for Intel XPU #72

Open

dvrogozh reviewed Jun 22, 2026

View reviewed changes

Comment thread packages/fbgemm-xpu/src/fbgemm_xpu/fbgemm_utils/torch_library.h Outdated

Comment thread packages/fbgemm-xpu/src/fbgemm_xpu/sycl_kernels/invert_permute_kernel.cpp Outdated

Comment thread packages/fbgemm-xpu/src/fbgemm_xpu/ops.py Outdated

flezaalv mentioned this pull request Jul 3, 2026

fbgemm-xpu: add CI workflow #86

Merged

flezaalv mentioned this pull request Jul 17, 2026

Add permute_2D_sparse_data operator with SYCL implementation for XPU #74

Draft

dvrogozh requested changes Jul 20, 2026

View reviewed changes

aagalleg force-pushed the feat/invert_permute branch from 6728c1e to df495b2 Compare July 27, 2026 23:11

dvrogozh reviewed Jul 28, 2026

View reviewed changes

dvrogozh mentioned this pull request Jul 28, 2026

UserWarning: XPU device count is zero! #104

Closed

aagalleg added 10 commits July 29, 2026 16:24

chore(fbgemm-xpu): remove .gitkeep files from populated directories

f7eea23

Remove .gitkeep placeholders

feat(fbgemm-xpu): register invert_permute operation

c050b11

- Add invert_permute kernel to CMake build - Implement invert_permute Python wrapper in ops.py - Register invert_permute operator with schema existence check - Add torch_library.h utility for schema validation

feat(fbgemm-xpu): add SYCL kernel implementation for invert_permute

cce1fee

Add SYCL/XPU kernel implementation for invert_permute operation.

style(fbgemm-xpu): add missing newline at end of torch_library.h

d280078

fix(fbgemm-xpu): remove redundant ops.py and ensure fbgemm_gpu import…

da64394

… order

ci(fbgemm-xpu): apply XPU patch and run tests in workflow

bacbaec

Replace TODO placeholders with actual commands to apply the FBGEMM XPU test patch and execute pytest for misc_ops_test.py in the CI pipeline.

aagalleg force-pushed the feat/invert_permute branch from 6cad528 to bacbaec Compare July 29, 2026 16:25

dvrogozh approved these changes Jul 29, 2026

View reviewed changes

dvrogozh merged commit a14ebda into intel:main Jul 29, 2026
16 checks passed

aagalleg deleted the feat/invert_permute branch July 29, 2026 22:31

Uh oh!

Conversation

aagalleg commented Jun 18, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Changes

Core Implementation

Infrastructure

Testing

Uh oh!

dvrogozh left a comment

Choose a reason for hiding this comment

Uh oh!

dvrogozh left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

flezaalv commented Jul 7, 2026

Uh oh!

aagalleg commented Jul 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

dvrogozh left a comment

Choose a reason for hiding this comment

Uh oh!

aagalleg commented Jul 20, 2026

Uh oh!

flezaalv commented Jul 21, 2026

Uh oh!

dvrogozh commented Jul 21, 2026

Uh oh!

aagalleg commented Jul 28, 2026

Uh oh!

dvrogozh left a comment

Choose a reason for hiding this comment

Uh oh!

dvrogozh commented Jul 28, 2026

Uh oh!

dvrogozh commented Jul 28, 2026

Uh oh!

dvrogozh commented Jul 29, 2026

Uh oh!

dvrogozh left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

aagalleg commented Jun 18, 2026 •

edited

Loading

aagalleg commented Jul 9, 2026 •

edited

Loading