Skip to content

Make CPU fallback type policy compile-time#29647

Open
GopalakrishnanN wants to merge 3 commits into
mainfrom
GopalakrishnanN/constexpr-kernel-capability-types
Open

Make CPU fallback type policy compile-time#29647
GopalakrishnanN wants to merge 3 commits into
mainfrom
GopalakrishnanN/constexpr-kernel-capability-types

Conversation

@GopalakrishnanN

Copy link
Copy Markdown
Contributor

Description

  • Move the CPU fallback unsupported-type policy from seven runtime ONNX type-registry lookups to a constexpr string-view table.
  • Validate at compile time that capability entries are non-empty, unique, and sorted.
  • Use a compact constexpr predicate in GetCpuPreferredNodes while preserving the existing full type-ID semantics.
  • Add compile-time coverage and a focused runtime test, including the intentional distinction between float16 and tensor(float16).

Motivation and Context

GetCpuPreferredNodes evaluated each candidate input against seven calls to DataTypeUtils::ToType, which accesses ONNX's global type registry. The capability policy is fixed metadata, so it can be represented and validated at compile time and queried without registry access.

This PR deliberately preserves the current string-ID behavior. It does not reinterpret the policy as tensor element-type matching; that would be a separate behavioral change.

Local Windows RelWithDebInfo measurements, using a stabilized build and forcing only fallback_cpu_capability.cc stale:

Metric Before After Change
Incremental build 11.13 s 9.85 s -11.5%
fallback_cpu_capability.obj 3,395,972 B 3,387,080 B -8,892 B
onnxruntime_framework.lib 196,933,848 B 196,926,104 B -7,744 B

These measurements are toolchain-specific; the compile-time policy validation and removal of registry lookups are the primary benefits.

Validation

  • lintrunner -a onnxruntime/core/framework/fallback_cpu_capability.h onnxruntime/core/framework/fallback_cpu_capability.cc onnxruntime/test/framework/fallback_cpu_capability_test.cc
  • cmake --build build/Windows/RelWithDebInfo --config RelWithDebInfo --target onnxruntime_test_all --parallel 8
  • onnxruntime_test_all.exe --gtest_filter="FallbackCpuCapabilityTest.*" (1/1 passed)

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR refactors the CPU-fallback “unsupported type” policy used by GetCpuPreferredNodes to be represented as a compile-time constexpr table instead of repeated runtime lookups into ONNX’s global type registry, and adds targeted compile-time/runtime validation in tests.

Changes:

  • Introduces a constexpr std::array<std::string_view> table of unsupported CPU fallback type IDs with compile-time validation (non-empty, unique, sorted).
  • Updates GetCpuPreferredNodes to use a compact predicate (IsUnsupportedCpuFallbackType) instead of multiple DataTypeUtils::ToType(...) lookups.
  • Adds a focused unit test with both compile-time (static_assert) and runtime checks, including coverage for the float16 vs tensor(float16) distinction.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 2 comments.

File Description
onnxruntime/core/framework/fallback_cpu_capability.h Adds the compile-time unsupported-type table and validation predicate.
onnxruntime/core/framework/fallback_cpu_capability.cc Replaces repeated ONNX type-registry lookups with the new table-based predicate in GetCpuPreferredNodes.
onnxruntime/test/framework/fallback_cpu_capability_test.cc Adds compile-time coverage checks and a small runtime classification test for type-name behavior.

Comment thread onnxruntime/core/framework/fallback_cpu_capability.cc Outdated
Comment thread onnxruntime/core/framework/fallback_cpu_capability.cc Outdated
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants