Skip to content

Support unequal-length samples in stochastic dominance and nns_norm#80

Merged
OVVO-Financial merged 3 commits into
mainfrom
claude/book-python-code-integration-xijzxt
Jul 5, 2026
Merged

Support unequal-length samples in stochastic dominance and nns_norm#80
OVVO-Financial merged 3 commits into
mainfrom
claude/book-python-code-integration-xijzxt

Conversation

@OVVO-Financial

Copy link
Copy Markdown
Owner

Summary

This PR extends stochastic dominance functions (fsd, ssd, tsd, and their *_uni wrappers) to support samples of unequal length, and enhances nns_norm to accept sequence inputs (lists/tuples of vectors) in addition to matrices.

Key Changes

Stochastic Dominance (src/nns/stochastic_dominance.py)

  • Unequal-length sample support: Pairwise dominance tests now evaluate each sample's LPM/CDF curve on a merged threshold grid (the union of both samples' sorted values), matching R's NNS.FSD/NNS.SSD/NNS.TSD behavior
  • Removed the length-equality constraint from _dominates_uni that previously raised ValueError
  • Updated docstrings to clarify that x and y may differ in length
  • Extended *_uni wrappers to support unequal lengths with the same merged-grid semantics (an intentional extension beyond R's C++ .uni routines which assume equal lengths)

NNS Norm (src/nns/norm.py)

  • Dual input convention support: nns_norm now accepts either:
    • A 2-D matrix (existing behavior)
    • A sequence (list/tuple) of 1-D vectors, matching R's list input convention
  • Smart path selection:
    • Equal-length vectors are column-stacked and normalized through the matrix path, returning a 2-D array (mirroring R's mapply simplification)
    • Unequal-length vectors force linear=True (as R does, since dependence-based scale factors require aligned columns) and return a list of scaled arrays
  • Added _as_vector helper for input validation with clear error messages
  • Added _norm_unequal_series helper implementing linear scaling for unequal-length sequences

Tests (tests/invariants/)

  • test_stochastic_dominance.py:

    • Added _r_sd_reference helper transcribing R's decision rule for validation
    • New test test_unequal_length_shifted_samples_dominate verifying dominance with different sample sizes
    • New test test_unequal_length_antisymmetry confirming antisymmetry property holds for unequal lengths
    • New test test_mtcars_transmission_groups_match_r validating against real R dataset results
    • Parametrized test test_unequal_length_matches_r_decision_rule comparing against R's reference implementation across multiple random seeds and dominance degrees
  • test_norm.py:

    • New test test_equal_length_sequence_matches_matrix_path verifying sequence and matrix paths produce identical results
    • New test test_unequal_length_sequence_forces_linear_scaling confirming linear scaling behavior and forced linear=True
    • New test test_zero_sum_length_differences_detected_as_unequal edge case for lengths with zero-sum pairwise differences
    • New test test_sequence_input_validation covering error cases (empty sequence, non-1D arrays, non-finite values)

Documentation

  • Updated API reference and conventions documentation to clarify unequal-length support and dual input conventions
  • Updated API status to reflect the extended capabilities

Implementation Details

  • The merged-grid approach for unequal-length dominance is mathematically equivalent to R's implementation: LPM(degree, sort(c(x, y)), sample) per sample
  • Linear scaling for unequal-length sequences uses a ratio grid of means to compute uniform scale factors, ensuring all scaled means equal the grand mean of input means
  • Input validation for sequence paths provides detailed error messages indicating which element failed and why

https://claude.ai/code/session_01JMQ9tSZyAMwerv7BdY6E5w

claude added 3 commits July 5, 2026 02:43
R's NNS.FSD/NNS.SSD/NNS.TSD evaluate each sample's LPM curve on the
merged threshold grid and never require equal sample sizes; only the
C++ .uni walkers assume equal lengths. The Python port carried that
equal-length restriction into all pairwise dominance tests, rejecting
valid comparisons such as mtcars mpg by transmission (19 vs 13 obs).

Drop the length gate in _dominates_uni: the merged-grid curve
evaluation is already size-agnostic, and the sorted-sample equality
guard degrades safely to False for mismatched shapes. Results for
equal-length inputs are unchanged.

Tests cover unequal-length dominance, antisymmetry, uni wrappers, the
mtcars case documented against R output, and randomized agreement with
a literal transcription of R's decision rule for degrees 1-3. Docs note
that the *_uni unequal-length path is an intentional extension over the
R C++ routines.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JMQ9tSZyAMwerv7BdY6E5w
nns_norm previously implemented only the numeric matrix path. R's
NNS.norm also accepts a list of per-variable vectors, so mirror that:

- A sequence of equal-length 1-D arrays is column-stacked and
  normalized through the existing matrix path (R's mapply simplifies
  the equal-length list result to a matrix; with the companion R fix,
  the nonlinear scale factors now work for lists there too).
- Unequal-length vectors force linear=True exactly as R does, since
  dependence-based scale factors need aligned columns, and return a
  list of scaled arrays. Length mismatch uses set semantics, covering
  patterns like (5, 6, 5) that R's sum(diff(lengths)) check missed.

Note the sequence convention: elements are variables (columns), not
observation rows, matching R's list semantics. ndarray input is
unchanged.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JMQ9tSZyAMwerv7BdY6E5w
Widening nns_norm's return annotation to include list broke mypy at
existing internal call sites (causation.py, diff.py) that pass 2-D
arrays and index the result directly. Overload the signature: ndarray
input returns ndarray; sequence input returns ndarray or list.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JMQ9tSZyAMwerv7BdY6E5w
@OVVO-Financial OVVO-Financial merged commit 18d5905 into main Jul 5, 2026
8 checks passed
@OVVO-Financial OVVO-Financial deleted the claude/book-python-code-integration-xijzxt branch July 5, 2026 15:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants