Support unequal-length samples in stochastic dominance and nns_norm#80
Merged
Merged
Conversation
R's NNS.FSD/NNS.SSD/NNS.TSD evaluate each sample's LPM curve on the merged threshold grid and never require equal sample sizes; only the C++ .uni walkers assume equal lengths. The Python port carried that equal-length restriction into all pairwise dominance tests, rejecting valid comparisons such as mtcars mpg by transmission (19 vs 13 obs). Drop the length gate in _dominates_uni: the merged-grid curve evaluation is already size-agnostic, and the sorted-sample equality guard degrades safely to False for mismatched shapes. Results for equal-length inputs are unchanged. Tests cover unequal-length dominance, antisymmetry, uni wrappers, the mtcars case documented against R output, and randomized agreement with a literal transcription of R's decision rule for degrees 1-3. Docs note that the *_uni unequal-length path is an intentional extension over the R C++ routines. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JMQ9tSZyAMwerv7BdY6E5w
nns_norm previously implemented only the numeric matrix path. R's NNS.norm also accepts a list of per-variable vectors, so mirror that: - A sequence of equal-length 1-D arrays is column-stacked and normalized through the existing matrix path (R's mapply simplifies the equal-length list result to a matrix; with the companion R fix, the nonlinear scale factors now work for lists there too). - Unequal-length vectors force linear=True exactly as R does, since dependence-based scale factors need aligned columns, and return a list of scaled arrays. Length mismatch uses set semantics, covering patterns like (5, 6, 5) that R's sum(diff(lengths)) check missed. Note the sequence convention: elements are variables (columns), not observation rows, matching R's list semantics. ndarray input is unchanged. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JMQ9tSZyAMwerv7BdY6E5w
Widening nns_norm's return annotation to include list broke mypy at existing internal call sites (causation.py, diff.py) that pass 2-D arrays and index the result directly. Overload the signature: ndarray input returns ndarray; sequence input returns ndarray or list. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JMQ9tSZyAMwerv7BdY6E5w
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This PR extends stochastic dominance functions (
fsd,ssd,tsd, and their*_uniwrappers) to support samples of unequal length, and enhancesnns_normto accept sequence inputs (lists/tuples of vectors) in addition to matrices.Key Changes
Stochastic Dominance (src/nns/stochastic_dominance.py)
NNS.FSD/NNS.SSD/NNS.TSDbehavior_dominates_unithat previously raisedValueErrorxandymay differ in length*_uniwrappers to support unequal lengths with the same merged-grid semantics (an intentional extension beyond R's C++.uniroutines which assume equal lengths)NNS Norm (src/nns/norm.py)
nns_normnow accepts either:mapplysimplification)linear=True(as R does, since dependence-based scale factors require aligned columns) and return a list of scaled arrays_as_vectorhelper for input validation with clear error messages_norm_unequal_serieshelper implementing linear scaling for unequal-length sequencesTests (tests/invariants/)
test_stochastic_dominance.py:
_r_sd_referencehelper transcribing R's decision rule for validationtest_unequal_length_shifted_samples_dominateverifying dominance with different sample sizestest_unequal_length_antisymmetryconfirming antisymmetry property holds for unequal lengthstest_mtcars_transmission_groups_match_rvalidating against real R dataset resultstest_unequal_length_matches_r_decision_rulecomparing against R's reference implementation across multiple random seeds and dominance degreestest_norm.py:
test_equal_length_sequence_matches_matrix_pathverifying sequence and matrix paths produce identical resultstest_unequal_length_sequence_forces_linear_scalingconfirming linear scaling behavior and forcedlinear=Truetest_zero_sum_length_differences_detected_as_unequaledge case for lengths with zero-sum pairwise differencestest_sequence_input_validationcovering error cases (empty sequence, non-1D arrays, non-finite values)Documentation
Implementation Details
LPM(degree, sort(c(x, y)), sample)per samplehttps://claude.ai/code/session_01JMQ9tSZyAMwerv7BdY6E5w