Skip to content

Add SlidingEuclideanDistanceNode for FFT-based sliding window distance computation (closes #78)#166

Open
ManojgowdaBY wants to merge 1 commit into
NVIDIA:mainfrom
ManojgowdaBY:main
Open

Add SlidingEuclideanDistanceNode for FFT-based sliding window distance computation (closes #78)#166
ManojgowdaBY wants to merge 1 commit into
NVIDIA:mainfrom
ManojgowdaBY:main

Conversation

@ManojgowdaBY

@ManojgowdaBY ManojgowdaBY commented Jul 21, 2026

Copy link
Copy Markdown

Summary

Closes #78

Adds SlidingEuclideanDistanceNode, a new gQuant transform node that computes the Euclidean distance between a query signal and every sliding window of the same length in a longer time series stream (the "distance profile"). This is a core primitive for shape-based pattern search and motif discovery in financial time series — e.g. finding where a known price/volume formation recurs in a stream.

Problem

There was no gQuant node for this. A naive per-position sliding comparison is O(n·m) (n = stream length, m = query length), which doesn't scale for large or frequently-updated streams, and hand-rolling it outside the TaskGraph loses column/type validation, GPU dataframe support, and gQuantLab UI integration.

Changes

  • Added transform/slidingDistanceNode.py implementing SlidingEuclideanDistanceNode, following the existing Node API conventions (columns_setup with required/addition/deletion/retention, process(self, inputs)).
  • Uses the FFT-based sliding dot-product trick (MASS algorithm) to compute the distance profile in O(n log n) instead of O(n·m):
    • moving sum of squares of the stream via cumulative sums
    • sliding dot product via FFT cross-correlation
  • Added an optional normalize config flag for z-normalized (amplitude/offset-invariant) distance, for shape-only comparison.
  • Registered the new node in transform/__init__.py (import + __all__).
  • Output is a new distance_profile column, front-padded with NaN so it aligns with the original stream index.

Node config

Key Type Description
stream_col str Column name of the longer time series to search
query_col str Column name of the short query/reference signal
normalize bool, optional (default False) Z-normalize each window for shape-only comparison

Testing

  • Unit test comparing FFT-based output against a naive O(n·m) loop on random data (atol=1e-6)
  • Benchmark showing speedup vs. naive loop at realistic stream sizes (10k–1M points)
  • Verified consistent results across CPU (numpy) and GPU (cupy/cudf) inputs

Alternatives considered

  1. Naive O(n·m) sliding loop — simple, useful as a correctness baseline, too slow at scale.
  2. numpy.correlate/scipy.signal.correlate alone — only covers the cross-correlation term, still needs pairing with the cumulative-sum-of-squares term.
  3. Skipping normalization — simpler, but loses shape-invariance to amplitude/offset differences.

Reference

Mueen's Algorithm for Similarity Search (MASS) — standard method for efficiently computing an all-subsequence Euclidean distance profile, basis for Matrix Profile motif discovery.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[FEA]Add sliding window Euclidean distance computation

1 participant