Add SlidingEuclideanDistanceNode for FFT-based sliding window distance computation (closes #78)#166
Open
ManojgowdaBY wants to merge 1 commit into
Open
Add SlidingEuclideanDistanceNode for FFT-based sliding window distance computation (closes #78)#166ManojgowdaBY wants to merge 1 commit into
ManojgowdaBY wants to merge 1 commit into
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Closes #78
Adds
SlidingEuclideanDistanceNode, a new gQuant transform node that computes the Euclidean distance between a query signal and every sliding window of the same length in a longer time series stream (the "distance profile"). This is a core primitive for shape-based pattern search and motif discovery in financial time series — e.g. finding where a known price/volume formation recurs in a stream.Problem
There was no gQuant node for this. A naive per-position sliding comparison is
O(n·m)(n= stream length,m= query length), which doesn't scale for large or frequently-updated streams, and hand-rolling it outside the TaskGraph loses column/type validation, GPU dataframe support, and gQuantLab UI integration.Changes
transform/slidingDistanceNode.pyimplementingSlidingEuclideanDistanceNode, following the existingNodeAPI conventions (columns_setupwithrequired/addition/deletion/retention,process(self, inputs)).O(n log n)instead ofO(n·m):normalizeconfig flag for z-normalized (amplitude/offset-invariant) distance, for shape-only comparison.transform/__init__.py(import +__all__).distance_profilecolumn, front-padded withNaNso it aligns with the original stream index.Node config
stream_colstrquery_colstrnormalizebool, optional (defaultFalse)Testing
O(n·m)loop on random data (atol=1e-6)numpy) and GPU (cupy/cudf) inputsAlternatives considered
O(n·m)sliding loop — simple, useful as a correctness baseline, too slow at scale.numpy.correlate/scipy.signal.correlatealone — only covers the cross-correlation term, still needs pairing with the cumulative-sum-of-squares term.Reference
Mueen's Algorithm for Similarity Search (MASS) — standard method for efficiently computing an all-subsequence Euclidean distance profile, basis for Matrix Profile motif discovery.