feat(studio): glue API coexistence layer for the NLE swap#2205
feat(studio): glue API coexistence layer for the NLE swap#2205ukimsanov wants to merge 2 commits into
Conversation
b35c4ba to
ed4eddc
Compare
cccd643 to
7f164b5
Compare
ed4eddc to
ba130cb
Compare
7f164b5 to
a2f4dd0
Compare
miga-heygen
left a comment
There was a problem hiding this comment.
Full Stack Review — NLE Overhaul (25 PRs, ~15,600 additions, ~10,100 deletions)
Reviewed the entire stack in four tiers: math/model layer (#2192–#2201), keystone API coexistence (#2205), UI components (#2202–#2210, #2214), and swap/cleanup (#2211–#2216). No blockers across the stack.
Keystone: #2205 — glue API coexistence (reviewed hardest)
This PR extends the existing studio APIs to support both the legacy engine and the new NLE simultaneously. It's the hinge between the "build the new" and "swap out the old" phases.
SSOT analysis — strong:
computeOverlayRootScaleextracted from inline code intoOverlayRect— now shared betweentoOverlayRectandelementCornerOverlayPoints. Single source for iframe→overlay mapping.restampManualOffsetDragGestureBaseandapplyManualOffsetCommitValueextracted fromapplyManualOffsetDragCommit— enables reuse for the new nudge path without duplicating the re-stamp logic.furthestClipEndFromDocumentextracted fromreadTimelineDurationFromDocument— the latter now delegates. Single computation of "furthest clip end." Correctly excludes the root element (prevents feedback loop where root duration is counted as clip content).patchDocumentRootDurationcorrectly finds the TOP-LEVEL root (no ancestor composition) — 6 well-targeted tests.- Union type on
onMoveElements(TimelineGroupMoveChange | TimelineGroupMoveEdit) allows both engines to coexist on the same callback without forking.
Edge case coverage:
orientedOverlayRectfalls back to plain AABB when corners can't be measured — safe degradation, pixel-identical to legacy at angle=0.readElementRotationDegreesguards NaN/Infinity on matrix components AND the output.resolveReloadSeekTimeuses??(not||) so explicit seek-to-0 isn't treated as missing.setTimelineScaleuses non-reactive mutation to avoid subscriber re-renders — documented as scratch state.
Coexistence hygiene: All TEMP fallow exemptions are marked "removed by studio-dnd/pr22" (#2213). The code-duplication ignores are clearly labeled as coexistence-window clones. #2213 confirms all ~30 exemptions are cleaned up.
One noted SSOT gap: readTimelineElementZIndex contains its own parseZ inline function and the comment says "Mirrors canvasContextMenuZOrder.parseZIndex semantics." Two z-index parsers — documented dependency, different packages. Acceptable.
Tier 1: Math/Model Layer (#2192–#2201) — all clean
Heavily unit-tested pure functions (2000+ test lines for ~1200 implementation lines). Each module owns one concern:
- #2192 — log-scale slider, zoom-pin, beat-source. Round-trip tests within 1%.
- #2193 — reload seek-restore. Double-seek guard trick for GSAP re-eval is clever.
- #2194 — batch files route.
shift-positions-batchfolds N shifts into one write. - #2195 — collision/placement. Half-open intervals, prefer-up search, insert-row detection. 292 lines of tests.
- #2196 — snapping. Priority dedup (playhead > clip > beat), clamped-snap-drop.
- #2197 — multi-drag preview. Rigid formation, leftmost-at-0 constraint,
-0avoidance. - #2198 — lane↔z stacking sync. Tie-aware cascade, multi-edit lower-first.
- #2199 — lane-zone model. Constrained packing. The z-to-lane round-trip convergence test is the kind of invariant test that prevents whole classes of regressions.
- #2200 — asset-click + nudge gate. Reference-counted claim/release, idempotent release.
- #2201 — gesture characterization. Center-anchored resize convergence proof + razor-split undo proof.
One nit: overlapsInTime (with epsilon) is copy-pasted identically between timelineStackingSync.ts (#2198) and timelineZones.ts (#2199). Should be a shared import. The exact-vs-epsilon distinction from timeRangesOverlap (#2195) is intentional and defensible.
Tier 2: UI Components (#2202–#2210, #2214) — all clean
Bottom-up build: pure math → state/context → drag engine → interaction hooks → chrome → shell → leaf UI.
- #2202 — z-order menu. Tie-aware DOM-order renumbering. Cross-realm
instanceoffix. - #2203 — canvas nudge. 1px / Shift 10px, composition-px not screen-px.
- #2204 — NLE shell + context. Nit: NLEContext has 25+ fields — future decomposition candidate.
- #2206 — clip-drag engine. Three-phase persist (patch-in-memory, atomic write, preview sync). Rollback on failure. Nit: fire-and-forget error path in
persistMoveEditsshould document that callers can't observe failure. - #2207 — timeline hooks + lanes. 40+ prop surface on TimelineLanes — consider grouping.
- #2208 — canvas chrome. Rotation-aware corner handles. Radial resize well-tested. Nit:
siblingZIndexEntrysilently drops untargetable elements — z patches can be partial. - #2209 — NLE shell assembly. Sub-comp coordinate rebasing (
toLocalElement) applied consistently across all edit paths. - #2210 — asset card. Click-vs-drag gate via pointer threshold.
- #2214 — clip thumbnails.
computeThumbnailStrip+resolveMediaPreviewUrlnow shared between image and video. SSOT win.
Tier 3: Swap + Cleanup (#2211–#2216)
- #2211 — canvas glue swap. AABB → OBB resize, nudge, context menu. All old consumers migrated.
- #2212 — timeline glue swap. Stacking-layer → flat-track model. Geometry hook extraction, multi-drag preview, sticky ruler.
- #2213 — app-shell swap + old engine deletion (-5522 lines). All ~30 TEMP fallow exemptions cleaned up. Nit: The old 963-line
useTimelineEditing.test.tsxintegration suite is deleted without same-scope replacement — the new paths (batched move, multi-delete, content-driven duration) should get integration tests in a follow-up. - #2215 — visual refresh. Pure CSS.
preview-regressionCI failure is expected (visual diff from intentional style changes). - #2216 — assets/blocks panel. Full-bleed media now selectable. Block drag-to-timeline.
Cross-Stack Architecture
The stack is disciplined:
- Pure math first, UI second, swap last. Every decision function is testable in isolation before it's wired into React.
- Coexistence window is explicit. Fallow exemptions, union types, optional fields — all labeled TEMP with cleanup PR reference.
- Swap is one-pass. #2213 removes the old engine and all exemptions in one commit. No lingering dead code.
- 2000+ lines of math tests including the z-to-lane convergence proof (#2199) and the resize oscillation proof (#2201).
Two things to watch post-merge:
- Integration test coverage for the new batched-move / multi-delete / content-driven-duration paths (gap from #2213's test deletion).
- The duplicated
overlapsInTime+ epsilon constant between stacking sync and zone model — worth extracting to a shared module.
CI: 0 failures across all 25 PRs (one preview-regression on #2215 is expected from the visual refresh).
Ship the stack.
— Miga
jrusso1020
left a comment
There was a problem hiding this comment.
Re-review — keystone deep dive, two structural NEW findings
Confirmed 6 of 8 prior claims cleanly (SSOT computeOverlayRootScale, root-excluded furthestClipEndFromDocument, NaN guards, safe AABB fallback, 37 TEMP fallow entries matching Via's count exactly + superset-invariant with #2213). Two structural NEW findings the first pass missed:
NEW #1 — DomEditOverlay dead prop surface = the upstream root cause of #2211's phantom z-menu
DomEditOverlay.tsx:90,97 declares onDeleteSelection? and onApplyZIndex? in the props interface (importing ZOrderPatch at :29), but neither is:
- destructured at
:101-122, - stored in any
useRef, - forwarded to
createDomEditOverlayGestureHandlers, or - referenced anywhere in the component body.
Search of the branch shows zero references beyond the interface declaration. So any caller who passes these callbacks today gets a silent no-op — which is exactly the manifestation Abhai flagged at #2211 (context menu writes to iframe DOM but nothing persists; Delete silently no-ops).
Impact: bounded by the wholesale deletion at #2213, but it's a real footgun for any reviewer reading the props contract in isolation. And fixing it here removes the footgun for any future caller that passes these props — cheaper than healing only at #2213.
Fix: destructure + forward both props into the gesture handler bag (~10 LOC).
NEW #2 — triple data-composition-id...data-duration regex fragility
Same regex appears in 3 places across this PR:
rootDuration.ts:4-6(extendRootDurationInSource)timelineAssetDrop.ts:145(setCompositionDurationToContent)timelineAssetDrop.ts:157(extendCompositionDurationIfNeeded)
All three: /(<[^>]*data-composition-id="[^"]*"[^>]*data-duration=")([^"]*)(")/
Two failure modes, both silent (return source on no-match):
- Attribute order dependency: requires
data-composition-idBEFOREdata-durationin source order. A hand-authored composition with the order swapped silently never extends duration. - Double-quotes only:
resolveTimelineAssetCompositionSize(same file,:194-195) accepts both quote styles via(["'])([^"']+)\1. A single-quoted composition (unusual but legal HTML5) reads w/h correctly but never extends duration.
Additional divergence: extendRootDurationInSource selects the FIRST [data-composition-id] tag by source text order, while patchDocumentRootDuration (timelineEditingHelpers.ts:622-624) selects by ancestor-closest. For pathological multi-root docs the two functions patch different roots — LIVE DOM and PERSISTED SOURCE end up disagreeing.
Fix: replace all 3 regexes with a single DOM-based patchSourceRootDuration that mirrors patchDocumentRootDuration's ancestry-aware root selection and uses DOMParser/serializer. DOMParser is already imported nearby in the same file (furthestClipEndFromSource). ~40 LOC eliminates the silent-fail class across the whole coexistence window.
Additional nits
-
playerStoredirect mutation (RDJ's finding stands + goes further):state.timelinePps = ppsbypassesset(). Beyond DevTools:useSyncExternalStoresnapshots use referential equality on the whole state object, so mutation is invisible to any consumer that reads this via useSyncExternalStore. Documented as non-reactive scratch, but nothing enforces that at the type level. -
onMoveElementsunion bivariance safety is purely social: no runtime type-guard, no discriminant. Coexistence holds only if the stack merges strictly in order. One out-of-order merge silently breaks group moves. -
15+ new pure exports, 1 tested:
applyTimelineAutoScrollStep,resolveTimelineAutoScrollLoopAction,resolveTimelineDragEscape,getTimelineRowTop,getTimelineRowFromY,getTimelineFitPps,getTimelineDisplayContentWidth,truncateMiddle,formatDuration,readTimelineElementZIndex,furthestClipEndFromSource,setCompositionDurationToContent,extendCompositionDurationIfNeeded,fitTimelineAssetGeometry,resolveTimelineAssetCompositionSize,applyManualOffsetNudgeDraft/Commit. All are pure with clear invariants — trivially unit-testable. Their consumers land later in the stack, but the coexistence layer's own invariants are undefended. Matches Via's #2201 model-bound-tests concern.
None are #2205's own merge blockers if you accept the coexistence framing. But NEW #1 (~10 LOC) is worth fixing here because it kills the root cause of the #2211 phantom-menu bug at the boundary. NEW #2 (~40 LOC) is worth fixing here because it closes a silent-fail class for hand-authored HTML that survives the whole coexistence window.
— Rames Jusso
0ab5336 to
5d3b082
Compare
dbb7d62 to
0a4df4f
Compare
|
Second fix wave landed — covering Ular's hands-on feel-test on a real project plus the Slack re-review round (Abhai + Rames). All 25 branches re-passed the full local gate set (tsc, full suite at the 18-failure baseline, verbatim fallow with zero suppressions, format:check, per-branch filesize, builds, exact tip parity). From the re-review — independently verified before acting:
From the hands-on session (fixed on a real project that reproduced them):
Not acted on (with reasons): Rames's "#2205 dead prop surface" — the props are forwarded at the tip; the finding describes the coexistence intermediate. "#2209 multi-select resize dropped" — restored in the previous wave. The triple duration-regex → DOMParser refactor and the per-member group-resize undo remain on the deferred list, alongside the integration-test suite. |
0a4df4f to
4675cb9
Compare
|
Third wave landed — hands-on regressions fixed + a lane-model change (product decision by Ular). The previous wave's hands-on testing surfaced two regressions the unit gates missed, plus the root cause of the long-standing drag jank. All were reproduced live in a browser against a real project before fixing, and re-verified the same way after (file diffs as ground truth). Regressions fixed:
Lane model change (the root cause, user-approved): display lanes were derived from global z-rank and re-persisted onto every clip on each edit — so one clip's horizontal move could rewrite other clips' track indexes and mint z values (file-diff-proven). The timeline now uses stable track lanes: a clip's lane is its Insert UX (user-specified): the insert indicator only triggers in the gap band between rows; inserting above the top row works; a drop whose aim is occupied creates an adjacent track (pointer-nearest gap) instead of snapping back to origin. Verification: full per-branch gates (tsc, suites at the known 18-failure baseline, fallow with zero suppressions, format, filesize, builds, exact tip parity) plus pointer-driven browser verification on a real project: marquee/collapse both directions, horizontal-drag one-element file-diff invariant, zero z-order requests, zero reload fetches (the edit blink is gone), no-op drags write nothing. |
4675cb9 to
6be93e9
Compare
|
Round-3 items landed. Owning the miss first: "all 25 re-gated green" was wrong — the gate script printed per-head vitest counts without asserting them. It now fails any branch whose count deviates from the 18-failure baseline (tolerating only the documented happy-dom GC flake by name), and this submit ran fully green under that assertion.
|
41d9a4b to
37f90ba
Compare
6be93e9 to
2a55fa3
Compare
|
Restacked onto current main (v0.7.53, Two conflict sites total, both resolved semantically (full per-file log in the stack's conflict report):
Post-restack verification: full per-branch gates on all 25 heads — tsc, complete suites at the known baseline with the per-head failure-count assertion, fallow (zero suppressions), format, filesize, builds — all green, including main's newer suites (graded-element, sdkCutover, variable-promote) running against this stack's engine. Standing: #2206→#2213 as one ordered unit; no stamps pending Abhay Zala per channel rule. Merge-ready from our side. |
miguel-heygen
left a comment
There was a problem hiding this comment.
Current head 2a55fa3 still has actionable structural issues from prior review: DomEditOverlay declares onDeleteSelection/onApplyZIndex but never forwards them (silent no-op boundary), and three duration-extension regexes duplicate fragile attribute-order/double-quote assumptions and can diverge from DOM-root selection. Resolve or explicitly supersede these before approval.
|
Two corrections on the final-sweep readout (verified at the current heads):
|
2a55fa3 to
4c1399b
Compare
|
Duration-regex triple: closed. One shared DOMParser-based reader ( |
37f90ba to
790de69
Compare
4c1399b to
fc1973d
Compare
What: extends 21 glue files so the OLD timeline/canvas engine and the NEW NLE components type-check side by side: playerStore (multi-select setters, zoom pin, snap toggle, non-reactive scale scratch), drag-state types gain optional NLE fields, timelineLayout/timelineAssetDrop/timelineEditingHelpers/ timelineEditing/timelineElementHelpers/studioHelpers/assetHelpers gain the NLE exports, DomEditOverlay + gestures + AssetContextMenu + Timeline props gain optional callbacks/params, contexts gain *Optional hooks, and TimelineEditCallbacks.onMoveElements becomes a bivariant method accepting both engines' change shapes. patchDocumentRootDuration's test rides along. Why: this is the keystone that dissolves the old "welded glue" problem — every symbol the NLE components need is ADDED next to what the old engine still uses, so the engine components and the swaps can land as separate reviewable PRs. How: 15 authored intermediate files (main content + additive symbols; no behavior changes — new fields optional, new callbacks unused until wired) plus 6 files whose final content is already purely additive. New exports without consumers yet carry TEMP(studio-dnd) ignoreExports entries, removed by the app-shell swap. Test plan: tsc --noEmit in studio + studio-server (verifies BOTH engines compile); bunx vitest run (full suite green incl. the 6 new patchDocumentRootDuration tests); fallow audit clean.
790de69 to
6a03b3c
Compare
fc1973d to
ee487d0
Compare
|
Superseded by current-head re-review; findings addressed.
miguel-heygen
left a comment
There was a problem hiding this comment.
Re-reviewed current head after removing dead DomEditOverlay action props and preserving duration parser fixes. Approval is for this exact head; required checks must still complete.

What
extends 21 glue files so the OLD timeline/canvas engine and the NEW NLE components type-check side by side: playerStore (multi-select setters, zoom pin, snap toggle, non-reactive scale scratch), drag-state types gain optional NLE fields, timelineLayout/timelineAssetDrop/timelineEditingHelpers/ timelineEditing/timelineElementHelpers/studioHelpers/assetHelpers gain the NLE exports, DomEditOverlay + gestures + AssetContextMenu + Timeline props gain optional callbacks/params, contexts gain *Optional hooks, and TimelineEditCallbacks.onMoveElements becomes a bivariant method accepting both engines' change shapes. patchDocumentRootDuration's test rides along.
Why
this is the keystone that dissolves the old "welded glue" problem — every symbol the NLE components need is ADDED next to what the old engine still uses, so the engine components and the swaps can land as separate reviewable PRs.
How
15 authored intermediate files (main content + additive symbols; no behavior changes — new fields optional, new callbacks unused until wired) plus 6 files whose final content is already purely additive. New exports without consumers yet carry TEMP(studio-dnd) ignoreExports entries, removed by the app-shell swap.
Test plan
tsc --noEmit in studio + studio-server (verifies BOTH engines compile); bunx vitest run (full suite green incl. the 6 new patchDocumentRootDuration tests); fallow audit clean.
Stack position 14/25 — studio NLE overhaul (CapCut-parity timeline + canvas). Graphite manages bases; merge from #2192 upward. Temporary
TEMP(studio-dnd)fallow entries (unwired-component windows) are all removed by #2213 (app-shell swap), whose tree is byte-identical to the fully-verified integration branch.