Skip to content

fix(linux): Hyprland cursor telemetry + anchor recording start to the first real frame#159

Open
eduardoaugustolb wants to merge 130 commits into
getopenscreen:mainfrom
eduardoaugustolb:feature/hyprland-cursor-telemetry
Open

fix(linux): Hyprland cursor telemetry + anchor recording start to the first real frame#159
eduardoaugustolb wants to merge 130 commits into
getopenscreen:mainfrom
eduardoaugustolb:feature/hyprland-cursor-telemetry

Conversation

@eduardoaugustolb

@eduardoaugustolb eduardoaugustolb commented Jul 26, 2026

Copy link
Copy Markdown

Summary

  • Adds Hyprland IPC-based cursor position telemetry so the cursor is visible in Wayland/Hyprland screen recordings.
  • Fixes the double-cursor regression by requesting cursor:never from the portal and relying on the editor's editable overlay on Linux.
  • Timestamps Hyprland cursor samples before the IPC round-trip instead of after, removing a fixed cursor/video lag bias.
  • Anchors recording start (duration accounting, recordingId, and cursor telemetry start time) to the first real captured frame instead of to Date.now() at button-press time. On Linux/PipeWire, the desktop-capture portal can still be negotiating (DMA-BUF modifier renegotiation) for several seconds after the track reports unmuted; anchoring to wall-clock "button pressed" time let the declared recording duration exceed the video's real content span (playback would jump to the end early) and let the cursor overlay drift out of sync with the picture. waitForFirstVideoFrame() now blocks on an actual rendered frame (via requestVideoFrameCallback, capped at 3s) before recorder.start() is called, so everything anchors to the same verified instant.

Fixes #99 (comment)

Test plan

  • tsc --noEmit passes clean
  • Full test suite passes (435 tests / 55 files)
  • Manually recorded on Hyprland/Wayland: cursor now visible and in sync with the video
  • Manually verified declared recording duration now matches actual playable content (no more early jump-to-end)

Summary by CodeRabbit

  • New Features

    • Added cursor tracking support for Hyprland on Linux, including normalized and smoothly sampled cursor positions during recordings.
    • Added Linux system-audio capture alongside screen video, with a video-only fallback if audio capture fails.
    • Cursor capture preferences now control whether the cursor appears in recordings.
  • Bug Fixes

    • Recording timing now begins when the first video frame is rendered, improving synchronization between screen content and recording metadata.
    • Improved handling of unavailable or delayed cursor data.

…s and snapping

- feat(ai-edition): multi-session chat history + multi-clip timeline with drag-drop
- feat(ai-edition): timeline interaction P0 — clip resize/delete/duplicate, region drag, Edit Clip dialog
- feat(ai-edition): timeline viewport spine — pan/zoom/reorder (T01-T09)
- feat(ai-edition): timeline lanes, navigator strip, Place-skip mode (T10-T18)
- feat(ai-edition): timeline polish — scrub preview, resequence, snap guides (T19-T25)
…atch the v4 design

- feat(ai-edition): refactor editor to OpenScreen Editor v4 (Media/Edit/Rec)
- style(ai-edition): flatten timeline region pills to v4 design
- style(ai-edition): match v4 timeline navigator to design
- style(ai-edition): align v4 timeline clip cards + ruler to design
- fix(ai-edition): hide timeline lane watermark when lane has regions
- style(ai-edition): float clip label as v4 blur chip over waveform
- docs(ai-edition): add v4 design-parity + import audit notes
…et, HUD drag and Wayland

- fix(wgc): stop holding the frame mutex across blocking WriteSample calls
- fix(recording): round window-capture dimensions up to even for WGC
- fix(recording): bound the video-writer frame wait as a hang safety net
- fix(overlay): stop the HUD drag from drifting away from the cursor
- test(launch): add regression coverage for HUD drag measurement suppression
- fix(recording): disable Vulkan on Wayland so PipeWire capture can import DMA-BUF frames
…ew limits

- fix(zoom): stop a manual zoom drifting during its ease-in, and smooth auto follow
- feat(timeline): bring back the global Auto-Focus toggle
- fix(editor): say what the preview actually does above 16x
- fix(editor): lock the per-zoom focus control under the global toggle, drop the 16x note
- docs(engineering): retract the webcam reactive-zoom gap, record the test pass
…ws, masks, images, text

- feat(annotations): send annotations to the native compositor
- feat(annotations): draw figure arrows in the compositor, and make font size proportional
- feat(annotations): render blur and mosaic masks in the compositor
- feat(annotations): render image annotations in the compositor
- feat(annotations): render text with DirectWrite, closing the preview/render gap
@coderabbitai

coderabbitai Bot commented Jul 26, 2026

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 620cf3e1-2ac4-4874-885e-22bcd856bc6c

📥 Commits

Reviewing files that changed from the base of the PR and between 40aa05c and 24231b6.

📒 Files selected for processing (5)
  • electron/native-bridge/cursor/recording/factory.ts
  • electron/native-bridge/cursor/recording/hyprlandCursorIpc.test.ts
  • electron/native-bridge/cursor/recording/hyprlandCursorIpc.ts
  • electron/native-bridge/cursor/recording/hyprlandCursorRecordingSession.test.ts
  • electron/native-bridge/cursor/recording/hyprlandCursorRecordingSession.ts
🚧 Files skipped from review as they are similar to previous changes (5)
  • electron/native-bridge/cursor/recording/hyprlandCursorIpc.ts
  • electron/native-bridge/cursor/recording/factory.ts
  • electron/native-bridge/cursor/recording/hyprlandCursorRecordingSession.test.ts
  • electron/native-bridge/cursor/recording/hyprlandCursorIpc.test.ts
  • electron/native-bridge/cursor/recording/hyprlandCursorRecordingSession.ts

📝 Walkthrough

Walkthrough

Hyprland cursor positions are captured through compositor IPC, normalized into cursor recording samples, and selected by the session factory. Linux screen/audio capture handling is updated, and recording timeline state now starts at the first produced video frame.

Changes

Cursor recording pipeline

Layer / File(s) Summary
Hyprland IPC cursor query
electron/native-bridge/cursor/recording/hyprlandCursorIpc.ts, electron/native-bridge/cursor/recording/hyprlandCursorIpc.test.ts
Resolves the Hyprland socket, sends j/cursorpos, validates cursor JSON, and handles connection, malformed-response, and timeout cases.
Hyprland recording session wiring
electron/native-bridge/cursor/recording/hyprlandCursorRecordingSession.ts, electron/native-bridge/cursor/recording/hyprlandCursorRecordingSession.test.ts, electron/native-bridge/cursor/recording/factory.ts
Adds periodic Hyprland cursor sampling with normalization, clamping, fallback positions, bounded sample storage, stop synchronization, tests, and factory selection through HYPRLAND_INSTANCE_SIGNATURE.
Linux capture and first-frame timing
src/hooks/useScreenRecorder.ts, src/hooks/waitForFirstVideoFrame.ts, src/hooks/waitForFirstVideoFrame.test.ts
Maps cursor capture constraints, separates Linux video and system-audio capture, adds first-frame waiting with fallback cleanup, and anchors recording state to the first frame timestamp.

Estimated code review effort: 4 (Complex) | ~45 minutes

Sequence Diagram(s)

sequenceDiagram
  participant ScreenRecorder
  participant VideoTrack
  participant ElectronAPI
  ScreenRecorder->>VideoTrack: waitForFirstVideoFrame
  VideoTrack-->>ScreenRecorder: first-frame timestamp
  ScreenRecorder->>ElectronAPI: setRecordingState with first-frame timestamp
Loading
sequenceDiagram
  participant RecordingFactory
  participant HyprlandCursorRecordingSession
  participant hyprlandCursorIpc
  participant HyprlandSocket
  RecordingFactory->>HyprlandCursorRecordingSession: create Hyprland session
  HyprlandCursorRecordingSession->>hyprlandCursorIpc: queryHyprlandCursorPos
  hyprlandCursorIpc->>HyprlandSocket: send j/cursorpos
  HyprlandSocket-->>hyprlandCursorIpc: cursor coordinates
  hyprlandCursorIpc-->>HyprlandCursorRecordingSession: validated position
Loading

Possibly related PRs

Suggested reviewers: etiennelescot

🚥 Pre-merge checks | ✅ 3 | ❌ 2

❌ Failed checks (2 warnings)

Check name Status Explanation Resolution
Description check ⚠️ Warning The description covers the change and testing, but it omits required template sections and uses a raw issue URL instead of the requested Fixes/Refs format. Add the missing Type of change, Release impact, Desktop impact, and Screenshots/video sections, and link the issue as Fixes/Refs #99.
Docstring Coverage ⚠️ Warning Docstring coverage is 18.18% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (3 passed)
Check name Status Explanation
Title check ✅ Passed The title is specific and matches the main changes: Hyprland cursor telemetry and first-frame recording anchoring.
Linked Issues check ✅ Passed The PR adds Hyprland IPC cursor telemetry, fixes Linux cursor-mode selection, and anchors timing to the first verified frame, matching issue #99.
Out of Scope Changes check ✅ Passed The first-frame anchoring and Linux audio handling are tied to the same recording-path timing and capture fixes, so no unrelated scope stands out.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 4

🧹 Nitpick comments (3)
electron/native-bridge/cursor/recording/hyprlandCursorIpc.test.ts (1)

47-117: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low value

Consider covering the timeout path.

Good coverage overall; the one gap is SOCKET_TIMEOUT_MS-triggered resolution (server accepts but never responds), which currently isn't exercised.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@electron/native-bridge/cursor/recording/hyprlandCursorIpc.test.ts` around
lines 47 - 117, Add a test in the queryHyprlandCursorPos suite that starts a
fake server which accepts the connection but never sends a response, then assert
queryHyprlandCursorPos resolves to null after the SOCKET_TIMEOUT_MS timeout
path. Ensure the test cleanup still closes the server and removes its temporary
socket directory.
electron/native-bridge/cursor/recording/hyprlandCursorRecordingSession.ts (2)

27-29: 🩺 Stability & Availability | 🔵 Trivial | ⚡ Quick win

Silent no-op when the Hyprland socket path can't be resolved.

If HYPRLAND_INSTANCE_SIGNATURE is set but XDG_RUNTIME_DIR isn't (or resolution otherwise fails), socketPath is null and captureSample() quietly no-ops on every tick (line 56-58) — the recording ends up with zero cursor samples and no trace of why, which is exactly the diagnosability gap behind the original missing-cursor bug (#99).

💡 Suggested diagnostic warning
 	constructor(private readonly options: HyprlandCursorRecordingSessionOptions) {
 		this.socketPath = options.socketPath ?? resolveHyprlandSocketPath();
+		if (!this.socketPath) {
+			console.warn(
+				"HyprlandCursorRecordingSession: could not resolve the Hyprland IPC socket path; cursor telemetry will be empty.",
+			);
+		}
 	}

Also applies to: 55-58

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@electron/native-bridge/cursor/recording/hyprlandCursorRecordingSession.ts`
around lines 27 - 29, Update the HyprlandCursorRecordingSession constructor and
captureSample flow to detect when socketPath cannot be resolved and emit a
diagnostic warning instead of silently no-oping. Preserve normal capture
behavior when a valid path exists, and ensure the warning clearly identifies the
missing Hyprland socket path and explains that cursor samples cannot be
captured.

19-91: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Add a test for HyprlandCursorRecordingSession.

The Hyprland cursor IPC layer has hyprlandCursorIpc.test.ts, but the new recording session class has no local test. Cover the interval-driven sampling, clamping/normalization, maxSamples capping, and stop/in-flight-capture behavior, with injectable getDisplayBounds/socketPath.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@electron/native-bridge/cursor/recording/hyprlandCursorRecordingSession.ts`
around lines 19 - 91, The HyprlandCursorRecordingSession class lacks coverage
for its recording lifecycle. Add local tests using injectable getDisplayBounds
and socketPath to verify immediate and interval-driven capture, coordinate
normalization and clamping, maxSamples eviction, and stop behavior including an
in-flight capture without leaving timers or sampling state active; mock the
Hyprland cursor query and timer behavior as needed.

Source: Coding guidelines

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@electron/native-bridge/cursor/recording/hyprlandCursorRecordingSession.ts`:
- Around line 41-90: Update stop() and captureSample() in the cursor recording
session to track any in-flight capture and await its completion before
returning. Return a snapshot copy of this.samples rather than the mutable array,
ensuring no later capture can alter the returned CursorRecordingData.

In `@src/hooks/useScreenRecorder.ts`:
- Around line 41-94: Add same-package tests covering waitForFirstVideoFrame’s
requestVideoFrameCallback success path, timeout fallback, and unavailable-API
fallback; verify Linux system versus editable cursor branching, audio fallback
cleanup, and that cursor recording receives the anchored first-frame timestamp.
Exercise the affected useScreenRecorder flows rather than adding unrelated
refactors.
- Around line 1273-1308: Update the display/audio capture flow around the
Promise.all call to retain the getUserMedia audio promise and, when
getDisplayMedia rejects, await its result and stop all resolved audio tracks
before propagating the display-capture failure. Preserve the existing video-only
fallback for audio-request failures and the combined stream behavior on success.
- Around line 1265-1266: Update the cursor selection logic in useScreenRecorder
so Linux system mode requests "always" and only cursorCaptureMode ===
"editable-overlay" requests "never". Remove the platform === "linux" condition
while preserving the existing "always" behavior for other non-overlay modes.

---

Nitpick comments:
In `@electron/native-bridge/cursor/recording/hyprlandCursorIpc.test.ts`:
- Around line 47-117: Add a test in the queryHyprlandCursorPos suite that starts
a fake server which accepts the connection but never sends a response, then
assert queryHyprlandCursorPos resolves to null after the SOCKET_TIMEOUT_MS
timeout path. Ensure the test cleanup still closes the server and removes its
temporary socket directory.

In `@electron/native-bridge/cursor/recording/hyprlandCursorRecordingSession.ts`:
- Around line 27-29: Update the HyprlandCursorRecordingSession constructor and
captureSample flow to detect when socketPath cannot be resolved and emit a
diagnostic warning instead of silently no-oping. Preserve normal capture
behavior when a valid path exists, and ensure the warning clearly identifies the
missing Hyprland socket path and explains that cursor samples cannot be
captured.
- Around line 19-91: The HyprlandCursorRecordingSession class lacks coverage for
its recording lifecycle. Add local tests using injectable getDisplayBounds and
socketPath to verify immediate and interval-driven capture, coordinate
normalization and clamping, maxSamples eviction, and stop behavior including an
in-flight capture without leaving timers or sampling state active; mock the
Hyprland cursor query and timer behavior as needed.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 2e4590be-7e17-4c8d-80b2-6c24b24ab3e2

📥 Commits

Reviewing files that changed from the base of the PR and between f508a21 and 08528a8.

📒 Files selected for processing (6)
  • electron/native-bridge/cursor/recording/factory.ts
  • electron/native-bridge/cursor/recording/hyprlandCursorIpc.test.ts
  • electron/native-bridge/cursor/recording/hyprlandCursorIpc.ts
  • electron/native-bridge/cursor/recording/hyprlandCursorRecordingSession.ts
  • src/components/video-editor/VideoEditor.tsx
  • src/hooks/useScreenRecorder.ts

Comment thread src/hooks/useScreenRecorder.ts Outdated
Comment on lines +41 to +94

const FIRST_FRAME_TIMEOUT_MS = 3000;

/**
* Resolve once `track` has produced a real, decodable frame -- not merely once
* `track.muted` reads false. On Linux/PipeWire the desktop-capture portal can still be
* negotiating (DMA-BUF modifier renegotiation, `no_hardware_cursors`, etc.) well after
* the track reports unmuted, so anchoring recording start to "now" makes MediaRecorder's
* own internal frame-0 arrive silently late: the declared recording duration (wall time
* from button-press to stop) ends up longer than the video's real content span (playback
* jumps to the end early), and cursor telemetry -- anchored to the same "now" -- drifts
* out of sync with the picture. Waiting for an actual rendered frame here, before
* `recorder.start()` is ever called, keeps duration accounting, cursor telemetry, and
* MediaRecorder's timeline all pointing at the same wall-clock instant.
*
* Falls back to `Date.now()` after `FIRST_FRAME_TIMEOUT_MS` if no frame arrives (e.g. a
* genuinely stalled capture), so recording start is never blocked indefinitely.
*/
function waitForFirstVideoFrame(track: MediaStreamTrack): Promise<number> {
return new Promise((resolve) => {
const video = document.createElement("video");
video.muted = true;
video.playsInline = true;
video.srcObject = new MediaStream([track]);

let settled = false;
const cleanup = () => {
video.srcObject = null;
video.remove();
};
const finish = (timeMs: number) => {
if (settled) return;
settled = true;
clearTimeout(timeoutId);
cleanup();
resolve(timeMs);
};

const timeoutId = setTimeout(() => finish(Date.now()), FIRST_FRAME_TIMEOUT_MS);

if (typeof video.requestVideoFrameCallback !== "function") {
// API unavailable (older Chromium); fall back to a short settle delay rather
// than blocking on the full timeout every time.
finish(Date.now());
return;
}

video.requestVideoFrameCallback(() => finish(Date.now()));
void video.play().catch(() => {
// Autoplay/play() rejection still lets requestVideoFrameCallback or the
// timeout resolve this promise; nothing else to do here.
});
});
}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

📐 Maintainability & Code Quality | 🟠 Major | 🏗️ Heavy lift

Add tests for the new capture and timing behaviors.

This change adds first-frame waiting, Linux cursor/audio branching, and timeline anchoring without included same-package test coverage. Cover the frame callback/timeout paths, Linux system versus editable cursor modes, audio fallback cleanup, and the timestamp passed to cursor recording.

As per coding guidelines, “Add a test for every new behavior in the same package as the code under test.”

Also applies to: 1250-1318, 1473-1542

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@src/hooks/useScreenRecorder.ts` around lines 41 - 94, Add same-package tests
covering waitForFirstVideoFrame’s requestVideoFrameCallback success path,
timeout fallback, and unavailable-API fallback; verify Linux system versus
editable cursor branching, audio fallback cleanup, and that cursor recording
receives the anchored first-frame timestamp. Exercise the affected
useScreenRecorder flows rather than adding unrelated refactors.

Source: Coding guidelines

Comment thread src/hooks/useScreenRecorder.ts Outdated
Comment thread src/hooks/useScreenRecorder.ts Outdated
…nd no more pre-v4 shell

- fix(editor): wire the New project starting points to the Media/Rec tabs
- feat(i18n): translate the right-rail settings panes and editor chrome
- refactor(ai-edition): drop the pre-v4 editor shell and its dnd-timeline dep
- fix(export): give the live preview back its pause instead of resuming it
… unify the zoom scale

- fix(zoom, annotations): the tilt was a quadratic formula collapsing, not a clipping rect
- fix(zoom): the tilt was never clipped — one edge was simply parallel to the frame
- fix(zoom): round the tilted shadow's corners like the plane's own
- fix(zoom): one zoom scale, so the focus can actually reach the recording's edge
main's 24 commits were already reimplemented on this branch, so the
resolution keeps the branch's code verbatim: not one file under src/,
electron/, tests/ or scripts/ differs from feat/ai-edition.

Only release infrastructure is inherited from main:

- package.json: version 1.7.0 and the build:win:store (MSIX) script
- electron-builder.json5: the appx target
- .github/workflows: build.yml release pins, prerelease.yml, promote.yml
- .gitignore, and docs/testing/rc-e2e-checklist.md

Dropped as superseded by this branch's own implementations: the pre-v4
editor shell (VideoEditor, VideoPlayback, TimelineEditor, videoEventHandlers,
rafCoalescer) and main's duplicate HUD-drag regression test.

build.yml and .gitignore are normalised back to LF, main's convention, so
they land on main without a whole-file rewrite.
@eduardoaugustolb

Copy link
Copy Markdown
Author

Addressed in c7adb1b:

  • HyprlandCursorRecordingSession.stop()/captureSample() race — fixed. stop() now awaits any capture still mid IPC-round-trip and returns a snapshot copy of samples.
  • Silent no-op on unresolved socket path — fixed, now logs a console.warn.
  • Linux system-audio Promise.all track leak on rejection — fixed, mirrors the existing pattern used for the screen/mic capture above it in the same file.
  • cursor: "always"|"never" logic — fixed as suggested. Turns out this was a real bug, not just a style nit: cursor telemetry and the editor's overlay only run in editable-overlay mode (see handlers.ts's set-recording-state handler), so Linux "system" mode was requesting "never" and ending up with no cursor at all. Now mirrors Windows.
  • Tests for waitForFirstVideoFrame — extracted it into its own module (src/hooks/waitForFirstVideoFrame.ts) and added unit tests covering all three resolution paths (rVFC success, timeout fallback, API-unavailable fallback).
  • Timeout-path test for queryHyprlandCursorPos — added.
  • Dedicated test suite for HyprlandCursorRecordingSession — added, covering sampling/clamping/eviction/stop behavior and the in-flight-capture race fixed above.

Not doing:

  • Splitting first-frame anchoring into a separate PR — the anchoring fix and the cursor telemetry fix share a root cause (both were anchored to "record button pressed" wall-clock time instead of when the capture track actually starts delivering frames) and were diagnosed and validated together against real Hyprland recordings. Splitting them would mean shipping the cursor fix with a known, already-diagnosed duration/sync bug still in place.
  • Docstring coverage threshold — this repo doesn't use docstring-style comments as a convention (see existing files in this PR and elsewhere); adding them here to hit an automated 80% threshold would be inconsistent with the codebase and isn't requested by a human reviewer.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In
`@electron/native-bridge/cursor/recording/hyprlandCursorRecordingSession.test.ts`:
- Around line 111-120: Update the recording-session test to advance timers by
another 100ms before calling stop(), while preserving the assertion that
stopPromise remains unresolved during the pending capture. In
hyprlandCursorRecordingSession.ts, ensure a skipped tick caused by isSampling
does not replace the existing captureInFlight promise, so stop() continues
waiting for the original IPC query.

In `@src/hooks/waitForFirstVideoFrame.test.ts`:
- Around line 61-64: Update the test case “resolves immediately when
requestVideoFrameCallback is unavailable” to remove the explicit any cast and
clear HTMLVideoElement.prototype.requestVideoFrameCallback via
Object.defineProperty, preserving strict typing and the simulated API gap.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: a4707dc5-f650-490e-833f-4470902362a7

📥 Commits

Reviewing files that changed from the base of the PR and between 08528a8 and c7adb1b.

📒 Files selected for processing (6)
  • electron/native-bridge/cursor/recording/hyprlandCursorIpc.test.ts
  • electron/native-bridge/cursor/recording/hyprlandCursorRecordingSession.test.ts
  • electron/native-bridge/cursor/recording/hyprlandCursorRecordingSession.ts
  • src/hooks/useScreenRecorder.ts
  • src/hooks/waitForFirstVideoFrame.test.ts
  • src/hooks/waitForFirstVideoFrame.ts
🚧 Files skipped from review as they are similar to previous changes (3)
  • electron/native-bridge/cursor/recording/hyprlandCursorIpc.test.ts
  • electron/native-bridge/cursor/recording/hyprlandCursorRecordingSession.ts
  • src/hooks/useScreenRecorder.ts

Comment on lines +111 to +120
await session.start();
await vi.advanceTimersByTimeAsync(100); // kicks off the second, still-pending capture

const stopPromise = session.stop();
let stopped = false;
void stopPromise.then(() => {
stopped = true;
});
await Promise.resolve();
expect(stopped).toBe(false); // must not resolve while the in-flight capture is pending

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎯 Functional Correctness | 🟠 Major | ⚡ Quick win

Cover the repeated-tick in-flight race.

The test stops after the first pending interval tick. A second tick hits isSampling, returns immediately, and replaces captureInFlight; stop() can then resolve before the real IPC query completes. Advance another 100ms before stop() and keep the non-settlement assertion; also prevent skipped ticks from replacing the in-flight promise in hyprlandCursorRecordingSession.ts.

Suggested regression step
 await session.start();
 await vi.advanceTimersByTimeAsync(100);
+await vi.advanceTimersByTimeAsync(100);
 
 const stopPromise = session.stop();
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
await session.start();
await vi.advanceTimersByTimeAsync(100); // kicks off the second, still-pending capture
const stopPromise = session.stop();
let stopped = false;
void stopPromise.then(() => {
stopped = true;
});
await Promise.resolve();
expect(stopped).toBe(false); // must not resolve while the in-flight capture is pending
await session.start();
await vi.advanceTimersByTimeAsync(100); // kicks off the second, still-pending capture
await vi.advanceTimersByTimeAsync(100);
const stopPromise = session.stop();
let stopped = false;
void stopPromise.then(() => {
stopped = true;
});
await Promise.resolve();
expect(stopped).toBe(false); // must not resolve while the in-flight capture is pending
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In
`@electron/native-bridge/cursor/recording/hyprlandCursorRecordingSession.test.ts`
around lines 111 - 120, Update the recording-session test to advance timers by
another 100ms before calling stop(), while preserving the assertion that
stopPromise remains unresolved during the pending capture. In
hyprlandCursorRecordingSession.ts, ensure a skipped tick caused by isSampling
does not replace the existing captureInFlight promise, so stop() continues
waiting for the original IPC query.

Comment thread src/hooks/waitForFirstVideoFrame.test.ts
Electron's screen.getCursorScreenPoint() is frozen at (0,0) under
Hyprland's xdg-desktop-portal, which is why cursor recordings there
have always shown a static cursor (issue getopenscreen#99). Hyprland exposes its
own IPC socket (`hyprctl`'s underlying protocol) that reports the real
cursor position at any time. When HYPRLAND_INSTANCE_SIGNATURE is set,
route Linux cursor recording through a new HyprlandCursorRecordingSession
that samples position from that socket instead of the broken Electron
API; other Wayland compositors (no equivalent IPC channel today) keep
using the existing TelemetryRecordingSession.

Samples are timestamped before the IPC round-trip completes (not
after), and the session's stop() awaits any capture still mid-flight
and returns a snapshot of its samples, so a race between the sampling
interval and stop() can't return a still-mutable array or drop a
sample that lands late.
The Linux capture path used the legacy chromeMediaSource constraints,
which have no cursor toggle at all. Switch Linux to getDisplayMedia +
setDisplayMediaRequestHandler (main.ts already supplies the pre-selected
source for any platform), mirroring the Windows path: editable-overlay
mode requests cursor:"never" so the editor's overlay (and Hyprland
telemetry) is the only cursor source and doesn't double up with one
baked into the frame; system mode requests "always". System audio on
Linux still goes through the legacy desktop-audio constraints (Electron's
getDisplayMedia loopback audio only supports win32/macOS) and gets
merged with the getDisplayMedia video track; if that video capture
rejects, the merge now awaits and stops the audio-only stream instead
of leaving it running with no consumer (mic indicator stuck on).
MediaRecorder's internal frame-0 timeline starts whenever the track
delivers its first real frame, not when recorder.start() is called.
On Linux/PipeWire the desktop-capture portal can still be negotiating
(DMA-BUF modifier renegotiation) for several seconds after the track
reports unmuted, so anchoring duration accounting and cursor telemetry
to Date.now() at button-press time let both drift out of sync with the
actual video content: declared duration exceeded the real content span
(playback jumped to the end early) and the cursor overlay lagged the
picture.

waitForFirstVideoFrame() now blocks on an actual rendered frame (via
requestVideoFrameCallback, capped at 3s) before recorder.start() is
ever called, so recordingId, segmentStartedAt, and the cursor-telemetry
start time all anchor to the same verified wall-clock instant.
@eduardoaugustolb
eduardoaugustolb force-pushed the feature/hyprland-cursor-telemetry branch from 40aa05c to 24231b6 Compare July 26, 2026 15:23
@eduardoaugustolb

Copy link
Copy Markdown
Author

Rebased onto current `main` (24231b6) — the branch had fallen behind the v3/v4 editor rewrite that landed while this PR was open.

What that rewrite changed and how this branch adapted:

  • `src/components/video-editor/VideoEditor.tsx` was deleted entirely (3328 lines, commit 09da429, "bootstrap the v3 editor model and shell, and make it the only editor"). This branch's platform-gating fix for that file (`hasEditableCursorRecording`) is gone along with it — the new editor's `CursorPreviewLayer.tsx` gates purely on `hasNativeCursorRecordingData` with no platform check, so the underlying problem that fix addressed doesn't exist anymore. No replacement change needed there.
  • The Linux `getDisplayMedia` capture path and the Hyprland IPC telemetry files were never ported into the new `v4` recording path at all — `electron/native-bridge/cursor/recording/factory.ts` on `main` still routes Linux to the generic `TelemetryRecordingSession` (the broken `screen.getCursorScreenPoint()` path this PR exists to fix), and `useScreenRecorder.ts`'s Linux capture had reverted to the legacy `chromeMediaSource` constraints with no cursor toggle. Reapplied both, plus the first-frame anchoring fix, as 3 commits on top of current `main`:
    • `ab4e0fdf` — Hyprland IPC cursor telemetry (unchanged from the reviewed version, including the `stop()`/in-flight-capture fix and socket-path warning from earlier review rounds)
    • `fdd081e2` — Linux `getDisplayMedia` + cursor-mode logic + audio-track-leak fix
    • `24231b6a` — first-frame anchoring (`waitForFirstVideoFrame`)

Verified: tsc --noEmit clean, full test suite passes (1349/1357 — the 8 failures are pre-existing/unrelated, reproduce identically on a clean main checkout with no changes: a localStorage env issue in RightPanes.i18n.test.tsx under this Node version). Branch is not built or run through Electron itself as part of this verification.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[BUG]: Mouse cursor not captured/recognized during screen recording on Hyprland (Wayland)

2 participants