Queue frames till Video Frame Observer is ready#1689
Open
ganeshprasads wants to merge 1 commit into
Open
Conversation
https://bugs.webkit.org/show_bug.cgi?id=317111 Reviewed by: E. Ocaña González Problem : WebRTC playback doesnot recover from a black screen and the issue is seen Intermittently Cause: The first video frame(s) (containing SPS/PPS/IDR) being dropped because libwebrtc starts delivering decoded frames before the downstream GStreamer pipeline's InternalSource registers as a VideoFrameObserver. Without first having the SPS/PPS and the first sync frame , the decoder can not decode any subsequent delta frames. In Following code snippet from mediastream/gstreamer/GStreamerMediaStreamSource.cpp, frames received while m_isObserving is false might be dropped. void videoFrameAvailable(VideoFrame& videoFrame, VideoFrameTimeMetadata) final { return; updateFirstVideoSampleSeenFlag(); Change : While this patch doesnot address the root cause of delayed start of VideoFrameObserver, add a small frame buffer to hold frames (currently max size set to 30 frames) that arrive before any observer registers. deliver the frames when observer is registered Note: From tests it is observed that only the first frame or 2 is actually needed to be queued. * Source/WebCore/platform/mediastream/RealtimeMediaSource.cpp: (RealtimeMediaSource::videoFrameAvailable) : Queue frames till observer is ready * Source/WebCore/platform/mediastream/RealtimeMediaSource.h: (struct PendingVideoFrame) : Added a buffer to hold the frames
eocanha
added
the
upstream
Member
|
Ported this PR upstream, with some modifications to avoid 4 layour test regressions, and submitted it as WebKit/WebKit#67460 for review there (on the https://bugs.webkit.org/show_bug.cgi?id=317111 upstream bug created by Ganesh). |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
https://bugs.webkit.org/show_bug.cgi?id=317111
Reviewed by: E. Ocaña González
Problem : WebRTC playback doesnot recover from a black screen and the issue is seen Intermittently
Cause: The first video frame(s) (containing SPS/PPS/IDR) being dropped
because libwebrtc starts delivering decoded frames before the downstream
GStreamer pipeline's InternalSource registers as a VideoFrameObserver.
Without first having the SPS/PPS and the first sync frame , the decoder
can not decode any subsequent delta frames.
In Following code snippet from mediastream/gstreamer/GStreamerMediaStreamSource.cpp,
frames received while m_isObserving is false might be dropped.
void videoFrameAvailable(VideoFrame& videoFrame, VideoFrameTimeMetadata) final {
return;
Change : While this patch doesnot address the root cause of delayed start of
VideoFrameObserver, add a small frame buffer to hold frames
(currently max size set to 30 frames) that arrive before any observer registers.
deliver the frames when observer is registered
Note: From tests it is observed that only the first frame or 2 is actually needed to be queued.
737e7dc