diff options
Diffstat (limited to 'third_party/libwebrtc/moz-patch-stack')
119 files changed, 13399 insertions, 0 deletions
diff --git a/third_party/libwebrtc/moz-patch-stack/0001.patch b/third_party/libwebrtc/moz-patch-stack/0001.patch new file mode 100644 index 0000000000..19eea5558b --- /dev/null +++ b/third_party/libwebrtc/moz-patch-stack/0001.patch @@ -0,0 +1,1950 @@ +From: Dan Minor <dminor@mozilla.com> +Date: Mon, 22 Jan 2018 13:31:00 -0500 +Subject: Bug 1376873 - Rollup of local modifications; r=ng + +MozReview-Commit-ID: 2euYzBEvuNb + +Differential Revision: https://phabricator.services.mozilla.com/D7425 +Mercurial Revision: https://hg.mozilla.org/mozilla-central/rev/28b57e3ba51de982a4663801a3935580114b5477 + +Bug 1376873 - Rollup conflict fixes for audio/video code; r=pehrsons + +MozReview-Commit-ID: 1T8mgqdkzq3 + +Differential Revision: https://phabricator.services.mozilla.com/D7427 +Mercurial Revision: https://hg.mozilla.org/mozilla-central/rev/93eec571640ee0810da8475ee37e417b88045574 + +Bug 1376873 - Rollup conflict fixes for rtp_rtcp module; r=ng + +MozReview-Commit-ID: D09534DOVLj + +Differential Revision: https://phabricator.services.mozilla.com/D7428 +Mercurial Revision: https://hg.mozilla.org/mozilla-central/rev/50f89f4e45b0af87fd6aa45aed60f02f3e69b951 + +Bug 1497552 - Remove support for 44100 Hz in dtmf_tone_generator; r=padenot + +Assertions in NetEqImpl::SetSampleRateAndChannels prevent us from requesting +tones at 44100 Hz, so this code can be safely removed. + +Differential Revision: https://phabricator.services.mozilla.com/D12982 +Mercurial Revision: https://hg.mozilla.org/mozilla-central/rev/8ea04ec01e9905d4714aa01ade891d552c56a3a6 + +Bug 1497577 - Remove code to detect zero size windows; r=ng + +This code was added by Bug 1196542 to fix a permanently failing test on our +Windows test machines. After this landed, upstream added a check for empty +windows in window_captuer_win.cc, so this should no longer be a problem on +Windows. As far as I know, this was never a problem on the other platforms, +so this code can be safely removed. + +Differential Revision: https://phabricator.services.mozilla.com/D13448 +Mercurial Revision: https://hg.mozilla.org/mozilla-central/rev/647ade4da0fba74a268aab079677cb47f20f036e + +Bug 1509994 - Move video_engine from webrtc to systemservices; r=pehrsons + +Historically this code was part of webrtc.org but has since been removed +from upstream. Rather than maintaining it as a local diff against upstream, +we should just move it to where it is used. + +Differential Revision: https://phabricator.services.mozilla.com/D13092 +Mercurial Revision: https://hg.mozilla.org/mozilla-central/rev/0fd399c6caabff60ac0fe53b920b7d26a9806750 + +Bug 1497606 - Remove disable_composition_ in screen_capturer_win_gdi; r=ng + +This removes disable_composition_ and instead uses the value of +composition_func_ to determine whether or not composition is +disabled. This is what is done by upstream webrtc.org. + +We call options.set_disable_effects(false) in desktop_capture_impl.cc. + +Differential Revision: https://phabricator.services.mozilla.com/D13839 +Mercurial Revision: https://hg.mozilla.org/mozilla-central/rev/b3f5cca4be44c01024b1ef7b5d4951c7297a112a + +Bug 1497974 - Remove local changes to jitter_buffer.cc; r=pehrsons + +These modifications are made to code which we do not use and so +can be removed. + +Differential Revision: https://phabricator.services.mozilla.com/D13989 +Mercurial Revision: https://hg.mozilla.org/mozilla-central/rev/2fb1dd0d8344ef24e2e0fca94725bbfaf59aa257 + +Bug 1512459 - Remove webrtc sndio audio device; r=padenot + +This code is unused and can be removed. + +Differential Revision: https://phabricator.services.mozilla.com/D13929 +Mercurial Revision: https://hg.mozilla.org/mozilla-central/rev/fa0a179388e96b1a47eeef2fda9676812c244d3b + +Bug 1497650 - Remove 100 bytes added to CalcBufferSize in vp8_impl.cc; r=ng + +In Bug 919979 we added 100 bytes to the size returned by CalcBufferSize +to work around an error with the calculated buffer size with small +resolutions. I verified that this extra buffer is no longer required with +a modified mochitest. Given the age of the bug this was working around, +I don't think a permanent test is required to prevent regressions from +upstream. + +Differential Revision: https://phabricator.services.mozilla.com/D14076 +Mercurial Revision: https://hg.mozilla.org/mozilla-central/rev/b20329b1ec0565b3b94db73e13a9c49e43c418c3 + +Bug 1368816 - Enable VideoCaptureExternalTest Rotation gtest; r=ng + +This test started failing after the 57 update and started passing +again after the 64 update, so we might as well enable it. + +Differential Revision: https://phabricator.services.mozilla.com/D13803 +Mercurial Revision: https://hg.mozilla.org/mozilla-central/rev/65a44b43b7b69c066abcb864076003252abc475e + +Bug 1497573 - Remove DesktopCapturer::Stop; r=ng + +Differential Revision: https://phabricator.services.mozilla.com/D14066 +Mercurial Revision: https://hg.mozilla.org/mozilla-central/rev/0d9c607f26195b7f63cca847224bcd137f73720d + +Bug 1497619 - Restore thread check in process_thread_impl.cc; r=ng + +Not not really needed. + +Differential Revision: https://phabricator.services.mozilla.com/D14097 +Mercurial Revision: https://hg.mozilla.org/mozilla-central/rev/923373fdf5e50d44c895141dc1db74709d6610fe + +Bug 1497992 - Remove VideoReceiver::Reset; r=pehrsons + +This ends up calling VCMReceiver::Reset() which resets the +state of the VCMJitterBuffer. We no longer use VCMJitterBuffer, +which is the old jitter buffer implementation, so this code +no longer has any effect and can be removed. + +Differential Revision: https://phabricator.services.mozilla.com/D14185 +Mercurial Revision: https://hg.mozilla.org/mozilla-central/rev/9e526b3093ee60791cf9c436ea06b6665eb5ef74 + +Bug 1497610 - Remove IsNewerOrSameTimestamp; r=bwc + +The affected functions are only used by VCMJitterBuffer, which is the older +jitter buffer that is no longer in use. We can safely remove these +modifications. + +Differential Revision: https://phabricator.services.mozilla.com/D14485 +Mercurial Revision: https://hg.mozilla.org/mozilla-central/rev/5ed0ed11b23d82ee026244a9a48e522fe38335e2 + +Bug 1498253 - Remove mozAvSyncDelay and mozJitterBufferDelay; r=ng + +The value for mozAvSyncDelay has been broken since the branch 57 update +(Bug 1341285). We added SetCurrentSyncOffset() but never called it from +anywhere. + +In the future we should be getting stats from AudioReceiveStream rather than +modifying the channel code, the delay_estimate_ms field provides almost the +same information. + +Since we're attempting to get rid of moz prefixed stats, it makes sense to just +remove this code rather than fix it. The associated telemetry code has been +broken since Bug 1341285 as well so I think it is safe to remove. + +Differential Revision: https://phabricator.services.mozilla.com/D14462 +Mercurial Revision: https://hg.mozilla.org/mozilla-central/rev/80e86c169d202e724cda74bbd9535b2d5236305b + +Bug 1498205 - Move PlatformUIThread from rtc_base to video_engine; r=pehrsons + +PlatformUIThread is only used by the video_engine code, so it makes sense to +move it there rather than maintain it as a diff against upstream webrtc.org. + +Differential Revision: https://phabricator.services.mozilla.com/D13531 +Mercurial Revision: https://hg.mozilla.org/mozilla-central/rev/65bf3e37a4409f3ca0350366d7be19368adfa21b + +Bug 1497602 - Enable DirectX screen capturer on Windows; r=pehrsons + +This enables support for the DirectX screen capturer. We use the default +DesktopCaptureOptions which do not set the option to use the DirectX screen +capturer so this change will have no effect with the current code. + +For what it's worth, I tested enabling the DirectX option and it worked fine on my +system, but I don't see any reason to not follow the defaults provided by +webrtc.org in this case. + +Differential Revision: https://phabricator.services.mozilla.com/D13303 +Mercurial Revision: https://hg.mozilla.org/mozilla-central/rev/eb07312cfbe868780ebed84fc83e5a5470e81fb4 + +Bug 1439997 - Remove old mac video capture code; r=jib + +This code is no longer used and has been removed upstream. We can remove +it as well. + +Differential Revision: https://phabricator.services.mozilla.com/D15196 +Mercurial Revision: https://hg.mozilla.org/mozilla-central/rev/e79251fa381cfc9f3425f6a746ac8d8c22046d6b + +Bug 1376873 - Updates to Android video capture; r=pehrsons + +Differential Revision: https://phabricator.services.mozilla.com/D7452 +Mercurial Revision: https://hg.mozilla.org/mozilla-central/rev/1e6d72de0587d24c20fefc142591d8b47c363f89 + +Bug 1578073 - Move android video capture code to dom/media/systemservices; r=jib + +Although originally part of webrtc.org, this code has subsequently been +removed by upstream. Moving it to under dom/media should make it clearer that +this is code that we are maintaining and simplify future upstream merges. + +Differential Revision: https://phabricator.services.mozilla.com/D61850 +Mercurial Revision: https://hg.mozilla.org/mozilla-central/rev/46c21affcbc14da30abb03b10573076aee6341c0 + +Bug 1652552 - Remove remaining application capture code; r=jib + +Differential Revision: https://phabricator.services.mozilla.com/D83491 +Mercurial Revision: https://hg.mozilla.org/mozilla-central/rev/26eee332d844bd3f9479d1db92d2f000255664c1 +--- + api/rtp_headers.cc | 3 +- + api/rtp_headers.h | 17 +- + api/rtp_parameters.cc | 3 +- + call/BUILD.gn | 4 +- + call/video_receive_stream.h | 2 + + modules/audio_coding/acm2/acm_receiver.h | 12 +- + modules/audio_coding/neteq/dtmf_buffer.cc | 6 +- + modules/audio_coding/neteq/merge.cc | 6 + + .../logging/apm_data_dumper.cc | 23 ++- + .../logging/apm_data_dumper.h | 67 ++++-- + modules/desktop_capture/desktop_capturer.h | 1 + + .../desktop_capture/fake_desktop_capturer.cc | 4 +- + .../linux/x11/mouse_cursor_monitor_x11.cc | 4 +- + .../linux/x11/screen_capturer_x11.cc | 2 +- + .../linux/x11/window_capturer_x11.cc | 12 +- + .../linux/x11/window_capturer_x11.h | 4 + + .../desktop_capture/linux/x11/x_error_trap.cc | 70 ++++--- + .../desktop_capture/linux/x11/x_error_trap.h | 29 ++- + .../mouse_cursor_monitor_win.cc | 7 +- + .../win/screen_capture_utils.cc | 3 +- + .../win/screen_capturer_win_gdi.cc | 1 + + .../win/screen_capturer_win_magnifier.cc | 16 +- + modules/rtp_rtcp/source/rtcp_sender.cc | 2 +- + .../rtp_rtcp/source/rtp_header_extensions.cc | 39 ++++ + .../rtp_rtcp/source/rtp_header_extensions.h | 15 ++ + modules/rtp_rtcp/source/rtp_packet.cc | 4 + + .../rtp_rtcp/source/rtp_packet_unittest.cc | 30 +++ + modules/rtp_rtcp/source/rtp_rtcp_config.h | 2 + + modules/rtp_rtcp/source/rtp_sender.cc | 4 + + modules/utility/source/jvm_android.cc | 28 ++- + modules/video_capture/device_info_impl.cc | 2 +- + modules/video_capture/device_info_impl.h | 1 + + .../video_capture/linux/device_info_v4l2.cc | 193 +++++++++++++++++- + .../video_capture/linux/device_info_v4l2.h | 18 +- + .../video_capture/linux/video_capture_v4l2.cc | 7 + + modules/video_capture/video_capture.h | 45 +++- + .../video_capture/video_capture_factory.cc | 8 - + modules/video_capture/video_capture_impl.cc | 36 +++- + modules/video_capture/video_capture_impl.h | 6 +- + .../codecs/vp9/libvpx_vp9_encoder.cc | 2 + + .../codecs/vp9/libvpx_vp9_encoder.h | 4 + + modules/video_coding/generic_decoder.cc | 5 +- + modules/video_coding/session_info.cc | 32 ++- + test/fuzzers/rtp_packet_fuzzer.cc | 5 + + test/vcm_capturer.cc | 2 +- + 45 files changed, 652 insertions(+), 134 deletions(-) + +diff --git a/api/rtp_headers.cc b/api/rtp_headers.cc +index e0ad9eb26e..8e1efc7262 100644 +--- a/api/rtp_headers.cc ++++ b/api/rtp_headers.cc +@@ -26,7 +26,8 @@ RTPHeaderExtension::RTPHeaderExtension() + videoRotation(kVideoRotation_0), + hasVideoContentType(false), + videoContentType(VideoContentType::UNSPECIFIED), +- has_video_timing(false) {} ++ has_video_timing(false), ++ csrcAudioLevels() {} + + RTPHeaderExtension::RTPHeaderExtension(const RTPHeaderExtension& other) = + default; +diff --git a/api/rtp_headers.h b/api/rtp_headers.h +index 743fd6d1b6..261c984d05 100644 +--- a/api/rtp_headers.h ++++ b/api/rtp_headers.h +@@ -89,6 +89,19 @@ inline bool operator!=(const AbsoluteCaptureTime& lhs, + return !(lhs == rhs); + } + ++enum { kRtpCsrcSize = 15 }; // RFC 3550 page 13 ++ ++// Audio level of CSRCs See: ++// https://tools.ietf.org/html/rfc6465 ++struct CsrcAudioLevelList { ++ CsrcAudioLevelList() : numAudioLevels(0) { } ++ CsrcAudioLevelList(const CsrcAudioLevelList&) = default; ++ CsrcAudioLevelList& operator=(const CsrcAudioLevelList&) = default; ++ uint8_t numAudioLevels; ++ // arrOfAudioLevels has the same ordering as RTPHeader.arrOfCSRCs ++ uint8_t arrOfAudioLevels[kRtpCsrcSize]; ++}; ++ + struct RTPHeaderExtension { + RTPHeaderExtension(); + RTPHeaderExtension(const RTPHeaderExtension& other); +@@ -144,9 +157,9 @@ struct RTPHeaderExtension { + std::string mid; + + absl::optional<ColorSpace> color_space; +-}; + +-enum { kRtpCsrcSize = 15 }; // RFC 3550 page 13 ++ CsrcAudioLevelList csrcAudioLevels; ++}; + + struct RTC_EXPORT RTPHeader { + RTPHeader(); +diff --git a/api/rtp_parameters.cc b/api/rtp_parameters.cc +index c48b8da02c..c1d12e5d8d 100644 +--- a/api/rtp_parameters.cc ++++ b/api/rtp_parameters.cc +@@ -148,7 +148,8 @@ bool RtpExtension::IsSupportedForAudio(absl::string_view uri) { + uri == webrtc::RtpExtension::kTransportSequenceNumberV2Uri || + uri == webrtc::RtpExtension::kMidUri || + uri == webrtc::RtpExtension::kRidUri || +- uri == webrtc::RtpExtension::kRepairedRidUri; ++ uri == webrtc::RtpExtension::kRepairedRidUri || ++ uri == webrtc::RtpExtension::kCsrcAudioLevelsUri; + } + + bool RtpExtension::IsSupportedForVideo(absl::string_view uri) { +diff --git a/call/BUILD.gn b/call/BUILD.gn +index f4c3295ff5..0e52e8fb3f 100644 +--- a/call/BUILD.gn ++++ b/call/BUILD.gn +@@ -20,6 +20,7 @@ rtc_library("call_interfaces") { + sources = [ + "audio_receive_stream.cc", + "audio_receive_stream.h", ++ "audio_send_stream.cc", + "audio_send_stream.h", + "audio_state.cc", + "audio_state.h", +@@ -32,9 +33,6 @@ rtc_library("call_interfaces") { + "syncable.cc", + "syncable.h", + ] +- if (!build_with_mozilla) { +- sources += [ "audio_send_stream.cc" ] +- } + + deps = [ + ":audio_sender_interface", +diff --git a/call/video_receive_stream.h b/call/video_receive_stream.h +index 0fa257e5ba..15b313a3b9 100644 +--- a/call/video_receive_stream.h ++++ b/call/video_receive_stream.h +@@ -199,6 +199,8 @@ class VideoReceiveStreamInterface : public MediaReceiveStreamInterface { + // disabled. + KeyFrameReqMethod keyframe_method = KeyFrameReqMethod::kPliRtcp; + ++ bool tmmbr = false; ++ + // See LntfConfig for description. + LntfConfig lntf; + +diff --git a/modules/audio_coding/acm2/acm_receiver.h b/modules/audio_coding/acm2/acm_receiver.h +index 18b662aed0..a61247627f 100644 +--- a/modules/audio_coding/acm2/acm_receiver.h ++++ b/modules/audio_coding/acm2/acm_receiver.h +@@ -18,6 +18,7 @@ + #include <string> + #include <utility> + #include <vector> ++#include <atomic> + + #include "absl/types/optional.h" + #include "api/array_view.h" +@@ -215,12 +216,15 @@ class AcmReceiver { + + mutable Mutex mutex_; + absl::optional<DecoderInfo> last_decoder_ RTC_GUARDED_BY(mutex_); +- ACMResampler resampler_ RTC_GUARDED_BY(mutex_); +- std::unique_ptr<int16_t[]> last_audio_buffer_ RTC_GUARDED_BY(mutex_); +- CallStatistics call_stats_ RTC_GUARDED_BY(mutex_); ++ ACMResampler resampler_; ++ ++ // After construction, this is only ever touched on the thread that calls ++ // AcmReceiver::GetAudio, and only modified in this method. ++ std::unique_ptr<int16_t[]> last_audio_buffer_; ++ CallStatistics call_stats_; + const std::unique_ptr<NetEq> neteq_; // NetEq is thread-safe; no lock needed. + Clock* const clock_; +- bool resampled_last_output_frame_ RTC_GUARDED_BY(mutex_); ++ std::atomic<bool> resampled_last_output_frame_; + }; + + } // namespace acm2 +diff --git a/modules/audio_coding/neteq/dtmf_buffer.cc b/modules/audio_coding/neteq/dtmf_buffer.cc +index 9f78aca6e2..115bfcf97b 100644 +--- a/modules/audio_coding/neteq/dtmf_buffer.cc ++++ b/modules/audio_coding/neteq/dtmf_buffer.cc +@@ -193,7 +193,11 @@ bool DtmfBuffer::Empty() const { + } + + int DtmfBuffer::SetSampleRate(int fs_hz) { +- if (fs_hz != 8000 && fs_hz != 16000 && fs_hz != 32000 && fs_hz != 48000) { ++ if (fs_hz != 8000 && ++ fs_hz != 16000 && ++ fs_hz != 32000 && ++ fs_hz != 44100 && ++ fs_hz != 48000) { + return kInvalidSampleRate; + } + max_extrapolation_samples_ = 7 * fs_hz / 100; +diff --git a/modules/audio_coding/neteq/merge.cc b/modules/audio_coding/neteq/merge.cc +index 22cf6a7754..0aec6d2597 100644 +--- a/modules/audio_coding/neteq/merge.cc ++++ b/modules/audio_coding/neteq/merge.cc +@@ -213,6 +213,12 @@ int16_t Merge::SignalScaling(const int16_t* input, + // Adjust muting factor if new vector is more or less of the BGN energy. + const auto mod_input_length = rtc::SafeMin<size_t>( + 64 * rtc::dchecked_cast<size_t>(fs_mult_), input_length); ++ ++ // Missing input, do no muting ++ if (mod_input_length == 0) { ++ return 16384; ++ } ++ + const int16_t expanded_max = + WebRtcSpl_MaxAbsValueW16(expanded_signal, mod_input_length); + int32_t factor = +diff --git a/modules/audio_processing/logging/apm_data_dumper.cc b/modules/audio_processing/logging/apm_data_dumper.cc +index 65d2167d37..f787b65604 100644 +--- a/modules/audio_processing/logging/apm_data_dumper.cc ++++ b/modules/audio_processing/logging/apm_data_dumper.cc +@@ -35,14 +35,20 @@ std::string FormFileName(absl::string_view output_dir, + int instance_index, + int reinit_index, + absl::string_view suffix) { +- char buf[1024]; +- rtc::SimpleStringBuilder ss(buf); +- if (!output_dir.empty()) { +- ss << output_dir; +- if (output_dir.back() != kPathDelimiter) { +- ss << kPathDelimiter; +- } ++#ifdef WEBRTC_WIN ++ char sep = '\\'; ++#else ++ char sep = '/'; ++#endif ++ ++ std::stringstream ss; ++ std::string base = webrtc::Trace::aec_debug_filename(); ++ ss << base; ++ ++ if (base.length() && base.back() != sep) { ++ ss << sep; + } ++ + ss << name << "_" << instance_index << "-" << reinit_index << suffix; + return ss.str(); + } +@@ -52,7 +58,8 @@ std::string FormFileName(absl::string_view output_dir, + + #if WEBRTC_APM_DEBUG_DUMP == 1 + ApmDataDumper::ApmDataDumper(int instance_index) +- : instance_index_(instance_index) {} ++ : instance_index_(instance_index) ++ , debug_written_(0) {} + #else + ApmDataDumper::ApmDataDumper(int instance_index) {} + #endif +diff --git a/modules/audio_processing/logging/apm_data_dumper.h b/modules/audio_processing/logging/apm_data_dumper.h +index 4ab6baad83..aa8496819b 100644 +--- a/modules/audio_processing/logging/apm_data_dumper.h ++++ b/modules/audio_processing/logging/apm_data_dumper.h +@@ -41,7 +41,7 @@ namespace webrtc { + // Functor used to use as a custom deleter in the map of file pointers to raw + // files. + struct RawFileCloseFunctor { +- void operator()(FILE* f) const { fclose(f); } ++ void operator()(FILE* f) const { if (f) fclose(f); } + }; + #endif + +@@ -100,6 +100,7 @@ class ApmDataDumper { + void InitiateNewSetOfRecordings() { + #if WEBRTC_APM_DEBUG_DUMP == 1 + ++recording_set_index_; ++ debug_written_ = 0; + #endif + } + +@@ -114,7 +115,9 @@ class ApmDataDumper { + + if (recording_activated_) { + FILE* file = GetRawFile(name); +- fwrite(&v, sizeof(v), 1, file); ++ if (file) { ++ fwrite(&v, sizeof(v), 1, file); ++ } + } + #endif + } +@@ -129,7 +132,9 @@ class ApmDataDumper { + + if (recording_activated_) { + FILE* file = GetRawFile(name); +- fwrite(v, sizeof(v[0]), v_length, file); ++ if (file) { ++ fwrite(v, sizeof(v[0]), v_length, file); ++ } + } + #endif + } +@@ -156,7 +161,9 @@ class ApmDataDumper { + + if (recording_activated_) { + FILE* file = GetRawFile(name); +- fwrite(&v, sizeof(v), 1, file); ++ if (file) { ++ fwrite(&v, sizeof(v), 1, file); ++ } + } + #endif + } +@@ -171,7 +178,9 @@ class ApmDataDumper { + + if (recording_activated_) { + FILE* file = GetRawFile(name); +- fwrite(v, sizeof(v[0]), v_length, file); ++ if (file) { ++ fwrite(v, sizeof(v[0]), v_length, file); ++ } + } + #endif + } +@@ -210,9 +219,11 @@ class ApmDataDumper { + + if (recording_activated_) { + FILE* file = GetRawFile(name); +- for (size_t k = 0; k < v_length; ++k) { +- int16_t value = static_cast<int16_t>(v[k]); +- fwrite(&value, sizeof(value), 1, file); ++ if (file) { ++ for (size_t k = 0; k < v_length; ++k) { ++ int16_t value = static_cast<int16_t>(v[k]); ++ fwrite(&value, sizeof(value), 1, file); ++ } + } + } + #endif +@@ -240,7 +251,9 @@ class ApmDataDumper { + + if (recording_activated_) { + FILE* file = GetRawFile(name); +- fwrite(&v, sizeof(v), 1, file); ++ if (file) { ++ fwrite(&v, sizeof(v), 1, file); ++ } + } + #endif + } +@@ -255,7 +268,9 @@ class ApmDataDumper { + + if (recording_activated_) { + FILE* file = GetRawFile(name); +- fwrite(v, sizeof(v[0]), v_length, file); ++ if (file) { ++ fwrite(v, sizeof(v[0]), v_length, file); ++ } + } + #endif + } +@@ -282,7 +297,9 @@ class ApmDataDumper { + + if (recording_activated_) { + FILE* file = GetRawFile(name); +- fwrite(&v, sizeof(v), 1, file); ++ if (file) { ++ fwrite(&v, sizeof(v), 1, file); ++ } + } + #endif + } +@@ -297,7 +314,9 @@ class ApmDataDumper { + + if (recording_activated_) { + FILE* file = GetRawFile(name); +- fwrite(v, sizeof(v[0]), v_length, file); ++ if (file) { ++ fwrite(v, sizeof(v[0]), v_length, file); ++ } + } + #endif + } +@@ -311,7 +330,9 @@ class ApmDataDumper { + + if (recording_activated_) { + FILE* file = GetRawFile(name); +- fwrite(&v, sizeof(v), 1, file); ++ if (file) { ++ fwrite(&v, sizeof(v), 1, file); ++ } + } + #endif + } +@@ -326,7 +347,9 @@ class ApmDataDumper { + + if (recording_activated_) { + FILE* file = GetRawFile(name); +- fwrite(v, sizeof(v[0]), v_length, file); ++ if (file) { ++ fwrite(v, sizeof(v[0]), v_length, file); ++ } + } + #endif + } +@@ -369,6 +392,12 @@ class ApmDataDumper { + WavWriter* file = GetWavFile(name, sample_rate_hz, num_channels, + WavFile::SampleFormat::kFloat); + file->WriteSamples(v, v_length); ++ // Cheat and use aec_near as a stand-in for "size of the largest file" ++ // in the dump. We're looking to limit the total time, and that's a ++ // reasonable stand-in. ++ if (strcmp(name, "aec_near") == 0) { ++ updateDebugWritten(v_length * sizeof(float)); ++ } + } + #endif + } +@@ -405,6 +434,16 @@ class ApmDataDumper { + int sample_rate_hz, + int num_channels, + WavFile::SampleFormat format); ++ ++ uint32_t debug_written_ = 0; ++ ++ void updateDebugWritten(uint32_t amount) { ++ debug_written_ += amount; ++ if (debug_written_ >= webrtc::Trace::aec_debug_size()) { ++ SetActivated(false); ++ } ++ } ++ + #endif + }; + +diff --git a/modules/desktop_capture/desktop_capturer.h b/modules/desktop_capture/desktop_capturer.h +index d336723f18..6909a57891 100644 +--- a/modules/desktop_capture/desktop_capturer.h ++++ b/modules/desktop_capture/desktop_capturer.h +@@ -79,6 +79,7 @@ class RTC_EXPORT DesktopCapturer { + struct Source { + // The unique id to represent a Source of current DesktopCapturer. + SourceId id; ++ pid_t pid; + + // Title of the window or screen in UTF-8 encoding, maybe empty. This field + // should not be used to identify a source. +diff --git a/modules/desktop_capture/fake_desktop_capturer.cc b/modules/desktop_capture/fake_desktop_capturer.cc +index f9d9dbd2c4..67149bfcb9 100644 +--- a/modules/desktop_capture/fake_desktop_capturer.cc ++++ b/modules/desktop_capture/fake_desktop_capturer.cc +@@ -72,8 +72,8 @@ void FakeDesktopCapturer::SetSharedMemoryFactory( + } + + bool FakeDesktopCapturer::GetSourceList(DesktopCapturer::SourceList* sources) { +- sources->push_back({kWindowId, "A-Fake-DesktopCapturer-Window"}); +- sources->push_back({kScreenId}); ++ sources->push_back({kWindowId, 1, "A-Fake-DesktopCapturer-Window"}); ++ sources->push_back({kScreenId, 1}); + return true; + } + +diff --git a/modules/desktop_capture/linux/x11/mouse_cursor_monitor_x11.cc b/modules/desktop_capture/linux/x11/mouse_cursor_monitor_x11.cc +index d9c7635c1d..d4b85af6bd 100644 +--- a/modules/desktop_capture/linux/x11/mouse_cursor_monitor_x11.cc ++++ b/modules/desktop_capture/linux/x11/mouse_cursor_monitor_x11.cc +@@ -38,6 +38,7 @@ namespace { + // searches up the list of the windows to find the root child that corresponds + // to `window`. + Window GetTopLevelWindow(Display* display, Window window) { ++ webrtc::XErrorTrap error_trap(display); + while (true) { + // If the window is in WithdrawnState then look at all of its children. + ::Window root, parent; +@@ -104,7 +105,7 @@ MouseCursorMonitorX11::~MouseCursorMonitorX11() { + } + + void MouseCursorMonitorX11::Init(Callback* callback, Mode mode) { +- // Init can be called only once per instance of MouseCursorMonitor. ++ // Init can be called only if not started + RTC_DCHECK(!callback_); + RTC_DCHECK(callback); + +@@ -116,6 +117,7 @@ void MouseCursorMonitorX11::Init(Callback* callback, Mode mode) { + + if (have_xfixes_) { + // Register for changes to the cursor shape. ++ XErrorTrap error_trap(display()); + XFixesSelectCursorInput(display(), window_, XFixesDisplayCursorNotifyMask); + x_display_->AddEventHandler(xfixes_event_base_ + XFixesCursorNotify, this); + +diff --git a/modules/desktop_capture/linux/x11/screen_capturer_x11.cc b/modules/desktop_capture/linux/x11/screen_capturer_x11.cc +index d5dcd7af86..fa6334e8ba 100644 +--- a/modules/desktop_capture/linux/x11/screen_capturer_x11.cc ++++ b/modules/desktop_capture/linux/x11/screen_capturer_x11.cc +@@ -302,7 +302,7 @@ bool ScreenCapturerX11::GetSourceList(SourceList* sources) { + char* monitor_title = XGetAtomName(display(), m.name); + + // Note name is an X11 Atom used to id the monitor. +- sources->push_back({static_cast<SourceId>(m.name), monitor_title}); ++ sources->push_back({static_cast<SourceId>(m.name), 0, monitor_title}); + XFree(monitor_title); + } + +diff --git a/modules/desktop_capture/linux/x11/window_capturer_x11.cc b/modules/desktop_capture/linux/x11/window_capturer_x11.cc +index b55f7e8fa9..2e17d44c0a 100644 +--- a/modules/desktop_capture/linux/x11/window_capturer_x11.cc ++++ b/modules/desktop_capture/linux/x11/window_capturer_x11.cc +@@ -57,6 +57,7 @@ bool WindowCapturerX11::GetSourceList(SourceList* sources) { + return GetWindowList(&atom_cache_, [this, sources](::Window window) { + Source w; + w.id = window; ++ w.pid = (pid_t)GetWindowProcessID(window); + if (this->GetWindowTitle(window, &w.title)) { + sources->push_back(w); + } +@@ -140,6 +141,7 @@ void WindowCapturerX11::Start(Callback* callback) { + + void WindowCapturerX11::CaptureFrame() { + TRACE_EVENT0("webrtc", "WindowCapturerX11::CaptureFrame"); ++ x_display_->ProcessPendingXEvents(); + + if (!x_server_pixel_buffer_.IsWindowValid()) { + RTC_LOG(LS_ERROR) << "The window is no longer valid."; +@@ -147,8 +149,6 @@ void WindowCapturerX11::CaptureFrame() { + return; + } + +- x_display_->ProcessPendingXEvents(); +- + if (!has_composite_extension_) { + // Without the Xcomposite extension we capture when the whole window is + // visible on screen and not covered by any other window. This is not +@@ -237,6 +237,14 @@ bool WindowCapturerX11::GetWindowTitle(::Window window, std::string* title) { + return result; + } + ++int WindowCapturerX11::GetWindowProcessID(::Window window) { ++ // Get _NET_WM_PID property of the window. ++ Atom process_atom = XInternAtom(display(), "_NET_WM_PID", True); ++ XWindowProperty<uint32_t> process_id(display(), window, process_atom); ++ ++ return process_id.is_valid() ? *process_id.data() : 0; ++} ++ + // static + std::unique_ptr<DesktopCapturer> WindowCapturerX11::CreateRawWindowCapturer( + const DesktopCaptureOptions& options) { +diff --git a/modules/desktop_capture/linux/x11/window_capturer_x11.h b/modules/desktop_capture/linux/x11/window_capturer_x11.h +index ac591c272e..cfd29eca66 100644 +--- a/modules/desktop_capture/linux/x11/window_capturer_x11.h ++++ b/modules/desktop_capture/linux/x11/window_capturer_x11.h +@@ -22,6 +22,7 @@ + #include "modules/desktop_capture/desktop_capturer.h" + #include "modules/desktop_capture/desktop_geometry.h" + #include "modules/desktop_capture/linux/x11/shared_x_display.h" ++#include "modules/desktop_capture/linux/x11/x_window_property.h" + #include "modules/desktop_capture/linux/x11/window_finder_x11.h" + #include "modules/desktop_capture/linux/x11/x_atom_cache.h" + #include "modules/desktop_capture/linux/x11/x_server_pixel_buffer.h" +@@ -57,6 +58,9 @@ class WindowCapturerX11 : public DesktopCapturer, + // Returns window title for the specified X `window`. + bool GetWindowTitle(::Window window, std::string* title); + ++ // Returns the id of the owning process. ++ int GetWindowProcessID(::Window window); ++ + Callback* callback_ = nullptr; + + rtc::scoped_refptr<SharedXDisplay> x_display_; +diff --git a/modules/desktop_capture/linux/x11/x_error_trap.cc b/modules/desktop_capture/linux/x11/x_error_trap.cc +index 94e3a03c73..3314dd286c 100644 +--- a/modules/desktop_capture/linux/x11/x_error_trap.cc ++++ b/modules/desktop_capture/linux/x11/x_error_trap.cc +@@ -10,50 +10,60 @@ + + #include "modules/desktop_capture/linux/x11/x_error_trap.h" + +-#include <atomic> +- + #include <stddef.h> + +-#include "rtc_base/checks.h" +- +-namespace webrtc { ++#include <limits> + +-namespace { ++#include "rtc_base/checks.h" + +-static int g_last_xserver_error_code = 0; +-static std::atomic<Display*> g_display_for_error_handler = nullptr; + +-Mutex* AcquireMutex() { +- static Mutex* mutex = new Mutex(); +- return mutex; +-} ++namespace webrtc { + +-int XServerErrorHandler(Display* display, XErrorEvent* error_event) { +- RTC_DCHECK_EQ(display, g_display_for_error_handler.load()); +- g_last_xserver_error_code = error_event->error_code; +- return 0; ++Bool XErrorTrap::XServerErrorHandler(Display* display, xReply* rep, ++ char* /* buf */, int /* len */, ++ XPointer data) { ++ XErrorTrap* self = reinterpret_cast<XErrorTrap*>(data); ++ if (rep->generic.type != X_Error || ++ // Overflow-safe last_request_read <= last_ignored_request_ for skipping ++ // async replies from requests before XErrorTrap was created. ++ self->last_ignored_request_ - display->last_request_read < ++ std::numeric_limits<unsigned long>::max() >> 1) ++ return False; ++ self->last_xserver_error_code_ = rep->error.errorCode; ++ return True; + } + +-} // namespace +- +-XErrorTrap::XErrorTrap(Display* display) : mutex_lock_(AcquireMutex()) { +- // We don't expect this class to be used in a nested fashion so therefore +- // g_display_for_error_handler should never be valid here. +- RTC_DCHECK(!g_display_for_error_handler.load()); +- RTC_DCHECK(display); +- g_display_for_error_handler.store(display); +- g_last_xserver_error_code = 0; +- original_error_handler_ = XSetErrorHandler(&XServerErrorHandler); ++XErrorTrap::XErrorTrap(Display* display) ++ : display_(display), ++ last_xserver_error_code_(0), ++ enabled_(true) { ++ // Use async_handlers instead of XSetErrorHandler(). async_handlers can ++ // remain in place and then be safely removed at the right time even if a ++ // handler change happens concurrently on another thread. async_handlers ++ // are processed first and so can prevent errors reaching the global ++ // XSetErrorHandler handler. They also will not see errors from or affect ++ // handling of errors on other Displays, which may be processed on other ++ // threads. ++ LockDisplay(display); ++ async_handler_.next = display->async_handlers; ++ async_handler_.handler = XServerErrorHandler; ++ async_handler_.data = reinterpret_cast<XPointer>(this); ++ display->async_handlers = &async_handler_; ++ last_ignored_request_ = display->request; ++ UnlockDisplay(display); + } + + int XErrorTrap::GetLastErrorAndDisable() { +- g_display_for_error_handler.store(nullptr); +- XSetErrorHandler(original_error_handler_); +- return g_last_xserver_error_code; ++ assert(enabled_); ++ enabled_ = false; ++ LockDisplay(display_); ++ DeqAsyncHandler(display_, &async_handler_); ++ UnlockDisplay(display_); ++ return last_xserver_error_code_; + } + + XErrorTrap::~XErrorTrap() { +- if (g_display_for_error_handler.load() != nullptr) ++ if (enabled_) + GetLastErrorAndDisable(); + } + +diff --git a/modules/desktop_capture/linux/x11/x_error_trap.h b/modules/desktop_capture/linux/x11/x_error_trap.h +index 1f21ab969c..df7e86bf03 100644 +--- a/modules/desktop_capture/linux/x11/x_error_trap.h ++++ b/modules/desktop_capture/linux/x11/x_error_trap.h +@@ -11,30 +11,39 @@ + #ifndef MODULES_DESKTOP_CAPTURE_LINUX_X11_X_ERROR_TRAP_H_ + #define MODULES_DESKTOP_CAPTURE_LINUX_X11_X_ERROR_TRAP_H_ + +-#include <X11/Xlib.h> +- +-#include "rtc_base/synchronization/mutex.h" ++#include <X11/Xlibint.h> ++#undef max // Xlibint.h defines this and it breaks std::max ++#undef min // Xlibint.h defines this and it breaks std::min + + namespace webrtc { + +-// Helper class that registers an X Window error handler. Caller can use ++// Helper class that registers X Window error handler. Caller can use + // GetLastErrorAndDisable() to get the last error that was caught, if any. ++// An XErrorTrap may be constructed on any thread, but errors are collected ++// from all threads and so |display| should be used only on one thread. ++// Other Displays are unaffected. + class XErrorTrap { + public: + explicit XErrorTrap(Display* display); ++ ~XErrorTrap(); + + XErrorTrap(const XErrorTrap&) = delete; + XErrorTrap& operator=(const XErrorTrap&) = delete; + +- ~XErrorTrap(); +- +- // Returns the last error if one was caught, otherwise 0. Also unregisters the +- // error handler and replaces it with `original_error_handler_`. ++ // Returns last error and removes unregisters the error handler. ++ // Must not be called more than once. + int GetLastErrorAndDisable(); + + private: +- MutexLock mutex_lock_; +- XErrorHandler original_error_handler_ = nullptr; ++ static Bool XServerErrorHandler(Display* display, xReply* rep, ++ char* /* buf */, int /* len */, ++ XPointer data); ++ ++ _XAsyncHandler async_handler_; ++ Display* display_; ++ unsigned long last_ignored_request_; ++ int last_xserver_error_code_; ++ bool enabled_; + }; + + } // namespace webrtc +diff --git a/modules/desktop_capture/mouse_cursor_monitor_win.cc b/modules/desktop_capture/mouse_cursor_monitor_win.cc +index c892d59955..18ef43eeb4 100644 +--- a/modules/desktop_capture/mouse_cursor_monitor_win.cc ++++ b/modules/desktop_capture/mouse_cursor_monitor_win.cc +@@ -88,6 +88,7 @@ MouseCursorMonitorWin::~MouseCursorMonitorWin() { + void MouseCursorMonitorWin::Init(Callback* callback, Mode mode) { + RTC_DCHECK(!callback_); + RTC_DCHECK(callback); ++ RTC_DCHECK(IsGUIThread(false)); + + callback_ = callback; + mode_ = mode; +@@ -96,6 +97,8 @@ void MouseCursorMonitorWin::Init(Callback* callback, Mode mode) { + } + + void MouseCursorMonitorWin::Capture() { ++// TODO: Bug 1666266. Commented out to pass new tests added in bug 1634044. ++// RTC_DCHECK(IsGUIThread(false)); + RTC_DCHECK(callback_); + + CURSORINFO cursor_info; +@@ -107,7 +110,8 @@ void MouseCursorMonitorWin::Capture() { + } + + if (!IsSameCursorShape(cursor_info, last_cursor_)) { +- if (cursor_info.flags == CURSOR_SUPPRESSED) { ++ // Mozilla - CURSOR_SUPPRESSED is win8 and above; so we seem not to be able to see the symbol ++ if (cursor_info.flags != CURSOR_SHOWING) { + // The cursor is intentionally hidden now, send an empty bitmap. + last_cursor_ = cursor_info; + callback_->OnMouseCursor(new MouseCursor( +@@ -168,6 +172,7 @@ void MouseCursorMonitorWin::Capture() { + } + + DesktopRect MouseCursorMonitorWin::GetScreenRect() { ++ RTC_DCHECK(IsGUIThread(false)); + RTC_DCHECK_NE(screen_, kInvalidScreenId); + if (screen_ == kFullDesktopScreenId) { + return DesktopRect::MakeXYWH(GetSystemMetrics(SM_XVIRTUALSCREEN), +diff --git a/modules/desktop_capture/win/screen_capture_utils.cc b/modules/desktop_capture/win/screen_capture_utils.cc +index 3d4aecf14d..1dc2918d08 100644 +--- a/modules/desktop_capture/win/screen_capture_utils.cc ++++ b/modules/desktop_capture/win/screen_capture_utils.cc +@@ -51,7 +51,7 @@ bool GetScreenList(DesktopCapturer::SourceList* screens, + continue; + } + +- screens->push_back({device_index, std::string()}); ++ screens->push_back({device_index, 0, std::string()}); + if (device_names) { + device_names->push_back(rtc::ToUtf8(device.DeviceName)); + } +@@ -147,6 +147,7 @@ DesktopRect GetFullscreenRect() { + + DesktopRect GetScreenRect(const DesktopCapturer::SourceId screen, + const std::wstring& device_key) { ++ RTC_DCHECK(IsGUIThread(false)); + if (screen == kFullDesktopScreenId) { + return GetFullscreenRect(); + } +diff --git a/modules/desktop_capture/win/screen_capturer_win_gdi.cc b/modules/desktop_capture/win/screen_capturer_win_gdi.cc +index 57b1f71b0d..c6d4a75931 100644 +--- a/modules/desktop_capture/win/screen_capturer_win_gdi.cc ++++ b/modules/desktop_capture/win/screen_capturer_win_gdi.cc +@@ -186,6 +186,7 @@ void ScreenCapturerWinGdi::PrepareCaptureResources() { + } + + bool ScreenCapturerWinGdi::CaptureImage() { ++ RTC_DCHECK(IsGUIThread(false)); + DesktopRect screen_rect = + GetScreenRect(current_screen_id_, current_device_key_); + if (screen_rect.is_empty()) { +diff --git a/modules/desktop_capture/win/screen_capturer_win_magnifier.cc b/modules/desktop_capture/win/screen_capturer_win_magnifier.cc +index 214eb0e463..ce747e0141 100644 +--- a/modules/desktop_capture/win/screen_capturer_win_magnifier.cc ++++ b/modules/desktop_capture/win/screen_capturer_win_magnifier.cc +@@ -49,17 +49,25 @@ ScreenCapturerWinMagnifier::ScreenCapturerWinMagnifier() = default; + ScreenCapturerWinMagnifier::~ScreenCapturerWinMagnifier() { + // DestroyWindow must be called before MagUninitialize. magnifier_window_ is + // destroyed automatically when host_window_ is destroyed. +- if (host_window_) ++ if (host_window_) { + DestroyWindow(host_window_); ++ host_window_ = NULL; ++ } + +- if (magnifier_initialized_) ++ if (magnifier_initialized_) { + mag_uninitialize_func_(); ++ magnifier_initialized_ = false; ++ } + +- if (mag_lib_handle_) ++ if (mag_lib_handle_) { + FreeLibrary(mag_lib_handle_); ++ mag_lib_handle_ = NULL; ++ } + +- if (desktop_dc_) ++ if (desktop_dc_) { + ReleaseDC(NULL, desktop_dc_); ++ desktop_dc_ = NULL; ++ } + } + + void ScreenCapturerWinMagnifier::Start(Callback* callback) { +diff --git a/modules/rtp_rtcp/source/rtcp_sender.cc b/modules/rtp_rtcp/source/rtcp_sender.cc +index 7983371097..983851a55b 100644 +--- a/modules/rtp_rtcp/source/rtcp_sender.cc ++++ b/modules/rtp_rtcp/source/rtcp_sender.cc +@@ -201,7 +201,7 @@ void RTCPSender::SetRTCPStatus(RtcpMode new_method) { + next_time_to_send_rtcp_ = absl::nullopt; + } else if (method_ == RtcpMode::kOff) { + // When switching on, reschedule the next packet +- SetNextRtcpSendEvaluationDuration(report_interval_ / 2); ++ SetNextRtcpSendEvaluationDuration(RTCP_INTERVAL_RAPID_SYNC_MS / 2); + } + method_ = new_method; + } +diff --git a/modules/rtp_rtcp/source/rtp_header_extensions.cc b/modules/rtp_rtcp/source/rtp_header_extensions.cc +index 81961c69aa..a57d9e7f62 100644 +--- a/modules/rtp_rtcp/source/rtp_header_extensions.cc ++++ b/modules/rtp_rtcp/source/rtp_header_extensions.cc +@@ -446,6 +446,45 @@ bool PlayoutDelayLimits::Write(rtc::ArrayView<uint8_t> data, + return true; + } + ++// CSRCAudioLevel ++// Sample Audio Level Encoding Using the One-Byte Header Format ++// Note that the range of len is 1 to 15 which is encoded as 0 to 14 ++// 0 1 2 3 ++// 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 ++// +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ ++// | ID | len=2 |0| level 1 |0| level 2 |0| level 3 | ++// +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ ++ ++ ++constexpr RTPExtensionType CsrcAudioLevel::kId; ++constexpr const char* CsrcAudioLevel::kUri; ++ ++bool CsrcAudioLevel::Parse(rtc::ArrayView<const uint8_t> data, ++ CsrcAudioLevelList* csrcAudioLevels) { ++ if (data.size() < 1 || data.size() > kRtpCsrcSize) ++ return false; ++ csrcAudioLevels->numAudioLevels = data.size(); ++ for(uint8_t i = 0; i < csrcAudioLevels->numAudioLevels; i++) { ++ // Ensure range is 0 to 127 inclusive ++ csrcAudioLevels->arrOfAudioLevels[i] = 0x7f & data[i]; ++ } ++ return true; ++} ++ ++size_t CsrcAudioLevel::ValueSize(const CsrcAudioLevelList& csrcAudioLevels) { ++ return csrcAudioLevels.numAudioLevels; ++} ++ ++bool CsrcAudioLevel::Write(rtc::ArrayView<uint8_t> data, ++ const CsrcAudioLevelList& csrcAudioLevels) { ++ RTC_DCHECK_GE(csrcAudioLevels.numAudioLevels, 0); ++ for(uint8_t i = 0; i < csrcAudioLevels.numAudioLevels; i++) { ++ data[i] = csrcAudioLevels.arrOfAudioLevels[i] & 0x7f; ++ } ++ // This extension if used must have at least one audio level ++ return csrcAudioLevels.numAudioLevels; ++} ++ + // Video Content Type. + // + // E.g. default video or screenshare. +diff --git a/modules/rtp_rtcp/source/rtp_header_extensions.h b/modules/rtp_rtcp/source/rtp_header_extensions.h +index d80e0da4f8..89c73955a2 100644 +--- a/modules/rtp_rtcp/source/rtp_header_extensions.h ++++ b/modules/rtp_rtcp/source/rtp_header_extensions.h +@@ -292,6 +292,21 @@ class ColorSpaceExtension { + static size_t WriteLuminance(uint8_t* data, float f, int denominator); + }; + ++class CsrcAudioLevel { ++ public: ++ static constexpr RTPExtensionType kId = kRtpExtensionCsrcAudioLevel; ++ static constexpr absl::string_view Uri() { ++ return RtpExtension::kCsrcAudioLevelsUri; ++ } ++ static constexpr const char* kUri = ++ "urn:ietf:params:rtp-hdrext:csrc-audio-level"; ++ ++ static bool Parse(rtc::ArrayView<const uint8_t> data, ++ CsrcAudioLevelList* csrcAudioLevels); ++ static size_t ValueSize(const CsrcAudioLevelList& csrcAudioLevels); ++ static bool Write(rtc::ArrayView<uint8_t> data, const CsrcAudioLevelList& csrcAudioLevels); ++}; ++ + // Base extension class for RTP header extensions which are strings. + // Subclasses must defined kId and kUri static constexpr members. + class BaseRtpStringExtension { +diff --git a/modules/rtp_rtcp/source/rtp_packet.cc b/modules/rtp_rtcp/source/rtp_packet.cc +index 6c7dff322b..9495841984 100644 +--- a/modules/rtp_rtcp/source/rtp_packet.cc ++++ b/modules/rtp_rtcp/source/rtp_packet.cc +@@ -205,6 +205,10 @@ void RtpPacket::ZeroMutableExtensions() { + // Non-mutable extension. Don't change it. + break; + } ++ case RTPExtensionType::kRtpExtensionCsrcAudioLevel: { ++ // TODO: This is a Mozilla addition, we need to add a handler for this. ++ RTC_CHECK(false); ++ } + } + } + } +diff --git a/modules/rtp_rtcp/source/rtp_packet_unittest.cc b/modules/rtp_rtcp/source/rtp_packet_unittest.cc +index 1d51d75662..41bc114efb 100644 +--- a/modules/rtp_rtcp/source/rtp_packet_unittest.cc ++++ b/modules/rtp_rtcp/source/rtp_packet_unittest.cc +@@ -121,6 +121,18 @@ constexpr uint8_t kPacketWithMid[] = { + 0xbe, 0xde, 0x00, 0x01, + 0xb2, 'm', 'i', 'd'}; + ++constexpr uint8_t kCsrcAudioLevelExtensionId = 0xc; ++constexpr uint8_t kCsrcAudioLevelsSize = 4; ++constexpr uint8_t kCsrcAudioLevels[] = {0x7f, 0x00, 0x10, 0x08}; ++constexpr uint8_t kPacketWithCsrcAudioLevels[] = { ++ 0x90, kPayloadType, kSeqNumFirstByte, kSeqNumSecondByte, ++ 0x65, 0x43, 0x12, 0x78, ++ 0x12, 0x34, 0x56, 0x78, ++ 0xbe, 0xde, 0x00, 0x02, ++ (kCsrcAudioLevelExtensionId << 4) | (kCsrcAudioLevelsSize - 1), ++ 0x7f, 0x00, 0x10, ++ 0x08, 0x00, 0x00, 0x00}; ++ + constexpr uint32_t kCsrcs[] = {0x34567890, 0x32435465}; + constexpr uint8_t kPayload[] = {'p', 'a', 'y', 'l', 'o', 'a', 'd'}; + constexpr uint8_t kPacketPaddingSize = 8; +@@ -385,6 +397,24 @@ TEST(RtpPacketTest, FailsToSetUnregisteredExtension) { + EXPECT_EQ(packet.GetExtension<TransportSequenceNumber>(), absl::nullopt); + } + ++TEST(RtpPacketTest, CreateWithDynamicSizedExtensionCsrcAudioLevel) { ++ RtpPacketToSend::ExtensionManager extensions; ++ extensions.Register<CsrcAudioLevel>(kCsrcAudioLevelExtensionId); ++ RtpPacketToSend packet(&extensions); ++ packet.SetPayloadType(kPayloadType); ++ packet.SetSequenceNumber(kSeqNum); ++ packet.SetTimestamp(kTimestamp); ++ packet.SetSsrc(kSsrc); ++ CsrcAudioLevelList levels; ++ levels.numAudioLevels = kCsrcAudioLevelsSize; ++ for (uint8_t i = 0; i < kCsrcAudioLevelsSize; i++) { ++ levels.arrOfAudioLevels[i] = kCsrcAudioLevels[i]; ++ } ++ packet.SetExtension<CsrcAudioLevel>(levels); ++ EXPECT_THAT(kPacketWithCsrcAudioLevels, ++ ElementsAreArray(packet.data(), packet.size())); ++} ++ + TEST(RtpPacketTest, SetReservedExtensionsAfterPayload) { + const size_t kPayloadSize = 4; + RtpPacketToSend::ExtensionManager extensions; +diff --git a/modules/rtp_rtcp/source/rtp_rtcp_config.h b/modules/rtp_rtcp/source/rtp_rtcp_config.h +index 3e6aa3baae..9ac7696ce9 100644 +--- a/modules/rtp_rtcp/source/rtp_rtcp_config.h ++++ b/modules/rtp_rtcp/source/rtp_rtcp_config.h +@@ -18,6 +18,8 @@ namespace webrtc { + constexpr int kDefaultMaxReorderingThreshold = 50; // In sequence numbers. + constexpr int kRtcpMaxNackFields = 253; + ++constexpr TimeDelta RTCP_INTERVAL_RAPID_SYNC_MS = ++ TimeDelta::Millis(100); // RFX 6051 + constexpr TimeDelta RTCP_SEND_BEFORE_KEY_FRAME = TimeDelta::Millis(100); + constexpr int RTCP_MAX_REPORT_BLOCKS = 31; // RFC 3550 page 37 + } // namespace webrtc +diff --git a/modules/rtp_rtcp/source/rtp_sender.cc b/modules/rtp_rtcp/source/rtp_sender.cc +index ac3bd55e05..0ed7243d0a 100644 +--- a/modules/rtp_rtcp/source/rtp_sender.cc ++++ b/modules/rtp_rtcp/source/rtp_sender.cc +@@ -132,6 +132,10 @@ bool IsNonVolatile(RTPExtensionType type) { + case kRtpExtensionNumberOfExtensions: + RTC_DCHECK_NOTREACHED(); + return false; ++ case kRtpExtensionCsrcAudioLevel: ++ // TODO: Mozilla implement for CsrcAudioLevel ++ RTC_CHECK(false); ++ return false; + } + RTC_CHECK_NOTREACHED(); + } +diff --git a/modules/utility/source/jvm_android.cc b/modules/utility/source/jvm_android.cc +index ee9930bcaa..39ef12f428 100644 +--- a/modules/utility/source/jvm_android.cc ++++ b/modules/utility/source/jvm_android.cc +@@ -18,6 +18,16 @@ + #include "rtc_base/logging.h" + #include "rtc_base/platform_thread.h" + ++namespace mozilla { ++namespace jni { ++jclass GetClassRef(JNIEnv* aEnv, const char* aClassName); ++} ++} ++ ++#define TAG "JVM" ++#define ALOGD(...) __android_log_print(ANDROID_LOG_DEBUG, TAG, __VA_ARGS__) ++#define ALOGE(...) __android_log_print(ANDROID_LOG_ERROR, TAG, __VA_ARGS__) ++ + namespace webrtc { + + JVM* g_jvm; +@@ -40,14 +50,11 @@ struct { + void LoadClasses(JNIEnv* jni) { + RTC_LOG(LS_INFO) << "LoadClasses:"; + for (auto& c : loaded_classes) { +- jclass localRef = FindClass(jni, c.name); +- RTC_LOG(LS_INFO) << "name: " << c.name; +- CHECK_EXCEPTION(jni) << "Error during FindClass: " << c.name; +- RTC_CHECK(localRef) << c.name; +- jclass globalRef = reinterpret_cast<jclass>(jni->NewGlobalRef(localRef)); +- CHECK_EXCEPTION(jni) << "Error during NewGlobalRef: " << c.name; +- RTC_CHECK(globalRef) << c.name; +- c.clazz = globalRef; ++ ALOGD("name: %s", c.name); ++ jclass clsRef = mozilla::jni::GetClassRef(jni, c.name); ++ RTC_CHECK(clsRef) << c.name; ++ c.clazz = static_cast<jclass>(jni->NewGlobalRef(clsRef)); ++ jni->DeleteLocalRef(clsRef); + } + } + +@@ -216,8 +223,9 @@ std::string JNIEnvironment::JavaToStdString(const jstring& j_string) { + + // static + void JVM::Initialize(JavaVM* jvm) { +- RTC_LOG(LS_INFO) << "JVM::Initialize"; +- RTC_CHECK(!g_jvm); ++ if (g_jvm) { ++ return; ++ } + g_jvm = new JVM(jvm); + } + +diff --git a/modules/video_capture/device_info_impl.cc b/modules/video_capture/device_info_impl.cc +index ff32a78580..7cccdb51a7 100644 +--- a/modules/video_capture/device_info_impl.cc ++++ b/modules/video_capture/device_info_impl.cc +@@ -65,7 +65,7 @@ int32_t DeviceInfoImpl::GetCapability(const char* deviceUniqueIdUTF8, + + // Make sure the number is valid + if (deviceCapabilityNumber >= (unsigned int)_captureCapabilities.size()) { +- RTC_LOG(LS_ERROR) << "Invalid deviceCapabilityNumber " ++ RTC_LOG(LS_ERROR) << deviceUniqueIdUTF8 << " Invalid deviceCapabilityNumber " + << deviceCapabilityNumber << ">= number of capabilities (" + << _captureCapabilities.size() << ")."; + return -1; +diff --git a/modules/video_capture/device_info_impl.h b/modules/video_capture/device_info_impl.h +index 546265049c..8acbef6d69 100644 +--- a/modules/video_capture/device_info_impl.h ++++ b/modules/video_capture/device_info_impl.h +@@ -42,6 +42,7 @@ class DeviceInfoImpl : public VideoCaptureModule::DeviceInfo { + /* Initialize this object*/ + + virtual int32_t Init() = 0; ++ int32_t Refresh() override { return 0; } + /* + * Fills the member variable _captureCapabilities with capabilities for the + * given device name. +diff --git a/modules/video_capture/linux/device_info_v4l2.cc b/modules/video_capture/linux/device_info_v4l2.cc +index 5af58015a7..28395a5a05 100644 +--- a/modules/video_capture/linux/device_info_v4l2.cc ++++ b/modules/video_capture/linux/device_info_v4l2.cc +@@ -27,15 +27,187 @@ + #include "modules/video_capture/video_capture_impl.h" + #include "rtc_base/logging.h" + ++#ifdef WEBRTC_LINUX ++#define EVENT_SIZE ( sizeof (struct inotify_event) ) ++#define BUF_LEN ( 1024 * ( EVENT_SIZE + 16 ) ) ++#endif ++ + namespace webrtc { + namespace videocapturemodule { +-DeviceInfoV4l2::DeviceInfoV4l2() : DeviceInfoImpl() {} ++#ifdef WEBRTC_LINUX ++void DeviceInfoV4l2::HandleEvent(inotify_event* event, int fd) ++{ ++ if (event->mask & IN_CREATE) { ++ if (fd == _fd_v4l || fd == _fd_snd) { ++ DeviceChange(); ++ } else if ((event->mask & IN_ISDIR) && (fd == _fd_dev)) { ++ if (_wd_v4l < 0) { ++ // Sometimes inotify_add_watch failed if we call it immediately after receiving this event ++ // Adding 5ms delay to let file system settle down ++ usleep(5*1000); ++ _wd_v4l = inotify_add_watch(_fd_v4l, "/dev/v4l/by-path/", IN_CREATE | IN_DELETE | IN_DELETE_SELF); ++ if (_wd_v4l >= 0) { ++ DeviceChange(); ++ } ++ } ++ if (_wd_snd < 0) { ++ usleep(5*1000); ++ _wd_snd = inotify_add_watch(_fd_snd, "/dev/snd/by-path/", IN_CREATE | IN_DELETE | IN_DELETE_SELF); ++ if (_wd_snd >= 0) { ++ DeviceChange(); ++ } ++ } ++ } ++ } else if (event->mask & IN_DELETE) { ++ if (fd == _fd_v4l || fd == _fd_snd) { ++ DeviceChange(); ++ } ++ } else if (event->mask & IN_DELETE_SELF) { ++ if (fd == _fd_v4l) { ++ inotify_rm_watch(_fd_v4l, _wd_v4l); ++ _wd_v4l = -1; ++ } else if (fd == _fd_snd) { ++ inotify_rm_watch(_fd_snd, _wd_snd); ++ _wd_snd = -1; ++ } else { ++ assert(false); ++ } ++ } ++} ++ ++int DeviceInfoV4l2::EventCheck(int fd) ++{ ++ struct timeval timeout; ++ fd_set rfds; ++ ++ timeout.tv_sec = 0; ++ timeout.tv_usec = 100000; ++ ++ FD_ZERO(&rfds); ++ FD_SET(fd, &rfds); ++ ++ return select(fd+1, &rfds, NULL, NULL, &timeout); ++} ++ ++int DeviceInfoV4l2::HandleEvents(int fd) ++{ ++ char buffer[BUF_LEN]; ++ ++ ssize_t r = read(fd, buffer, BUF_LEN); ++ ++ if (r <= 0) { ++ return r; ++ } ++ ++ ssize_t buffer_i = 0; ++ inotify_event* pevent; ++ size_t eventSize; ++ int count = 0; ++ ++ while (buffer_i < r) ++ { ++ pevent = (inotify_event *) (&buffer[buffer_i]); ++ eventSize = sizeof(inotify_event) + pevent->len; ++ char event[sizeof(inotify_event) + FILENAME_MAX + 1] // null-terminated ++ __attribute__ ((aligned(__alignof__(struct inotify_event)))); ++ ++ memcpy(event, pevent, eventSize); ++ ++ HandleEvent((inotify_event*)(event), fd); ++ ++ buffer_i += eventSize; ++ count++; ++ } ++ ++ return count; ++} ++ ++int DeviceInfoV4l2::ProcessInotifyEvents() ++{ ++ while (!_isShutdown) { ++ if (EventCheck(_fd_dev) > 0) { ++ if (HandleEvents(_fd_dev) < 0) { ++ break; ++ } ++ } ++ if (EventCheck(_fd_v4l) > 0) { ++ if (HandleEvents(_fd_v4l) < 0) { ++ break; ++ } ++ } ++ if (EventCheck(_fd_snd) > 0) { ++ if (HandleEvents(_fd_snd) < 0) { ++ break; ++ } ++ } ++ } ++ return 0; ++} ++ ++void DeviceInfoV4l2::InotifyEventThread(void* obj) ++{ ++ static_cast<DeviceInfoLinux*> (obj)->InotifyProcess(); ++} ++ ++void DeviceInfoV4l2::InotifyProcess() ++{ ++ _fd_v4l = inotify_init(); ++ _fd_snd = inotify_init(); ++ _fd_dev = inotify_init(); ++ if (_fd_v4l >= 0 && _fd_snd >= 0 && _fd_dev >= 0) { ++ _wd_v4l = inotify_add_watch(_fd_v4l, "/dev/v4l/by-path/", IN_CREATE | IN_DELETE | IN_DELETE_SELF); ++ _wd_snd = inotify_add_watch(_fd_snd, "/dev/snd/by-path/", IN_CREATE | IN_DELETE | IN_DELETE_SELF); ++ _wd_dev = inotify_add_watch(_fd_dev, "/dev/", IN_CREATE); ++ ProcessInotifyEvents(); ++ ++ if (_wd_v4l >= 0) { ++ inotify_rm_watch(_fd_v4l, _wd_v4l); ++ } ++ ++ if (_wd_snd >= 0) { ++ inotify_rm_watch(_fd_snd, _wd_snd); ++ } ++ ++ if (_wd_dev >= 0) { ++ inotify_rm_watch(_fd_dev, _wd_dev); ++ } ++ ++ close(_fd_v4l); ++ close(_fd_snd); ++ close(_fd_dev); ++ } ++} ++#endif ++ ++DeviceInfoV4l2::DeviceInfoV4l2() : DeviceInfoImpl() ++#ifdef WEBRTC_LINUX ++ , _inotifyEventThread(new rtc::PlatformThread( ++ InotifyEventThread, this, "InotifyEventThread")) ++ , _isShutdown(false) ++#endif ++{ ++#ifdef WEBRTC_LINUX ++ if (_inotifyEventThread) ++ { ++ _inotifyEventThread->Start(); ++ } ++} ++#endif + + int32_t DeviceInfoV4l2::Init() { + return 0; + } + +-DeviceInfoV4l2::~DeviceInfoV4l2() {} ++DeviceInfoV4l2::~DeviceInfoV4l2() { ++#ifdef WEBRTC_LINUX ++ _isShutdown = true; ++ ++ if (_inotifyEventThread) { ++ _inotifyEventThread->Stop(); ++ _inotifyEventThread = nullptr; ++ } ++#endif ++} + + uint32_t DeviceInfoV4l2::NumberOfDevices() { + uint32_t count = 0; +@@ -68,15 +240,17 @@ int32_t DeviceInfoV4l2::GetDeviceName(uint32_t deviceNumber, + char* deviceUniqueIdUTF8, + uint32_t deviceUniqueIdUTF8Length, + char* /*productUniqueIdUTF8*/, +- uint32_t /*productUniqueIdUTF8Length*/) { ++ uint32_t /*productUniqueIdUTF8Length*/, ++ pid_t* /*pid*/) { + // Travel through /dev/video [0-63] + uint32_t count = 0; + char device[20]; + int fd = -1; + bool found = false; + struct v4l2_capability cap; +- for (int n = 0; n < 64; n++) { +- snprintf(device, sizeof(device), "/dev/video%d", n); ++ int device_index; ++ for (device_index = 0; device_index < 64; device_index++) { ++ sprintf(device, "/dev/video%d", device_index); + if ((fd = open(device, O_RDONLY)) != -1) { + // query device capabilities and make sure this is a video capture device + if (ioctl(fd, VIDIOC_QUERYCAP, &cap) < 0 || +@@ -129,8 +303,15 @@ int32_t DeviceInfoV4l2::GetDeviceName(uint32_t deviceNumber, + RTC_LOG(LS_INFO) << "buffer passed is too small"; + return -1; + } ++ } else { ++ // if there's no bus info to use for uniqueId, invent one - and it has to be repeatable ++ if (snprintf(deviceUniqueIdUTF8, ++ deviceUniqueIdUTF8Length, "fake_%u", device_index) >= ++ (int) deviceUniqueIdUTF8Length) ++ { ++ return -1; ++ } + } +- + return 0; + } + +diff --git a/modules/video_capture/linux/device_info_v4l2.h b/modules/video_capture/linux/device_info_v4l2.h +index fb95a6020d..95432a509d 100644 +--- a/modules/video_capture/linux/device_info_v4l2.h ++++ b/modules/video_capture/linux/device_info_v4l2.h +@@ -15,6 +15,9 @@ + + #include "modules/video_capture/device_info_impl.h" + ++#include "rtc_base/platform_thread.h" ++#include <sys/inotify.h> ++ + namespace webrtc { + namespace videocapturemodule { + class DeviceInfoV4l2 : public DeviceInfoImpl { +@@ -28,7 +31,8 @@ class DeviceInfoV4l2 : public DeviceInfoImpl { + char* deviceUniqueIdUTF8, + uint32_t deviceUniqueIdUTF8Length, + char* productUniqueIdUTF8 = 0, +- uint32_t productUniqueIdUTF8Length = 0) override; ++ uint32_t productUniqueIdUTF8Length = 0, ++ pid_t* pid=0) override; + /* + * Fills the membervariable _captureCapabilities with capabilites for the + * given device name. +@@ -45,6 +49,18 @@ class DeviceInfoV4l2 : public DeviceInfoImpl { + + private: + bool IsDeviceNameMatches(const char* name, const char* deviceUniqueIdUTF8); ++ ++#ifdef WEBRTC_LINUX ++ void HandleEvent(inotify_event* event, int fd); ++ int EventCheck(int fd); ++ int HandleEvents(int fd); ++ int ProcessInotifyEvents(); ++ std::unique_ptr<rtc::PlatformThread> _inotifyEventThread; ++ static void InotifyEventThread(void*); ++ void InotifyProcess(); ++ int _fd_v4l, _fd_dev, _wd_v4l, _wd_dev; /* accessed on InotifyEventThread thread */ ++ std::atomic<bool> _isShutdown; ++#endif + }; + } // namespace videocapturemodule + } // namespace webrtc +diff --git a/modules/video_capture/linux/video_capture_v4l2.cc b/modules/video_capture/linux/video_capture_v4l2.cc +index 5101a67e0c..1dc13b01aa 100644 +--- a/modules/video_capture/linux/video_capture_v4l2.cc ++++ b/modules/video_capture/linux/video_capture_v4l2.cc +@@ -50,6 +50,13 @@ int32_t VideoCaptureModuleV4L2::Init(const char* deviceUniqueIdUTF8) { + memcpy(_deviceUniqueId, deviceUniqueIdUTF8, len + 1); + } + ++ int device_index; ++ if (sscanf(deviceUniqueIdUTF8,"fake_%d", &device_index) == 1) ++ { ++ _deviceId = device_index; ++ return 0; ++ } ++ + int fd; + char device[32]; + bool found = false; +diff --git a/modules/video_capture/video_capture.h b/modules/video_capture/video_capture.h +index eddc31414a..187eba76a0 100644 +--- a/modules/video_capture/video_capture.h ++++ b/modules/video_capture/video_capture.h +@@ -15,15 +15,48 @@ + #include "api/video/video_sink_interface.h" + #include "modules/video_capture/raw_video_sink_interface.h" + #include "modules/video_capture/video_capture_defines.h" ++#include <set> ++ ++#if defined(ANDROID) ++#include <jni.h> ++#endif + + namespace webrtc { + ++class VideoInputFeedBack ++{ ++public: ++ virtual void OnDeviceChange() = 0; ++protected: ++ virtual ~VideoInputFeedBack(){} ++}; ++ ++#if defined(ANDROID) && !defined(WEBRTC_CHROMIUM_BUILD) ++ int32_t SetCaptureAndroidVM(JavaVM* javaVM); ++#endif ++ + class VideoCaptureModule : public rtc::RefCountInterface { + public: + // Interface for receiving information about available camera devices. + class DeviceInfo { + public: + virtual uint32_t NumberOfDevices() = 0; ++ virtual int32_t Refresh() = 0; ++ virtual void DeviceChange() { ++ for (auto inputCallBack : _inputCallBacks) { ++ inputCallBack->OnDeviceChange(); ++ } ++ } ++ virtual void RegisterVideoInputFeedBack(VideoInputFeedBack* callBack) { ++ _inputCallBacks.insert(callBack); ++ } ++ ++ virtual void DeRegisterVideoInputFeedBack(VideoInputFeedBack* callBack) { ++ auto it = _inputCallBacks.find(callBack); ++ if (it != _inputCallBacks.end()) { ++ _inputCallBacks.erase(it); ++ } ++ } + + // Returns the available capture devices. + // deviceNumber - Index of capture device. +@@ -38,7 +71,8 @@ class VideoCaptureModule : public rtc::RefCountInterface { + char* deviceUniqueIdUTF8, + uint32_t deviceUniqueIdUTF8Length, + char* productUniqueIdUTF8 = 0, +- uint32_t productUniqueIdUTF8Length = 0) = 0; ++ uint32_t productUniqueIdUTF8Length = 0, ++ pid_t* pid = 0) = 0; + + // Returns the number of capabilities this device. + virtual int32_t NumberOfCapabilities(const char* deviceUniqueIdUTF8) = 0; +@@ -70,6 +104,8 @@ class VideoCaptureModule : public rtc::RefCountInterface { + uint32_t positionY) = 0; + + virtual ~DeviceInfo() {} ++ private: ++ std::set<VideoInputFeedBack*> _inputCallBacks; + }; + + // Register capture data callback +@@ -79,11 +115,16 @@ class VideoCaptureModule : public rtc::RefCountInterface { + RawVideoSinkInterface* dataCallback) = 0; + + // Remove capture data callback +- virtual void DeRegisterCaptureDataCallback() = 0; ++ virtual void DeRegisterCaptureDataCallback( ++ rtc::VideoSinkInterface<VideoFrame> *dataCallback) = 0; + + // Start capture device + virtual int32_t StartCapture(const VideoCaptureCapability& capability) = 0; + ++ virtual int32_t StopCaptureIfAllClientsClose() = 0; ++ ++ virtual bool FocusOnSelectedSource() { return false; } ++ + virtual int32_t StopCapture() = 0; + + // Returns the name of the device used by this module. +diff --git a/modules/video_capture/video_capture_factory.cc b/modules/video_capture/video_capture_factory.cc +index c0e1479caf..e4a46902e0 100644 +--- a/modules/video_capture/video_capture_factory.cc ++++ b/modules/video_capture/video_capture_factory.cc +@@ -16,19 +16,11 @@ namespace webrtc { + + rtc::scoped_refptr<VideoCaptureModule> VideoCaptureFactory::Create( + const char* deviceUniqueIdUTF8) { +-#if defined(WEBRTC_ANDROID) || defined(WEBRTC_MAC) +- return nullptr; +-#else + return videocapturemodule::VideoCaptureImpl::Create(deviceUniqueIdUTF8); +-#endif + } + + VideoCaptureModule::DeviceInfo* VideoCaptureFactory::CreateDeviceInfo() { +-#if defined(WEBRTC_ANDROID) || defined(WEBRTC_MAC) +- return nullptr; +-#else + return videocapturemodule::VideoCaptureImpl::CreateDeviceInfo(); +-#endif + } + + } // namespace webrtc +diff --git a/modules/video_capture/video_capture_impl.cc b/modules/video_capture/video_capture_impl.cc +index d539b38264..d5ec4daae1 100644 +--- a/modules/video_capture/video_capture_impl.cc ++++ b/modules/video_capture/video_capture_impl.cc +@@ -76,7 +76,6 @@ VideoCaptureImpl::VideoCaptureImpl() + _requestedCapability(), + _lastProcessTimeNanos(rtc::TimeNanos()), + _lastFrameRateCallbackTimeNanos(rtc::TimeNanos()), +- _dataCallBack(NULL), + _rawDataCallBack(NULL), + _lastProcessFrameTimeNanos(rtc::TimeNanos()), + _rotateFrame(kVideoRotation_0), +@@ -89,7 +88,6 @@ VideoCaptureImpl::VideoCaptureImpl() + } + + VideoCaptureImpl::~VideoCaptureImpl() { +- DeRegisterCaptureDataCallback(); + if (_deviceUniqueId) + delete[] _deviceUniqueId; + } +@@ -98,26 +96,39 @@ void VideoCaptureImpl::RegisterCaptureDataCallback( + rtc::VideoSinkInterface<VideoFrame>* dataCallBack) { + MutexLock lock(&api_lock_); + RTC_DCHECK(!_rawDataCallBack); +- _dataCallBack = dataCallBack; ++ _dataCallBacks.insert(dataCallBack); + } + + void VideoCaptureImpl::RegisterCaptureDataCallback( + RawVideoSinkInterface* dataCallBack) { + MutexLock lock(&api_lock_); +- RTC_DCHECK(!_dataCallBack); ++ RTC_DCHECK(_dataCallBacks.empty()); + _rawDataCallBack = dataCallBack; + } + +-void VideoCaptureImpl::DeRegisterCaptureDataCallback() { ++void VideoCaptureImpl::DeRegisterCaptureDataCallback( ++ rtc::VideoSinkInterface<VideoFrame>* dataCallBack) { + MutexLock lock(&api_lock_); +- _dataCallBack = NULL; ++ auto it = _dataCallBacks.find(dataCallBack); ++ if (it != _dataCallBacks.end()) { ++ _dataCallBacks.erase(it); ++ } + _rawDataCallBack = NULL; + } ++ ++int32_t VideoCaptureImpl::StopCaptureIfAllClientsClose() { ++ if (_dataCallBacks.empty()) { ++ return StopCapture(); ++ } else { ++ return 0; ++ } ++} ++ + int32_t VideoCaptureImpl::DeliverCapturedFrame(VideoFrame& captureFrame) { + UpdateFrameCount(); // frame count used for local frame rate callback. + +- if (_dataCallBack) { +- _dataCallBack->OnFrame(captureFrame); ++ for (auto dataCallBack : _dataCallBacks) { ++ dataCallBack->OnFrame(captureFrame); + } + + return 0; +@@ -204,7 +215,7 @@ int32_t VideoCaptureImpl::IncomingFrame(uint8_t* videoFrame, + buffer.get()->StrideV(), 0, 0, // No Cropping + width, height, target_width, target_height, rotation_mode, + ConvertVideoType(frameInfo.videoType)); +- if (conversionResult < 0) { ++ if (conversionResult != 0) { + RTC_LOG(LS_ERROR) << "Failed to convert capture frame from type " + << static_cast<int>(frameInfo.videoType) << "to I420."; + return -1; +@@ -219,6 +230,13 @@ int32_t VideoCaptureImpl::IncomingFrame(uint8_t* videoFrame, + .build(); + captureFrame.set_ntp_time_ms(captureTime); + ++ // This is one ugly hack to let CamerasParent know what rotation ++ // the frame was captured at. Note that this goes against the intended ++ // meaning of rotation of the frame (how to rotate it before rendering). ++ // We do this so CamerasChild can scale to the proper dimensions ++ // later on in the pipe. ++ captureFrame.set_rotation(_rotateFrame); ++ + DeliverCapturedFrame(captureFrame); + + return 0; +diff --git a/modules/video_capture/video_capture_impl.h b/modules/video_capture/video_capture_impl.h +index fee93396d3..f874580471 100644 +--- a/modules/video_capture/video_capture_impl.h ++++ b/modules/video_capture/video_capture_impl.h +@@ -55,8 +55,10 @@ class VideoCaptureImpl : public VideoCaptureModule { + rtc::VideoSinkInterface<VideoFrame>* dataCallback) override; + virtual void RegisterCaptureDataCallback( + RawVideoSinkInterface* dataCallback) override; +- void DeRegisterCaptureDataCallback() override; ++ void DeRegisterCaptureDataCallback( ++ rtc::VideoSinkInterface<VideoFrame>* dataCallback) override; + ++ int32_t StopCaptureIfAllClientsClose() override; + int32_t SetCaptureRotation(VideoRotation rotation) override; + bool SetApplyRotation(bool enable) override; + bool GetApplyRotation() override; +@@ -98,7 +100,7 @@ class VideoCaptureImpl : public VideoCaptureModule { + // last time the frame rate callback function was called. + int64_t _lastFrameRateCallbackTimeNanos; + +- rtc::VideoSinkInterface<VideoFrame>* _dataCallBack; ++ std::set<rtc::VideoSinkInterface<VideoFrame>*> _dataCallBacks; + RawVideoSinkInterface* _rawDataCallBack; + + int64_t _lastProcessFrameTimeNanos; +diff --git a/modules/video_coding/codecs/vp9/libvpx_vp9_encoder.cc b/modules/video_coding/codecs/vp9/libvpx_vp9_encoder.cc +index c2884c0395..35b13058a2 100644 +--- a/modules/video_coding/codecs/vp9/libvpx_vp9_encoder.cc ++++ b/modules/video_coding/codecs/vp9/libvpx_vp9_encoder.cc +@@ -254,6 +254,7 @@ LibvpxVp9Encoder::LibvpxVp9Encoder(const cricket::VideoCodec& codec, + first_frame_in_picture_(true), + ss_info_needed_(false), + force_all_active_layers_(false), ++ num_cores_(0), + is_flexible_mode_(false), + variable_framerate_experiment_(ParseVariableFramerateConfig(trials)), + variable_framerate_controller_( +@@ -577,6 +578,7 @@ int LibvpxVp9Encoder::InitEncode(const VideoCodec* inst, + + force_key_frame_ = true; + pics_since_key_ = 0; ++ num_cores_ = settings.number_of_cores; + + scalability_mode_ = inst->GetScalabilityMode(); + if (scalability_mode_.has_value()) { +diff --git a/modules/video_coding/codecs/vp9/libvpx_vp9_encoder.h b/modules/video_coding/codecs/vp9/libvpx_vp9_encoder.h +index 6b662ae8f9..bb871f8498 100644 +--- a/modules/video_coding/codecs/vp9/libvpx_vp9_encoder.h ++++ b/modules/video_coding/codecs/vp9/libvpx_vp9_encoder.h +@@ -67,6 +67,9 @@ class LibvpxVp9Encoder : public VP9Encoder { + // Call encoder initialize function and set control settings. + int InitAndSetControlSettings(const VideoCodec* inst); + ++ // Update frame size for codec. ++ int UpdateCodecFrameSize(const VideoFrame& input_image); ++ + bool PopulateCodecSpecific(CodecSpecificInfo* codec_specific, + absl::optional<int>* spatial_idx, + absl::optional<int>* temporal_idx, +@@ -150,6 +153,7 @@ class LibvpxVp9Encoder : public VP9Encoder { + VideoBitrateAllocation current_bitrate_allocation_; + bool ss_info_needed_; + bool force_all_active_layers_; ++ uint8_t num_cores_; + + std::unique_ptr<ScalableVideoController> svc_controller_; + absl::optional<ScalabilityMode> scalability_mode_; +diff --git a/modules/video_coding/generic_decoder.cc b/modules/video_coding/generic_decoder.cc +index b660e02b72..54467d1477 100644 +--- a/modules/video_coding/generic_decoder.cc ++++ b/modules/video_coding/generic_decoder.cc +@@ -32,7 +32,10 @@ namespace webrtc { + + namespace { + +-constexpr size_t kDecoderFrameMemoryLength = 10; ++// Changed from 10 to 30 in Mozilla Bug 989944: Increase decode ++// timestamp map to handle delayed decode on 8x10. The map is ++// now a deque (as of libwebrtc upstream commit 1c51ec4d74). ++constexpr size_t kDecoderFrameMemoryLength = 30; + + } + +diff --git a/modules/video_coding/session_info.cc b/modules/video_coding/session_info.cc +index 854230ae0a..e31b8b1d25 100644 +--- a/modules/video_coding/session_info.cc ++++ b/modules/video_coding/session_info.cc +@@ -210,16 +210,26 @@ size_t VCMSessionInfo::InsertBuffer(uint8_t* frame_buffer, + if (h264 && h264->packetization_type == kH264StapA) { + size_t required_length = 0; + const uint8_t* nalu_ptr = packet_buffer + kH264NALHeaderLengthInBytes; +- while (nalu_ptr < packet_buffer + packet.sizeBytes) { ++ // Must check that incoming data length doesn't extend past end of buffer. ++ // We allow for 100 bytes of expansion due to startcodes being longer than ++ // length fields. ++ while (nalu_ptr + kLengthFieldLength <= packet_buffer + packet.sizeBytes) { + size_t length = BufferToUWord16(nalu_ptr); +- required_length += ++ if (nalu_ptr + kLengthFieldLength + length <= packet_buffer + packet.sizeBytes) { ++ required_length += + length + (packet.insertStartCode ? kH264StartCodeLengthBytes : 0); +- nalu_ptr += kLengthFieldLength + length; ++ nalu_ptr += kLengthFieldLength + length; ++ } else { ++ // Something is very wrong! ++ RTC_LOG(LS_ERROR) << "Failed to insert packet due to corrupt H264 STAP-A"; ++ return 0; ++ } + } + ShiftSubsequentPackets(packet_it, required_length); + nalu_ptr = packet_buffer + kH264NALHeaderLengthInBytes; + uint8_t* frame_buffer_ptr = frame_buffer + offset; +- while (nalu_ptr < packet_buffer + packet.sizeBytes) { ++ // we already know we won't go past end-of-buffer ++ while (nalu_ptr + kLengthFieldLength <= packet_buffer + packet.sizeBytes) { + size_t length = BufferToUWord16(nalu_ptr); + nalu_ptr += kLengthFieldLength; + frame_buffer_ptr += Insert(nalu_ptr, length, packet.insertStartCode, +@@ -447,12 +457,23 @@ int VCMSessionInfo::InsertPacket(const VCMPacket& packet, + return -2; + + if (packet.codec() == kVideoCodecH264) { +- frame_type_ = packet.video_header.frame_type; ++ // H.264 can have leading or trailing non-VCL (Video Coding Layer) ++ // NALUs, such as SPS/PPS/SEI and others. Also, the RTP marker bit is ++ // not reliable for the last packet of a frame (RFC 6184 5.1 - "Decoders ++ // [] MUST NOT rely on this property"), so allow out-of-order packets to ++ // update the first and last seq# range. Also mark as a key frame if ++ // any packet is of that type. ++ if (frame_type_ != VideoFrameType::kVideoFrameKey) { ++ frame_type_ = packet.video_header.frame_type; ++ } + if (packet.is_first_packet_in_frame() && + (first_packet_seq_num_ == -1 || + IsNewerSequenceNumber(first_packet_seq_num_, packet.seqNum))) { + first_packet_seq_num_ = packet.seqNum; + } ++ // Note: the code does *not* currently handle the Marker bit being totally ++ // absent from a frame. It does not, however, depend on it being on the last ++ // packet of the 'frame'/'session'. + if (packet.markerBit && + (last_packet_seq_num_ == -1 || + IsNewerSequenceNumber(packet.seqNum, last_packet_seq_num_))) { +@@ -499,7 +520,6 @@ int VCMSessionInfo::InsertPacket(const VCMPacket& packet, + + size_t returnLength = InsertBuffer(frame_buffer, packet_list_it); + UpdateCompleteSession(); +- + return static_cast<int>(returnLength); + } + +diff --git a/test/fuzzers/rtp_packet_fuzzer.cc b/test/fuzzers/rtp_packet_fuzzer.cc +index 60afb986de..0e10a8fa3a 100644 +--- a/test/fuzzers/rtp_packet_fuzzer.cc ++++ b/test/fuzzers/rtp_packet_fuzzer.cc +@@ -164,6 +164,11 @@ void FuzzOneInput(const uint8_t* data, size_t size) { + // This extension requires state to read and so complicated that + // deserves own fuzzer. + break; ++ case kRtpExtensionCsrcAudioLevel: { ++ CsrcAudioLevelList levels; ++ packet.GetExtension<CsrcAudioLevel>(&levels); ++ break; ++ } + } + } + +diff --git a/test/vcm_capturer.cc b/test/vcm_capturer.cc +index a037f9eff6..e02fc722b2 100644 +--- a/test/vcm_capturer.cc ++++ b/test/vcm_capturer.cc +@@ -81,7 +81,7 @@ void VcmCapturer::Destroy() { + return; + + vcm_->StopCapture(); +- vcm_->DeRegisterCaptureDataCallback(); ++ vcm_->DeRegisterCaptureDataCallback(this); + // Release reference to VCM. + vcm_ = nullptr; + } +-- +2.34.1 + diff --git a/third_party/libwebrtc/moz-patch-stack/0002.patch b/third_party/libwebrtc/moz-patch-stack/0002.patch new file mode 100644 index 0000000000..fb6cf038ae --- /dev/null +++ b/third_party/libwebrtc/moz-patch-stack/0002.patch @@ -0,0 +1,46 @@ +From: Nico Grunbaum <na-g@nostrum.com> +Date: Sat, 13 Feb 2021 04:20:00 -0800 +Subject: Bug 1654112 - Add pid_t to desktop_capture_types.h; r=pehrsons + +Upstreaming bug 1697385 + +Also includes: +Bug 1654112 - Clarifying prev. rev that moved pid_t into the global namespace; r=dminor + +Differential Revision: https://phabricator.services.mozilla.com/D107897 +Mercurial Revision: https://hg.mozilla.org/mozilla-central/rev/539b69f0e21118a1898f1ef689577c7193ff7be7 +--- + modules/desktop_capture/desktop_capture_types.h | 4 ++++ + modules/video_capture/video_capture.h | 1 + + 2 files changed, 5 insertions(+) + +diff --git a/modules/desktop_capture/desktop_capture_types.h b/modules/desktop_capture/desktop_capture_types.h +index 9627076eea..381d1021c4 100644 +--- a/modules/desktop_capture/desktop_capture_types.h ++++ b/modules/desktop_capture/desktop_capture_types.h +@@ -13,6 +13,10 @@ + + #include <stdint.h> + ++#ifdef XP_WIN // Moving this into the global namespace ++typedef int pid_t; // matching what used to be in ++#endif // video_capture_defines.h ++ + namespace webrtc { + + enum class CaptureType { kWindow, kScreen }; +diff --git a/modules/video_capture/video_capture.h b/modules/video_capture/video_capture.h +index 187eba76a0..258bc7f810 100644 +--- a/modules/video_capture/video_capture.h ++++ b/modules/video_capture/video_capture.h +@@ -13,6 +13,7 @@ + + #include "api/video/video_rotation.h" + #include "api/video/video_sink_interface.h" ++#include "modules/desktop_capture/desktop_capture_types.h" + #include "modules/video_capture/raw_video_sink_interface.h" + #include "modules/video_capture/video_capture_defines.h" + #include <set> +-- +2.34.1 + diff --git a/third_party/libwebrtc/moz-patch-stack/0003.patch b/third_party/libwebrtc/moz-patch-stack/0003.patch new file mode 100644 index 0000000000..bb8c8ba81a --- /dev/null +++ b/third_party/libwebrtc/moz-patch-stack/0003.patch @@ -0,0 +1,102 @@ +From: Dan Minor <dminor@mozilla.com> +Date: Tue, 21 Aug 2018 13:39:00 -0400 +Subject: Bug 1376873 - Fix up logging in WebrtcLog.cpp; r=ng + +The webrtc::Trace code is removed by this update. We already had support for +LOG (now RTC_LOG) in WebrtcLog.cpp. This removes the trace code from +WebRtcLog.cpp and moves the aec logging code from webrtc::Trace to +rtc::LogMessage. + +This also disables logging to stderr in rtc_base/logging.cc. We could disable +it using the API, but that happens through peerconnection resulting in some +logging occuring during getusermedia. + +The aec logs were testing with --disable-e10s. Rather than trying to +work around sandboxing, I think it makes more sense to fix Bug 1404982 and +store the logs in memory for retrieval from about:webrtc. + +Differential Revision: https://phabricator.services.mozilla.com/D7429 +Mercurial Revision: https://hg.mozilla.org/mozilla-central/rev/e84c60e2c9373f4d2dc24e769375a92c17c2a0ad +--- + .../audio_processing/logging/apm_data_dumper.cc | 2 +- + rtc_base/logging.cc | 11 ++++++++++- + rtc_base/logging.h | 14 ++++++++++++++ + 3 files changed, 25 insertions(+), 2 deletions(-) + +diff --git a/modules/audio_processing/logging/apm_data_dumper.cc b/modules/audio_processing/logging/apm_data_dumper.cc +index f787b65604..a15321ad48 100644 +--- a/modules/audio_processing/logging/apm_data_dumper.cc ++++ b/modules/audio_processing/logging/apm_data_dumper.cc +@@ -42,7 +42,7 @@ std::string FormFileName(absl::string_view output_dir, + #endif + + std::stringstream ss; +- std::string base = webrtc::Trace::aec_debug_filename(); ++ std::string base = rtc::LogMessage::aec_debug_filename(); + ss << base; + + if (base.length() && base.back() != sep) { +diff --git a/rtc_base/logging.cc b/rtc_base/logging.cc +index 4bc9183d97..d71d6a3e1b 100644 +--- a/rtc_base/logging.cc ++++ b/rtc_base/logging.cc +@@ -54,6 +54,15 @@ static const int kMaxLogLineSize = 1024 - 60; + #include "rtc_base/time_utils.h" + + namespace rtc { ++ ++bool LogMessage::aec_debug_ = false; ++uint32_t LogMessage::aec_debug_size_ = 4*1024*1024; ++std::string LogMessage::aec_filename_base_; ++ ++std::string LogMessage::aec_debug_filename() { ++ return aec_filename_base_; ++} ++ + namespace { + + // By default, release builds don't log, debug builds at info level +@@ -114,7 +123,7 @@ std::string LogLineRef::DefaultLogLine() const { + // LogMessage + ///////////////////////////////////////////////////////////////////////////// + +-bool LogMessage::log_to_stderr_ = true; ++bool LogMessage::log_to_stderr_ = false; + + // The list of logging streams currently configured. + // Note: we explicitly do not clean this up, because of the uncertain ordering +diff --git a/rtc_base/logging.h b/rtc_base/logging.h +index d59b9a0ef7..8f490c44a2 100644 +--- a/rtc_base/logging.h ++++ b/rtc_base/logging.h +@@ -581,6 +581,16 @@ class LogMessage { + } + #endif // RTC_LOG_ENABLED() + ++ // Enable dumping of AEC inputs and outputs. Can be changed in mid-call ++ static void set_aec_debug(bool enable) { aec_debug_ = enable; } ++ static void set_aec_debug_size(uint32_t size) { aec_debug_size_ = size; } ++ static bool aec_debug() { return aec_debug_; } ++ static uint32_t aec_debug_size() { return aec_debug_size_; } ++ static std::string aec_debug_filename(); ++ static void set_aec_debug_filename(const char* filename) { ++ aec_filename_base_ = filename; ++ } ++ + private: + friend class LogMessageForTesting; + +@@ -636,6 +646,10 @@ class LogMessage { + + // The stringbuilder that buffers the formatted message before output + rtc::StringBuilder print_stream_; ++ ++ static bool aec_debug_; ++ static uint32_t aec_debug_size_; ++ static std::string aec_filename_base_; + }; + + ////////////////////////////////////////////////////////////////////// +-- +2.34.1 + diff --git a/third_party/libwebrtc/moz-patch-stack/0004.patch b/third_party/libwebrtc/moz-patch-stack/0004.patch new file mode 100644 index 0000000000..1144dc2ea4 --- /dev/null +++ b/third_party/libwebrtc/moz-patch-stack/0004.patch @@ -0,0 +1,37 @@ +From: Michael Froman <mfroman@mozilla.com> +Date: Fri, 9 Jul 2021 18:14:00 -0500 +Subject: Bug 1654112 - mutex changes to fix tsan errors. r=ng + +Differential Revision: https://phabricator.services.mozilla.com/D119674 +Mercurial Revision: https://hg.mozilla.org/mozilla-central/rev/3d5503acf9a4b22e02c4300f29e4fbfed406ea2c +--- + rtc_base/logging.cc | 5 ++++- + 1 file changed, 4 insertions(+), 1 deletion(-) + +diff --git a/rtc_base/logging.cc b/rtc_base/logging.cc +index d71d6a3e1b..d6ae7612fc 100644 +--- a/rtc_base/logging.cc ++++ b/rtc_base/logging.cc +@@ -265,8 +265,8 @@ void LogMessage::LogTimestamps(bool on) { + } + + void LogMessage::LogToDebug(LoggingSeverity min_sev) { +- g_dbg_sev = min_sev; + webrtc::MutexLock lock(&GetLoggingLock()); ++ g_dbg_sev = min_sev; + UpdateMinLogSeverity(); + } + +@@ -455,6 +455,9 @@ void LogMessage::OutputToDebug(const LogLineRef& log_line) { + + // static + bool LogMessage::IsNoop(LoggingSeverity severity) { ++ // Added MutexLock to fix tsan warnings on accessing g_dbg_sev. (mjf) ++ // See https://bugs.chromium.org/p/chromium/issues/detail?id=1228729 ++ webrtc::MutexLock lock(&GetLoggingLock()); + if (severity >= g_dbg_sev || severity >= g_min_sev) + return false; + return streams_empty_.load(std::memory_order_relaxed); +-- +2.34.1 + diff --git a/third_party/libwebrtc/moz-patch-stack/0005.patch b/third_party/libwebrtc/moz-patch-stack/0005.patch new file mode 100644 index 0000000000..990094bc0d --- /dev/null +++ b/third_party/libwebrtc/moz-patch-stack/0005.patch @@ -0,0 +1,49 @@ +From: Dan Minor <dminor@mozilla.com> +Date: Wed, 18 Nov 2020 13:33:00 -0500 +Subject: Bug 1654112 - Suppress -Wclass-varargs warning in logging.h. r=ng + +This needs some investigation to see why we get this warning when it is not +present upstream. + +Since both were doing the same thing for different compiler chains, +also includes: +Bug 1654112 - linux build fix (pragmas) for base-toolchains* . r=ng + +Differential Revision: https://phabricator.services.mozilla.com/D130086 +Mercurial Revision: https://hg.mozilla.org/mozilla-central/rev/8d832e832ffe513246c0763a56376a8022b2447b +--- + rtc_base/logging.h | 13 +++++++++++++ + 1 file changed, 13 insertions(+) + +diff --git a/rtc_base/logging.h b/rtc_base/logging.h +index 8f490c44a2..9340fe2c55 100644 +--- a/rtc_base/logging.h ++++ b/rtc_base/logging.h +@@ -48,6 +48,14 @@ + #ifndef RTC_BASE_LOGGING_H_ + #define RTC_BASE_LOGGING_H_ + ++#pragma GCC diagnostic push ++#pragma GCC diagnostic ignored "-Wvarargs" ++ ++#if defined(__clang__) ++# pragma clang diagnostic push ++# pragma clang diagnostic ignored "-Wclass-varargs" ++#endif ++ + #include <errno.h> + + #include <atomic> +@@ -769,4 +777,9 @@ inline const char* AdaptString(const std::string& str) { + + } // namespace rtc + ++#pragma GCC diagnostic pop ++#if defined(__clang__) ++# pragma clang diagnostic pop ++#endif ++ + #endif // RTC_BASE_LOGGING_H_ +-- +2.34.1 + diff --git a/third_party/libwebrtc/moz-patch-stack/0006.patch b/third_party/libwebrtc/moz-patch-stack/0006.patch new file mode 100644 index 0000000000..5b2c6f8c8a --- /dev/null +++ b/third_party/libwebrtc/moz-patch-stack/0006.patch @@ -0,0 +1,52 @@ +From: Dan Minor <dminor@mozilla.com> +Date: Tue, 27 Mar 2018 15:43:00 -0400 +Subject: Bug 1376873 - Disable Mid support in RtpDemuxer; r=mjf + +The only use of Mid in the current webrtc.org code is in the unit tests. +RtpStreamReceiverController only allows adding sinks using SSRCs. Because +of this, we'll end up dropping packets in the RtpDemuxer with the current +code as none of our Mids will be recognized. + +Tip of webrtc.org fully supports using Mids, so we'll be able to enable this +code again after the next update. + +Differential Revision: https://phabricator.services.mozilla.com/D7442 +Mercurial Revision: https://hg.mozilla.org/mozilla-central/rev/b3ba8452e77105c72f6ddbc49cbe5a53dbea1507 +--- + call/rtp_demuxer.cc | 7 ++++++- + 1 file changed, 6 insertions(+), 1 deletion(-) + +diff --git a/call/rtp_demuxer.cc b/call/rtp_demuxer.cc +index 0b74f2ac0a..5c53f48144 100644 +--- a/call/rtp_demuxer.cc ++++ b/call/rtp_demuxer.cc +@@ -272,13 +272,17 @@ RtpPacketSinkInterface* RtpDemuxer::ResolveSink( + // RSID and RRID are routed to the same sinks. If an RSID is specified on a + // repair packet, it should be ignored and the RRID should be used. + std::string packet_mid, packet_rsid; +- bool has_mid = use_mid_ && packet.GetExtension<RtpMid>(&packet_mid); ++ //bool has_mid = use_mid_ && packet.GetExtension<RtpMid>(&packet_mid); + bool has_rsid = packet.GetExtension<RepairedRtpStreamId>(&packet_rsid); + if (!has_rsid) { + has_rsid = packet.GetExtension<RtpStreamId>(&packet_rsid); + } + uint32_t ssrc = packet.Ssrc(); + ++ // Mid support is half-baked in branch 64. RtpStreamReceiverController only ++ // supports adding sinks by ssrc, so our mids will never show up in ++ // known_mids_, causing us to drop packets here. ++#if 0 + // The BUNDLE spec says to drop any packets with unknown MIDs, even if the + // SSRC is known/latched. + if (has_mid && known_mids_.find(packet_mid) == known_mids_.end()) { +@@ -352,6 +356,7 @@ RtpPacketSinkInterface* RtpDemuxer::ResolveSink( + } + } + ++#endif + // We trust signaled SSRC more than payload type which is likely to conflict + // between streams. + const auto ssrc_sink_it = sink_by_ssrc_.find(ssrc); +-- +2.34.1 + diff --git a/third_party/libwebrtc/moz-patch-stack/0007.patch b/third_party/libwebrtc/moz-patch-stack/0007.patch new file mode 100644 index 0000000000..c396dae974 --- /dev/null +++ b/third_party/libwebrtc/moz-patch-stack/0007.patch @@ -0,0 +1,142 @@ +From: Dan Minor <dminor@mozilla.com> +Date: Wed, 7 Feb 2018 15:00:00 -0500 +Subject: Bug 1376873 - Fix GetRTCPSenderReport; r=ng + +Differential Revision: https://phabricator.services.mozilla.com/D7431 +Mercurial Revision: https://hg.mozilla.org/mozilla-central/rev/b55b0368d9f21849fa465fa9b3f028285c9ea6ae +--- + call/video_receive_stream.h | 3 +++ + modules/rtp_rtcp/source/rtcp_receiver.cc | 7 +++++++ + modules/rtp_rtcp/source/rtcp_receiver.h | 4 ++++ + modules/rtp_rtcp/source/rtp_rtcp_impl.cc | 5 +++++ + modules/rtp_rtcp/source/rtp_rtcp_impl.h | 3 +++ + modules/rtp_rtcp/source/rtp_rtcp_impl2.cc | 5 +++++ + modules/rtp_rtcp/source/rtp_rtcp_impl2.h | 3 +++ + modules/rtp_rtcp/source/rtp_rtcp_interface.h | 4 ++++ + 8 files changed, 34 insertions(+) + +diff --git a/call/video_receive_stream.h b/call/video_receive_stream.h +index 15b313a3b9..a7e82665c3 100644 +--- a/call/video_receive_stream.h ++++ b/call/video_receive_stream.h +@@ -144,6 +144,9 @@ class VideoReceiveStreamInterface : public MediaReceiveStreamInterface { + RtpReceiveStats rtp_stats; + RtcpPacketTypeCounter rtcp_packet_type_counts; + ++ uint32_t rtcp_sender_packets_sent; ++ uint32_t rtcp_sender_octets_sent; ++ + // Timing frame info: all important timestamps for a full lifetime of a + // single 'timing frame'. + absl::optional<webrtc::TimingFrameInfo> timing_frame_info; +diff --git a/modules/rtp_rtcp/source/rtcp_receiver.cc b/modules/rtp_rtcp/source/rtcp_receiver.cc +index 0a24481762..7dfe4f0b5d 100644 +--- a/modules/rtp_rtcp/source/rtcp_receiver.cc ++++ b/modules/rtp_rtcp/source/rtcp_receiver.cc +@@ -428,6 +428,13 @@ RTCPReceiver::ConsumeReceivedXrReferenceTimeInfo() { + return last_xr_rtis; + } + ++void RTCPReceiver::RemoteRTCPSenderInfo(uint32_t* packet_count, ++ uint32_t* octet_count) const { ++ MutexLock lock(&rtcp_receiver_lock_); ++ *packet_count = remote_sender_packet_count_; ++ *octet_count = remote_sender_octet_count_; ++} ++ + std::vector<ReportBlockData> RTCPReceiver::GetLatestReportBlockData() const { + std::vector<ReportBlockData> result; + MutexLock lock(&rtcp_receiver_lock_); +diff --git a/modules/rtp_rtcp/source/rtcp_receiver.h b/modules/rtp_rtcp/source/rtcp_receiver.h +index cdf4cbadf8..f68e57479b 100644 +--- a/modules/rtp_rtcp/source/rtcp_receiver.h ++++ b/modules/rtp_rtcp/source/rtcp_receiver.h +@@ -132,6 +132,10 @@ class RTCPReceiver final { + + std::vector<rtcp::ReceiveTimeInfo> ConsumeReceivedXrReferenceTimeInfo(); + ++ // Get received sender packet and octet counts ++ void RemoteRTCPSenderInfo(uint32_t* packet_count, ++ uint32_t* octet_count) const; ++ + // Get rtt. + int32_t RTT(uint32_t remote_ssrc, + int64_t* last_rtt_ms, +diff --git a/modules/rtp_rtcp/source/rtp_rtcp_impl.cc b/modules/rtp_rtcp/source/rtp_rtcp_impl.cc +index b7f23236fe..54fb82c2a1 100644 +--- a/modules/rtp_rtcp/source/rtp_rtcp_impl.cc ++++ b/modules/rtp_rtcp/source/rtp_rtcp_impl.cc +@@ -527,6 +527,11 @@ void ModuleRtpRtcpImpl::GetSendStreamDataCounters( + } + + // Received RTCP report. ++void ModuleRtpRtcpImpl::RemoteRTCPSenderInfo(uint32_t* packet_count, ++ uint32_t* octet_count) const { ++ return rtcp_receiver_.RemoteRTCPSenderInfo(packet_count, octet_count); ++} ++ + std::vector<ReportBlockData> ModuleRtpRtcpImpl::GetLatestReportBlockData() + const { + return rtcp_receiver_.GetLatestReportBlockData(); +diff --git a/modules/rtp_rtcp/source/rtp_rtcp_impl.h b/modules/rtp_rtcp/source/rtp_rtcp_impl.h +index f164195168..dd916fbe40 100644 +--- a/modules/rtp_rtcp/source/rtp_rtcp_impl.h ++++ b/modules/rtp_rtcp/source/rtp_rtcp_impl.h +@@ -192,6 +192,9 @@ class ABSL_DEPRECATED("") ModuleRtpRtcpImpl + StreamDataCounters* rtp_counters, + StreamDataCounters* rtx_counters) const override; + ++ void RemoteRTCPSenderInfo(uint32_t* packet_count, ++ uint32_t* octet_count) const override; ++ + // A snapshot of the most recent Report Block with additional data of + // interest to statistics. Used to implement RTCRemoteInboundRtpStreamStats. + // Within this list, the ReportBlockData::RTCPReportBlock::source_ssrc(), +diff --git a/modules/rtp_rtcp/source/rtp_rtcp_impl2.cc b/modules/rtp_rtcp/source/rtp_rtcp_impl2.cc +index 31dd1499d5..d0f9c8ed1a 100644 +--- a/modules/rtp_rtcp/source/rtp_rtcp_impl2.cc ++++ b/modules/rtp_rtcp/source/rtp_rtcp_impl2.cc +@@ -508,6 +508,11 @@ void ModuleRtpRtcpImpl2::GetSendStreamDataCounters( + } + + // Received RTCP report. ++void ModuleRtpRtcpImpl2::RemoteRTCPSenderInfo(uint32_t* packet_count, ++ uint32_t* octet_count) const { ++ return rtcp_receiver_.RemoteRTCPSenderInfo(packet_count, octet_count); ++} ++ + std::vector<ReportBlockData> ModuleRtpRtcpImpl2::GetLatestReportBlockData() + const { + return rtcp_receiver_.GetLatestReportBlockData(); +diff --git a/modules/rtp_rtcp/source/rtp_rtcp_impl2.h b/modules/rtp_rtcp/source/rtp_rtcp_impl2.h +index e7a3ac03e8..3ef76ab66a 100644 +--- a/modules/rtp_rtcp/source/rtp_rtcp_impl2.h ++++ b/modules/rtp_rtcp/source/rtp_rtcp_impl2.h +@@ -204,6 +204,9 @@ class ModuleRtpRtcpImpl2 final : public RtpRtcpInterface, + StreamDataCounters* rtp_counters, + StreamDataCounters* rtx_counters) const override; + ++ void RemoteRTCPSenderInfo(uint32_t* packet_count, ++ uint32_t* octet_count) const override; ++ + // A snapshot of the most recent Report Block with additional data of + // interest to statistics. Used to implement RTCRemoteInboundRtpStreamStats. + // Within this list, the ReportBlockData::RTCPReportBlock::source_ssrc(), +diff --git a/modules/rtp_rtcp/source/rtp_rtcp_interface.h b/modules/rtp_rtcp/source/rtp_rtcp_interface.h +index 2024b308dd..f23d4d0758 100644 +--- a/modules/rtp_rtcp/source/rtp_rtcp_interface.h ++++ b/modules/rtp_rtcp/source/rtp_rtcp_interface.h +@@ -399,6 +399,10 @@ class RtpRtcpInterface : public RtcpFeedbackSenderInterface { + StreamDataCounters* rtp_counters, + StreamDataCounters* rtx_counters) const = 0; + ++ ++ // Returns packet count and octet count from RTCP sender report. ++ virtual void RemoteRTCPSenderInfo(uint32_t* packet_count, ++ uint32_t* octet_count) const = 0; + // A snapshot of Report Blocks with additional data of interest to statistics. + // Within this list, the sender-source SSRC pair is unique and per-pair the + // ReportBlockData represents the latest Report Block that was received for +-- +2.34.1 + diff --git a/third_party/libwebrtc/moz-patch-stack/0008.patch b/third_party/libwebrtc/moz-patch-stack/0008.patch new file mode 100644 index 0000000000..ba1a816d30 --- /dev/null +++ b/third_party/libwebrtc/moz-patch-stack/0008.patch @@ -0,0 +1,148 @@ +From: Nico Grunbaum <na-g@nostrum.com> +Date: Fri, 5 Jun 2020 11:41:00 +0000 +Subject: Bug 1615191 - P0 - implement remoteTimestamp for + RTCRemoteOutboundRtpStreamStats in libwebrtc;r=dminor + +Differential Revision: https://phabricator.services.mozilla.com/D78004 +Mercurial Revision: https://hg.mozilla.org/mozilla-central/rev/cd901e96d23e004e4bee19b96c8c2f1ca3b42830 + +This patch also absorbs two additional patches that collapse into using ms for the remote timestamp: +Bug 1654112 - Plumb RemoteRTCPSenderInfo through also for audio and change unit to ms. r=ng +Bug 1654112 - Replace custom upstream code for remote received audio stats with cherry-pick. r=ng +--- + call/video_receive_stream.h | 1 + + modules/rtp_rtcp/source/rtcp_receiver.cc | 4 +++- + modules/rtp_rtcp/source/rtcp_receiver.h | 3 ++- + modules/rtp_rtcp/source/rtp_rtcp_impl.cc | 6 ++++-- + modules/rtp_rtcp/source/rtp_rtcp_impl.h | 3 ++- + modules/rtp_rtcp/source/rtp_rtcp_impl2.cc | 6 ++++-- + modules/rtp_rtcp/source/rtp_rtcp_impl2.h | 3 ++- + modules/rtp_rtcp/source/rtp_rtcp_interface.h | 5 +++-- + 8 files changed, 21 insertions(+), 10 deletions(-) + +diff --git a/call/video_receive_stream.h b/call/video_receive_stream.h +index a7e82665c3..3125993d4b 100644 +--- a/call/video_receive_stream.h ++++ b/call/video_receive_stream.h +@@ -146,6 +146,7 @@ class VideoReceiveStreamInterface : public MediaReceiveStreamInterface { + + uint32_t rtcp_sender_packets_sent; + uint32_t rtcp_sender_octets_sent; ++ int64_t rtcp_sender_ntp_timestamp_ms; + + // Timing frame info: all important timestamps for a full lifetime of a + // single 'timing frame'. +diff --git a/modules/rtp_rtcp/source/rtcp_receiver.cc b/modules/rtp_rtcp/source/rtcp_receiver.cc +index 7dfe4f0b5d..68171d1c2a 100644 +--- a/modules/rtp_rtcp/source/rtcp_receiver.cc ++++ b/modules/rtp_rtcp/source/rtcp_receiver.cc +@@ -429,10 +429,12 @@ RTCPReceiver::ConsumeReceivedXrReferenceTimeInfo() { + } + + void RTCPReceiver::RemoteRTCPSenderInfo(uint32_t* packet_count, +- uint32_t* octet_count) const { ++ uint32_t* octet_count, ++ int64_t* ntp_timestamp_ms) const { + MutexLock lock(&rtcp_receiver_lock_); + *packet_count = remote_sender_packet_count_; + *octet_count = remote_sender_octet_count_; ++ *ntp_timestamp_ms = remote_sender_ntp_time_.ToMs(); + } + + std::vector<ReportBlockData> RTCPReceiver::GetLatestReportBlockData() const { +diff --git a/modules/rtp_rtcp/source/rtcp_receiver.h b/modules/rtp_rtcp/source/rtcp_receiver.h +index f68e57479b..6912912cfc 100644 +--- a/modules/rtp_rtcp/source/rtcp_receiver.h ++++ b/modules/rtp_rtcp/source/rtcp_receiver.h +@@ -134,7 +134,8 @@ class RTCPReceiver final { + + // Get received sender packet and octet counts + void RemoteRTCPSenderInfo(uint32_t* packet_count, +- uint32_t* octet_count) const; ++ uint32_t* octet_count, ++ int64_t* ntp_timestamp_ms) const; + + // Get rtt. + int32_t RTT(uint32_t remote_ssrc, +diff --git a/modules/rtp_rtcp/source/rtp_rtcp_impl.cc b/modules/rtp_rtcp/source/rtp_rtcp_impl.cc +index 54fb82c2a1..bf9e2b3bf9 100644 +--- a/modules/rtp_rtcp/source/rtp_rtcp_impl.cc ++++ b/modules/rtp_rtcp/source/rtp_rtcp_impl.cc +@@ -528,8 +528,10 @@ void ModuleRtpRtcpImpl::GetSendStreamDataCounters( + + // Received RTCP report. + void ModuleRtpRtcpImpl::RemoteRTCPSenderInfo(uint32_t* packet_count, +- uint32_t* octet_count) const { +- return rtcp_receiver_.RemoteRTCPSenderInfo(packet_count, octet_count); ++ uint32_t* octet_count, ++ int64_t* ntp_timestamp_ms) const { ++ return rtcp_receiver_.RemoteRTCPSenderInfo(packet_count, octet_count, ++ ntp_timestamp_ms); + } + + std::vector<ReportBlockData> ModuleRtpRtcpImpl::GetLatestReportBlockData() +diff --git a/modules/rtp_rtcp/source/rtp_rtcp_impl.h b/modules/rtp_rtcp/source/rtp_rtcp_impl.h +index dd916fbe40..5cf558717e 100644 +--- a/modules/rtp_rtcp/source/rtp_rtcp_impl.h ++++ b/modules/rtp_rtcp/source/rtp_rtcp_impl.h +@@ -193,7 +193,8 @@ class ABSL_DEPRECATED("") ModuleRtpRtcpImpl + StreamDataCounters* rtx_counters) const override; + + void RemoteRTCPSenderInfo(uint32_t* packet_count, +- uint32_t* octet_count) const override; ++ uint32_t* octet_count, ++ int64_t* ntp_timestamp_ms) const override; + + // A snapshot of the most recent Report Block with additional data of + // interest to statistics. Used to implement RTCRemoteInboundRtpStreamStats. +diff --git a/modules/rtp_rtcp/source/rtp_rtcp_impl2.cc b/modules/rtp_rtcp/source/rtp_rtcp_impl2.cc +index d0f9c8ed1a..8378a76133 100644 +--- a/modules/rtp_rtcp/source/rtp_rtcp_impl2.cc ++++ b/modules/rtp_rtcp/source/rtp_rtcp_impl2.cc +@@ -509,8 +509,10 @@ void ModuleRtpRtcpImpl2::GetSendStreamDataCounters( + + // Received RTCP report. + void ModuleRtpRtcpImpl2::RemoteRTCPSenderInfo(uint32_t* packet_count, +- uint32_t* octet_count) const { +- return rtcp_receiver_.RemoteRTCPSenderInfo(packet_count, octet_count); ++ uint32_t* octet_count, ++ int64_t* ntp_timestamp_ms) const { ++ return rtcp_receiver_.RemoteRTCPSenderInfo(packet_count, octet_count, ++ ntp_timestamp_ms); + } + + std::vector<ReportBlockData> ModuleRtpRtcpImpl2::GetLatestReportBlockData() +diff --git a/modules/rtp_rtcp/source/rtp_rtcp_impl2.h b/modules/rtp_rtcp/source/rtp_rtcp_impl2.h +index 3ef76ab66a..4ef67d4647 100644 +--- a/modules/rtp_rtcp/source/rtp_rtcp_impl2.h ++++ b/modules/rtp_rtcp/source/rtp_rtcp_impl2.h +@@ -205,7 +205,8 @@ class ModuleRtpRtcpImpl2 final : public RtpRtcpInterface, + StreamDataCounters* rtx_counters) const override; + + void RemoteRTCPSenderInfo(uint32_t* packet_count, +- uint32_t* octet_count) const override; ++ uint32_t* octet_count, ++ int64_t* ntp_timestamp_ms) const override; + + // A snapshot of the most recent Report Block with additional data of + // interest to statistics. Used to implement RTCRemoteInboundRtpStreamStats. +diff --git a/modules/rtp_rtcp/source/rtp_rtcp_interface.h b/modules/rtp_rtcp/source/rtp_rtcp_interface.h +index f23d4d0758..c6854937cb 100644 +--- a/modules/rtp_rtcp/source/rtp_rtcp_interface.h ++++ b/modules/rtp_rtcp/source/rtp_rtcp_interface.h +@@ -400,9 +400,10 @@ class RtpRtcpInterface : public RtcpFeedbackSenderInterface { + StreamDataCounters* rtx_counters) const = 0; + + +- // Returns packet count and octet count from RTCP sender report. ++ // Returns packet count, octet count, and timestamp from RTCP sender report. + virtual void RemoteRTCPSenderInfo(uint32_t* packet_count, +- uint32_t* octet_count) const = 0; ++ uint32_t* octet_count, ++ int64_t* ntp_timestamp_ms) const = 0; + // A snapshot of Report Blocks with additional data of interest to statistics. + // Within this list, the sender-source SSRC pair is unique and per-pair the + // ReportBlockData represents the latest Report Block that was received for +-- +2.34.1 + diff --git a/third_party/libwebrtc/moz-patch-stack/0009.patch b/third_party/libwebrtc/moz-patch-stack/0009.patch new file mode 100644 index 0000000000..9d2bd64f02 --- /dev/null +++ b/third_party/libwebrtc/moz-patch-stack/0009.patch @@ -0,0 +1,842 @@ +From: Dan Minor <dminor@mozilla.com> +Date: Tue, 31 Jul 2018 13:32:00 -0400 +Subject: Bug 1376873 - OS X desktop capture fixes; r=pehrsons + +Differential Revision: https://phabricator.services.mozilla.com/D7464 +Mercurial Revision: https://hg.mozilla.org/mozilla-central/rev/02c038eca65c1218b56fdf8937fdeab3d8767fe6 +--- + .../desktop_capture/mac/screen_capturer_mac.h | 7 + + .../mac/screen_capturer_mac.mm | 4 +- + .../mouse_cursor_monitor_mac.mm | 2 +- + .../desktop_capture/screen_capturer_mac.mm | 766 ++++++++++++++++++ + 4 files changed, 777 insertions(+), 2 deletions(-) + create mode 100644 modules/desktop_capture/screen_capturer_mac.mm + +diff --git a/modules/desktop_capture/mac/screen_capturer_mac.h b/modules/desktop_capture/mac/screen_capturer_mac.h +index d9a5966efa..7be05cc639 100644 +--- a/modules/desktop_capture/mac/screen_capturer_mac.h ++++ b/modules/desktop_capture/mac/screen_capturer_mac.h +@@ -114,6 +114,13 @@ class ScreenCapturerMac final : public DesktopCapturer { + + // Start, CaptureFrame and destructor have to called in the same thread. + SequenceChecker thread_checker_; ++ ++ // Used to force CaptureFrame to update it's screen configuration ++ // and reregister event handlers. This ensure that this ++ // occurs on the ScreenCapture thread. Read and written from ++ // both the VideoCapture thread and ScreenCapture thread. ++ // Protected by desktop_config_monitor_. ++ bool update_screen_configuration_ = false; + }; + + } // namespace webrtc +diff --git a/modules/desktop_capture/mac/screen_capturer_mac.mm b/modules/desktop_capture/mac/screen_capturer_mac.mm +index 634849122e..115f6440b1 100644 +--- a/modules/desktop_capture/mac/screen_capturer_mac.mm ++++ b/modules/desktop_capture/mac/screen_capturer_mac.mm +@@ -182,6 +182,7 @@ void ScreenCapturerMac::Start(Callback* callback) { + "webrtc", "ScreenCapturermac::Start", "target display id ", current_display_); + + callback_ = callback; ++ update_screen_configuration_ = false; + // Start and operate CGDisplayStream handler all from capture thread. + if (!RegisterRefreshAndMoveHandlers()) { + RTC_LOG(LS_ERROR) << "Failed to register refresh and move handlers."; +@@ -202,7 +203,8 @@ void ScreenCapturerMac::CaptureFrame() { + } + + MacDesktopConfiguration new_config = desktop_config_monitor_->desktop_configuration(); +- if (!desktop_config_.Equals(new_config)) { ++ if (update_screen_configuration_ || !desktop_config_.Equals(new_config)) { ++ update_screen_configuration_ = false; + desktop_config_ = new_config; + // If the display configuraiton has changed then refresh capturer data + // structures. Occasionally, the refresh and move handlers are lost when +diff --git a/modules/desktop_capture/mouse_cursor_monitor_mac.mm b/modules/desktop_capture/mouse_cursor_monitor_mac.mm +index 3db4332cd1..512103ab5e 100644 +--- a/modules/desktop_capture/mouse_cursor_monitor_mac.mm ++++ b/modules/desktop_capture/mouse_cursor_monitor_mac.mm +@@ -133,7 +133,7 @@ void MouseCursorMonitorMac::CaptureImage(float scale) { + NSSize nssize = [nsimage size]; // DIP size + + // No need to caputre cursor image if it's unchanged since last capture. +- if ([[nsimage TIFFRepresentation] isEqual:[last_cursor_ TIFFRepresentation]]) return; ++ if (last_cursor_ && [[nsimage TIFFRepresentation] isEqual:[last_cursor_ TIFFRepresentation]]) return; + last_cursor_ = nsimage; + + DesktopSize size(round(nssize.width * scale), +diff --git a/modules/desktop_capture/screen_capturer_mac.mm b/modules/desktop_capture/screen_capturer_mac.mm +new file mode 100644 +index 0000000000..285086ffa6 +--- /dev/null ++++ b/modules/desktop_capture/screen_capturer_mac.mm +@@ -0,0 +1,766 @@ ++/* ++ * Copyright (c) 2013 The WebRTC project authors. All Rights Reserved. ++ * ++ * Use of this source code is governed by a BSD-style license ++ * that can be found in the LICENSE file in the root of the source ++ * tree. An additional intellectual property rights grant can be found ++ * in the file PATENTS. All contributing project authors may ++ * be found in the AUTHORS file in the root of the source tree. ++ */ ++ ++#include <stddef.h> ++ ++#include <memory> ++#include <set> ++#include <utility> ++ ++#include <ApplicationServices/ApplicationServices.h> ++#include <Cocoa/Cocoa.h> ++#include <CoreGraphics/CoreGraphics.h> ++ ++#include "modules/desktop_capture/desktop_capture_options.h" ++#include "modules/desktop_capture/desktop_capturer.h" ++#include "modules/desktop_capture/desktop_frame.h" ++#include "modules/desktop_capture/desktop_geometry.h" ++#include "modules/desktop_capture/desktop_region.h" ++#include "modules/desktop_capture/mac/desktop_configuration.h" ++#include "modules/desktop_capture/mac/desktop_configuration_monitor.h" ++#include "modules/desktop_capture/mac/scoped_pixel_buffer_object.h" ++#include "modules/desktop_capture/screen_capture_frame_queue.h" ++#include "modules/desktop_capture/screen_capturer_helper.h" ++#include "modules/desktop_capture/shared_desktop_frame.h" ++#include "rtc_base/checks.h" ++#include "rtc_base/constructormagic.h" ++#include "rtc_base/logging.h" ++#include "rtc_base/macutils.h" ++#include "rtc_base/timeutils.h" ++ ++namespace webrtc { ++ ++namespace { ++ ++// CGDisplayStreamRefs need to be destroyed asynchronously after receiving a ++// kCGDisplayStreamFrameStatusStopped callback from CoreGraphics. This may ++// happen after the ScreenCapturerMac has been destroyed. DisplayStreamManager ++// is responsible for destroying all extant CGDisplayStreamRefs, and will ++// destroy itself once it's done. ++class DisplayStreamManager { ++ public: ++ int GetUniqueId() { return ++unique_id_generator_; } ++ void DestroyStream(int unique_id) { ++ auto it = display_stream_wrappers_.find(unique_id); ++ RTC_CHECK(it != display_stream_wrappers_.end()); ++ RTC_CHECK(!it->second.active); ++ CFRelease(it->second.stream); ++ display_stream_wrappers_.erase(it); ++ ++ if (ready_for_self_destruction_ && display_stream_wrappers_.empty()) ++ delete this; ++ } ++ ++ void SaveStream(int unique_id, ++ CGDisplayStreamRef stream) { ++ RTC_CHECK(unique_id <= unique_id_generator_); ++ DisplayStreamWrapper wrapper; ++ wrapper.stream = stream; ++ display_stream_wrappers_[unique_id] = wrapper; ++ } ++ ++ void UnregisterActiveStreams() { ++ for (auto& pair : display_stream_wrappers_) { ++ DisplayStreamWrapper& wrapper = pair.second; ++ if (wrapper.active) { ++ wrapper.active = false; ++ CFRunLoopSourceRef source = ++ CGDisplayStreamGetRunLoopSource(wrapper.stream); ++ CFRunLoopRemoveSource(CFRunLoopGetCurrent(), source, ++ kCFRunLoopCommonModes); ++ CGDisplayStreamStop(wrapper.stream); ++ } ++ } ++ } ++ ++ void PrepareForSelfDestruction() { ++ ready_for_self_destruction_ = true; ++ ++ if (display_stream_wrappers_.empty()) ++ delete this; ++ } ++ ++ // Once the DisplayStreamManager is ready for destruction, the ++ // ScreenCapturerMac is no longer present. Any updates should be ignored. ++ bool ShouldIgnoreUpdates() { return ready_for_self_destruction_; } ++ ++ private: ++ struct DisplayStreamWrapper { ++ // The registered CGDisplayStreamRef. ++ CGDisplayStreamRef stream = nullptr; ++ ++ // Set to false when the stream has been stopped. An asynchronous callback ++ // from CoreGraphics will let us destroy the CGDisplayStreamRef. ++ bool active = true; ++ }; ++ ++ std::map<int, DisplayStreamWrapper> display_stream_wrappers_; ++ int unique_id_generator_ = 0; ++ bool ready_for_self_destruction_ = false; ++}; ++ ++// Standard Mac displays have 72dpi, but we report 96dpi for ++// consistency with Windows and Linux. ++const int kStandardDPI = 96; ++ ++// Scales all coordinates of a rect by a specified factor. ++DesktopRect ScaleAndRoundCGRect(const CGRect& rect, float scale) { ++ return DesktopRect::MakeLTRB( ++ static_cast<int>(floor(rect.origin.x * scale)), ++ static_cast<int>(floor(rect.origin.y * scale)), ++ static_cast<int>(ceil((rect.origin.x + rect.size.width) * scale)), ++ static_cast<int>(ceil((rect.origin.y + rect.size.height) * scale))); ++} ++ ++// Copy pixels in the |rect| from |src_place| to |dest_plane|. |rect| should be ++// relative to the origin of |src_plane| and |dest_plane|. ++void CopyRect(const uint8_t* src_plane, ++ int src_plane_stride, ++ uint8_t* dest_plane, ++ int dest_plane_stride, ++ int bytes_per_pixel, ++ const DesktopRect& rect) { ++ // Get the address of the starting point. ++ const int src_y_offset = src_plane_stride * rect.top(); ++ const int dest_y_offset = dest_plane_stride * rect.top(); ++ const int x_offset = bytes_per_pixel * rect.left(); ++ src_plane += src_y_offset + x_offset; ++ dest_plane += dest_y_offset + x_offset; ++ ++ // Copy pixels in the rectangle line by line. ++ const int bytes_per_line = bytes_per_pixel * rect.width(); ++ const int height = rect.height(); ++ for (int i = 0 ; i < height; ++i) { ++ memcpy(dest_plane, src_plane, bytes_per_line); ++ src_plane += src_plane_stride; ++ dest_plane += dest_plane_stride; ++ } ++} ++ ++// Returns an array of CGWindowID for all the on-screen windows except ++// |window_to_exclude|, or NULL if the window is not found or it fails. The ++// caller should release the returned CFArrayRef. ++CFArrayRef CreateWindowListWithExclusion(CGWindowID window_to_exclude) { ++ if (!window_to_exclude) ++ return nullptr; ++ ++ CFArrayRef all_windows = CGWindowListCopyWindowInfo( ++ kCGWindowListOptionOnScreenOnly, kCGNullWindowID); ++ if (!all_windows) ++ return nullptr; ++ ++ CFMutableArrayRef returned_array = ++ CFArrayCreateMutable(nullptr, CFArrayGetCount(all_windows), nullptr); ++ ++ bool found = false; ++ for (CFIndex i = 0; i < CFArrayGetCount(all_windows); ++i) { ++ CFDictionaryRef window = reinterpret_cast<CFDictionaryRef>( ++ CFArrayGetValueAtIndex(all_windows, i)); ++ ++ CFNumberRef id_ref = reinterpret_cast<CFNumberRef>( ++ CFDictionaryGetValue(window, kCGWindowNumber)); ++ ++ CGWindowID id; ++ CFNumberGetValue(id_ref, kCFNumberIntType, &id); ++ if (id == window_to_exclude) { ++ found = true; ++ continue; ++ } ++ CFArrayAppendValue(returned_array, reinterpret_cast<void *>(id)); ++ } ++ CFRelease(all_windows); ++ ++ if (!found) { ++ CFRelease(returned_array); ++ returned_array = nullptr; ++ } ++ return returned_array; ++} ++ ++// Returns the bounds of |window| in physical pixels, enlarged by a small amount ++// on four edges to take account of the border/shadow effects. ++DesktopRect GetExcludedWindowPixelBounds(CGWindowID window, ++ float dip_to_pixel_scale) { ++ // The amount of pixels to add to the actual window bounds to take into ++ // account of the border/shadow effects. ++ static const int kBorderEffectSize = 20; ++ CGRect rect; ++ CGWindowID ids[1]; ++ ids[0] = window; ++ ++ CFArrayRef window_id_array = ++ CFArrayCreate(nullptr, reinterpret_cast<const void**>(&ids), 1, nullptr); ++ CFArrayRef window_array = ++ CGWindowListCreateDescriptionFromArray(window_id_array); ++ ++ if (CFArrayGetCount(window_array) > 0) { ++ CFDictionaryRef window = reinterpret_cast<CFDictionaryRef>( ++ CFArrayGetValueAtIndex(window_array, 0)); ++ CFDictionaryRef bounds_ref = reinterpret_cast<CFDictionaryRef>( ++ CFDictionaryGetValue(window, kCGWindowBounds)); ++ CGRectMakeWithDictionaryRepresentation(bounds_ref, &rect); ++ } ++ ++ CFRelease(window_id_array); ++ CFRelease(window_array); ++ ++ rect.origin.x -= kBorderEffectSize; ++ rect.origin.y -= kBorderEffectSize; ++ rect.size.width += kBorderEffectSize * 2; ++ rect.size.height += kBorderEffectSize * 2; ++ // |rect| is in DIP, so convert to physical pixels. ++ return ScaleAndRoundCGRect(rect, dip_to_pixel_scale); ++} ++ ++// Create an image of the given region using the given |window_list|. ++// |pixel_bounds| should be in the primary display's coordinate in physical ++// pixels. The caller should release the returned CGImageRef and CFDataRef. ++CGImageRef CreateExcludedWindowRegionImage(const DesktopRect& pixel_bounds, ++ float dip_to_pixel_scale, ++ CFArrayRef window_list) { ++ CGRect window_bounds; ++ // The origin is in DIP while the size is in physical pixels. That's what ++ // CGWindowListCreateImageFromArray expects. ++ window_bounds.origin.x = pixel_bounds.left() / dip_to_pixel_scale; ++ window_bounds.origin.y = pixel_bounds.top() / dip_to_pixel_scale; ++ window_bounds.size.width = pixel_bounds.width(); ++ window_bounds.size.height = pixel_bounds.height(); ++ ++ return CGWindowListCreateImageFromArray( ++ window_bounds, window_list, kCGWindowImageDefault); ++} ++ ++// A class to perform video frame capturing for mac. ++class ScreenCapturerMac : public DesktopCapturer { ++ public: ++ explicit ScreenCapturerMac( ++ rtc::scoped_refptr<DesktopConfigurationMonitor> desktop_config_monitor, ++ bool detect_updated_region); ++ ~ScreenCapturerMac() override; ++ ++ bool Init(); ++ ++ // DesktopCapturer interface. ++ void Start(Callback* callback) override; ++ void CaptureFrame() override; ++ void SetExcludedWindow(WindowId window) override; ++ bool GetSourceList(SourceList* screens) override; ++ bool SelectSource(SourceId id) override; ++ ++ private: ++ // Returns false if the selected screen is no longer valid. ++ bool CgBlit(const DesktopFrame& frame, const DesktopRegion& region); ++ ++ // Called when the screen configuration is changed. ++ void ScreenConfigurationChanged(); ++ ++ bool RegisterRefreshAndMoveHandlers(); ++ void UnregisterRefreshAndMoveHandlers(); ++ ++ void ScreenRefresh(CGRectCount count, ++ const CGRect *rect_array, ++ DesktopVector display_origin); ++ void ReleaseBuffers(); ++ ++ std::unique_ptr<DesktopFrame> CreateFrame(); ++ ++ const bool detect_updated_region_; ++ ++ Callback* callback_ = nullptr; ++ ++ ScopedPixelBufferObject pixel_buffer_object_; ++ ++ // Queue of the frames buffers. ++ ScreenCaptureFrameQueue<SharedDesktopFrame> queue_; ++ ++ // Current display configuration. ++ MacDesktopConfiguration desktop_config_; ++ ++ // Currently selected display, or 0 if the full desktop is selected. On OS X ++ // 10.6 and before, this is always 0. ++ CGDirectDisplayID current_display_ = 0; ++ ++ // The physical pixel bounds of the current screen. ++ DesktopRect screen_pixel_bounds_; ++ ++ // The dip to physical pixel scale of the current screen. ++ float dip_to_pixel_scale_ = 1.0f; ++ ++ // A thread-safe list of invalid rectangles, and the size of the most ++ // recently captured screen. ++ ScreenCapturerHelper helper_; ++ ++ // Contains an invalid region from the previous capture. ++ DesktopRegion last_invalid_region_; ++ ++ // Monitoring display reconfiguration. ++ rtc::scoped_refptr<DesktopConfigurationMonitor> desktop_config_monitor_; ++ ++ CGWindowID excluded_window_ = 0; ++ ++ // A self-owned object that will destroy itself after ScreenCapturerMac and ++ // all display streams have been destroyed.. ++ DisplayStreamManager* display_stream_manager_; ++ ++ // Used to force CaptureFrame to update it's screen configuration ++ // and reregister event handlers. This ensure that this ++ // occurs on the ScreenCapture thread. Read and written from ++ // both the VideoCapture thread and ScreenCapture thread. ++ // Protected by desktop_config_monitor_. ++ bool update_screen_configuration_ = false; ++ ++ RTC_DISALLOW_COPY_AND_ASSIGN(ScreenCapturerMac); ++}; ++ ++// DesktopFrame wrapper that flips wrapped frame upside down by inverting ++// stride. ++class InvertedDesktopFrame : public DesktopFrame { ++ public: ++ InvertedDesktopFrame(std::unique_ptr<DesktopFrame> frame) ++ : DesktopFrame( ++ frame->size(), ++ -frame->stride(), ++ frame->data() + (frame->size().height() - 1) * frame->stride(), ++ frame->shared_memory()) { ++ original_frame_ = std::move(frame); ++ MoveFrameInfoFrom(original_frame_.get()); ++ } ++ ~InvertedDesktopFrame() override {} ++ ++ private: ++ std::unique_ptr<DesktopFrame> original_frame_; ++ ++ RTC_DISALLOW_COPY_AND_ASSIGN(InvertedDesktopFrame); ++}; ++ ++ScreenCapturerMac::ScreenCapturerMac( ++ rtc::scoped_refptr<DesktopConfigurationMonitor> desktop_config_monitor, ++ bool detect_updated_region) ++ : detect_updated_region_(detect_updated_region), ++ desktop_config_monitor_(desktop_config_monitor) { ++ display_stream_manager_ = new DisplayStreamManager; ++} ++ ++ScreenCapturerMac::~ScreenCapturerMac() { ++ ReleaseBuffers(); ++ UnregisterRefreshAndMoveHandlers(); ++ display_stream_manager_->PrepareForSelfDestruction(); ++} ++ ++bool ScreenCapturerMac::Init() { ++ desktop_config_monitor_->Lock(); ++ desktop_config_ = desktop_config_monitor_->desktop_configuration(); ++ desktop_config_monitor_->Unlock(); ++ if (!RegisterRefreshAndMoveHandlers()) { ++ return false; ++ } ++ ScreenConfigurationChanged(); ++ return true; ++} ++ ++void ScreenCapturerMac::ReleaseBuffers() { ++ // The buffers might be in use by the encoder, so don't delete them here. ++ // Instead, mark them as "needs update"; next time the buffers are used by ++ // the capturer, they will be recreated if necessary. ++ queue_.Reset(); ++} ++ ++void ScreenCapturerMac::Start(Callback* callback) { ++ assert(!callback_); ++ assert(callback); ++ ++ callback_ = callback; ++ desktop_config_monitor_->Lock(); ++ update_screen_configuration_ = true; ++ desktop_config_monitor_->Unlock(); ++} ++ ++void ScreenCapturerMac::CaptureFrame() { ++ int64_t capture_start_time_nanos = rtc::TimeNanos(); ++ ++ // Spin RunLoop for 1/100th of a second, handling at most one source ++ CFRunLoopRunInMode(kCFRunLoopDefaultMode, 0.01, true); ++ ++ queue_.MoveToNextFrame(); ++ RTC_DCHECK(!queue_.current_frame() || !queue_.current_frame()->IsShared()); ++ ++ desktop_config_monitor_->Lock(); ++ MacDesktopConfiguration new_config = ++ desktop_config_monitor_->desktop_configuration(); ++ if (update_screen_configuration_ || !desktop_config_.Equals(new_config)) { ++ update_screen_configuration_ = false; ++ desktop_config_ = new_config; ++ // If the display configuraiton has changed then refresh capturer data ++ // structures. Occasionally, the refresh and move handlers are lost when ++ // the screen mode changes, so re-register them here. ++ UnregisterRefreshAndMoveHandlers(); ++ RegisterRefreshAndMoveHandlers(); ++ ScreenConfigurationChanged(); ++ } ++ ++ DesktopRegion region; ++ helper_.TakeInvalidRegion(®ion); ++ ++ // If the current buffer is from an older generation then allocate a new one. ++ // Note that we can't reallocate other buffers at this point, since the caller ++ // may still be reading from them. ++ if (!queue_.current_frame()) ++ queue_.ReplaceCurrentFrame(SharedDesktopFrame::Wrap(CreateFrame())); ++ ++ DesktopFrame* current_frame = queue_.current_frame(); ++ ++ if (!CgBlit(*current_frame, region)) { ++ desktop_config_monitor_->Unlock(); ++ callback_->OnCaptureResult(Result::ERROR_PERMANENT, nullptr); ++ return; ++ } ++ std::unique_ptr<DesktopFrame> new_frame = queue_.current_frame()->Share(); ++ if (detect_updated_region_) { ++ *new_frame->mutable_updated_region() = region; ++ } else { ++ new_frame->mutable_updated_region()->AddRect( ++ DesktopRect::MakeSize(new_frame->size())); ++ } ++ ++ if (current_display_) { ++ const MacDisplayConfiguration* config = ++ desktop_config_.FindDisplayConfigurationById(current_display_); ++ if (config) { ++ new_frame->set_top_left(config->bounds.top_left().subtract( ++ desktop_config_.bounds.top_left())); ++ } ++ } ++ ++ helper_.set_size_most_recent(new_frame->size()); ++ ++ // Signal that we are done capturing data from the display framebuffer, ++ // and accessing display structures. ++ desktop_config_monitor_->Unlock(); ++ ++ new_frame->set_capture_time_ms((rtc::TimeNanos() - capture_start_time_nanos) / ++ rtc::kNumNanosecsPerMillisec); ++ callback_->OnCaptureResult(Result::SUCCESS, std::move(new_frame)); ++} ++ ++void ScreenCapturerMac::SetExcludedWindow(WindowId window) { ++ excluded_window_ = window; ++} ++ ++bool ScreenCapturerMac::GetSourceList(SourceList* screens) { ++ assert(screens->size() == 0); ++ ++ for (MacDisplayConfigurations::iterator it = desktop_config_.displays.begin(); ++ it != desktop_config_.displays.end(); ++it) { ++ screens->push_back({it->id}); ++ } ++ return true; ++} ++ ++bool ScreenCapturerMac::SelectSource(SourceId id) { ++ if (id == kFullDesktopScreenId) { ++ current_display_ = 0; ++ } else { ++ const MacDisplayConfiguration* config = ++ desktop_config_.FindDisplayConfigurationById( ++ static_cast<CGDirectDisplayID>(id)); ++ if (!config) ++ return false; ++ current_display_ = config->id; ++ } ++ ++ ScreenConfigurationChanged(); ++ return true; ++} ++ ++bool ScreenCapturerMac::CgBlit(const DesktopFrame& frame, const DesktopRegion& region) { ++ // Copy the entire contents of the previous capture buffer, to capture over. ++ // TODO(wez): Get rid of this as per crbug.com/145064, or implement ++ // crbug.com/92354. ++ if (queue_.previous_frame()) { ++ memcpy(frame.data(), queue_.previous_frame()->data(), ++ frame.stride() * frame.size().height()); ++ } ++ ++ MacDisplayConfigurations displays_to_capture; ++ if (current_display_) { ++ // Capturing a single screen. Note that the screen id may change when ++ // screens are added or removed. ++ const MacDisplayConfiguration* config = ++ desktop_config_.FindDisplayConfigurationById(current_display_); ++ if (config) { ++ displays_to_capture.push_back(*config); ++ } else { ++ RTC_LOG(LS_ERROR) << "The selected screen cannot be found for capturing."; ++ return false; ++ } ++ } else { ++ // Capturing the whole desktop. ++ displays_to_capture = desktop_config_.displays; ++ } ++ ++ // Create the window list once for all displays. ++ CFArrayRef window_list = CreateWindowListWithExclusion(excluded_window_); ++ ++ for (size_t i = 0; i < displays_to_capture.size(); ++i) { ++ const MacDisplayConfiguration& display_config = displays_to_capture[i]; ++ ++ // Capturing mixed-DPI on one surface is hard, so we only return displays ++ // that match the "primary" display's DPI. The primary display is always ++ // the first in the list. ++ if (i > 0 && display_config.dip_to_pixel_scale != ++ displays_to_capture[0].dip_to_pixel_scale) { ++ continue; ++ } ++ // Determine the display's position relative to the desktop, in pixels. ++ DesktopRect display_bounds = display_config.pixel_bounds; ++ display_bounds.Translate(-screen_pixel_bounds_.left(), ++ -screen_pixel_bounds_.top()); ++ ++ // Determine which parts of the blit region, if any, lay within the monitor. ++ DesktopRegion copy_region = region; ++ copy_region.IntersectWith(display_bounds); ++ if (copy_region.is_empty()) ++ continue; ++ ++ // Translate the region to be copied into display-relative coordinates. ++ copy_region.Translate(-display_bounds.left(), -display_bounds.top()); ++ ++ DesktopRect excluded_window_bounds; ++ CGImageRef excluded_image = nullptr; ++ if (excluded_window_ && window_list) { ++ // Get the region of the excluded window relative the primary display. ++ excluded_window_bounds = GetExcludedWindowPixelBounds( ++ excluded_window_, display_config.dip_to_pixel_scale); ++ excluded_window_bounds.IntersectWith(display_config.pixel_bounds); ++ ++ // Create the image under the excluded window first, because it's faster ++ // than captuing the whole display. ++ if (!excluded_window_bounds.is_empty()) { ++ excluded_image = CreateExcludedWindowRegionImage( ++ excluded_window_bounds, display_config.dip_to_pixel_scale, ++ window_list); ++ } ++ } ++ ++ // Create an image containing a snapshot of the display. ++ CGImageRef image = CGDisplayCreateImage(display_config.id); ++ if (!image) { ++ if (excluded_image) ++ CFRelease(excluded_image); ++ continue; ++ } ++ ++ // Verify that the image has 32-bit depth. ++ int bits_per_pixel = CGImageGetBitsPerPixel(image); ++ if (bits_per_pixel / 8 != DesktopFrame::kBytesPerPixel) { ++ RTC_LOG(LS_ERROR) << "CGDisplayCreateImage() returned imaged with " << bits_per_pixel ++ << " bits per pixel. Only 32-bit depth is supported."; ++ CFRelease(image); ++ if (excluded_image) ++ CFRelease(excluded_image); ++ return false; ++ } ++ ++ // Request access to the raw pixel data via the image's DataProvider. ++ CGDataProviderRef provider = CGImageGetDataProvider(image); ++ CFDataRef data = CGDataProviderCopyData(provider); ++ assert(data); ++ ++ const uint8_t* display_base_address = CFDataGetBytePtr(data); ++ int src_bytes_per_row = CGImageGetBytesPerRow(image); ++ ++ // |image| size may be different from display_bounds in case the screen was ++ // resized recently. ++ copy_region.IntersectWith( ++ DesktopRect::MakeWH(CGImageGetWidth(image), CGImageGetHeight(image))); ++ ++ // Copy the dirty region from the display buffer into our desktop buffer. ++ uint8_t* out_ptr = frame.GetFrameDataAtPos(display_bounds.top_left()); ++ for (DesktopRegion::Iterator i(copy_region); !i.IsAtEnd(); i.Advance()) { ++ CopyRect(display_base_address, src_bytes_per_row, out_ptr, frame.stride(), ++ DesktopFrame::kBytesPerPixel, i.rect()); ++ } ++ ++ CFRelease(data); ++ CFRelease(image); ++ ++ if (excluded_image) { ++ CGDataProviderRef provider = CGImageGetDataProvider(excluded_image); ++ CFDataRef excluded_image_data = CGDataProviderCopyData(provider); ++ assert(excluded_image_data); ++ display_base_address = CFDataGetBytePtr(excluded_image_data); ++ src_bytes_per_row = CGImageGetBytesPerRow(excluded_image); ++ ++ // Translate the bounds relative to the desktop, because |frame| data ++ // starts from the desktop top-left corner. ++ DesktopRect window_bounds_relative_to_desktop(excluded_window_bounds); ++ window_bounds_relative_to_desktop.Translate(-screen_pixel_bounds_.left(), ++ -screen_pixel_bounds_.top()); ++ ++ DesktopRect rect_to_copy = ++ DesktopRect::MakeSize(excluded_window_bounds.size()); ++ rect_to_copy.IntersectWith(DesktopRect::MakeWH( ++ CGImageGetWidth(excluded_image), CGImageGetHeight(excluded_image))); ++ ++ if (CGImageGetBitsPerPixel(excluded_image) / 8 == ++ DesktopFrame::kBytesPerPixel) { ++ CopyRect(display_base_address, src_bytes_per_row, ++ frame.GetFrameDataAtPos( ++ window_bounds_relative_to_desktop.top_left()), ++ frame.stride(), DesktopFrame::kBytesPerPixel, rect_to_copy); ++ } ++ ++ CFRelease(excluded_image_data); ++ CFRelease(excluded_image); ++ } ++ } ++ if (window_list) ++ CFRelease(window_list); ++ return true; ++} ++ ++void ScreenCapturerMac::ScreenConfigurationChanged() { ++ if (current_display_) { ++ const MacDisplayConfiguration* config = ++ desktop_config_.FindDisplayConfigurationById(current_display_); ++ screen_pixel_bounds_ = config ? config->pixel_bounds : DesktopRect(); ++ dip_to_pixel_scale_ = config ? config->dip_to_pixel_scale : 1.0f; ++ } else { ++ screen_pixel_bounds_ = desktop_config_.pixel_bounds; ++ dip_to_pixel_scale_ = desktop_config_.dip_to_pixel_scale; ++ } ++ ++ // Release existing buffers, which will be of the wrong size. ++ ReleaseBuffers(); ++ ++ // Clear the dirty region, in case the display is down-sizing. ++ helper_.ClearInvalidRegion(); ++ ++ // Re-mark the entire desktop as dirty. ++ helper_.InvalidateScreen(screen_pixel_bounds_.size()); ++ ++ // Make sure the frame buffers will be reallocated. ++ queue_.Reset(); ++} ++ ++bool ScreenCapturerMac::RegisterRefreshAndMoveHandlers() { ++ desktop_config_ = desktop_config_monitor_->desktop_configuration(); ++ for (const auto& config : desktop_config_.displays) { ++ size_t pixel_width = config.pixel_bounds.width(); ++ size_t pixel_height = config.pixel_bounds.height(); ++ if (pixel_width == 0 || pixel_height == 0) ++ continue; ++ // Using a local variable forces the block to capture the raw pointer. ++ DisplayStreamManager* manager = display_stream_manager_; ++ int unique_id = manager->GetUniqueId(); ++ CGDirectDisplayID display_id = config.id; ++ DesktopVector display_origin = config.pixel_bounds.top_left(); ++ ++ CGDisplayStreamFrameAvailableHandler handler = ++ ^(CGDisplayStreamFrameStatus status, uint64_t display_time, ++ IOSurfaceRef frame_surface, CGDisplayStreamUpdateRef updateRef) { ++ if (status == kCGDisplayStreamFrameStatusStopped) { ++ manager->DestroyStream(unique_id); ++ return; ++ } ++ ++ if (manager->ShouldIgnoreUpdates()) ++ return; ++ ++ // Only pay attention to frame updates. ++ if (status != kCGDisplayStreamFrameStatusFrameComplete) ++ return; ++ ++ size_t count = 0; ++ const CGRect* rects = CGDisplayStreamUpdateGetRects( ++ updateRef, kCGDisplayStreamUpdateDirtyRects, &count); ++ if (count != 0) { ++ // According to CGDisplayStream.h, it's safe to call ++ // CGDisplayStreamStop() from within the callback. ++ ScreenRefresh(count, rects, display_origin); ++ } ++ }; ++ CGDisplayStreamRef display_stream = CGDisplayStreamCreate( ++ display_id, pixel_width, pixel_height, 'BGRA', nullptr, handler); ++ ++ if (display_stream) { ++ CGError error = CGDisplayStreamStart(display_stream); ++ if (error != kCGErrorSuccess) ++ return false; ++ ++ CFRunLoopSourceRef source = ++ CGDisplayStreamGetRunLoopSource(display_stream); ++ CFRunLoopAddSource(CFRunLoopGetCurrent(), source, kCFRunLoopCommonModes); ++ display_stream_manager_->SaveStream(unique_id, display_stream); ++ } ++ } ++ ++ return true; ++} ++ ++void ScreenCapturerMac::UnregisterRefreshAndMoveHandlers() { ++ display_stream_manager_->UnregisterActiveStreams(); ++} ++ ++void ScreenCapturerMac::ScreenRefresh(CGRectCount count, ++ const CGRect* rect_array, ++ DesktopVector display_origin) { ++ if (screen_pixel_bounds_.is_empty()) ++ ScreenConfigurationChanged(); ++ ++ // The refresh rects are in display coordinates. We want to translate to ++ // framebuffer coordinates. If a specific display is being captured, then no ++ // change is necessary. If all displays are being captured, then we want to ++ // translate by the origin of the display. ++ DesktopVector translate_vector; ++ if (!current_display_) ++ translate_vector = display_origin; ++ ++ DesktopRegion region; ++ for (CGRectCount i = 0; i < count; ++i) { ++ // All rects are already in physical pixel coordinates. ++ DesktopRect rect = DesktopRect::MakeXYWH( ++ rect_array[i].origin.x, rect_array[i].origin.y, ++ rect_array[i].size.width, rect_array[i].size.height); ++ ++ rect.Translate(translate_vector); ++ ++ region.AddRect(rect); ++ } ++ ++ helper_.InvalidateRegion(region); ++} ++ ++std::unique_ptr<DesktopFrame> ScreenCapturerMac::CreateFrame() { ++ std::unique_ptr<DesktopFrame> frame( ++ new BasicDesktopFrame(screen_pixel_bounds_.size())); ++ frame->set_dpi(DesktopVector(kStandardDPI * dip_to_pixel_scale_, ++ kStandardDPI * dip_to_pixel_scale_)); ++ return frame; ++} ++ ++} // namespace ++ ++// static ++std::unique_ptr<DesktopCapturer> DesktopCapturer::CreateRawScreenCapturer( ++ const DesktopCaptureOptions& options) { ++ if (!options.configuration_monitor()) ++ return nullptr; ++ ++ std::unique_ptr<ScreenCapturerMac> capturer(new ScreenCapturerMac( ++ options.configuration_monitor(), options.detect_updated_region())); ++ if (!capturer.get()->Init()) { ++ return nullptr; ++ } ++ ++ return capturer; ++} ++ ++} // namespace webrtc +-- +2.34.1 + diff --git a/third_party/libwebrtc/moz-patch-stack/0010.patch b/third_party/libwebrtc/moz-patch-stack/0010.patch new file mode 100644 index 0000000000..13103c1ab4 --- /dev/null +++ b/third_party/libwebrtc/moz-patch-stack/0010.patch @@ -0,0 +1,56 @@ +From: "Byron Campen [:bwc]" <docfaraday@gmail.com> +Date: Fri, 12 Feb 2021 14:27:00 +0000 +Subject: Bug 1654112 - Get OS X build working. + r=ng,firefox-build-system-reviewers,glandium + +* Pull in sdk/objc/base and sdk/objc/helpers +* Add gclient_args.gni to keep build happy. +* Add a missing include path for libyuv +* Support .m files in build. + +Differential Revision: https://phabricator.services.mozilla.com/D105015 +Mercurial Revision: https://hg.mozilla.org/mozilla-central/rev/9314046d89ebc0836a50ab7addece71abbf0b5e4 +--- + modules/desktop_capture/mac/screen_capturer_mac.mm | 3 ++- + modules/desktop_capture/mac/window_list_utils.cc | 4 ++-- + 2 files changed, 4 insertions(+), 3 deletions(-) + +diff --git a/modules/desktop_capture/mac/screen_capturer_mac.mm b/modules/desktop_capture/mac/screen_capturer_mac.mm +index 115f6440b1..cad0c5b65b 100644 +--- a/modules/desktop_capture/mac/screen_capturer_mac.mm ++++ b/modules/desktop_capture/mac/screen_capturer_mac.mm +@@ -276,7 +276,8 @@ bool ScreenCapturerMac::GetSourceList(SourceList* screens) { + for (MacDisplayConfigurations::iterator it = desktop_config_.displays.begin(); + it != desktop_config_.displays.end(); + ++it) { +- screens->push_back({it->id, std::string()}); ++ Source value = {it->id, 0, std::string()}; ++ screens->push_back(value); + } + return true; + } +diff --git a/modules/desktop_capture/mac/window_list_utils.cc b/modules/desktop_capture/mac/window_list_utils.cc +index 5d881662ea..989ec7ea54 100644 +--- a/modules/desktop_capture/mac/window_list_utils.cc ++++ b/modules/desktop_capture/mac/window_list_utils.cc +@@ -198,7 +198,7 @@ bool GetWindowList(DesktopCapturer::SourceList* windows, + // the check in the map. Also skip the window if owner name is + // empty too. + if (!owner_name.empty() && (itr == pid_itr_map.end())) { +- sources.push_back(DesktopCapturer::Source{window_id, owner_name}); ++ sources.push_back(DesktopCapturer::Source{window_id, pid, owner_name}); + RTC_DCHECK(!sources.empty()); + // Get an iterator on the last valid element in the source list. + std::list<DesktopCapturer::Source>::const_iterator last_source = +@@ -209,7 +209,7 @@ bool GetWindowList(DesktopCapturer::SourceList* windows, + pid, last_source)); + } + } else { +- sources.push_back(DesktopCapturer::Source{window_id, title}); ++ sources.push_back(DesktopCapturer::Source{window_id, pid, title}); + // Once the window with empty title has been removed no other empty + // windows are allowed for the same pid. + if (itr != pid_itr_map.end() && (itr->second != sources.end())) { +-- +2.34.1 + diff --git a/third_party/libwebrtc/moz-patch-stack/0011.patch b/third_party/libwebrtc/moz-patch-stack/0011.patch new file mode 100644 index 0000000000..f1c49df481 --- /dev/null +++ b/third_party/libwebrtc/moz-patch-stack/0011.patch @@ -0,0 +1,26 @@ +From: Dan Minor <dminor@mozilla.com> +Date: Mon, 13 Aug 2018 08:34:00 -0400 +Subject: Bug 1376873 - Allow single channel opus; r=padenot + +Differential Revision: https://phabricator.services.mozilla.com/D7469 +Mercurial Revision: https://hg.mozilla.org/mozilla-central/rev/92a7c3eee9f0c80ffbf79fdab8f8f2b8f6bd7701 +--- + modules/audio_coding/codecs/opus/audio_encoder_opus.cc | 2 +- + 1 file changed, 1 insertion(+), 1 deletion(-) + +diff --git a/modules/audio_coding/codecs/opus/audio_encoder_opus.cc b/modules/audio_coding/codecs/opus/audio_encoder_opus.cc +index 51b0fcd492..17e0e33b1d 100644 +--- a/modules/audio_coding/codecs/opus/audio_encoder_opus.cc ++++ b/modules/audio_coding/codecs/opus/audio_encoder_opus.cc +@@ -240,7 +240,7 @@ std::unique_ptr<AudioEncoder> AudioEncoderOpusImpl::MakeAudioEncoder( + absl::optional<AudioEncoderOpusConfig> AudioEncoderOpusImpl::SdpToConfig( + const SdpAudioFormat& format) { + if (!absl::EqualsIgnoreCase(format.name, "opus") || +- format.clockrate_hz != kRtpTimestampRateHz || format.num_channels != 2) { ++ format.clockrate_hz != kRtpTimestampRateHz) { + return absl::nullopt; + } + +-- +2.34.1 + diff --git a/third_party/libwebrtc/moz-patch-stack/0012.patch b/third_party/libwebrtc/moz-patch-stack/0012.patch new file mode 100644 index 0000000000..646a3d07d8 --- /dev/null +++ b/third_party/libwebrtc/moz-patch-stack/0012.patch @@ -0,0 +1,37 @@ +From: Dan Minor <dminor@mozilla.com> +Date: Mon, 13 Aug 2018 10:24:00 -0400 +Subject: Bug 1376873 - Fix warning in mean_variance_estimator.cc; r=padenot + +Differential Revision: https://phabricator.services.mozilla.com/D7470 +Mercurial Revision: https://hg.mozilla.org/mozilla-central/rev/cb6c555654c0bca72999f1e414d8d6d0b59186c9 +--- + .../echo_detector/mean_variance_estimator.cc | 6 +++--- + 1 file changed, 3 insertions(+), 3 deletions(-) + +diff --git a/modules/audio_processing/echo_detector/mean_variance_estimator.cc b/modules/audio_processing/echo_detector/mean_variance_estimator.cc +index a85740387b..a9ebb8cd92 100644 +--- a/modules/audio_processing/echo_detector/mean_variance_estimator.cc ++++ b/modules/audio_processing/echo_detector/mean_variance_estimator.cc +@@ -10,7 +10,7 @@ + + #include "modules/audio_processing/echo_detector/mean_variance_estimator.h" + +-#include <math.h> ++#include <cmath> + + #include "rtc_base/checks.h" + +@@ -26,8 +26,8 @@ void MeanVarianceEstimator::Update(float value) { + mean_ = (1.f - kAlpha) * mean_ + kAlpha * value; + variance_ = + (1.f - kAlpha) * variance_ + kAlpha * (value - mean_) * (value - mean_); +- RTC_DCHECK(isfinite(mean_)); +- RTC_DCHECK(isfinite(variance_)); ++ RTC_DCHECK(std::isfinite(mean_)); ++ RTC_DCHECK(std::isfinite(variance_)); + } + + float MeanVarianceEstimator::std_deviation() const { +-- +2.34.1 + diff --git a/third_party/libwebrtc/moz-patch-stack/0013.patch b/third_party/libwebrtc/moz-patch-stack/0013.patch new file mode 100644 index 0000000000..edf3a049e8 --- /dev/null +++ b/third_party/libwebrtc/moz-patch-stack/0013.patch @@ -0,0 +1,45 @@ +From: Dan Minor <dminor@mozilla.com> +Date: Wed, 19 Sep 2018 15:06:00 -0400 +Subject: Bug 1376873 - Fix up rotation in video_capture_impl.cc; r=pehrsons + +This fixes a bug in the upstream code introduced when they removed the +ConvertToI420 helper method from webrtc_libyuv.cc. The buffer size is +passed into libyuv::ConvertI420 incorrectly when rotation is applied, which +causes bad rendering and instabilities. + +Differential Revision: https://phabricator.services.mozilla.com/D7478 +Mercurial Revision: https://hg.mozilla.org/mozilla-central/rev/c56cb66f86518dfb046c642ebeb07cf51c23a3cb +--- + modules/video_capture/video_capture_impl.cc | 11 ++++++++++- + 1 file changed, 10 insertions(+), 1 deletion(-) + +diff --git a/modules/video_capture/video_capture_impl.cc b/modules/video_capture/video_capture_impl.cc +index d5ec4daae1..5dbc5e4e27 100644 +--- a/modules/video_capture/video_capture_impl.cc ++++ b/modules/video_capture/video_capture_impl.cc +@@ -208,12 +208,21 @@ int32_t VideoCaptureImpl::IncomingFrame(uint8_t* videoFrame, + } + } + ++ int dst_width = buffer->width(); ++ int dst_height = buffer->height(); ++ ++ // LibYuv expects pre-rotation_mode values for dst. ++ // Stride values should correspond to the destination values. ++ if (rotation_mode == libyuv::kRotate90 || rotation_mode == libyuv::kRotate270) { ++ std::swap(dst_width, dst_height); ++ } ++ + const int conversionResult = libyuv::ConvertToI420( + videoFrame, videoFrameLength, buffer.get()->MutableDataY(), + buffer.get()->StrideY(), buffer.get()->MutableDataU(), + buffer.get()->StrideU(), buffer.get()->MutableDataV(), + buffer.get()->StrideV(), 0, 0, // No Cropping +- width, height, target_width, target_height, rotation_mode, ++ width, height, dst_width, dst_height, rotation_mode, + ConvertVideoType(frameInfo.videoType)); + if (conversionResult != 0) { + RTC_LOG(LS_ERROR) << "Failed to convert capture frame from type " +-- +2.34.1 + diff --git a/third_party/libwebrtc/moz-patch-stack/0014.patch b/third_party/libwebrtc/moz-patch-stack/0014.patch new file mode 100644 index 0000000000..91e7aab915 --- /dev/null +++ b/third_party/libwebrtc/moz-patch-stack/0014.patch @@ -0,0 +1,153 @@ +From: Jan Beich <jbeich@FreeBSD.org> +Date: Fri, 12 Oct 2018 12:56:00 -0400 +Subject: Bug 1376873 - Unbreak WebRTC 64 build on BSDs. r=dminor f=gaston + +Also includes: +Bug 1554949 - Fix WebRTC build failure with newer linux kernel. r=dminor + +Mercurial Revision: https://hg.mozilla.org/mozilla-central/rev/a636ecdcb91afb1c5d436bbcbd87da4f10f7df78 +--- + modules/video_capture/linux/device_info_linux.cc | 6 ++++++ + modules/video_capture/linux/device_info_v4l2.cc | 8 +++++++- + modules/video_capture/linux/video_capture_linux.cc | 9 ++++++++- + rtc_base/byte_order.h | 2 ++ + rtc_base/physical_socket_server.cc | 7 +++++-- + system_wrappers/source/cpu_info.cc | 4 ++-- + 6 files changed, 30 insertions(+), 6 deletions(-) + +diff --git a/modules/video_capture/linux/device_info_linux.cc b/modules/video_capture/linux/device_info_linux.cc +index ccbbeae3ab..9c12b2183e 100644 +--- a/modules/video_capture/linux/device_info_linux.cc ++++ b/modules/video_capture/linux/device_info_linux.cc +@@ -16,7 +16,13 @@ + #include <sys/ioctl.h> + #include <unistd.h> + // v4l includes ++#if defined(__NetBSD__) || defined(__OpenBSD__) // WEBRTC_BSD ++#include <sys/videoio.h> ++#elif defined(__sun) ++#include <sys/videodev2.h> ++#else + #include <linux/videodev2.h> ++#endif + + #include <vector> + +diff --git a/modules/video_capture/linux/device_info_v4l2.cc b/modules/video_capture/linux/device_info_v4l2.cc +index 28395a5a05..d836747b4a 100644 +--- a/modules/video_capture/linux/device_info_v4l2.cc ++++ b/modules/video_capture/linux/device_info_v4l2.cc +@@ -18,7 +18,13 @@ + #include <sys/ioctl.h> + #include <unistd.h> + // v4l includes ++#if defined(__NetBSD__) || defined(__OpenBSD__) // WEBRTC_BSD ++#include <sys/videoio.h> ++#elif defined(__sun) ++#include <sys/videodev2.h> ++#else + #include <linux/videodev2.h> ++#endif + + #include <vector> + +@@ -191,8 +197,8 @@ DeviceInfoV4l2::DeviceInfoV4l2() : DeviceInfoImpl() + { + _inotifyEventThread->Start(); + } +-} + #endif ++} + + int32_t DeviceInfoV4l2::Init() { + return 0; +diff --git a/modules/video_capture/linux/video_capture_linux.cc b/modules/video_capture/linux/video_capture_linux.cc +index b6c4017927..4895a1ab71 100644 +--- a/modules/video_capture/linux/video_capture_linux.cc ++++ b/modules/video_capture/linux/video_capture_linux.cc +@@ -10,7 +10,6 @@ + + #include <errno.h> + #include <fcntl.h> +-#include <linux/videodev2.h> + #include <stdio.h> + #include <string.h> + #include <sys/ioctl.h> +@@ -18,6 +17,14 @@ + #include <sys/select.h> + #include <time.h> + #include <unistd.h> ++// v4l includes ++#if defined(__NetBSD__) || defined(__OpenBSD__) // WEBRTC_BSD ++#include <sys/videoio.h> ++#elif defined(__sun) ++#include <sys/videodev2.h> ++#else ++#include <linux/videodev2.h> ++#endif + + #include <new> + #include <string> +diff --git a/rtc_base/byte_order.h b/rtc_base/byte_order.h +index b8f8ae9f7a..382511daeb 100644 +--- a/rtc_base/byte_order.h ++++ b/rtc_base/byte_order.h +@@ -90,6 +90,8 @@ + #error WEBRTC_ARCH_BIG_ENDIAN or WEBRTC_ARCH_LITTLE_ENDIAN must be defined. + #endif // defined(WEBRTC_ARCH_LITTLE_ENDIAN) + ++#elif defined(WEBRTC_BSD) && !defined(__OpenBSD__) ++#include <sys/endian.h> + #elif defined(WEBRTC_POSIX) + #include <endian.h> + #else +diff --git a/rtc_base/physical_socket_server.cc b/rtc_base/physical_socket_server.cc +index b7d69140e0..4ed4fd0cbb 100644 +--- a/rtc_base/physical_socket_server.cc ++++ b/rtc_base/physical_socket_server.cc +@@ -73,7 +73,10 @@ typedef void* SockOptArg; + + #endif // WEBRTC_POSIX + +-#if defined(WEBRTC_POSIX) && !defined(WEBRTC_MAC) && !defined(__native_client__) ++#if defined(WEBRTC_POSIX) && !defined(WEBRTC_MAC) && !defined(WEBRTC_BSD) && !defined(__native_client__) ++#if defined(WEBRTC_LINUX) ++#include <linux/sockios.h> ++#endif + + int64_t GetSocketRecvTimestamp(int socket) { + struct timeval tv_ioctl; +@@ -641,7 +644,7 @@ int PhysicalSocket::TranslateOption(Option opt, int* slevel, int* sopt) { + *slevel = IPPROTO_IP; + *sopt = IP_DONTFRAGMENT; + break; +-#elif defined(WEBRTC_MAC) || defined(BSD) || defined(__native_client__) ++#elif defined(WEBRTC_MAC) || defined(WEBRTC_BSD) || defined(__native_client__) + RTC_LOG(LS_WARNING) << "Socket::OPT_DONTFRAGMENT not supported."; + return -1; + #elif defined(WEBRTC_POSIX) +diff --git a/system_wrappers/source/cpu_info.cc b/system_wrappers/source/cpu_info.cc +index eff720371a..94aed09c48 100644 +--- a/system_wrappers/source/cpu_info.cc ++++ b/system_wrappers/source/cpu_info.cc +@@ -12,7 +12,7 @@ + + #if defined(WEBRTC_WIN) + #include <windows.h> +-#elif defined(WEBRTC_LINUX) ++#elif defined(WEBRTC_LINUX) || defined(WEBRTC_BSD) + #include <unistd.h> + #elif defined(WEBRTC_MAC) + #include <sys/sysctl.h> +@@ -30,7 +30,7 @@ static int DetectNumberOfCores() { + SYSTEM_INFO si; + GetNativeSystemInfo(&si); + number_of_cores = static_cast<int>(si.dwNumberOfProcessors); +-#elif defined(WEBRTC_LINUX) || defined(WEBRTC_ANDROID) ++#elif defined(WEBRTC_LINUX) || defined(WEBRTC_ANDROID) || defined(WEBRTC_BSD) + number_of_cores = static_cast<int>(sysconf(_SC_NPROCESSORS_ONLN)); + if (number_of_cores <= 0) { + RTC_LOG(LS_ERROR) << "Failed to get number of cores"; +-- +2.34.1 + diff --git a/third_party/libwebrtc/moz-patch-stack/0015.patch b/third_party/libwebrtc/moz-patch-stack/0015.patch new file mode 100644 index 0000000000..65bcc4bcee --- /dev/null +++ b/third_party/libwebrtc/moz-patch-stack/0015.patch @@ -0,0 +1,32 @@ +From: Jed Davis <jld@mozilla.com> +Date: Sun, 17 Nov 2019 03:40:00 +0000 +Subject: Bug 1545504 - Strengthen bounds check in WebRTC PhysicalSocketServer. + r=jesup + +PhysicalSocketServer isn't currently used by Mozilla's WebRTC +integration, but just in case, let's make sure that this array index is +bounds-checked in actual use, not just in debug builds (which tend to +never see realistic test conditions). + +Differential Revision: https://phabricator.services.mozilla.com/D52745 +Mercurial Revision: https://hg.mozilla.org/mozilla-central/rev/2b079067648bfe0d064a334bf19bdcc233e26b6e +--- + rtc_base/physical_socket_server.cc | 2 +- + 1 file changed, 1 insertion(+), 1 deletion(-) + +diff --git a/rtc_base/physical_socket_server.cc b/rtc_base/physical_socket_server.cc +index 4ed4fd0cbb..60d024c769 100644 +--- a/rtc_base/physical_socket_server.cc ++++ b/rtc_base/physical_socket_server.cc +@@ -1397,7 +1397,7 @@ bool PhysicalSocketServer::WaitSelect(int cmsWait, bool process_io) { + int fd = pdispatcher->GetDescriptor(); + // "select"ing a file descriptor that is equal to or larger than + // FD_SETSIZE will result in undefined behavior. +- RTC_DCHECK_LT(fd, FD_SETSIZE); ++ RTC_CHECK_LT(fd, FD_SETSIZE); + if (fd > fdmax) + fdmax = fd; + +-- +2.34.1 + diff --git a/third_party/libwebrtc/moz-patch-stack/0016.patch b/third_party/libwebrtc/moz-patch-stack/0016.patch new file mode 100644 index 0000000000..1369ad01cd --- /dev/null +++ b/third_party/libwebrtc/moz-patch-stack/0016.patch @@ -0,0 +1,49 @@ +From: Dan Minor <dminor@mozilla.com> +Date: Mon, 5 Nov 2018 10:33:00 -0500 +Subject: Bug 1376873 - Reduce thread stack size in platform_thread.cc; r=bwc + +Summary: +The current default stack size of 1M results in intermittent OOMs on win32 +builds while running web-platform tests. The value of 256k was chosen for +consistency with the default value used elsewhere in Gecko, which is defined in +nsIThreadManager.idl. + +Reviewers: bwc + +Tags: #secure-revision + +Bug #: 1376873 + +Differential Revision: https://phabricator.services.mozilla.com/D11090 +Mercurial Revision: https://hg.mozilla.org/mozilla-central/rev/e83c311e5293902be4db4ecea17cff87c633f7cf +--- + rtc_base/platform_thread.cc | 6 ++++-- + 1 file changed, 4 insertions(+), 2 deletions(-) + +diff --git a/rtc_base/platform_thread.cc b/rtc_base/platform_thread.cc +index 6d369d747e..556204ac89 100644 +--- a/rtc_base/platform_thread.cc ++++ b/rtc_base/platform_thread.cc +@@ -189,15 +189,17 @@ PlatformThread PlatformThread::SpawnThread( + // Set the reserved stack stack size to 1M, which is the default on Windows + // and Linux. + DWORD thread_id = 0; ++ // Mozilla: Set to 256kb for consistency with nsIThreadManager.idl + PlatformThread::Handle handle = ::CreateThread( +- nullptr, 1024 * 1024, &RunPlatformThread, start_thread_function_ptr, ++ nullptr, 256 * 1024, &RunPlatformThread, start_thread_function_ptr, + STACK_SIZE_PARAM_IS_A_RESERVATION, &thread_id); + RTC_CHECK(handle) << "CreateThread failed"; + #else + pthread_attr_t attr; + pthread_attr_init(&attr); + // Set the stack stack size to 1M. +- pthread_attr_setstacksize(&attr, 1024 * 1024); ++ // Mozilla: Set to 256kb for consistency with nsIThreadManager.idl ++ pthread_attr_setstacksize(&attr, 256 * 1024); + pthread_attr_setdetachstate( + &attr, joinable ? PTHREAD_CREATE_JOINABLE : PTHREAD_CREATE_DETACHED); + PlatformThread::Handle handle; +-- +2.34.1 + diff --git a/third_party/libwebrtc/moz-patch-stack/0017.patch b/third_party/libwebrtc/moz-patch-stack/0017.patch new file mode 100644 index 0000000000..06daef4cc4 --- /dev/null +++ b/third_party/libwebrtc/moz-patch-stack/0017.patch @@ -0,0 +1,37 @@ +From: Dan Minor <dminor@mozilla.com> +Date: Thu, 31 Jan 2019 15:37:00 -0500 +Subject: Bug 1524208 - Calculate stride based upon target_width in + video_capture_impl.cc; r=pehrsons + +Differential Revision: https://phabricator.services.mozilla.com/D18270 +Mercurial Revision: https://hg.mozilla.org/mozilla-central/rev/51d12094d825c4c4467cb132d03d4f3cad4b1b82 +--- + modules/video_capture/video_capture_impl.cc | 5 +++-- + 1 file changed, 3 insertions(+), 2 deletions(-) + +diff --git a/modules/video_capture/video_capture_impl.cc b/modules/video_capture/video_capture_impl.cc +index 5dbc5e4e27..d227d41c34 100644 +--- a/modules/video_capture/video_capture_impl.cc ++++ b/modules/video_capture/video_capture_impl.cc +@@ -167,8 +167,6 @@ int32_t VideoCaptureImpl::IncomingFrame(uint8_t* videoFrame, + return -1; + } + +- int stride_y = width; +- int stride_uv = (width + 1) / 2; + int target_width = width; + int target_height = abs(height); + +@@ -184,6 +182,9 @@ int32_t VideoCaptureImpl::IncomingFrame(uint8_t* videoFrame, + } + } + ++ int stride_y = target_width; ++ int stride_uv = (target_width + 1) / 2; ++ + // Setting absolute height (in case it was negative). + // In Windows, the image starts bottom left, instead of top left. + // Setting a negative source height, inverts the image (within LibYuv). +-- +2.34.1 + diff --git a/third_party/libwebrtc/moz-patch-stack/0018.patch b/third_party/libwebrtc/moz-patch-stack/0018.patch new file mode 100644 index 0000000000..c097e221ad --- /dev/null +++ b/third_party/libwebrtc/moz-patch-stack/0018.patch @@ -0,0 +1,81 @@ +From: Dan Minor <dminor@mozilla.com> +Date: Thu, 21 Mar 2019 15:48:00 +0000 +Subject: Bug 1535584 - Restore UpdateCodecFrameSize to vp9_impl.cc; r=bwc + +Differential Revision: https://phabricator.services.mozilla.com/D23713 +Mercurial Revision: https://hg.mozilla.org/mozilla-central/rev/0898f9cfe09273e1d86c38abdd576cdf273009f0 +--- + .../codecs/vp9/libvpx_vp9_encoder.cc | 50 +++++++++++++++++++ + 1 file changed, 50 insertions(+) + +diff --git a/modules/video_coding/codecs/vp9/libvpx_vp9_encoder.cc b/modules/video_coding/codecs/vp9/libvpx_vp9_encoder.cc +index 35b13058a2..5877373b76 100644 +--- a/modules/video_coding/codecs/vp9/libvpx_vp9_encoder.cc ++++ b/modules/video_coding/codecs/vp9/libvpx_vp9_encoder.cc +@@ -1159,6 +1159,14 @@ int LibvpxVp9Encoder::Encode(const VideoFrame& input_image, + config_changed_ = false; + } + ++ if (input_image.width() != codec_.width || ++ input_image.height() != codec_.height) { ++ int ret = UpdateCodecFrameSize(input_image); ++ if (ret < 0) { ++ return ret; ++ } ++ } ++ + RTC_DCHECK_EQ(input_image.width(), raw_->d_w); + RTC_DCHECK_EQ(input_image.height(), raw_->d_h); + +@@ -1286,6 +1294,48 @@ int LibvpxVp9Encoder::Encode(const VideoFrame& input_image, + return WEBRTC_VIDEO_CODEC_OK; + } + ++int LibvpxVp9Encoder::UpdateCodecFrameSize( ++ const VideoFrame& input_image) { ++ RTC_LOG(LS_INFO) << "Reconfiging VP from " << ++ codec_.width << "x" << codec_.height << " to " << ++ input_image.width() << "x" << input_image.height(); ++ // Preserve latest bitrate/framerate setting ++ // TODO: Mozilla - see below, we need to save more state here. ++ //uint32_t old_bitrate_kbit = config_->rc_target_bitrate; ++ //uint32_t old_framerate = codec_.maxFramerate; ++ ++ codec_.width = input_image.width(); ++ codec_.height = input_image.height(); ++ ++ vpx_img_free(raw_); ++ raw_ = vpx_img_wrap(NULL, VPX_IMG_FMT_I420, codec_.width, codec_.height, ++ 1, NULL); ++ // Update encoder context for new frame size. ++ config_->g_w = codec_.width; ++ config_->g_h = codec_.height; ++ ++ // Determine number of threads based on the image size and #cores. ++ config_->g_threads = NumberOfThreads(codec_.width, codec_.height, ++ num_cores_); ++ ++ // NOTE: We would like to do this the same way vp8 does it ++ // (with vpx_codec_enc_config_set()), but that causes asserts ++ // in AQ 3 (cyclic); and in AQ 0 it works, but on a resize to smaller ++ // than 1/2 x 1/2 original it asserts in convolve(). Given these ++ // bugs in trying to do it the "right" way, we basically re-do ++ // the initialization. ++ vpx_codec_destroy(encoder_); // clean up old state ++ int result = InitAndSetControlSettings(&codec_); ++ if (result == WEBRTC_VIDEO_CODEC_OK) { ++ // TODO: Mozilla rates have become much more complicated, we need to store ++ // more state or find another way of doing this. ++ //return SetRates(old_bitrate_kbit, old_framerate); ++ RTC_CHECK(false); ++ return WEBRTC_VIDEO_CODEC_UNINITIALIZED; ++ } ++ return result; ++} ++ + bool LibvpxVp9Encoder::PopulateCodecSpecific(CodecSpecificInfo* codec_specific, + absl::optional<int>* spatial_idx, + absl::optional<int>* temporal_idx, +-- +2.34.1 + diff --git a/third_party/libwebrtc/moz-patch-stack/0019.patch b/third_party/libwebrtc/moz-patch-stack/0019.patch new file mode 100644 index 0000000000..e8538290e8 --- /dev/null +++ b/third_party/libwebrtc/moz-patch-stack/0019.patch @@ -0,0 +1,47 @@ +From: Dan Minor <dminor@mozilla.com> +Date: Fri, 29 Mar 2019 18:43:00 +0000 +Subject: Bug 1539220 - Prefer non-RGB24 capabilities when available; + r=pehrsons + +We've hit a number of problems with handling of RGB24 video capture on +Windows. This adds a check that will ignore any RGB24 capture capabilities +when determining a best match if there are other capabilities available to +workaround the problems. + +Differential Revision: https://phabricator.services.mozilla.com/D25449 +Mercurial Revision: https://hg.mozilla.org/mozilla-central/rev/c12307f3817bec87712ab5930493c3135c76b0a0 +--- + modules/video_capture/device_info_impl.cc | 13 +++++++++++++ + 1 file changed, 13 insertions(+) + +diff --git a/modules/video_capture/device_info_impl.cc b/modules/video_capture/device_info_impl.cc +index 7cccdb51a7..2a6afb3147 100644 +--- a/modules/video_capture/device_info_impl.cc ++++ b/modules/video_capture/device_info_impl.cc +@@ -100,10 +100,23 @@ int32_t DeviceInfoImpl::GetBestMatchedCapability( + const int32_t numberOfCapabilies = + static_cast<int32_t>(_captureCapabilities.size()); + ++ bool hasNonRGB24Capability = false; + for (int32_t tmp = 0; tmp < numberOfCapabilies; + ++tmp) // Loop through all capabilities + { + VideoCaptureCapability& capability = _captureCapabilities[tmp]; ++ if (capability.videoType != VideoType::kRGB24) { ++ hasNonRGB24Capability = true; ++ } ++ } ++ ++ for (int32_t tmp = 0; tmp < numberOfCapabilies; ++ ++tmp) // Loop through all capabilities ++ { ++ VideoCaptureCapability& capability = _captureCapabilities[tmp]; ++ if (hasNonRGB24Capability && capability.videoType == VideoType::kRGB24) { ++ continue; ++ } + + const int32_t diffWidth = capability.width - requested.width; + const int32_t diffHeight = capability.height - requested.height; +-- +2.34.1 + diff --git a/third_party/libwebrtc/moz-patch-stack/0020.patch b/third_party/libwebrtc/moz-patch-stack/0020.patch new file mode 100644 index 0000000000..72b949b3c1 --- /dev/null +++ b/third_party/libwebrtc/moz-patch-stack/0020.patch @@ -0,0 +1,96 @@ +From: Dan Minor <dminor@mozilla.com> +Date: Tue, 17 Sep 2019 06:47:00 +0000 +Subject: Bug 1581193 - Fix devicechange event on Windows; r=achronop + +This restores the code for generating devicechange events that was +accidentally removed as part of updating the Windows video capture code +in Bug 1552755. + +Differential Revision: https://phabricator.services.mozilla.com/D46033 +Mercurial Revision: https://hg.mozilla.org/mozilla-central/rev/840c4edafa021eeac6a7e6ae0e828d0adcfea92e +--- + .../video_capture/windows/device_info_ds.cc | 39 +++++++++++++++++++ + .../video_capture/windows/device_info_ds.h | 3 ++ + 2 files changed, 42 insertions(+) + +diff --git a/modules/video_capture/windows/device_info_ds.cc b/modules/video_capture/windows/device_info_ds.cc +index fb8d55137f..8543dce746 100644 +--- a/modules/video_capture/windows/device_info_ds.cc ++++ b/modules/video_capture/windows/device_info_ds.cc +@@ -20,6 +20,29 @@ + namespace webrtc { + namespace videocapturemodule { + ++LRESULT CALLBACK WndProc(HWND hWnd, UINT uiMsg, WPARAM wParam, LPARAM lParam) ++{ ++ DeviceInfoDS* pParent; ++ if (uiMsg == WM_CREATE) ++ { ++ pParent = (DeviceInfoDS*)((LPCREATESTRUCT)lParam)->lpCreateParams; ++ SetWindowLongPtr(hWnd, GWLP_USERDATA, (LONG_PTR)pParent); ++ } ++ else if (uiMsg == WM_DESTROY) ++ { ++ SetWindowLongPtr(hWnd, GWLP_USERDATA, NULL); ++ } ++ else if (uiMsg == WM_DEVICECHANGE) ++ { ++ pParent = (DeviceInfoDS*)GetWindowLongPtr(hWnd, GWLP_USERDATA); ++ if (pParent) ++ { ++ pParent->DeviceChange(); ++ } ++ } ++ return DefWindowProc(hWnd, uiMsg, wParam, lParam); ++} ++ + // static + DeviceInfoDS* DeviceInfoDS::Create() { + DeviceInfoDS* dsInfo = new DeviceInfoDS(); +@@ -77,6 +100,18 @@ DeviceInfoDS::DeviceInfoDS() + << rtc::ToHex(hr); + } + } ++ ++ _hInstance = reinterpret_cast<HINSTANCE>(GetModuleHandle(NULL)); ++ _wndClass = {0}; ++ _wndClass.lpfnWndProc = &WndProc; ++ _wndClass.lpszClassName = TEXT("DeviceInfoDS"); ++ _wndClass.hInstance = _hInstance; ++ ++ if (RegisterClass(&_wndClass)) { ++ _hwnd = CreateWindow(_wndClass.lpszClassName, NULL, 0, CW_USEDEFAULT, ++ CW_USEDEFAULT, CW_USEDEFAULT, CW_USEDEFAULT, NULL, ++ NULL, _hInstance, this); ++ } + } + + DeviceInfoDS::~DeviceInfoDS() { +@@ -85,6 +120,10 @@ DeviceInfoDS::~DeviceInfoDS() { + if (_CoUninitializeIsRequired) { + CoUninitialize(); + } ++ if (_hwnd != NULL) { ++ DestroyWindow(_hwnd); ++ } ++ UnregisterClass(_wndClass.lpszClassName, _hInstance); + } + + int32_t DeviceInfoDS::Init() { +diff --git a/modules/video_capture/windows/device_info_ds.h b/modules/video_capture/windows/device_info_ds.h +index 1b52645cde..dc7b9b1a24 100644 +--- a/modules/video_capture/windows/device_info_ds.h ++++ b/modules/video_capture/windows/device_info_ds.h +@@ -93,6 +93,9 @@ class DeviceInfoDS : public DeviceInfoImpl { + IEnumMoniker* _dsMonikerDevEnum; + bool _CoUninitializeIsRequired; + std::vector<VideoCaptureCapabilityWindows> _captureCapabilitiesWindows; ++ HWND _hwnd; ++ WNDCLASS _wndClass; ++ HINSTANCE _hInstance; + }; + } // namespace videocapturemodule + } // namespace webrtc +-- +2.34.1 + diff --git a/third_party/libwebrtc/moz-patch-stack/0021.patch b/third_party/libwebrtc/moz-patch-stack/0021.patch new file mode 100644 index 0000000000..d9fdd2cb9d --- /dev/null +++ b/third_party/libwebrtc/moz-patch-stack/0021.patch @@ -0,0 +1,97 @@ +From: Alex Chronopoulos <achronop@gmail.com> +Date: Tue, 17 Sep 2019 14:31:00 +0000 +Subject: Bug 1581806 - Trigger devicechange event for audio and video input + devices only. r=dminor + +After Bug 1581193 devicechange notifications were triggered with any device change, not just video and audio devices. This patch limits down the notifications to only video and audio input devices change. + +Differential Revision: https://phabricator.services.mozilla.com/D46147 +Mercurial Revision: https://hg.mozilla.org/mozilla-central/rev/1feec83ee6f92a35de0d4b27ebea04e68a0d7ef0 +--- + .../video_capture/windows/device_info_ds.cc | 29 +++++++++++++++++-- + .../video_capture/windows/device_info_ds.h | 1 + + 2 files changed, 28 insertions(+), 2 deletions(-) + +diff --git a/modules/video_capture/windows/device_info_ds.cc b/modules/video_capture/windows/device_info_ds.cc +index 8543dce746..96db60c968 100644 +--- a/modules/video_capture/windows/device_info_ds.cc ++++ b/modules/video_capture/windows/device_info_ds.cc +@@ -20,6 +20,18 @@ + namespace webrtc { + namespace videocapturemodule { + ++BOOL isCaptureDevice(DEV_BROADCAST_HDR *pHdr) ++{ ++ if (pHdr == NULL) { ++ return FALSE; ++ } ++ if (pHdr->dbch_devicetype != DBT_DEVTYP_DEVICEINTERFACE) { ++ return FALSE; ++ } ++ DEV_BROADCAST_DEVICEINTERFACE* pDi = (DEV_BROADCAST_DEVICEINTERFACE*)pHdr; ++ return pDi->dbcc_classguid == KSCATEGORY_CAPTURE; ++} ++ + LRESULT CALLBACK WndProc(HWND hWnd, UINT uiMsg, WPARAM wParam, LPARAM lParam) + { + DeviceInfoDS* pParent; +@@ -35,7 +47,7 @@ LRESULT CALLBACK WndProc(HWND hWnd, UINT uiMsg, WPARAM wParam, LPARAM lParam) + else if (uiMsg == WM_DEVICECHANGE) + { + pParent = (DeviceInfoDS*)GetWindowLongPtr(hWnd, GWLP_USERDATA); +- if (pParent) ++ if (pParent && isCaptureDevice((PDEV_BROADCAST_HDR)lParam)) + { + pParent->DeviceChange(); + } +@@ -56,7 +68,8 @@ DeviceInfoDS* DeviceInfoDS::Create() { + DeviceInfoDS::DeviceInfoDS() + : _dsDevEnum(NULL), + _dsMonikerDevEnum(NULL), +- _CoUninitializeIsRequired(true) { ++ _CoUninitializeIsRequired(true), ++ _hdevnotify(NULL) { + // 1) Initialize the COM library (make Windows load the DLLs). + // + // CoInitializeEx must be called at least once, and is usually called only +@@ -111,6 +124,14 @@ DeviceInfoDS::DeviceInfoDS() + _hwnd = CreateWindow(_wndClass.lpszClassName, NULL, 0, CW_USEDEFAULT, + CW_USEDEFAULT, CW_USEDEFAULT, CW_USEDEFAULT, NULL, + NULL, _hInstance, this); ++ ++ DEV_BROADCAST_DEVICEINTERFACE di = { 0 }; ++ di.dbcc_size = sizeof(di); ++ di.dbcc_devicetype = DBT_DEVTYP_DEVICEINTERFACE; ++ di.dbcc_classguid = KSCATEGORY_CAPTURE; ++ ++ _hdevnotify = RegisterDeviceNotification(_hwnd, &di, ++ DEVICE_NOTIFY_WINDOW_HANDLE); + } + } + +@@ -120,6 +141,10 @@ DeviceInfoDS::~DeviceInfoDS() { + if (_CoUninitializeIsRequired) { + CoUninitialize(); + } ++ if (_hdevnotify) ++ { ++ UnregisterDeviceNotification(_hdevnotify); ++ } + if (_hwnd != NULL) { + DestroyWindow(_hwnd); + } +diff --git a/modules/video_capture/windows/device_info_ds.h b/modules/video_capture/windows/device_info_ds.h +index dc7b9b1a24..ed2a726d6f 100644 +--- a/modules/video_capture/windows/device_info_ds.h ++++ b/modules/video_capture/windows/device_info_ds.h +@@ -96,6 +96,7 @@ class DeviceInfoDS : public DeviceInfoImpl { + HWND _hwnd; + WNDCLASS _wndClass; + HINSTANCE _hInstance; ++ HDEVNOTIFY _hdevnotify; + }; + } // namespace videocapturemodule + } // namespace webrtc +-- +2.34.1 + diff --git a/third_party/libwebrtc/moz-patch-stack/0022.patch b/third_party/libwebrtc/moz-patch-stack/0022.patch new file mode 100644 index 0000000000..59784f8734 --- /dev/null +++ b/third_party/libwebrtc/moz-patch-stack/0022.patch @@ -0,0 +1,98 @@ +From: Alex Chronopoulos <achronop@gmail.com> +Date: Wed, 18 Sep 2019 13:16:00 +0000 +Subject: Bug 1572281 - Remove audio device change notifications from video + capture in Linux. r=dminor + +Video capture used to provide device change notifications for audio and video devices. From now on, CubebDeviceEnumerator will provide audio device change notifications thus video capture is updated to notify only changes of the video device. This is the Linux part. + +Differential Revision: https://phabricator.services.mozilla.com/D46272 +Mercurial Revision: https://hg.mozilla.org/mozilla-central/rev/7bf7263db30b794139332691f4fbc98b4bfcfdd7 +--- + .../video_capture/linux/device_info_v4l2.cc | 28 ++----------------- + 1 file changed, 3 insertions(+), 25 deletions(-) + +diff --git a/modules/video_capture/linux/device_info_v4l2.cc b/modules/video_capture/linux/device_info_v4l2.cc +index d836747b4a..77968b7eaf 100644 +--- a/modules/video_capture/linux/device_info_v4l2.cc ++++ b/modules/video_capture/linux/device_info_v4l2.cc +@@ -44,7 +44,7 @@ namespace videocapturemodule { + void DeviceInfoV4l2::HandleEvent(inotify_event* event, int fd) + { + if (event->mask & IN_CREATE) { +- if (fd == _fd_v4l || fd == _fd_snd) { ++ if (fd == _fd_v4l) { + DeviceChange(); + } else if ((event->mask & IN_ISDIR) && (fd == _fd_dev)) { + if (_wd_v4l < 0) { +@@ -56,25 +56,15 @@ void DeviceInfoV4l2::HandleEvent(inotify_event* event, int fd) + DeviceChange(); + } + } +- if (_wd_snd < 0) { +- usleep(5*1000); +- _wd_snd = inotify_add_watch(_fd_snd, "/dev/snd/by-path/", IN_CREATE | IN_DELETE | IN_DELETE_SELF); +- if (_wd_snd >= 0) { +- DeviceChange(); +- } +- } + } + } else if (event->mask & IN_DELETE) { +- if (fd == _fd_v4l || fd == _fd_snd) { ++ if (fd == _fd_v4l) { + DeviceChange(); + } + } else if (event->mask & IN_DELETE_SELF) { + if (fd == _fd_v4l) { + inotify_rm_watch(_fd_v4l, _wd_v4l); + _wd_v4l = -1; +- } else if (fd == _fd_snd) { +- inotify_rm_watch(_fd_snd, _wd_snd); +- _wd_snd = -1; + } else { + assert(false); + } +@@ -141,11 +131,6 @@ int DeviceInfoV4l2::ProcessInotifyEvents() + break; + } + } +- if (EventCheck(_fd_snd) > 0) { +- if (HandleEvents(_fd_snd) < 0) { +- break; +- } +- } + } + return 0; + } +@@ -158,11 +143,9 @@ void DeviceInfoV4l2::InotifyEventThread(void* obj) + void DeviceInfoV4l2::InotifyProcess() + { + _fd_v4l = inotify_init(); +- _fd_snd = inotify_init(); + _fd_dev = inotify_init(); +- if (_fd_v4l >= 0 && _fd_snd >= 0 && _fd_dev >= 0) { ++ if (_fd_v4l >= 0 && _fd_dev >= 0) { + _wd_v4l = inotify_add_watch(_fd_v4l, "/dev/v4l/by-path/", IN_CREATE | IN_DELETE | IN_DELETE_SELF); +- _wd_snd = inotify_add_watch(_fd_snd, "/dev/snd/by-path/", IN_CREATE | IN_DELETE | IN_DELETE_SELF); + _wd_dev = inotify_add_watch(_fd_dev, "/dev/", IN_CREATE); + ProcessInotifyEvents(); + +@@ -170,16 +153,11 @@ void DeviceInfoV4l2::InotifyProcess() + inotify_rm_watch(_fd_v4l, _wd_v4l); + } + +- if (_wd_snd >= 0) { +- inotify_rm_watch(_fd_snd, _wd_snd); +- } +- + if (_wd_dev >= 0) { + inotify_rm_watch(_fd_dev, _wd_dev); + } + + close(_fd_v4l); +- close(_fd_snd); + close(_fd_dev); + } + } +-- +2.34.1 + diff --git a/third_party/libwebrtc/moz-patch-stack/0023.patch b/third_party/libwebrtc/moz-patch-stack/0023.patch new file mode 100644 index 0000000000..83497607c2 --- /dev/null +++ b/third_party/libwebrtc/moz-patch-stack/0023.patch @@ -0,0 +1,56 @@ +From: Alex Chronopoulos <achronop@gmail.com> +Date: Wed, 18 Sep 2019 13:12:00 +0000 +Subject: Bug 1572281 - Remove audio device change notifications from video + capture in Windows. r=dminor + +Video capture used to provide device change notifications for audio and video devices. From now on, CubebDeviceEnumerator will provide audio device change notifications thus video capture is updated to notify only changes of the video device. This is the windows part. + +Differential Revision: https://phabricator.services.mozilla.com/D46274 +Mercurial Revision: https://hg.mozilla.org/mozilla-central/rev/a5c23245837ebdd99532e8bfaca05774c5c96d9d +--- + modules/video_capture/windows/device_info_ds.cc | 8 ++++---- + 1 file changed, 4 insertions(+), 4 deletions(-) + +diff --git a/modules/video_capture/windows/device_info_ds.cc b/modules/video_capture/windows/device_info_ds.cc +index 96db60c968..3ab95837c0 100644 +--- a/modules/video_capture/windows/device_info_ds.cc ++++ b/modules/video_capture/windows/device_info_ds.cc +@@ -20,7 +20,7 @@ + namespace webrtc { + namespace videocapturemodule { + +-BOOL isCaptureDevice(DEV_BROADCAST_HDR *pHdr) ++BOOL isVideoDevice(DEV_BROADCAST_HDR *pHdr) + { + if (pHdr == NULL) { + return FALSE; +@@ -29,7 +29,7 @@ BOOL isCaptureDevice(DEV_BROADCAST_HDR *pHdr) + return FALSE; + } + DEV_BROADCAST_DEVICEINTERFACE* pDi = (DEV_BROADCAST_DEVICEINTERFACE*)pHdr; +- return pDi->dbcc_classguid == KSCATEGORY_CAPTURE; ++ return pDi->dbcc_classguid == KSCATEGORY_VIDEO_CAMERA; + } + + LRESULT CALLBACK WndProc(HWND hWnd, UINT uiMsg, WPARAM wParam, LPARAM lParam) +@@ -47,7 +47,7 @@ LRESULT CALLBACK WndProc(HWND hWnd, UINT uiMsg, WPARAM wParam, LPARAM lParam) + else if (uiMsg == WM_DEVICECHANGE) + { + pParent = (DeviceInfoDS*)GetWindowLongPtr(hWnd, GWLP_USERDATA); +- if (pParent && isCaptureDevice((PDEV_BROADCAST_HDR)lParam)) ++ if (pParent && isVideoDevice((PDEV_BROADCAST_HDR)lParam)) + { + pParent->DeviceChange(); + } +@@ -128,7 +128,7 @@ DeviceInfoDS::DeviceInfoDS() + DEV_BROADCAST_DEVICEINTERFACE di = { 0 }; + di.dbcc_size = sizeof(di); + di.dbcc_devicetype = DBT_DEVTYP_DEVICEINTERFACE; +- di.dbcc_classguid = KSCATEGORY_CAPTURE; ++ di.dbcc_classguid = KSCATEGORY_VIDEO_CAMERA; + + _hdevnotify = RegisterDeviceNotification(_hwnd, &di, + DEVICE_NOTIFY_WINDOW_HANDLE); +-- +2.34.1 + diff --git a/third_party/libwebrtc/moz-patch-stack/0024.patch b/third_party/libwebrtc/moz-patch-stack/0024.patch new file mode 100644 index 0000000000..7e5ce6fd32 --- /dev/null +++ b/third_party/libwebrtc/moz-patch-stack/0024.patch @@ -0,0 +1,58 @@ +From: Wang Qing <wangqing-hf@loongson.cn> +Date: Wed, 25 Sep 2019 14:15:00 +0000 +Subject: Bug 1579834 - [WebRTC] Add mips64 support; r=dminor + +Differential Revision: https://phabricator.services.mozilla.com/D45620 +Mercurial Revision: https://hg.mozilla.org/mozilla-central/rev/1387b2c480b55ecca3fbdf62bd7649dafc62438d +--- + modules/video_coding/codecs/vp8/libvpx_vp8_decoder.cc | 2 +- + modules/video_coding/codecs/vp8/libvpx_vp8_encoder.cc | 6 +++--- + 2 files changed, 4 insertions(+), 4 deletions(-) + +diff --git a/modules/video_coding/codecs/vp8/libvpx_vp8_decoder.cc b/modules/video_coding/codecs/vp8/libvpx_vp8_decoder.cc +index 1ac71899ee..3fe86f2f85 100644 +--- a/modules/video_coding/codecs/vp8/libvpx_vp8_decoder.cc ++++ b/modules/video_coding/codecs/vp8/libvpx_vp8_decoder.cc +@@ -47,7 +47,7 @@ const char kVp8PostProcArmFieldTrial[] = "WebRTC-VP8-Postproc-Config-Arm"; + const char kVp8PostProcFieldTrial[] = "WebRTC-VP8-Postproc-Config"; + + #if defined(WEBRTC_ARCH_ARM) || defined(WEBRTC_ARCH_ARM64) || \ +- defined(WEBRTC_ANDROID) ++ defined(WEBRTC_ANDROID) || defined(WEBRTC_ARCH_MIPS) + constexpr bool kIsArm = true; + #else + constexpr bool kIsArm = false; +diff --git a/modules/video_coding/codecs/vp8/libvpx_vp8_encoder.cc b/modules/video_coding/codecs/vp8/libvpx_vp8_encoder.cc +index 8e401fcc7b..cc84605ce7 100644 +--- a/modules/video_coding/codecs/vp8/libvpx_vp8_encoder.cc ++++ b/modules/video_coding/codecs/vp8/libvpx_vp8_encoder.cc +@@ -687,7 +687,7 @@ int LibvpxVp8Encoder::InitEncode(const VideoCodec* inst, + + int LibvpxVp8Encoder::GetCpuSpeed(int width, int height) { + #if defined(WEBRTC_ARCH_ARM) || defined(WEBRTC_ARCH_ARM64) || \ +- defined(WEBRTC_ANDROID) ++ defined(WEBRTC_ANDROID) || defined(WEBRTC_ARCH_MIPS) + // On mobile platform, use a lower speed setting for lower resolutions for + // CPUs with 4 or more cores. + RTC_DCHECK_GT(number_of_cores_, 0); +@@ -720,7 +720,7 @@ int LibvpxVp8Encoder::GetCpuSpeed(int width, int height) { + } + + int LibvpxVp8Encoder::NumberOfThreads(int width, int height, int cpus) { +-#if defined(WEBRTC_ANDROID) ++#if defined(WEBRTC_ANDROID) || defined(WEBRTC_ARCH_MIPS) + if (width * height >= 320 * 180) { + if (cpus >= 4) { + // 3 threads for CPUs with 4 and more cores since most of times only 4 +@@ -794,7 +794,7 @@ int LibvpxVp8Encoder::InitAndSetControlSettings() { + // multi-res encoding feature? + denoiserState denoiser_state = kDenoiserOnYOnly; + #if defined(WEBRTC_ARCH_ARM) || defined(WEBRTC_ARCH_ARM64) || \ +- defined(WEBRTC_ANDROID) ++ defined(WEBRTC_ANDROID) || defined(WEBRTC_ARCH_MIPS) + denoiser_state = kDenoiserOnYOnly; + #else + denoiser_state = kDenoiserOnAdaptive; +-- +2.34.1 + diff --git a/third_party/libwebrtc/moz-patch-stack/0025.patch b/third_party/libwebrtc/moz-patch-stack/0025.patch new file mode 100644 index 0000000000..d90bb77010 --- /dev/null +++ b/third_party/libwebrtc/moz-patch-stack/0025.patch @@ -0,0 +1,29 @@ +From: Dan Minor <dminor@mozilla.com> +Date: Wed, 9 Oct 2019 20:12:00 +0000 +Subject: Bug 1587159 - Fix undefined shift in g722_encode.c; r=ng + +Left shifting a negative value results in undefined behaviour. It is safer to +multiply in this case. + +Differential Revision: https://phabricator.services.mozilla.com/D48751 +Mercurial Revision: https://hg.mozilla.org/mozilla-central/rev/385d660fac359c907986e08d1d89ab5a353f30b2 +--- + modules/third_party/g722/g722_encode.c | 2 +- + 1 file changed, 1 insertion(+), 1 deletion(-) + +diff --git a/modules/third_party/g722/g722_encode.c b/modules/third_party/g722/g722_encode.c +index 10a5bcfe7c..fedf9f5961 100644 +--- a/modules/third_party/g722/g722_encode.c ++++ b/modules/third_party/g722/g722_encode.c +@@ -74,7 +74,7 @@ static void block4(G722EncoderState *s, int band, int d) + /* Block 4, UPPOL2 */ + for (i = 0; i < 3; i++) + s->band[band].sg[i] = s->band[band].p[i] >> 15; +- wd1 = saturate(s->band[band].a[1] << 2); ++ wd1 = saturate(s->band[band].a[1] * 4); + + wd2 = (s->band[band].sg[0] == s->band[band].sg[1]) ? -wd1 : wd1; + if (wd2 > 32767) +-- +2.34.1 + diff --git a/third_party/libwebrtc/moz-patch-stack/0026.patch b/third_party/libwebrtc/moz-patch-stack/0026.patch new file mode 100644 index 0000000000..ba8c50ed73 --- /dev/null +++ b/third_party/libwebrtc/moz-patch-stack/0026.patch @@ -0,0 +1,148 @@ +From: Gabriele Svelto <gsvelto@mozilla.com> +Date: Mon, 28 Oct 2019 23:26:00 +0000 +Subject: Bug 1590984 - Use poll() instead of select() in WebRTC code r=drno + +The use of select() was leading to crashes when the file descriptor value was +larger than FD_SETSIZE. Recent versions of glibc have checks in the FD_CLR(), +FD_SET() and FD_ISSET() macros that will abort() the program instead of doing +an out-of-bounds access. poll() doesn't have limitations on the file +descriptor values and provides behavior that is otherwise identical to +select() thus solving the problem. + +Differential Revision: https://phabricator.services.mozilla.com/D50798 +Mercurial Revision: https://hg.mozilla.org/mozilla-central/rev/59fb6760bb6785a6f8a51be6fc66bf04cfba3e16 +--- + .../video_capture/linux/device_info_linux.cc | 1 + + .../video_capture/linux/device_info_v4l2.cc | 16 +++++----- + .../linux/video_capture_linux.cc | 1 + + .../video_capture/linux/video_capture_v4l2.cc | 29 +++++++++++-------- + 4 files changed, 26 insertions(+), 21 deletions(-) + +diff --git a/modules/video_capture/linux/device_info_linux.cc b/modules/video_capture/linux/device_info_linux.cc +index 9c12b2183e..4821cbccd5 100644 +--- a/modules/video_capture/linux/device_info_linux.cc ++++ b/modules/video_capture/linux/device_info_linux.cc +@@ -10,6 +10,7 @@ + + #include <errno.h> + #include <fcntl.h> ++#include <poll.h> + #include <stdio.h> + #include <stdlib.h> + #include <string.h> +diff --git a/modules/video_capture/linux/device_info_v4l2.cc b/modules/video_capture/linux/device_info_v4l2.cc +index 77968b7eaf..f87fe53abe 100644 +--- a/modules/video_capture/linux/device_info_v4l2.cc ++++ b/modules/video_capture/linux/device_info_v4l2.cc +@@ -12,6 +12,7 @@ + + #include <errno.h> + #include <fcntl.h> ++#include <poll.h> + #include <stdio.h> + #include <stdlib.h> + #include <string.h> +@@ -73,16 +74,13 @@ void DeviceInfoV4l2::HandleEvent(inotify_event* event, int fd) + + int DeviceInfoV4l2::EventCheck(int fd) + { +- struct timeval timeout; +- fd_set rfds; ++ struct pollfd fds = { ++ .fd = fd, ++ .events = POLLIN, ++ .revents = 0 ++ }; + +- timeout.tv_sec = 0; +- timeout.tv_usec = 100000; +- +- FD_ZERO(&rfds); +- FD_SET(fd, &rfds); +- +- return select(fd+1, &rfds, NULL, NULL, &timeout); ++ return poll(&fds, 1, 100); + } + + int DeviceInfoV4l2::HandleEvents(int fd) +diff --git a/modules/video_capture/linux/video_capture_linux.cc b/modules/video_capture/linux/video_capture_linux.cc +index 4895a1ab71..f3324a8e68 100644 +--- a/modules/video_capture/linux/video_capture_linux.cc ++++ b/modules/video_capture/linux/video_capture_linux.cc +@@ -10,6 +10,7 @@ + + #include <errno.h> + #include <fcntl.h> ++#include <poll.h> + #include <stdio.h> + #include <string.h> + #include <sys/ioctl.h> +diff --git a/modules/video_capture/linux/video_capture_v4l2.cc b/modules/video_capture/linux/video_capture_v4l2.cc +index 1dc13b01aa..b527a331e4 100644 +--- a/modules/video_capture/linux/video_capture_v4l2.cc ++++ b/modules/video_capture/linux/video_capture_v4l2.cc +@@ -12,7 +12,7 @@ + + #include <errno.h> + #include <fcntl.h> +-#include <linux/videodev2.h> ++#include <poll.h> + #include <stdio.h> + #include <string.h> + #include <sys/ioctl.h> +@@ -20,6 +20,14 @@ + #include <sys/select.h> + #include <time.h> + #include <unistd.h> ++// v4l includes ++#if defined(__NetBSD__) || defined(__OpenBSD__) // WEBRTC_BSD ++#include <sys/videoio.h> ++#elif defined(__sun) ++#include <sys/videodev2.h> ++#else ++#include <linux/videodev2.h> ++#endif + + #include <new> + #include <string> +@@ -359,16 +367,13 @@ bool VideoCaptureModuleV4L2::CaptureStarted() { + + bool VideoCaptureModuleV4L2::CaptureProcess() { + int retVal = 0; +- fd_set rSet; +- struct timeval timeout; ++ struct pollfd rSet; + +- FD_ZERO(&rSet); +- FD_SET(_deviceFd, &rSet); +- timeout.tv_sec = 1; +- timeout.tv_usec = 0; ++ rSet.fd = _deviceFd; ++ rSet.events = POLLIN; ++ rSet.revents = 0; + +- // _deviceFd written only in StartCapture, when this thread isn't running. +- retVal = select(_deviceFd + 1, &rSet, NULL, NULL, &timeout); ++ retVal = poll(&rSet, 1, 1000); + + { + MutexLock lock(&capture_lock_); +@@ -378,12 +383,12 @@ bool VideoCaptureModuleV4L2::CaptureProcess() { + } + + if (retVal < 0 && errno != EINTR) { // continue if interrupted +- // select failed ++ // poll failed + return false; + } else if (retVal == 0) { +- // select timed out ++ // poll timed out + return true; +- } else if (!FD_ISSET(_deviceFd, &rSet)) { ++ } else if (!(rSet.revents & POLLIN)) { + // not event on camera handle + return true; + } +-- +2.34.1 + diff --git a/third_party/libwebrtc/moz-patch-stack/0027.patch b/third_party/libwebrtc/moz-patch-stack/0027.patch new file mode 100644 index 0000000000..2558a1c609 --- /dev/null +++ b/third_party/libwebrtc/moz-patch-stack/0027.patch @@ -0,0 +1,28 @@ +From: Dan Minor <dminor@mozilla.com> +Date: Tue, 11 Feb 2020 17:04:00 +0000 +Subject: Bug 1578073 - Fix warning: [cast] redundant cast to int; r=ng + +Differential Revision: https://phabricator.services.mozilla.com/D61852 +Mercurial Revision: https://hg.mozilla.org/mozilla-central/rev/82877c8a864072c03f45234c6649645ed609e098 +--- + sdk/android/api/org/webrtc/TextureBufferImpl.java | 4 ++-- + 1 file changed, 2 insertions(+), 2 deletions(-) + +diff --git a/sdk/android/api/org/webrtc/TextureBufferImpl.java b/sdk/android/api/org/webrtc/TextureBufferImpl.java +index c08fc4c29b..8e0e40ef70 100644 +--- a/sdk/android/api/org/webrtc/TextureBufferImpl.java ++++ b/sdk/android/api/org/webrtc/TextureBufferImpl.java +@@ -136,8 +136,8 @@ public class TextureBufferImpl implements VideoFrame.TextureBuffer { + cropAndScaleMatrix.preScale(cropWidth / (float) width, cropHeight / (float) height); + + return applyTransformMatrix(cropAndScaleMatrix, +- (int) Math.round(unscaledWidth * cropWidth / (float) width), +- (int) Math.round(unscaledHeight * cropHeight / (float) height), scaleWidth, scaleHeight); ++ Math.round(unscaledWidth * cropWidth / (float) width), ++ Math.round(unscaledHeight * cropHeight / (float) height), scaleWidth, scaleHeight); + } + + /** +-- +2.34.1 + diff --git a/third_party/libwebrtc/moz-patch-stack/0028.patch b/third_party/libwebrtc/moz-patch-stack/0028.patch new file mode 100644 index 0000000000..d33bb501e7 --- /dev/null +++ b/third_party/libwebrtc/moz-patch-stack/0028.patch @@ -0,0 +1,43 @@ +From: Dan Minor <dminor@mozilla.com> +Date: Tue, 11 Feb 2020 17:07:00 +0000 +Subject: Bug 1578073 - Remove native calls in Histogram.java; r=ng + +Getting the JNI calls here to work requires a good amount of webrtc.org +machinery. It might be worth setting that up the next time we do an upstream +merge, but for now, it is a lot simpler to just remove the affected code, +given that we are not interested in collecting this data anyway. + +Differential Revision: https://phabricator.services.mozilla.com/D61860 +Mercurial Revision: https://hg.mozilla.org/mozilla-central/rev/7a9b07dec9f9d435416b06829fa5063aca3a476e +--- + sdk/android/src/java/org/webrtc/Histogram.java | 9 ++------- + 1 file changed, 2 insertions(+), 7 deletions(-) + +diff --git a/sdk/android/src/java/org/webrtc/Histogram.java b/sdk/android/src/java/org/webrtc/Histogram.java +index 877986134a..c1d2d61a71 100644 +--- a/sdk/android/src/java/org/webrtc/Histogram.java ++++ b/sdk/android/src/java/org/webrtc/Histogram.java +@@ -27,18 +27,13 @@ class Histogram { + } + + static public Histogram createCounts(String name, int min, int max, int bucketCount) { +- return new Histogram(nativeCreateCounts(name, min, max, bucketCount)); ++ return new Histogram(0); + } + + static public Histogram createEnumeration(String name, int max) { +- return new Histogram(nativeCreateEnumeration(name, max)); ++ return new Histogram(0); + } + + public void addSample(int sample) { +- nativeAddSample(handle, sample); + } +- +- private static native long nativeCreateCounts(String name, int min, int max, int bucketCount); +- private static native long nativeCreateEnumeration(String name, int max); +- private static native void nativeAddSample(long handle, int sample); + } +-- +2.34.1 + diff --git a/third_party/libwebrtc/moz-patch-stack/0029.patch b/third_party/libwebrtc/moz-patch-stack/0029.patch new file mode 100644 index 0000000000..407fcccce2 --- /dev/null +++ b/third_party/libwebrtc/moz-patch-stack/0029.patch @@ -0,0 +1,36 @@ +From: Dan Minor <dminor@mozilla.com> +Date: Wed, 12 Feb 2020 17:19:00 +0000 +Subject: Bug 1578073 - Suppress MissingPermission lint in Camera2Session; + r=snorp + +Depends on D61861 + +Differential Revision: https://phabricator.services.mozilla.com/D62457 +Mercurial Revision: https://hg.mozilla.org/mozilla-central/rev/722c4a6d1dad867f9ce47fe96d71b5dedb4cbaa8 +--- + sdk/android/src/java/org/webrtc/Camera2Session.java | 2 ++ + 1 file changed, 2 insertions(+) + +diff --git a/sdk/android/src/java/org/webrtc/Camera2Session.java b/sdk/android/src/java/org/webrtc/Camera2Session.java +index dec97a2c25..d5ee80c73e 100644 +--- a/sdk/android/src/java/org/webrtc/Camera2Session.java ++++ b/sdk/android/src/java/org/webrtc/Camera2Session.java +@@ -10,6 +10,7 @@ + + package org.webrtc; + ++import android.annotation.SuppressLint; + import android.content.Context; + import android.hardware.camera2.CameraAccessException; + import android.hardware.camera2.CameraCaptureSession; +@@ -347,6 +348,7 @@ class Camera2Session implements CameraSession { + Logging.d(TAG, "Using capture format: " + captureFormat); + } + ++ @SuppressLint("MissingPermission") + private void openCamera() { + checkIsOnCameraThread(); + +-- +2.34.1 + diff --git a/third_party/libwebrtc/moz-patch-stack/0030.patch b/third_party/libwebrtc/moz-patch-stack/0030.patch new file mode 100644 index 0000000000..75ddb7fbf8 --- /dev/null +++ b/third_party/libwebrtc/moz-patch-stack/0030.patch @@ -0,0 +1,90 @@ +From: Dan Minor <dminor@mozilla.com> +Date: Wed, 8 Jul 2020 17:35:00 +0000 +Subject: Bug 1650572 - Check V4L2_CAP_DEVICE_CAPS before accessing + device_caps; r=ng + +The capabilities field is for the physical device, device_caps is for the +specific /dev/videoX device that has been opened. The device_caps field is +only populated if V4L2_CAP_DEVICE_CAPS is set, so we should check that, and +fall back to capabilities if it is not set. + +Differential Revision: https://phabricator.services.mozilla.com/D82377 +Mercurial Revision: https://hg.mozilla.org/mozilla-central/rev/b5acbf536c46a66c939a61bde34ad93b1977a604 +--- + modules/video_capture/linux/device_info_v4l2.cc | 17 ++++++++++++----- + modules/video_capture/linux/device_info_v4l2.h | 3 +++ + 2 files changed, 15 insertions(+), 5 deletions(-) + +diff --git a/modules/video_capture/linux/device_info_v4l2.cc b/modules/video_capture/linux/device_info_v4l2.cc +index f87fe53abe..d506f3a448 100644 +--- a/modules/video_capture/linux/device_info_v4l2.cc ++++ b/modules/video_capture/linux/device_info_v4l2.cc +@@ -202,8 +202,7 @@ uint32_t DeviceInfoV4l2::NumberOfDevices() { + snprintf(device, sizeof(device), "/dev/video%d", n); + if ((fd = open(device, O_RDONLY)) != -1) { + // query device capabilities and make sure this is a video capture device +- if (ioctl(fd, VIDIOC_QUERYCAP, &cap) < 0 || +- !(cap.device_caps & V4L2_CAP_VIDEO_CAPTURE)) { ++ if (ioctl(fd, VIDIOC_QUERYCAP, &cap) < 0 || !IsVideoCaptureDevice(&cap)) { + close(fd); + continue; + } +@@ -235,8 +234,7 @@ int32_t DeviceInfoV4l2::GetDeviceName(uint32_t deviceNumber, + sprintf(device, "/dev/video%d", device_index); + if ((fd = open(device, O_RDONLY)) != -1) { + // query device capabilities and make sure this is a video capture device +- if (ioctl(fd, VIDIOC_QUERYCAP, &cap) < 0 || +- !(cap.device_caps & V4L2_CAP_VIDEO_CAPTURE)) { ++ if (ioctl(fd, VIDIOC_QUERYCAP, &cap) < 0 || !IsVideoCaptureDevice(&cap)) { + close(fd); + continue; + } +@@ -321,7 +319,7 @@ int32_t DeviceInfoV4l2::CreateCapabilityMap(const char* deviceUniqueIdUTF8) { + struct v4l2_capability cap; + if (ioctl(fd, VIDIOC_QUERYCAP, &cap) == 0) { + // skip devices without video capture capability +- if (!(cap.device_caps & V4L2_CAP_VIDEO_CAPTURE)) { ++ if (!IsVideoCaptureDevice(&cap)) { + continue; + } + +@@ -383,6 +381,15 @@ bool DeviceInfoV4l2::IsDeviceNameMatches(const char* name, + return false; + } + ++bool DeviceInfoV4l2::IsVideoCaptureDevice(struct v4l2_capability* cap) ++{ ++ if (cap->capabilities & V4L2_CAP_DEVICE_CAPS) { ++ return cap->device_caps & V4L2_CAP_VIDEO_CAPTURE; ++ } else { ++ return cap->capabilities & V4L2_CAP_VIDEO_CAPTURE; ++ } ++} ++ + int32_t DeviceInfoV4l2::FillCapabilities(int fd) { + // set image format + struct v4l2_format video_fmt; +diff --git a/modules/video_capture/linux/device_info_v4l2.h b/modules/video_capture/linux/device_info_v4l2.h +index 95432a509d..e3c2395f49 100644 +--- a/modules/video_capture/linux/device_info_v4l2.h ++++ b/modules/video_capture/linux/device_info_v4l2.h +@@ -18,6 +18,8 @@ + #include "rtc_base/platform_thread.h" + #include <sys/inotify.h> + ++struct v4l2_capability; ++ + namespace webrtc { + namespace videocapturemodule { + class DeviceInfoV4l2 : public DeviceInfoImpl { +@@ -49,6 +51,7 @@ class DeviceInfoV4l2 : public DeviceInfoImpl { + + private: + bool IsDeviceNameMatches(const char* name, const char* deviceUniqueIdUTF8); ++ bool IsVideoCaptureDevice(struct v4l2_capability* cap); + + #ifdef WEBRTC_LINUX + void HandleEvent(inotify_event* event, int fd); +-- +2.34.1 + diff --git a/third_party/libwebrtc/moz-patch-stack/0031.patch b/third_party/libwebrtc/moz-patch-stack/0031.patch new file mode 100644 index 0000000000..00f1479dce --- /dev/null +++ b/third_party/libwebrtc/moz-patch-stack/0031.patch @@ -0,0 +1,66 @@ +From: Randell Jesup <rjesup@wgate.com> +Date: Sat, 11 Jul 2020 12:31:00 +0000 +Subject: Bug 1112392 - Move webrtc Tab Sharing to work in e10s/fission + r=dminor + +Also we drop support for an independent-of-scroll/viewport capture, which +the old Tab Sharing supported, for security reasons (and we don't need it). + +Differential Revision: https://phabricator.services.mozilla.com/D80974 +Mercurial Revision: https://hg.mozilla.org/mozilla-central/rev/d57a030e6e3ae9ff56f14e8cc732b0e1d3869858 +--- + modules/desktop_capture/desktop_capturer.cc | 11 +++++++++++ + modules/desktop_capture/desktop_capturer.h | 8 ++++++++ + 2 files changed, 19 insertions(+) + +diff --git a/modules/desktop_capture/desktop_capturer.cc b/modules/desktop_capture/desktop_capturer.cc +index 5211f1acec..4baa93cab9 100644 +--- a/modules/desktop_capture/desktop_capturer.cc ++++ b/modules/desktop_capture/desktop_capturer.cc +@@ -101,6 +101,17 @@ std::unique_ptr<DesktopCapturer> DesktopCapturer::CreateScreenCapturer( + return capturer; + } + ++// static ++std::unique_ptr<DesktopCapturer> DesktopCapturer::CreateTabCapturer( ++ const DesktopCaptureOptions& options) { ++ std::unique_ptr<DesktopCapturer> capturer = CreateRawTabCapturer(options); ++ if (capturer && options.detect_updated_region()) { ++ capturer.reset(new DesktopCapturerDifferWrapper(std::move(capturer))); ++ } ++ ++ return capturer; ++} ++ + #if defined(WEBRTC_USE_PIPEWIRE) || defined(WEBRTC_USE_X11) + bool DesktopCapturer::IsRunningUnderWayland() { + const char* xdg_session_type = getenv("XDG_SESSION_TYPE"); +diff --git a/modules/desktop_capture/desktop_capturer.h b/modules/desktop_capture/desktop_capturer.h +index 6909a57891..cf75004af5 100644 +--- a/modules/desktop_capture/desktop_capturer.h ++++ b/modules/desktop_capture/desktop_capturer.h +@@ -178,6 +178,10 @@ class RTC_EXPORT DesktopCapturer { + static std::unique_ptr<DesktopCapturer> CreateScreenCapturer( + const DesktopCaptureOptions& options); + ++ // Creates a DesktopCapturer instance which targets to capture tab. ++ static std::unique_ptr<DesktopCapturer> CreateTabCapturer( ++ const DesktopCaptureOptions& options); ++ + #if defined(WEBRTC_USE_PIPEWIRE) || defined(WEBRTC_USE_X11) + static bool IsRunningUnderWayland(); + +@@ -203,6 +207,10 @@ class RTC_EXPORT DesktopCapturer { + // capture screens. + static std::unique_ptr<DesktopCapturer> CreateRawScreenCapturer( + const DesktopCaptureOptions& options); ++ ++ // Creates a DesktopCapturer instance which targets to capture tabs ++ static std::unique_ptr<DesktopCapturer> CreateRawTabCapturer( ++ const DesktopCaptureOptions& options); + }; + + } // namespace webrtc +-- +2.34.1 + diff --git a/third_party/libwebrtc/moz-patch-stack/0032.patch b/third_party/libwebrtc/moz-patch-stack/0032.patch new file mode 100644 index 0000000000..6ef6e0e6a8 --- /dev/null +++ b/third_party/libwebrtc/moz-patch-stack/0032.patch @@ -0,0 +1,40 @@ +From: James Willcox <snorp@snorp.net> +Date: Fri, 18 Sep 2020 22:29:00 +0000 +Subject: Bug 1553459 - Migrate to AndroidX r=geckoview-reviewers,agi + +Differential Revision: https://phabricator.services.mozilla.com/D90711 +Mercurial Revision: https://hg.mozilla.org/mozilla-central/rev/cf8917364050fc7d981b190cdc1db5ab57357f9b +--- + sdk/android/src/java/org/webrtc/EglBase14Impl.java | 1 - + sdk/android/src/java/org/webrtc/GlGenericDrawer.java | 3 --- + 2 files changed, 4 deletions(-) + +diff --git a/sdk/android/src/java/org/webrtc/EglBase14Impl.java b/sdk/android/src/java/org/webrtc/EglBase14Impl.java +index e53dda6e4c..caf45b091e 100644 +--- a/sdk/android/src/java/org/webrtc/EglBase14Impl.java ++++ b/sdk/android/src/java/org/webrtc/EglBase14Impl.java +@@ -21,7 +21,6 @@ import android.opengl.GLException; + import android.os.Build; + import android.view.Surface; + import androidx.annotation.Nullable; +-import org.webrtc.EglBase; + + /** + * Holds EGL state and utility methods for handling an EGL14 EGLContext, an EGLDisplay, +diff --git a/sdk/android/src/java/org/webrtc/GlGenericDrawer.java b/sdk/android/src/java/org/webrtc/GlGenericDrawer.java +index b70a3728b9..34144e2f75 100644 +--- a/sdk/android/src/java/org/webrtc/GlGenericDrawer.java ++++ b/sdk/android/src/java/org/webrtc/GlGenericDrawer.java +@@ -14,9 +14,6 @@ import android.opengl.GLES11Ext; + import android.opengl.GLES20; + import androidx.annotation.Nullable; + import java.nio.FloatBuffer; +-import org.webrtc.GlShader; +-import org.webrtc.GlUtil; +-import org.webrtc.RendererCommon; + + /** + * Helper class to implement an instance of RendererCommon.GlDrawer that can accept multiple input +-- +2.34.1 + diff --git a/third_party/libwebrtc/moz-patch-stack/0033.patch b/third_party/libwebrtc/moz-patch-stack/0033.patch new file mode 100644 index 0000000000..88c0ca9151 --- /dev/null +++ b/third_party/libwebrtc/moz-patch-stack/0033.patch @@ -0,0 +1,67 @@ +From: Paul Adenot <paul@paul.cx> +Date: Wed, 4 Nov 2020 13:03:00 +0000 +Subject: Bug 1675042 - Put IR camera last in the device selection list, so + that they are never the default. r=dminor + +Differential Revision: https://phabricator.services.mozilla.com/D95764 +Mercurial Revision: https://hg.mozilla.org/mozilla-central/rev/754900ff2a2b1c32878baf3c76d7c0e8219419ff +--- + sdk/android/api/org/webrtc/Camera1Enumerator.java | 5 +++++ + sdk/android/api/org/webrtc/Camera2Enumerator.java | 12 ++++++++++++ + sdk/android/api/org/webrtc/CameraEnumerator.java | 1 + + 3 files changed, 18 insertions(+) + +diff --git a/sdk/android/api/org/webrtc/Camera1Enumerator.java b/sdk/android/api/org/webrtc/Camera1Enumerator.java +index fb1a21f323..4a1aacdb05 100644 +--- a/sdk/android/api/org/webrtc/Camera1Enumerator.java ++++ b/sdk/android/api/org/webrtc/Camera1Enumerator.java +@@ -63,6 +63,11 @@ public class Camera1Enumerator implements CameraEnumerator { + return info != null && info.facing == android.hardware.Camera.CameraInfo.CAMERA_FACING_BACK; + } + ++ @Override ++ public boolean isInfrared(String deviceName) { ++ return false; ++ } ++ + @Override + public List<CaptureFormat> getSupportedFormats(String deviceName) { + return getSupportedFormats(getCameraIndex(deviceName)); +diff --git a/sdk/android/api/org/webrtc/Camera2Enumerator.java b/sdk/android/api/org/webrtc/Camera2Enumerator.java +index 456d8cd060..44e239ad8e 100644 +--- a/sdk/android/api/org/webrtc/Camera2Enumerator.java ++++ b/sdk/android/api/org/webrtc/Camera2Enumerator.java +@@ -74,6 +74,18 @@ public class Camera2Enumerator implements CameraEnumerator { + == CameraMetadata.LENS_FACING_BACK; + } + ++ @Override ++ public boolean isInfrared(String deviceName) { ++ CameraCharacteristics characteristics = getCameraCharacteristics(deviceName); ++ ++ if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.Q) { ++ Integer colors = characteristics.get(CameraCharacteristics.SENSOR_INFO_COLOR_FILTER_ARRANGEMENT); ++ return colors != null && colors.equals(CameraCharacteristics.SENSOR_INFO_COLOR_FILTER_ARRANGEMENT_NIR); ++ } ++ ++ return false; ++ } ++ + @Nullable + @Override + public List<CaptureFormat> getSupportedFormats(String deviceName) { +diff --git a/sdk/android/api/org/webrtc/CameraEnumerator.java b/sdk/android/api/org/webrtc/CameraEnumerator.java +index dc954b62e0..db34d542c8 100644 +--- a/sdk/android/api/org/webrtc/CameraEnumerator.java ++++ b/sdk/android/api/org/webrtc/CameraEnumerator.java +@@ -18,6 +18,7 @@ public interface CameraEnumerator { + public String[] getDeviceNames(); + public boolean isFrontFacing(String deviceName); + public boolean isBackFacing(String deviceName); ++ public boolean isInfrared(String deviceName); + public List<CaptureFormat> getSupportedFormats(String deviceName); + + public CameraVideoCapturer createCapturer( +-- +2.34.1 + diff --git a/third_party/libwebrtc/moz-patch-stack/0034.patch b/third_party/libwebrtc/moz-patch-stack/0034.patch new file mode 100644 index 0000000000..c704a1ecd2 --- /dev/null +++ b/third_party/libwebrtc/moz-patch-stack/0034.patch @@ -0,0 +1,1099 @@ +From: Michael Froman <mjfroman@mac.com> +Date: Mon, 14 Feb 2022 17:50:52 -0600 +Subject: (fix) reconcile github with what was actually committed in + moz-central + +--- + .../signal_processing/filter_ar_fast_q12.c | 2 +- + .../linux/wayland/base_capturer_pipewire.cc | 11 +- + .../video_capture/linux/device_info_v4l2.cc | 1 + + rtc_base/trace_event.h | 1025 +---------------- + 4 files changed, 15 insertions(+), 1024 deletions(-) + +diff --git a/common_audio/signal_processing/filter_ar_fast_q12.c b/common_audio/signal_processing/filter_ar_fast_q12.c +index 8b8bdb1af5..9010f1ce82 100644 +--- a/common_audio/signal_processing/filter_ar_fast_q12.c ++++ b/common_audio/signal_processing/filter_ar_fast_q12.c +@@ -8,7 +8,7 @@ + * be found in the AUTHORS file in the root of the source tree. + */ + +-#include "stddef.h" ++#include <stddef.h> + + #include "rtc_base/checks.h" + #include "common_audio/signal_processing/include/signal_processing_library.h" +diff --git a/modules/desktop_capture/linux/wayland/base_capturer_pipewire.cc b/modules/desktop_capture/linux/wayland/base_capturer_pipewire.cc +index cf4f7dc9aa..dae2b70510 100644 +--- a/modules/desktop_capture/linux/wayland/base_capturer_pipewire.cc ++++ b/modules/desktop_capture/linux/wayland/base_capturer_pipewire.cc +@@ -165,6 +165,15 @@ void BaseCapturerPipeWire::CaptureFrame() { + callback_->OnCaptureResult(Result::SUCCESS, std::move(frame)); + } + ++// Keep in sync with defines at browser/actors/WebRTCParent.jsm ++// With PipeWire we can't select which system resource is shared so ++// we don't create a window/screen list. Instead we place these constants ++// as window name/id so frontend code can identify PipeWire backend ++// and does not try to create screen/window preview. ++ ++#define PIPEWIRE_ID 0xaffffff ++#define PIPEWIRE_NAME "####_PIPEWIRE_PORTAL_####" ++ + bool BaseCapturerPipeWire::GetSourceList(SourceList* sources) { + RTC_DCHECK(sources->size() == 0); + // List of available screens is already presented by the xdg-desktop-portal, +@@ -181,7 +190,7 @@ bool BaseCapturerPipeWire::GetSourceList(SourceList* sources) { + bool BaseCapturerPipeWire::SelectSource(SourceId id) { + // Screen selection is handled by the xdg-desktop-portal. + selected_source_id_ = id; +- return true; ++ return id == PIPEWIRE_ID; + } + + DelegatedSourceListController* +diff --git a/modules/video_capture/linux/device_info_v4l2.cc b/modules/video_capture/linux/device_info_v4l2.cc +index d506f3a448..e8abcdda78 100644 +--- a/modules/video_capture/linux/device_info_v4l2.cc ++++ b/modules/video_capture/linux/device_info_v4l2.cc +@@ -320,6 +320,7 @@ int32_t DeviceInfoV4l2::CreateCapabilityMap(const char* deviceUniqueIdUTF8) { + if (ioctl(fd, VIDIOC_QUERYCAP, &cap) == 0) { + // skip devices without video capture capability + if (!IsVideoCaptureDevice(&cap)) { ++ close(fd); + continue; + } + +diff --git a/rtc_base/trace_event.h b/rtc_base/trace_event.h +index e9a5c4c6f2..b34df0c93f 100644 +--- a/rtc_base/trace_event.h ++++ b/rtc_base/trace_event.h +@@ -1,1022 +1,3 @@ +-// Copyright (c) 2012 The Chromium Authors. All rights reserved. +-// Use of this source code is governed by a BSD-style license that can be +-// found in the LICENSE file under third_party_mods/chromium or at: +-// http://src.chromium.org/svn/trunk/src/LICENSE +- +-#ifndef RTC_BASE_TRACE_EVENT_H_ +-#define RTC_BASE_TRACE_EVENT_H_ +- +-#include <string> +- +-#include "rtc_base/event_tracer.h" +- +-#if defined(TRACE_EVENT0) +-#error "Another copy of trace_event.h has already been included." +-#endif +- +-#if defined(RTC_DISABLE_TRACE_EVENTS) +-#define RTC_TRACE_EVENTS_ENABLED 0 +-#else +-#define RTC_TRACE_EVENTS_ENABLED 1 +-#endif +- +-// Type values for identifying types in the TraceValue union. +-#define TRACE_VALUE_TYPE_BOOL (static_cast<unsigned char>(1)) +-#define TRACE_VALUE_TYPE_UINT (static_cast<unsigned char>(2)) +-#define TRACE_VALUE_TYPE_INT (static_cast<unsigned char>(3)) +-#define TRACE_VALUE_TYPE_DOUBLE (static_cast<unsigned char>(4)) +-#define TRACE_VALUE_TYPE_POINTER (static_cast<unsigned char>(5)) +-#define TRACE_VALUE_TYPE_STRING (static_cast<unsigned char>(6)) +-#define TRACE_VALUE_TYPE_COPY_STRING (static_cast<unsigned char>(7)) +- +-#if RTC_TRACE_EVENTS_ENABLED +- +-// Extracted from Chromium's src/base/debug/trace_event.h. +- +-// This header is designed to give you trace_event macros without specifying +-// how the events actually get collected and stored. If you need to expose trace +-// event to some other universe, you can copy-and-paste this file, +-// implement the TRACE_EVENT_API macros, and do any other necessary fixup for +-// the target platform. The end result is that multiple libraries can funnel +-// events through to a shared trace event collector. +- +-// Trace events are for tracking application performance and resource usage. +-// Macros are provided to track: +-// Begin and end of function calls +-// Counters +-// +-// Events are issued against categories. Whereas RTC_LOG's +-// categories are statically defined, TRACE categories are created +-// implicitly with a string. For example: +-// TRACE_EVENT_INSTANT0("MY_SUBSYSTEM", "SomeImportantEvent") +-// +-// Events can be INSTANT, or can be pairs of BEGIN and END in the same scope: +-// TRACE_EVENT_BEGIN0("MY_SUBSYSTEM", "SomethingCostly") +-// doSomethingCostly() +-// TRACE_EVENT_END0("MY_SUBSYSTEM", "SomethingCostly") +-// Note: our tools can't always determine the correct BEGIN/END pairs unless +-// these are used in the same scope. Use ASYNC_BEGIN/ASYNC_END macros if you +-// need them to be in separate scopes. +-// +-// A common use case is to trace entire function scopes. This +-// issues a trace BEGIN and END automatically: +-// void doSomethingCostly() { +-// TRACE_EVENT0("MY_SUBSYSTEM", "doSomethingCostly"); +-// ... +-// } +-// +-// Additional parameters can be associated with an event: +-// void doSomethingCostly2(int howMuch) { +-// TRACE_EVENT1("MY_SUBSYSTEM", "doSomethingCostly", +-// "howMuch", howMuch); +-// ... +-// } +-// +-// The trace system will automatically add to this information the +-// current process id, thread id, and a timestamp in microseconds. +-// +-// To trace an asynchronous procedure such as an IPC send/receive, use +-// ASYNC_BEGIN and ASYNC_END: +-// [single threaded sender code] +-// static int send_count = 0; +-// ++send_count; +-// TRACE_EVENT_ASYNC_BEGIN0("ipc", "message", send_count); +-// Send(new MyMessage(send_count)); +-// [receive code] +-// void OnMyMessage(send_count) { +-// TRACE_EVENT_ASYNC_END0("ipc", "message", send_count); +-// } +-// The third parameter is a unique ID to match ASYNC_BEGIN/ASYNC_END pairs. +-// ASYNC_BEGIN and ASYNC_END can occur on any thread of any traced process. +-// Pointers can be used for the ID parameter, and they will be mangled +-// internally so that the same pointer on two different processes will not +-// match. For example: +-// class MyTracedClass { +-// public: +-// MyTracedClass() { +-// TRACE_EVENT_ASYNC_BEGIN0("category", "MyTracedClass", this); +-// } +-// ~MyTracedClass() { +-// TRACE_EVENT_ASYNC_END0("category", "MyTracedClass", this); +-// } +-// } +-// +-// Trace event also supports counters, which is a way to track a quantity +-// as it varies over time. Counters are created with the following macro: +-// TRACE_COUNTER1("MY_SUBSYSTEM", "myCounter", g_myCounterValue); +-// +-// Counters are process-specific. The macro itself can be issued from any +-// thread, however. +-// +-// Sometimes, you want to track two counters at once. You can do this with two +-// counter macros: +-// TRACE_COUNTER1("MY_SUBSYSTEM", "myCounter0", g_myCounterValue[0]); +-// TRACE_COUNTER1("MY_SUBSYSTEM", "myCounter1", g_myCounterValue[1]); +-// Or you can do it with a combined macro: +-// TRACE_COUNTER2("MY_SUBSYSTEM", "myCounter", +-// "bytesPinned", g_myCounterValue[0], +-// "bytesAllocated", g_myCounterValue[1]); +-// This indicates to the tracing UI that these counters should be displayed +-// in a single graph, as a summed area chart. +-// +-// Since counters are in a global namespace, you may want to disembiguate with a +-// unique ID, by using the TRACE_COUNTER_ID* variations. +-// +-// By default, trace collection is compiled in, but turned off at runtime. +-// Collecting trace data is the responsibility of the embedding +-// application. In Chrome's case, navigating to about:tracing will turn on +-// tracing and display data collected across all active processes. +-// +-// +-// Memory scoping note: +-// Tracing copies the pointers, not the string content, of the strings passed +-// in for category, name, and arg_names. Thus, the following code will +-// cause problems: +-// char* str = strdup("impprtantName"); +-// TRACE_EVENT_INSTANT0("SUBSYSTEM", str); // BAD! +-// free(str); // Trace system now has dangling pointer +-// +-// To avoid this issue with the `name` and `arg_name` parameters, use the +-// TRACE_EVENT_COPY_XXX overloads of the macros at additional runtime overhead. +-// Notes: The category must always be in a long-lived char* (i.e. static const). +-// The `arg_values`, when used, are always deep copied with the _COPY +-// macros. +-// +-// When are string argument values copied: +-// const char* arg_values are only referenced by default: +-// TRACE_EVENT1("category", "name", +-// "arg1", "literal string is only referenced"); +-// Use TRACE_STR_COPY to force copying of a const char*: +-// TRACE_EVENT1("category", "name", +-// "arg1", TRACE_STR_COPY("string will be copied")); +-// std::string arg_values are always copied: +-// TRACE_EVENT1("category", "name", +-// "arg1", std::string("string will be copied")); +-// +-// +-// Thread Safety: +-// Thread safety is provided by methods defined in event_tracer.h. See the file +-// for details. +- +-// By default, const char* argument values are assumed to have long-lived scope +-// and will not be copied. Use this macro to force a const char* to be copied. +-#define TRACE_STR_COPY(str) \ +- webrtc::trace_event_internal::TraceStringWithCopy(str) +- +-// This will mark the trace event as disabled by default. The user will need +-// to explicitly enable the event. +-#define TRACE_DISABLED_BY_DEFAULT(name) "disabled-by-default-" name +- +-// By default, uint64 ID argument values are not mangled with the Process ID in +-// TRACE_EVENT_ASYNC macros. Use this macro to force Process ID mangling. +-#define TRACE_ID_MANGLE(id) \ +- webrtc::trace_event_internal::TraceID::ForceMangle(id) +- +-// Records a pair of begin and end events called "name" for the current +-// scope, with 0, 1 or 2 associated arguments. If the category is not +-// enabled, then this does nothing. +-// - category and name strings must have application lifetime (statics or +-// literals). They may not include " chars. +-#define TRACE_EVENT0(category, name) \ +- INTERNAL_TRACE_EVENT_ADD_SCOPED(category, name) +-#define TRACE_EVENT1(category, name, arg1_name, arg1_val) \ +- INTERNAL_TRACE_EVENT_ADD_SCOPED(category, name, arg1_name, arg1_val) +-#define TRACE_EVENT2(category, name, arg1_name, arg1_val, arg2_name, arg2_val) \ +- INTERNAL_TRACE_EVENT_ADD_SCOPED(category, name, arg1_name, arg1_val, \ +- arg2_name, arg2_val) +- +-// Records a single event called "name" immediately, with 0, 1 or 2 +-// associated arguments. If the category is not enabled, then this +-// does nothing. +-// - category and name strings must have application lifetime (statics or +-// literals). They may not include " chars. +-#define TRACE_EVENT_INSTANT0(category, name) \ +- INTERNAL_TRACE_EVENT_ADD(TRACE_EVENT_PHASE_INSTANT, \ +- category, name, TRACE_EVENT_FLAG_NONE) +-#define TRACE_EVENT_INSTANT1(category, name, arg1_name, arg1_val) \ +- INTERNAL_TRACE_EVENT_ADD(TRACE_EVENT_PHASE_INSTANT, \ +- category, name, TRACE_EVENT_FLAG_NONE, arg1_name, arg1_val) +-#define TRACE_EVENT_INSTANT2(category, name, arg1_name, arg1_val, \ +- arg2_name, arg2_val) \ +- INTERNAL_TRACE_EVENT_ADD(TRACE_EVENT_PHASE_INSTANT, \ +- category, name, TRACE_EVENT_FLAG_NONE, arg1_name, arg1_val, \ +- arg2_name, arg2_val) +-#define TRACE_EVENT_COPY_INSTANT0(category, name) \ +- INTERNAL_TRACE_EVENT_ADD(TRACE_EVENT_PHASE_INSTANT, \ +- category, name, TRACE_EVENT_FLAG_COPY) +-#define TRACE_EVENT_COPY_INSTANT1(category, name, arg1_name, arg1_val) \ +- INTERNAL_TRACE_EVENT_ADD(TRACE_EVENT_PHASE_INSTANT, \ +- category, name, TRACE_EVENT_FLAG_COPY, arg1_name, arg1_val) +-#define TRACE_EVENT_COPY_INSTANT2(category, name, arg1_name, arg1_val, \ +- arg2_name, arg2_val) \ +- INTERNAL_TRACE_EVENT_ADD(TRACE_EVENT_PHASE_INSTANT, \ +- category, name, TRACE_EVENT_FLAG_COPY, arg1_name, arg1_val, \ +- arg2_name, arg2_val) +- +-// Records a single BEGIN event called "name" immediately, with 0, 1 or 2 +-// associated arguments. If the category is not enabled, then this +-// does nothing. +-// - category and name strings must have application lifetime (statics or +-// literals). They may not include " chars. +-#define TRACE_EVENT_BEGIN0(category, name) \ +- INTERNAL_TRACE_EVENT_ADD(TRACE_EVENT_PHASE_BEGIN, \ +- category, name, TRACE_EVENT_FLAG_NONE) +-#define TRACE_EVENT_BEGIN1(category, name, arg1_name, arg1_val) \ +- INTERNAL_TRACE_EVENT_ADD(TRACE_EVENT_PHASE_BEGIN, \ +- category, name, TRACE_EVENT_FLAG_NONE, arg1_name, arg1_val) +-#define TRACE_EVENT_BEGIN2(category, name, arg1_name, arg1_val, \ +- arg2_name, arg2_val) \ +- INTERNAL_TRACE_EVENT_ADD(TRACE_EVENT_PHASE_BEGIN, \ +- category, name, TRACE_EVENT_FLAG_NONE, arg1_name, arg1_val, \ +- arg2_name, arg2_val) +-#define TRACE_EVENT_COPY_BEGIN0(category, name) \ +- INTERNAL_TRACE_EVENT_ADD(TRACE_EVENT_PHASE_BEGIN, \ +- category, name, TRACE_EVENT_FLAG_COPY) +-#define TRACE_EVENT_COPY_BEGIN1(category, name, arg1_name, arg1_val) \ +- INTERNAL_TRACE_EVENT_ADD(TRACE_EVENT_PHASE_BEGIN, \ +- category, name, TRACE_EVENT_FLAG_COPY, arg1_name, arg1_val) +-#define TRACE_EVENT_COPY_BEGIN2(category, name, arg1_name, arg1_val, \ +- arg2_name, arg2_val) \ +- INTERNAL_TRACE_EVENT_ADD(TRACE_EVENT_PHASE_BEGIN, \ +- category, name, TRACE_EVENT_FLAG_COPY, arg1_name, arg1_val, \ +- arg2_name, arg2_val) +- +-// Records a single END event for "name" immediately. If the category +-// is not enabled, then this does nothing. +-// - category and name strings must have application lifetime (statics or +-// literals). They may not include " chars. +-#define TRACE_EVENT_END0(category, name) \ +- INTERNAL_TRACE_EVENT_ADD(TRACE_EVENT_PHASE_END, \ +- category, name, TRACE_EVENT_FLAG_NONE) +-#define TRACE_EVENT_END1(category, name, arg1_name, arg1_val) \ +- INTERNAL_TRACE_EVENT_ADD(TRACE_EVENT_PHASE_END, \ +- category, name, TRACE_EVENT_FLAG_NONE, arg1_name, arg1_val) +-#define TRACE_EVENT_END2(category, name, arg1_name, arg1_val, \ +- arg2_name, arg2_val) \ +- INTERNAL_TRACE_EVENT_ADD(TRACE_EVENT_PHASE_END, \ +- category, name, TRACE_EVENT_FLAG_NONE, arg1_name, arg1_val, \ +- arg2_name, arg2_val) +-#define TRACE_EVENT_COPY_END0(category, name) \ +- INTERNAL_TRACE_EVENT_ADD(TRACE_EVENT_PHASE_END, \ +- category, name, TRACE_EVENT_FLAG_COPY) +-#define TRACE_EVENT_COPY_END1(category, name, arg1_name, arg1_val) \ +- INTERNAL_TRACE_EVENT_ADD(TRACE_EVENT_PHASE_END, \ +- category, name, TRACE_EVENT_FLAG_COPY, arg1_name, arg1_val) +-#define TRACE_EVENT_COPY_END2(category, name, arg1_name, arg1_val, \ +- arg2_name, arg2_val) \ +- INTERNAL_TRACE_EVENT_ADD(TRACE_EVENT_PHASE_END, \ +- category, name, TRACE_EVENT_FLAG_COPY, arg1_name, arg1_val, \ +- arg2_name, arg2_val) +- +-// Records the value of a counter called "name" immediately. Value +-// must be representable as a 32 bit integer. +-// - category and name strings must have application lifetime (statics or +-// literals). They may not include " chars. +-#define TRACE_COUNTER1(category, name, value) \ +- INTERNAL_TRACE_EVENT_ADD(TRACE_EVENT_PHASE_COUNTER, \ +- category, name, TRACE_EVENT_FLAG_NONE, \ +- "value", static_cast<int>(value)) +-#define TRACE_COPY_COUNTER1(category, name, value) \ +- INTERNAL_TRACE_EVENT_ADD(TRACE_EVENT_PHASE_COUNTER, \ +- category, name, TRACE_EVENT_FLAG_COPY, \ +- "value", static_cast<int>(value)) +- +-// Records the values of a multi-parted counter called "name" immediately. +-// The UI will treat value1 and value2 as parts of a whole, displaying their +-// values as a stacked-bar chart. +-// - category and name strings must have application lifetime (statics or +-// literals). They may not include " chars. +-#define TRACE_COUNTER2(category, name, value1_name, value1_val, \ +- value2_name, value2_val) \ +- INTERNAL_TRACE_EVENT_ADD(TRACE_EVENT_PHASE_COUNTER, \ +- category, name, TRACE_EVENT_FLAG_NONE, \ +- value1_name, static_cast<int>(value1_val), \ +- value2_name, static_cast<int>(value2_val)) +-#define TRACE_COPY_COUNTER2(category, name, value1_name, value1_val, \ +- value2_name, value2_val) \ +- INTERNAL_TRACE_EVENT_ADD(TRACE_EVENT_PHASE_COUNTER, \ +- category, name, TRACE_EVENT_FLAG_COPY, \ +- value1_name, static_cast<int>(value1_val), \ +- value2_name, static_cast<int>(value2_val)) +- +-// Records the value of a counter called "name" immediately. Value +-// must be representable as a 32 bit integer. +-// - category and name strings must have application lifetime (statics or +-// literals). They may not include " chars. +-// - `id` is used to disambiguate counters with the same name. It must either +-// be a pointer or an integer value up to 64 bits. If it's a pointer, the bits +-// will be xored with a hash of the process ID so that the same pointer on +-// two different processes will not collide. +-#define TRACE_COUNTER_ID1(category, name, id, value) \ +- INTERNAL_TRACE_EVENT_ADD_WITH_ID(TRACE_EVENT_PHASE_COUNTER, \ +- category, name, id, TRACE_EVENT_FLAG_NONE, \ +- "value", static_cast<int>(value)) +-#define TRACE_COPY_COUNTER_ID1(category, name, id, value) \ +- INTERNAL_TRACE_EVENT_ADD_WITH_ID(TRACE_EVENT_PHASE_COUNTER, \ +- category, name, id, TRACE_EVENT_FLAG_COPY, \ +- "value", static_cast<int>(value)) +- +-// Records the values of a multi-parted counter called "name" immediately. +-// The UI will treat value1 and value2 as parts of a whole, displaying their +-// values as a stacked-bar chart. +-// - category and name strings must have application lifetime (statics or +-// literals). They may not include " chars. +-// - `id` is used to disambiguate counters with the same name. It must either +-// be a pointer or an integer value up to 64 bits. If it's a pointer, the bits +-// will be xored with a hash of the process ID so that the same pointer on +-// two different processes will not collide. +-#define TRACE_COUNTER_ID2(category, name, id, value1_name, value1_val, \ +- value2_name, value2_val) \ +- INTERNAL_TRACE_EVENT_ADD_WITH_ID(TRACE_EVENT_PHASE_COUNTER, \ +- category, name, id, TRACE_EVENT_FLAG_NONE, \ +- value1_name, static_cast<int>(value1_val), \ +- value2_name, static_cast<int>(value2_val)) +-#define TRACE_COPY_COUNTER_ID2(category, name, id, value1_name, value1_val, \ +- value2_name, value2_val) \ +- INTERNAL_TRACE_EVENT_ADD_WITH_ID(TRACE_EVENT_PHASE_COUNTER, \ +- category, name, id, TRACE_EVENT_FLAG_COPY, \ +- value1_name, static_cast<int>(value1_val), \ +- value2_name, static_cast<int>(value2_val)) +- +- +-// Records a single ASYNC_BEGIN event called "name" immediately, with 0, 1 or 2 +-// associated arguments. If the category is not enabled, then this +-// does nothing. +-// - category and name strings must have application lifetime (statics or +-// literals). They may not include " chars. +-// - `id` is used to match the ASYNC_BEGIN event with the ASYNC_END event. ASYNC +-// events are considered to match if their category, name and id values all +-// match. `id` must either be a pointer or an integer value up to 64 bits. If +-// it's a pointer, the bits will be xored with a hash of the process ID so +-// that the same pointer on two different processes will not collide. +-// An asynchronous operation can consist of multiple phases. The first phase is +-// defined by the ASYNC_BEGIN calls. Additional phases can be defined using the +-// ASYNC_STEP macros. When the operation completes, call ASYNC_END. +-// An ASYNC trace typically occur on a single thread (if not, they will only be +-// drawn on the thread defined in the ASYNC_BEGIN event), but all events in that +-// operation must use the same `name` and `id`. Each event can have its own +-// args. +-#define TRACE_EVENT_ASYNC_BEGIN0(category, name, id) \ +- INTERNAL_TRACE_EVENT_ADD_WITH_ID(TRACE_EVENT_PHASE_ASYNC_BEGIN, \ +- category, name, id, TRACE_EVENT_FLAG_NONE) +-#define TRACE_EVENT_ASYNC_BEGIN1(category, name, id, arg1_name, arg1_val) \ +- INTERNAL_TRACE_EVENT_ADD_WITH_ID(TRACE_EVENT_PHASE_ASYNC_BEGIN, \ +- category, name, id, TRACE_EVENT_FLAG_NONE, arg1_name, arg1_val) +-#define TRACE_EVENT_ASYNC_BEGIN2(category, name, id, arg1_name, arg1_val, \ +- arg2_name, arg2_val) \ +- INTERNAL_TRACE_EVENT_ADD_WITH_ID(TRACE_EVENT_PHASE_ASYNC_BEGIN, \ +- category, name, id, TRACE_EVENT_FLAG_NONE, \ +- arg1_name, arg1_val, arg2_name, arg2_val) +-#define TRACE_EVENT_COPY_ASYNC_BEGIN0(category, name, id) \ +- INTERNAL_TRACE_EVENT_ADD_WITH_ID(TRACE_EVENT_PHASE_ASYNC_BEGIN, \ +- category, name, id, TRACE_EVENT_FLAG_COPY) +-#define TRACE_EVENT_COPY_ASYNC_BEGIN1(category, name, id, arg1_name, arg1_val) \ +- INTERNAL_TRACE_EVENT_ADD_WITH_ID(TRACE_EVENT_PHASE_ASYNC_BEGIN, \ +- category, name, id, TRACE_EVENT_FLAG_COPY, \ +- arg1_name, arg1_val) +-#define TRACE_EVENT_COPY_ASYNC_BEGIN2(category, name, id, arg1_name, arg1_val, \ +- arg2_name, arg2_val) \ +- INTERNAL_TRACE_EVENT_ADD_WITH_ID(TRACE_EVENT_PHASE_ASYNC_BEGIN, \ +- category, name, id, TRACE_EVENT_FLAG_COPY, \ +- arg1_name, arg1_val, arg2_name, arg2_val) +- +-// Records a single ASYNC_STEP event for `step` immediately. If the category +-// is not enabled, then this does nothing. The `name` and `id` must match the +-// ASYNC_BEGIN event above. The `step` param identifies this step within the +-// async event. This should be called at the beginning of the next phase of an +-// asynchronous operation. +-#define TRACE_EVENT_ASYNC_STEP0(category, name, id, step) \ +- INTERNAL_TRACE_EVENT_ADD_WITH_ID(TRACE_EVENT_PHASE_ASYNC_STEP, \ +- category, name, id, TRACE_EVENT_FLAG_NONE, "step", step) +-#define TRACE_EVENT_ASYNC_STEP1(category, name, id, step, \ +- arg1_name, arg1_val) \ +- INTERNAL_TRACE_EVENT_ADD_WITH_ID(TRACE_EVENT_PHASE_ASYNC_STEP, \ +- category, name, id, TRACE_EVENT_FLAG_NONE, "step", step, \ +- arg1_name, arg1_val) +-#define TRACE_EVENT_COPY_ASYNC_STEP0(category, name, id, step) \ +- INTERNAL_TRACE_EVENT_ADD_WITH_ID(TRACE_EVENT_PHASE_ASYNC_STEP, \ +- category, name, id, TRACE_EVENT_FLAG_COPY, "step", step) +-#define TRACE_EVENT_COPY_ASYNC_STEP1(category, name, id, step, \ +- arg1_name, arg1_val) \ +- INTERNAL_TRACE_EVENT_ADD_WITH_ID(TRACE_EVENT_PHASE_ASYNC_STEP, \ +- category, name, id, TRACE_EVENT_FLAG_COPY, "step", step, \ +- arg1_name, arg1_val) +- +-// Records a single ASYNC_END event for "name" immediately. If the category +-// is not enabled, then this does nothing. +-#define TRACE_EVENT_ASYNC_END0(category, name, id) \ +- INTERNAL_TRACE_EVENT_ADD_WITH_ID(TRACE_EVENT_PHASE_ASYNC_END, \ +- category, name, id, TRACE_EVENT_FLAG_NONE) +-#define TRACE_EVENT_ASYNC_END1(category, name, id, arg1_name, arg1_val) \ +- INTERNAL_TRACE_EVENT_ADD_WITH_ID(TRACE_EVENT_PHASE_ASYNC_END, \ +- category, name, id, TRACE_EVENT_FLAG_NONE, arg1_name, arg1_val) +-#define TRACE_EVENT_ASYNC_END2(category, name, id, arg1_name, arg1_val, \ +- arg2_name, arg2_val) \ +- INTERNAL_TRACE_EVENT_ADD_WITH_ID(TRACE_EVENT_PHASE_ASYNC_END, \ +- category, name, id, TRACE_EVENT_FLAG_NONE, \ +- arg1_name, arg1_val, arg2_name, arg2_val) +-#define TRACE_EVENT_COPY_ASYNC_END0(category, name, id) \ +- INTERNAL_TRACE_EVENT_ADD_WITH_ID(TRACE_EVENT_PHASE_ASYNC_END, \ +- category, name, id, TRACE_EVENT_FLAG_COPY) +-#define TRACE_EVENT_COPY_ASYNC_END1(category, name, id, arg1_name, arg1_val) \ +- INTERNAL_TRACE_EVENT_ADD_WITH_ID(TRACE_EVENT_PHASE_ASYNC_END, \ +- category, name, id, TRACE_EVENT_FLAG_COPY, \ +- arg1_name, arg1_val) +-#define TRACE_EVENT_COPY_ASYNC_END2(category, name, id, arg1_name, arg1_val, \ +- arg2_name, arg2_val) \ +- INTERNAL_TRACE_EVENT_ADD_WITH_ID(TRACE_EVENT_PHASE_ASYNC_END, \ +- category, name, id, TRACE_EVENT_FLAG_COPY, \ +- arg1_name, arg1_val, arg2_name, arg2_val) +- +- +-// Records a single FLOW_BEGIN event called "name" immediately, with 0, 1 or 2 +-// associated arguments. If the category is not enabled, then this +-// does nothing. +-// - category and name strings must have application lifetime (statics or +-// literals). They may not include " chars. +-// - `id` is used to match the FLOW_BEGIN event with the FLOW_END event. FLOW +-// events are considered to match if their category, name and id values all +-// match. `id` must either be a pointer or an integer value up to 64 bits. If +-// it's a pointer, the bits will be xored with a hash of the process ID so +-// that the same pointer on two different processes will not collide. +-// FLOW events are different from ASYNC events in how they are drawn by the +-// tracing UI. A FLOW defines asynchronous data flow, such as posting a task +-// (FLOW_BEGIN) and later executing that task (FLOW_END). Expect FLOWs to be +-// drawn as lines or arrows from FLOW_BEGIN scopes to FLOW_END scopes. Similar +-// to ASYNC, a FLOW can consist of multiple phases. The first phase is defined +-// by the FLOW_BEGIN calls. Additional phases can be defined using the FLOW_STEP +-// macros. When the operation completes, call FLOW_END. An async operation can +-// span threads and processes, but all events in that operation must use the +-// same `name` and `id`. Each event can have its own args. +-#define TRACE_EVENT_FLOW_BEGIN0(category, name, id) \ +- INTERNAL_TRACE_EVENT_ADD_WITH_ID(TRACE_EVENT_PHASE_FLOW_BEGIN, \ +- category, name, id, TRACE_EVENT_FLAG_NONE) +-#define TRACE_EVENT_FLOW_BEGIN1(category, name, id, arg1_name, arg1_val) \ +- INTERNAL_TRACE_EVENT_ADD_WITH_ID(TRACE_EVENT_PHASE_FLOW_BEGIN, \ +- category, name, id, TRACE_EVENT_FLAG_NONE, arg1_name, arg1_val) +-#define TRACE_EVENT_FLOW_BEGIN2(category, name, id, arg1_name, arg1_val, \ +- arg2_name, arg2_val) \ +- INTERNAL_TRACE_EVENT_ADD_WITH_ID(TRACE_EVENT_PHASE_FLOW_BEGIN, \ +- category, name, id, TRACE_EVENT_FLAG_NONE, \ +- arg1_name, arg1_val, arg2_name, arg2_val) +-#define TRACE_EVENT_COPY_FLOW_BEGIN0(category, name, id) \ +- INTERNAL_TRACE_EVENT_ADD_WITH_ID(TRACE_EVENT_PHASE_FLOW_BEGIN, \ +- category, name, id, TRACE_EVENT_FLAG_COPY) +-#define TRACE_EVENT_COPY_FLOW_BEGIN1(category, name, id, arg1_name, arg1_val) \ +- INTERNAL_TRACE_EVENT_ADD_WITH_ID(TRACE_EVENT_PHASE_FLOW_BEGIN, \ +- category, name, id, TRACE_EVENT_FLAG_COPY, \ +- arg1_name, arg1_val) +-#define TRACE_EVENT_COPY_FLOW_BEGIN2(category, name, id, arg1_name, arg1_val, \ +- arg2_name, arg2_val) \ +- INTERNAL_TRACE_EVENT_ADD_WITH_ID(TRACE_EVENT_PHASE_FLOW_BEGIN, \ +- category, name, id, TRACE_EVENT_FLAG_COPY, \ +- arg1_name, arg1_val, arg2_name, arg2_val) +- +-// Records a single FLOW_STEP event for `step` immediately. If the category +-// is not enabled, then this does nothing. The `name` and `id` must match the +-// FLOW_BEGIN event above. The `step` param identifies this step within the +-// async event. This should be called at the beginning of the next phase of an +-// asynchronous operation. +-#define TRACE_EVENT_FLOW_STEP0(category, name, id, step) \ +- INTERNAL_TRACE_EVENT_ADD_WITH_ID(TRACE_EVENT_PHASE_FLOW_STEP, \ +- category, name, id, TRACE_EVENT_FLAG_NONE, "step", step) +-#define TRACE_EVENT_FLOW_STEP1(category, name, id, step, \ +- arg1_name, arg1_val) \ +- INTERNAL_TRACE_EVENT_ADD_WITH_ID(TRACE_EVENT_PHASE_FLOW_STEP, \ +- category, name, id, TRACE_EVENT_FLAG_NONE, "step", step, \ +- arg1_name, arg1_val) +-#define TRACE_EVENT_COPY_FLOW_STEP0(category, name, id, step) \ +- INTERNAL_TRACE_EVENT_ADD_WITH_ID(TRACE_EVENT_PHASE_FLOW_STEP, \ +- category, name, id, TRACE_EVENT_FLAG_COPY, "step", step) +-#define TRACE_EVENT_COPY_FLOW_STEP1(category, name, id, step, \ +- arg1_name, arg1_val) \ +- INTERNAL_TRACE_EVENT_ADD_WITH_ID(TRACE_EVENT_PHASE_FLOW_STEP, \ +- category, name, id, TRACE_EVENT_FLAG_COPY, "step", step, \ +- arg1_name, arg1_val) +- +-// Records a single FLOW_END event for "name" immediately. If the category +-// is not enabled, then this does nothing. +-#define TRACE_EVENT_FLOW_END0(category, name, id) \ +- INTERNAL_TRACE_EVENT_ADD_WITH_ID(TRACE_EVENT_PHASE_FLOW_END, \ +- category, name, id, TRACE_EVENT_FLAG_NONE) +-#define TRACE_EVENT_FLOW_END1(category, name, id, arg1_name, arg1_val) \ +- INTERNAL_TRACE_EVENT_ADD_WITH_ID(TRACE_EVENT_PHASE_FLOW_END, \ +- category, name, id, TRACE_EVENT_FLAG_NONE, arg1_name, arg1_val) +-#define TRACE_EVENT_FLOW_END2(category, name, id, arg1_name, arg1_val, \ +- arg2_name, arg2_val) \ +- INTERNAL_TRACE_EVENT_ADD_WITH_ID(TRACE_EVENT_PHASE_FLOW_END, \ +- category, name, id, TRACE_EVENT_FLAG_NONE, \ +- arg1_name, arg1_val, arg2_name, arg2_val) +-#define TRACE_EVENT_COPY_FLOW_END0(category, name, id) \ +- INTERNAL_TRACE_EVENT_ADD_WITH_ID(TRACE_EVENT_PHASE_FLOW_END, \ +- category, name, id, TRACE_EVENT_FLAG_COPY) +-#define TRACE_EVENT_COPY_FLOW_END1(category, name, id, arg1_name, arg1_val) \ +- INTERNAL_TRACE_EVENT_ADD_WITH_ID(TRACE_EVENT_PHASE_FLOW_END, \ +- category, name, id, TRACE_EVENT_FLAG_COPY, \ +- arg1_name, arg1_val) +-#define TRACE_EVENT_COPY_FLOW_END2(category, name, id, arg1_name, arg1_val, \ +- arg2_name, arg2_val) \ +- INTERNAL_TRACE_EVENT_ADD_WITH_ID(TRACE_EVENT_PHASE_FLOW_END, \ +- category, name, id, TRACE_EVENT_FLAG_COPY, \ +- arg1_name, arg1_val, arg2_name, arg2_val) +- +- +-//////////////////////////////////////////////////////////////////////////////// +-// Implementation specific tracing API definitions. +- +-// Get a pointer to the enabled state of the given trace category. Only +-// long-lived literal strings should be given as the category name. The returned +-// pointer can be held permanently in a local static for example. If the +-// unsigned char is non-zero, tracing is enabled. If tracing is enabled, +-// TRACE_EVENT_API_ADD_TRACE_EVENT can be called. It's OK if tracing is disabled +-// between the load of the tracing state and the call to +-// TRACE_EVENT_API_ADD_TRACE_EVENT, because this flag only provides an early out +-// for best performance when tracing is disabled. +-// const unsigned char* +-// TRACE_EVENT_API_GET_CATEGORY_ENABLED(const char* category_name) +-#define TRACE_EVENT_API_GET_CATEGORY_ENABLED \ +- webrtc::EventTracer::GetCategoryEnabled +- +-// Add a trace event to the platform tracing system. +-// void TRACE_EVENT_API_ADD_TRACE_EVENT( +-// char phase, +-// const unsigned char* category_enabled, +-// const char* name, +-// unsigned long long id, +-// int num_args, +-// const char** arg_names, +-// const unsigned char* arg_types, +-// const unsigned long long* arg_values, +-// unsigned char flags) +-#define TRACE_EVENT_API_ADD_TRACE_EVENT webrtc::EventTracer::AddTraceEvent +- +-//////////////////////////////////////////////////////////////////////////////// +- +-// Implementation detail: trace event macros create temporary variables +-// to keep instrumentation overhead low. These macros give each temporary +-// variable a unique name based on the line number to prevent name collissions. +-#define INTERNAL_TRACE_EVENT_UID3(a,b) \ +- trace_event_unique_##a##b +-#define INTERNAL_TRACE_EVENT_UID2(a,b) \ +- INTERNAL_TRACE_EVENT_UID3(a,b) +-#define INTERNAL_TRACE_EVENT_UID(name_prefix) \ +- INTERNAL_TRACE_EVENT_UID2(name_prefix, __LINE__) +- +-#if WEBRTC_NON_STATIC_TRACE_EVENT_HANDLERS +-#define INTERNAL_TRACE_EVENT_INFO_TYPE const unsigned char* +-#else +-#define INTERNAL_TRACE_EVENT_INFO_TYPE static const unsigned char* +-#endif // WEBRTC_NON_STATIC_TRACE_EVENT_HANDLERS +- +-// Implementation detail: internal macro to create static category. +-#define INTERNAL_TRACE_EVENT_GET_CATEGORY_INFO(category) \ +- INTERNAL_TRACE_EVENT_INFO_TYPE INTERNAL_TRACE_EVENT_UID(catstatic) = \ +- TRACE_EVENT_API_GET_CATEGORY_ENABLED(category); +- +-// Implementation detail: internal macro to create static category and add +-// event if the category is enabled. +-#define INTERNAL_TRACE_EVENT_ADD(phase, category, name, flags, ...) \ +- do { \ +- INTERNAL_TRACE_EVENT_GET_CATEGORY_INFO(category); \ +- if (*INTERNAL_TRACE_EVENT_UID(catstatic)) { \ +- webrtc::trace_event_internal::AddTraceEvent( \ +- phase, INTERNAL_TRACE_EVENT_UID(catstatic), name, \ +- webrtc::trace_event_internal::kNoEventId, flags, ##__VA_ARGS__); \ +- } \ +- } while (0) +- +-// Implementation detail: internal macro to create static category and add begin +-// event if the category is enabled. Also adds the end event when the scope +-// ends. +-#define INTERNAL_TRACE_EVENT_ADD_SCOPED(category, name, ...) \ +- INTERNAL_TRACE_EVENT_GET_CATEGORY_INFO(category); \ +- webrtc::trace_event_internal::TraceEndOnScopeClose \ +- INTERNAL_TRACE_EVENT_UID(profileScope); \ +- if (*INTERNAL_TRACE_EVENT_UID(catstatic)) { \ +- webrtc::trace_event_internal::AddTraceEvent( \ +- TRACE_EVENT_PHASE_BEGIN, \ +- INTERNAL_TRACE_EVENT_UID(catstatic), \ +- name, webrtc::trace_event_internal::kNoEventId, \ +- TRACE_EVENT_FLAG_NONE, ##__VA_ARGS__); \ +- INTERNAL_TRACE_EVENT_UID(profileScope).Initialize( \ +- INTERNAL_TRACE_EVENT_UID(catstatic), name); \ +- } +- +-// Implementation detail: internal macro to create static category and add +-// event if the category is enabled. +-#define INTERNAL_TRACE_EVENT_ADD_WITH_ID(phase, category, name, id, flags, \ +- ...) \ +- do { \ +- INTERNAL_TRACE_EVENT_GET_CATEGORY_INFO(category); \ +- if (*INTERNAL_TRACE_EVENT_UID(catstatic)) { \ +- unsigned char trace_event_flags = flags | TRACE_EVENT_FLAG_HAS_ID; \ +- webrtc::trace_event_internal::TraceID trace_event_trace_id( \ +- id, &trace_event_flags); \ +- webrtc::trace_event_internal::AddTraceEvent( \ +- phase, INTERNAL_TRACE_EVENT_UID(catstatic), \ +- name, trace_event_trace_id.data(), trace_event_flags, \ +- ##__VA_ARGS__); \ +- } \ +- } while (0) +- +-// Notes regarding the following definitions: +-// New values can be added and propagated to third party libraries, but existing +-// definitions must never be changed, because third party libraries may use old +-// definitions. +- +-// Phase indicates the nature of an event entry. E.g. part of a begin/end pair. +-#define TRACE_EVENT_PHASE_BEGIN ('B') +-#define TRACE_EVENT_PHASE_END ('E') +-#define TRACE_EVENT_PHASE_INSTANT ('I') +-#define TRACE_EVENT_PHASE_ASYNC_BEGIN ('S') +-#define TRACE_EVENT_PHASE_ASYNC_STEP ('T') +-#define TRACE_EVENT_PHASE_ASYNC_END ('F') +-#define TRACE_EVENT_PHASE_FLOW_BEGIN ('s') +-#define TRACE_EVENT_PHASE_FLOW_STEP ('t') +-#define TRACE_EVENT_PHASE_FLOW_END ('f') +-#define TRACE_EVENT_PHASE_METADATA ('M') +-#define TRACE_EVENT_PHASE_COUNTER ('C') +- +-// Flags for changing the behavior of TRACE_EVENT_API_ADD_TRACE_EVENT. +-#define TRACE_EVENT_FLAG_NONE (static_cast<unsigned char>(0)) +-#define TRACE_EVENT_FLAG_COPY (static_cast<unsigned char>(1 << 0)) +-#define TRACE_EVENT_FLAG_HAS_ID (static_cast<unsigned char>(1 << 1)) +-#define TRACE_EVENT_FLAG_MANGLE_ID (static_cast<unsigned char>(1 << 2)) +- +-namespace webrtc { +-namespace trace_event_internal { +- +-// Specify these values when the corresponding argument of AddTraceEvent is not +-// used. +-const int kZeroNumArgs = 0; +-const unsigned long long kNoEventId = 0; +- +-// TraceID encapsulates an ID that can either be an integer or pointer. Pointers +-// are mangled with the Process ID so that they are unlikely to collide when the +-// same pointer is used on different processes. +-class TraceID { +- public: +- class ForceMangle { +- public: +- explicit ForceMangle(unsigned long long id) : data_(id) {} +- explicit ForceMangle(unsigned long id) : data_(id) {} +- explicit ForceMangle(unsigned int id) : data_(id) {} +- explicit ForceMangle(unsigned short id) : data_(id) {} +- explicit ForceMangle(unsigned char id) : data_(id) {} +- explicit ForceMangle(long long id) +- : data_(static_cast<unsigned long long>(id)) {} +- explicit ForceMangle(long id) +- : data_(static_cast<unsigned long long>(id)) {} +- explicit ForceMangle(int id) +- : data_(static_cast<unsigned long long>(id)) {} +- explicit ForceMangle(short id) +- : data_(static_cast<unsigned long long>(id)) {} +- explicit ForceMangle(signed char id) +- : data_(static_cast<unsigned long long>(id)) {} +- +- unsigned long long data() const { return data_; } +- +- private: +- unsigned long long data_; +- }; +- +- explicit TraceID(const void* id, unsigned char* flags) +- : data_(static_cast<unsigned long long>( +- reinterpret_cast<uintptr_t>(id))) { +- *flags |= TRACE_EVENT_FLAG_MANGLE_ID; +- } +- explicit TraceID(ForceMangle id, unsigned char* flags) : data_(id.data()) { +- *flags |= TRACE_EVENT_FLAG_MANGLE_ID; +- } +- explicit TraceID(unsigned long long id, unsigned char* flags) +- : data_(id) { (void)flags; } +- explicit TraceID(unsigned long id, unsigned char* flags) +- : data_(id) { (void)flags; } +- explicit TraceID(unsigned int id, unsigned char* flags) +- : data_(id) { (void)flags; } +- explicit TraceID(unsigned short id, unsigned char* flags) +- : data_(id) { (void)flags; } +- explicit TraceID(unsigned char id, unsigned char* flags) +- : data_(id) { (void)flags; } +- explicit TraceID(long long id, unsigned char* flags) +- : data_(static_cast<unsigned long long>(id)) { (void)flags; } +- explicit TraceID(long id, unsigned char* flags) +- : data_(static_cast<unsigned long long>(id)) { (void)flags; } +- explicit TraceID(int id, unsigned char* flags) +- : data_(static_cast<unsigned long long>(id)) { (void)flags; } +- explicit TraceID(short id, unsigned char* flags) +- : data_(static_cast<unsigned long long>(id)) { (void)flags; } +- explicit TraceID(signed char id, unsigned char* flags) +- : data_(static_cast<unsigned long long>(id)) { (void)flags; } +- +- unsigned long long data() const { return data_; } +- +- private: +- unsigned long long data_; +-}; +- +-// Simple union to store various types as unsigned long long. +-union TraceValueUnion { +- bool as_bool; +- unsigned long long as_uint; +- long long as_int; +- double as_double; +- const void* as_pointer; +- const char* as_string; +-}; +- +-// Simple container for const char* that should be copied instead of retained. +-class TraceStringWithCopy { +- public: +- explicit TraceStringWithCopy(const char* str) : str_(str) {} +- operator const char* () const { return str_; } +- private: +- const char* str_; +-}; +- +-// Define SetTraceValue for each allowed type. It stores the type and +-// value in the return arguments. This allows this API to avoid declaring any +-// structures so that it is portable to third_party libraries. +-#define INTERNAL_DECLARE_SET_TRACE_VALUE(actual_type, \ +- union_member, \ +- value_type_id) \ +- static inline void SetTraceValue(actual_type arg, \ +- unsigned char* type, \ +- unsigned long long* value) { \ +- TraceValueUnion type_value; \ +- type_value.union_member = arg; \ +- *type = value_type_id; \ +- *value = type_value.as_uint; \ +- } +-// Simpler form for int types that can be safely casted. +-#define INTERNAL_DECLARE_SET_TRACE_VALUE_INT(actual_type, \ +- value_type_id) \ +- static inline void SetTraceValue(actual_type arg, \ +- unsigned char* type, \ +- unsigned long long* value) { \ +- *type = value_type_id; \ +- *value = static_cast<unsigned long long>(arg); \ +- } +- +-INTERNAL_DECLARE_SET_TRACE_VALUE_INT(unsigned long long, TRACE_VALUE_TYPE_UINT) +-INTERNAL_DECLARE_SET_TRACE_VALUE_INT(unsigned long, TRACE_VALUE_TYPE_UINT) +-INTERNAL_DECLARE_SET_TRACE_VALUE_INT(unsigned int, TRACE_VALUE_TYPE_UINT) +-INTERNAL_DECLARE_SET_TRACE_VALUE_INT(unsigned short, TRACE_VALUE_TYPE_UINT) +-INTERNAL_DECLARE_SET_TRACE_VALUE_INT(unsigned char, TRACE_VALUE_TYPE_UINT) +-INTERNAL_DECLARE_SET_TRACE_VALUE_INT(long long, TRACE_VALUE_TYPE_INT) +-INTERNAL_DECLARE_SET_TRACE_VALUE_INT(long, TRACE_VALUE_TYPE_INT) +-INTERNAL_DECLARE_SET_TRACE_VALUE_INT(int, TRACE_VALUE_TYPE_INT) +-INTERNAL_DECLARE_SET_TRACE_VALUE_INT(short, TRACE_VALUE_TYPE_INT) +-INTERNAL_DECLARE_SET_TRACE_VALUE_INT(signed char, TRACE_VALUE_TYPE_INT) +-INTERNAL_DECLARE_SET_TRACE_VALUE(bool, as_bool, TRACE_VALUE_TYPE_BOOL) +-INTERNAL_DECLARE_SET_TRACE_VALUE(double, as_double, TRACE_VALUE_TYPE_DOUBLE) +-INTERNAL_DECLARE_SET_TRACE_VALUE(const void*, as_pointer, +- TRACE_VALUE_TYPE_POINTER) +-INTERNAL_DECLARE_SET_TRACE_VALUE(const char*, as_string, +- TRACE_VALUE_TYPE_STRING) +-INTERNAL_DECLARE_SET_TRACE_VALUE(const TraceStringWithCopy&, as_string, +- TRACE_VALUE_TYPE_COPY_STRING) +- +-#undef INTERNAL_DECLARE_SET_TRACE_VALUE +-#undef INTERNAL_DECLARE_SET_TRACE_VALUE_INT +- +-// std::string version of SetTraceValue so that trace arguments can be strings. +-static inline void SetTraceValue(const std::string& arg, +- unsigned char* type, +- unsigned long long* value) { +- TraceValueUnion type_value; +- type_value.as_string = arg.c_str(); +- *type = TRACE_VALUE_TYPE_COPY_STRING; +- *value = type_value.as_uint; +-} +- +-// These AddTraceEvent template functions are defined here instead of in the +-// macro, because the arg_values could be temporary objects, such as +-// std::string. In order to store pointers to the internal c_str and pass +-// through to the tracing API, the arg_values must live throughout +-// these procedures. +- +-static inline void AddTraceEvent(char phase, +- const unsigned char* category_enabled, +- const char* name, +- unsigned long long id, +- unsigned char flags) { +- TRACE_EVENT_API_ADD_TRACE_EVENT(phase, category_enabled, name, id, +- kZeroNumArgs, nullptr, nullptr, nullptr, +- flags); +-} +- +-template<class ARG1_TYPE> +-static inline void AddTraceEvent(char phase, +- const unsigned char* category_enabled, +- const char* name, +- unsigned long long id, +- unsigned char flags, +- const char* arg1_name, +- const ARG1_TYPE& arg1_val) { +- const int num_args = 1; +- unsigned char arg_types[1]; +- unsigned long long arg_values[1]; +- SetTraceValue(arg1_val, &arg_types[0], &arg_values[0]); +- TRACE_EVENT_API_ADD_TRACE_EVENT( +- phase, category_enabled, name, id, +- num_args, &arg1_name, arg_types, arg_values, +- flags); +-} +- +-template<class ARG1_TYPE, class ARG2_TYPE> +-static inline void AddTraceEvent(char phase, +- const unsigned char* category_enabled, +- const char* name, +- unsigned long long id, +- unsigned char flags, +- const char* arg1_name, +- const ARG1_TYPE& arg1_val, +- const char* arg2_name, +- const ARG2_TYPE& arg2_val) { +- const int num_args = 2; +- const char* arg_names[2] = { arg1_name, arg2_name }; +- unsigned char arg_types[2]; +- unsigned long long arg_values[2]; +- SetTraceValue(arg1_val, &arg_types[0], &arg_values[0]); +- SetTraceValue(arg2_val, &arg_types[1], &arg_values[1]); +- TRACE_EVENT_API_ADD_TRACE_EVENT( +- phase, category_enabled, name, id, +- num_args, arg_names, arg_types, arg_values, +- flags); +-} +- +-// Used by TRACE_EVENTx macro. Do not use directly. +-class TraceEndOnScopeClose { +- public: +- // Note: members of data_ intentionally left uninitialized. See Initialize. +- TraceEndOnScopeClose() : p_data_(nullptr) {} +- ~TraceEndOnScopeClose() { +- if (p_data_) +- AddEventIfEnabled(); +- } +- +- void Initialize(const unsigned char* category_enabled, +- const char* name) { +- data_.category_enabled = category_enabled; +- data_.name = name; +- p_data_ = &data_; +- } +- +- private: +- // Add the end event if the category is still enabled. +- void AddEventIfEnabled() { +- // Only called when p_data_ is non-null. +- if (*p_data_->category_enabled) { +- TRACE_EVENT_API_ADD_TRACE_EVENT(TRACE_EVENT_PHASE_END, +- p_data_->category_enabled, p_data_->name, +- kNoEventId, kZeroNumArgs, nullptr, +- nullptr, nullptr, TRACE_EVENT_FLAG_NONE); +- } +- } +- +- // This Data struct workaround is to avoid initializing all the members +- // in Data during construction of this object, since this object is always +- // constructed, even when tracing is disabled. If the members of Data were +- // members of this class instead, compiler warnings occur about potential +- // uninitialized accesses. +- struct Data { +- const unsigned char* category_enabled; +- const char* name; +- }; +- Data* p_data_; +- Data data_; +-}; +- +-} // namespace trace_event_internal +-} // namespace webrtc +-#else +- +-//////////////////////////////////////////////////////////////////////////////// +-// This section defines no-op alternatives to the tracing macros when +-// RTC_DISABLE_TRACE_EVENTS is defined. +- +-#define RTC_NOOP() do {} while (0) +- +-#define TRACE_STR_COPY(str) RTC_NOOP() +- +-#define TRACE_DISABLED_BY_DEFAULT(name) "disabled-by-default-" name +- +-#define TRACE_ID_MANGLE(id) 0 +- +-#define TRACE_EVENT0(category, name) RTC_NOOP() +-#define TRACE_EVENT1(category, name, arg1_name, arg1_val) RTC_NOOP() +-#define TRACE_EVENT2(category, name, arg1_name, arg1_val, arg2_name, arg2_val) \ +- RTC_NOOP() +- +-#define TRACE_EVENT_INSTANT0(category, name) RTC_NOOP() +-#define TRACE_EVENT_INSTANT1(category, name, arg1_name, arg1_val) RTC_NOOP() +- +-#define TRACE_EVENT_INSTANT2(category, name, arg1_name, arg1_val, \ +- arg2_name, arg2_val) RTC_NOOP() +- +-#define TRACE_EVENT_COPY_INSTANT0(category, name) RTC_NOOP() +-#define TRACE_EVENT_COPY_INSTANT1(category, name, arg1_name, arg1_val) \ +- RTC_NOOP() +-#define TRACE_EVENT_COPY_INSTANT2(category, name, arg1_name, arg1_val, \ +- arg2_name, arg2_val) RTC_NOOP() +- +-#define TRACE_EVENT_BEGIN0(category, name) RTC_NOOP() +-#define TRACE_EVENT_BEGIN1(category, name, arg1_name, arg1_val) RTC_NOOP() +-#define TRACE_EVENT_BEGIN2(category, name, arg1_name, arg1_val, \ +- arg2_name, arg2_val) RTC_NOOP() +-#define TRACE_EVENT_COPY_BEGIN0(category, name) RTC_NOOP() +-#define TRACE_EVENT_COPY_BEGIN1(category, name, arg1_name, arg1_val) RTC_NOOP() +-#define TRACE_EVENT_COPY_BEGIN2(category, name, arg1_name, arg1_val, \ +- arg2_name, arg2_val) RTC_NOOP() +- +-#define TRACE_EVENT_END0(category, name) RTC_NOOP() +-#define TRACE_EVENT_END1(category, name, arg1_name, arg1_val) RTC_NOOP() +-#define TRACE_EVENT_END2(category, name, arg1_name, arg1_val, \ +- arg2_name, arg2_val) RTC_NOOP() +-#define TRACE_EVENT_COPY_END0(category, name) RTC_NOOP() +-#define TRACE_EVENT_COPY_END1(category, name, arg1_name, arg1_val) RTC_NOOP() +-#define TRACE_EVENT_COPY_END2(category, name, arg1_name, arg1_val, \ +- arg2_name, arg2_val) RTC_NOOP() +- +-#define TRACE_COUNTER1(category, name, value) RTC_NOOP() +-#define TRACE_COPY_COUNTER1(category, name, value) RTC_NOOP() +- +-#define TRACE_COUNTER2(category, name, value1_name, value1_val, \ +- value2_name, value2_val) RTC_NOOP() +-#define TRACE_COPY_COUNTER2(category, name, value1_name, value1_val, \ +- value2_name, value2_val) RTC_NOOP() +- +-#define TRACE_COUNTER_ID1(category, name, id, value) RTC_NOOP() +-#define TRACE_COPY_COUNTER_ID1(category, name, id, value) RTC_NOOP() +- +-#define TRACE_COUNTER_ID2(category, name, id, value1_name, value1_val, \ +- value2_name, value2_val) RTC_NOOP() +-#define TRACE_COPY_COUNTER_ID2(category, name, id, value1_name, value1_val, \ +- value2_name, value2_val) RTC_NOOP() +- +-#define TRACE_EVENT_ASYNC_BEGIN0(category, name, id) RTC_NOOP() +-#define TRACE_EVENT_ASYNC_BEGIN1(category, name, id, arg1_name, arg1_val) \ +- RTC_NOOP() +-#define TRACE_EVENT_ASYNC_BEGIN2(category, name, id, arg1_name, arg1_val, \ +- arg2_name, arg2_val) RTC_NOOP() +-#define TRACE_EVENT_COPY_ASYNC_BEGIN0(category, name, id) RTC_NOOP() +-#define TRACE_EVENT_COPY_ASYNC_BEGIN1(category, name, id, arg1_name, arg1_val) \ +- RTC_NOOP() +-#define TRACE_EVENT_COPY_ASYNC_BEGIN2(category, name, id, arg1_name, arg1_val, \ +- arg2_name, arg2_val) RTC_NOOP() +- +-#define TRACE_EVENT_ASYNC_STEP0(category, name, id, step) RTC_NOOP() +-#define TRACE_EVENT_ASYNC_STEP1(category, name, id, step, \ +- arg1_name, arg1_val) RTC_NOOP() +-#define TRACE_EVENT_COPY_ASYNC_STEP0(category, name, id, step) RTC_NOOP() +-#define TRACE_EVENT_COPY_ASYNC_STEP1(category, name, id, step, \ +- arg1_name, arg1_val) RTC_NOOP() +- +-#define TRACE_EVENT_ASYNC_END0(category, name, id) RTC_NOOP() +-#define TRACE_EVENT_ASYNC_END1(category, name, id, arg1_name, arg1_val) \ +- RTC_NOOP() +-#define TRACE_EVENT_ASYNC_END2(category, name, id, arg1_name, arg1_val, \ +- arg2_name, arg2_val) RTC_NOOP() +-#define TRACE_EVENT_COPY_ASYNC_END0(category, name, id) RTC_NOOP() +-#define TRACE_EVENT_COPY_ASYNC_END1(category, name, id, arg1_name, arg1_val) \ +- RTC_NOOP() +-#define TRACE_EVENT_COPY_ASYNC_END2(category, name, id, arg1_name, arg1_val, \ +- arg2_name, arg2_val) RTC_NOOP() +- +-#define TRACE_EVENT_FLOW_BEGIN0(category, name, id) RTC_NOOP() +-#define TRACE_EVENT_FLOW_BEGIN1(category, name, id, arg1_name, arg1_val) \ +- RTC_NOOP() +-#define TRACE_EVENT_FLOW_BEGIN2(category, name, id, arg1_name, arg1_val, \ +- arg2_name, arg2_val) RTC_NOOP() +-#define TRACE_EVENT_COPY_FLOW_BEGIN0(category, name, id) RTC_NOOP() +-#define TRACE_EVENT_COPY_FLOW_BEGIN1(category, name, id, arg1_name, arg1_val) \ +- RTC_NOOP() +-#define TRACE_EVENT_COPY_FLOW_BEGIN2(category, name, id, arg1_name, arg1_val, \ +- arg2_name, arg2_val) RTC_NOOP() +- +-#define TRACE_EVENT_FLOW_STEP0(category, name, id, step) RTC_NOOP() +-#define TRACE_EVENT_FLOW_STEP1(category, name, id, step, \ +- arg1_name, arg1_val) RTC_NOOP() +-#define TRACE_EVENT_COPY_FLOW_STEP0(category, name, id, step) RTC_NOOP() +-#define TRACE_EVENT_COPY_FLOW_STEP1(category, name, id, step, \ +- arg1_name, arg1_val) RTC_NOOP() +- +-#define TRACE_EVENT_FLOW_END0(category, name, id) RTC_NOOP() +-#define TRACE_EVENT_FLOW_END1(category, name, id, arg1_name, arg1_val) \ +- RTC_NOOP() +-#define TRACE_EVENT_FLOW_END2(category, name, id, arg1_name, arg1_val, \ +- arg2_name, arg2_val) RTC_NOOP() +-#define TRACE_EVENT_COPY_FLOW_END0(category, name, id) RTC_NOOP() +-#define TRACE_EVENT_COPY_FLOW_END1(category, name, id, arg1_name, arg1_val) \ +- RTC_NOOP() +-#define TRACE_EVENT_COPY_FLOW_END2(category, name, id, arg1_name, arg1_val, \ +- arg2_name, arg2_val) RTC_NOOP() +- +-#define TRACE_EVENT_API_GET_CATEGORY_ENABLED "" +- +-#define TRACE_EVENT_API_ADD_TRACE_EVENT RTC_NOOP() +- +-#endif // RTC_TRACE_EVENTS_ENABLED +- +-#endif // RTC_BASE_TRACE_EVENT_H_ ++// This header is diverted to a similar header in Gecko, that is defining the ++// same macros, modified to talk to the Gecko Profiler. ++#include "GeckoTraceEvent.h" +-- +2.34.1 + diff --git a/third_party/libwebrtc/moz-patch-stack/0035.patch b/third_party/libwebrtc/moz-patch-stack/0035.patch new file mode 100644 index 0000000000..c215ed8b8a --- /dev/null +++ b/third_party/libwebrtc/moz-patch-stack/0035.patch @@ -0,0 +1,778 @@ +From: stransky <stransky@redhat.com> +Date: Thu, 5 Nov 2020 07:47:00 +0000 +Subject: Bug 1654112 - Tweak upstream gn files for Firefox build. r=ng + +Differential Revision: https://phabricator.services.mozilla.com/D130075 +Mercurial Revision: https://hg.mozilla.org/mozilla-central/rev/127ace4d8887f11abb201d300a849772a2b519f8 +--- + .gn | 2 ++ + BUILD.gn | 26 +++++++++++++- + api/BUILD.gn | 8 ++++- + api/task_queue/BUILD.gn | 2 ++ + api/transport/BUILD.gn | 2 ++ + common_audio/BUILD.gn | 4 --- + common_audio/fir_filter_avx2.cc | 2 ++ + common_audio/intrin.h | 8 +++++ + media/BUILD.gn | 30 ++++++++++++++++ + modules/audio_coding/BUILD.gn | 2 +- + modules/audio_device/BUILD.gn | 8 ++--- + modules/audio_processing/aec3/BUILD.gn | 13 +++---- + .../aec3/adaptive_fir_filter_avx2.cc | 3 +- + .../audio_processing/agc2/rnn_vad/BUILD.gn | 2 +- + modules/desktop_capture/BUILD.gn | 29 +--------------- + modules/portal/BUILD.gn | 24 +++++++++++++ + modules/video_capture/BUILD.gn | 11 +----- + modules/video_coding/BUILD.gn | 6 ++++ + rtc_base/BUILD.gn | 4 ++- + rtc_base/system/BUILD.gn | 2 +- + test/BUILD.gn | 14 ++++++++ + webrtc.gni | 34 ++++++++++++------- + 22 files changed, 162 insertions(+), 74 deletions(-) + create mode 100644 common_audio/intrin.h + +diff --git a/.gn b/.gn +index c9824916ad..d35ea79ced 100644 +--- a/.gn ++++ b/.gn +@@ -69,6 +69,8 @@ default_args = { + # Prevent jsoncpp to pass -Wno-deprecated-declarations to users + jsoncpp_no_deprecated_declarations = false + ++ use_custom_libcxx = false ++ + # Fixes the abi-revision issue. + # TODO(https://bugs.webrtc.org/14437): Remove this section if general + # Chromium fix resolves the problem. +diff --git a/BUILD.gn b/BUILD.gn +index 5817d22227..8b2648a306 100644 +--- a/BUILD.gn ++++ b/BUILD.gn +@@ -33,7 +33,7 @@ if (is_android) { + import("//build/config/android/rules.gni") + } + +-if (!build_with_chromium) { ++if (!build_with_chromium && !build_with_mozilla) { + # This target should (transitively) cause everything to be built; if you run + # 'ninja default' and then 'ninja all', the second build should do no work. + group("default") { +@@ -152,6 +152,10 @@ config("common_inherited_config") { + defines += [ "WEBRTC_ENABLE_OBJC_SYMBOL_EXPORT" ] + } + ++ if (build_with_mozilla) { ++ defines += [ "WEBRTC_MOZILLA_BUILD" ] ++ } ++ + if (!rtc_builtin_ssl_root_certificates) { + defines += [ "WEBRTC_EXCLUDE_BUILT_IN_SSL_ROOT_CERTS" ] + } +@@ -460,9 +464,11 @@ config("common_config") { + } + } + ++if (is_mac) { + config("common_objc") { + frameworks = [ "Foundation.framework" ] + } ++} + + if (!build_with_chromium) { + # Target to build all the WebRTC production code. +@@ -502,6 +508,23 @@ if (!build_with_chromium) { + "sdk", + "video", + ] ++ if (build_with_mozilla) { ++ deps -= [ ++ "api:create_peerconnection_factory", ++ "api:rtc_error", ++ "api:transport_api", ++ "api/crypto", ++ "api/rtc_event_log:rtc_event_log_factory", ++ "api/task_queue", ++ "api/task_queue:default_task_queue_factory", ++ "api/test/metrics", ++ "logging:rtc_event_log_api", ++ "p2p:rtc_p2p", ++ "pc:libjingle_peerconnection", ++ "pc:rtc_pc", ++ "sdk", ++ ] ++ } + + if (rtc_include_builtin_audio_codecs) { + deps += [ +@@ -521,6 +544,7 @@ if (!build_with_chromium) { + deps += [ + "api/video:video_frame", + "api/video:video_rtp_headers", ++ "test:rtp_test_utils", + ] + } else { + deps += [ +diff --git a/api/BUILD.gn b/api/BUILD.gn +index 33a6b0aaa6..ab5d6c91ce 100644 +--- a/api/BUILD.gn ++++ b/api/BUILD.gn +@@ -35,7 +35,7 @@ rtc_source_set("callfactory_api") { + ] + } + +-if (!build_with_chromium) { ++if (!build_with_chromium && !build_with_mozilla) { + rtc_library("create_peerconnection_factory") { + visibility = [ "*" ] + allow_poison = [ "default_task_queue" ] +@@ -227,6 +227,7 @@ rtc_library("rtp_sender_interface") { + } + + rtc_library("libjingle_peerconnection_api") { ++if (!build_with_mozilla) { + visibility = [ "*" ] + cflags = [] + sources = [ +@@ -343,6 +344,7 @@ rtc_library("libjingle_peerconnection_api") { + "//third_party/abseil-cpp/absl/types:optional", + ] + } ++} + + rtc_source_set("frame_transformer_interface") { + visibility = [ "*" ] +@@ -550,6 +552,7 @@ rtc_source_set("peer_network_dependencies") { + } + + rtc_source_set("peer_connection_quality_test_fixture_api") { ++if (!build_with_mozilla) { + visibility = [ "*" ] + testonly = true + sources = [ "test/peerconnection_quality_test_fixture.h" ] +@@ -600,6 +603,7 @@ rtc_source_set("peer_connection_quality_test_fixture_api") { + "//third_party/abseil-cpp/absl/types:optional", + ] + } ++} + + rtc_source_set("frame_generator_api") { + visibility = [ "*" ] +@@ -885,6 +889,7 @@ rtc_source_set("refcountedbase") { + ] + } + ++if (!build_with_mozilla) { + rtc_library("ice_transport_factory") { + visibility = [ "*" ] + sources = [ +@@ -903,6 +908,7 @@ rtc_library("ice_transport_factory") { + "rtc_event_log:rtc_event_log", + ] + } ++} + + rtc_library("neteq_simulator_api") { + visibility = [ "*" ] +diff --git a/api/task_queue/BUILD.gn b/api/task_queue/BUILD.gn +index 69393b80ff..c9b4a5d0ec 100644 +--- a/api/task_queue/BUILD.gn ++++ b/api/task_queue/BUILD.gn +@@ -30,6 +30,7 @@ rtc_library("task_queue") { + ] + } + ++if (rtc_include_tests) { + rtc_library("task_queue_test") { + visibility = [ "*" ] + testonly = true +@@ -78,6 +79,7 @@ rtc_library("task_queue_test") { + ] + } + } ++} + + rtc_library("default_task_queue_factory") { + visibility = [ "*" ] +diff --git a/api/transport/BUILD.gn b/api/transport/BUILD.gn +index 08f3d6d1d0..e0b31122b2 100644 +--- a/api/transport/BUILD.gn ++++ b/api/transport/BUILD.gn +@@ -90,6 +90,7 @@ rtc_source_set("sctp_transport_factory_interface") { + } + + rtc_source_set("stun_types") { ++if (!build_with_mozilla) { + visibility = [ "*" ] + sources = [ + "stun.cc", +@@ -110,6 +111,7 @@ rtc_source_set("stun_types") { + ] + absl_deps = [ "//third_party/abseil-cpp/absl/strings" ] + } ++} + + if (rtc_include_tests) { + rtc_source_set("test_feedback_generator_interface") { +diff --git a/common_audio/BUILD.gn b/common_audio/BUILD.gn +index 2ae6d32710..a45214f754 100644 +--- a/common_audio/BUILD.gn ++++ b/common_audio/BUILD.gn +@@ -267,14 +267,10 @@ if (current_cpu == "x86" || current_cpu == "x64") { + "resampler/sinc_resampler_avx2.cc", + ] + +- if (is_win) { +- cflags = [ "/arch:AVX2" ] +- } else { + cflags = [ + "-mavx2", + "-mfma", + ] +- } + + deps = [ + ":fir_filter", +diff --git a/common_audio/fir_filter_avx2.cc b/common_audio/fir_filter_avx2.cc +index 9cb0f770ca..0031392f8a 100644 +--- a/common_audio/fir_filter_avx2.cc ++++ b/common_audio/fir_filter_avx2.cc +@@ -15,6 +15,8 @@ + #include <string.h> + #include <xmmintrin.h> + ++#include "common_audio/intrin.h" ++ + #include "rtc_base/checks.h" + #include "rtc_base/memory/aligned_malloc.h" + +diff --git a/common_audio/intrin.h b/common_audio/intrin.h +new file mode 100644 +index 0000000000..f6ff7f218f +--- /dev/null ++++ b/common_audio/intrin.h +@@ -0,0 +1,8 @@ ++#if defined (__SSE__) ++ #include <immintrin.h> ++ #if defined (__clang__) ++ #include <avxintrin.h> ++ #include <avx2intrin.h> ++ #include <fmaintrin.h> ++ #endif ++#endif +diff --git a/media/BUILD.gn b/media/BUILD.gn +index 2e5c0bb872..4ddc8349a8 100644 +--- a/media/BUILD.gn ++++ b/media/BUILD.gn +@@ -147,6 +147,27 @@ rtc_library("rtc_media_base") { + "base/video_source_base.cc", + "base/video_source_base.h", + ] ++ if (build_with_mozilla) { ++ sources -= [ ++ "base/adapted_video_track_source.cc", ++ "base/adapted_video_track_source.h", ++ "base/audio_source.h", ++ "base/delayable.h", ++ "base/media_channel.h", ++ "base/media_channel_impl.cc", ++ "base/media_channel_impl.h", ++ "base/media_engine.cc", ++ "base/media_engine.h", ++ "base/rid_description.cc", ++ "base/rid_description.h", ++ "base/rtp_utils.cc", ++ "base/rtp_utils.h", ++ "base/stream_params.cc", ++ "base/stream_params.h", ++ "base/turn_utils.cc", ++ "base/turn_utils.h", ++ ] ++ } + } + + rtc_library("media_channel_impl") { +@@ -420,6 +441,9 @@ rtc_library("rtc_internal_video_codecs") { + "../system_wrappers:field_trial", + "../test:fake_video_codecs", + ] ++ if (build_with_mozilla) { ++ deps -= [ "../test:fake_video_codecs" ] ++ } + + if (enable_libaom) { + defines += [ "RTC_USE_LIBAOM_AV1_ENCODER" ] +@@ -449,6 +473,12 @@ rtc_library("rtc_internal_video_codecs") { + # targets depend on :rtc_encoder_simulcast_proxy directly. + "engine/encoder_simulcast_proxy.h", + ] ++ if (build_with_mozilla) { ++ sources -= [ ++ "engine/fake_video_codec_factory.cc", ++ "engine/fake_video_codec_factory.h", ++ ] ++ } + } + + rtc_library("rtc_audio_video") { +diff --git a/modules/audio_coding/BUILD.gn b/modules/audio_coding/BUILD.gn +index dddc3edd83..eac0650a26 100644 +--- a/modules/audio_coding/BUILD.gn ++++ b/modules/audio_coding/BUILD.gn +@@ -553,7 +553,7 @@ rtc_library("webrtc_opus_wrapper") { + deps += [ rtc_opus_dir ] + public_configs = [ "//third_party/opus:opus_config" ] + } else if (build_with_mozilla) { +- include_dirs = [ getenv("DIST") + "/include/opus" ] ++ public_configs = [ "//third_party/opus:opus_config" ] + } + } + +diff --git a/modules/audio_device/BUILD.gn b/modules/audio_device/BUILD.gn +index 4a6a0ab41c..e35a442025 100644 +--- a/modules/audio_device/BUILD.gn ++++ b/modules/audio_device/BUILD.gn +@@ -233,9 +233,9 @@ rtc_library("audio_device_impl") { + ] + + if (build_with_mozilla) { +- sources += [ +- "opensl/single_rw_fifo.cc", +- "opensl/single_rw_fifo.h", ++ sources -= [ ++ "include/test_audio_device.cc", ++ "include/test_audio_device.h", + ] + } + +@@ -477,7 +477,7 @@ if (rtc_include_tests && !build_with_chromium) { + } + } + +-if (!build_with_chromium && is_android) { ++if ((!build_with_chromium && !build_with_mozilla) && is_android) { + rtc_android_library("audio_device_java") { + sources = [ + "android/java/src/org/webrtc/voiceengine/BuildInfo.java", +diff --git a/modules/audio_processing/aec3/BUILD.gn b/modules/audio_processing/aec3/BUILD.gn +index f5eb5d5951..3e11a245a1 100644 +--- a/modules/audio_processing/aec3/BUILD.gn ++++ b/modules/audio_processing/aec3/BUILD.gn +@@ -264,14 +264,11 @@ if (current_cpu == "x86" || current_cpu == "x64") { + "vector_math_avx2.cc", + ] + +- if (is_win) { +- cflags = [ "/arch:AVX2" ] +- } else { +- cflags = [ +- "-mavx2", +- "-mfma", +- ] +- } ++ cflags = [ ++ "-mavx", ++ "-mavx2", ++ "-mfma", ++ ] + + deps = [ + ":adaptive_fir_filter", +diff --git a/modules/audio_processing/aec3/adaptive_fir_filter_avx2.cc b/modules/audio_processing/aec3/adaptive_fir_filter_avx2.cc +index 6c8c948026..44d4514275 100644 +--- a/modules/audio_processing/aec3/adaptive_fir_filter_avx2.cc ++++ b/modules/audio_processing/aec3/adaptive_fir_filter_avx2.cc +@@ -10,8 +10,7 @@ + + #include "modules/audio_processing/aec3/adaptive_fir_filter.h" + +-#include <immintrin.h> +- ++#include "common_audio/intrin.h" + #include "rtc_base/checks.h" + + namespace webrtc { +diff --git a/modules/audio_processing/agc2/rnn_vad/BUILD.gn b/modules/audio_processing/agc2/rnn_vad/BUILD.gn +index 9093a68cf3..3003a585bd 100644 +--- a/modules/audio_processing/agc2/rnn_vad/BUILD.gn ++++ b/modules/audio_processing/agc2/rnn_vad/BUILD.gn +@@ -122,7 +122,7 @@ rtc_source_set("vector_math") { + if (current_cpu == "x86" || current_cpu == "x64") { + rtc_library("vector_math_avx2") { + sources = [ "vector_math_avx2.cc" ] +- if (is_win) { ++ if (is_win && !build_with_mozilla) { + cflags = [ "/arch:AVX2" ] + } else { + cflags = [ +diff --git a/modules/desktop_capture/BUILD.gn b/modules/desktop_capture/BUILD.gn +index c9993dd47b..4aa9186527 100644 +--- a/modules/desktop_capture/BUILD.gn ++++ b/modules/desktop_capture/BUILD.gn +@@ -342,37 +342,12 @@ rtc_library("desktop_capture") { + ] + deps += [ ":desktop_capture_objc" ] + } +- +- if (build_with_mozilla) { +- sources += [ +- "desktop_device_info.cc", +- "desktop_device_info.h", +- ] +- if (is_win) { +- sources += [ +- "app_capturer_win.cc", +- "win/desktop_device_info_win.cc", +- "win/win_shared.cc", +- ] +- } +- } + if (rtc_use_x11_extensions || rtc_use_pipewire) { + sources += [ + "mouse_cursor_monitor_linux.cc", + "screen_capturer_linux.cc", + "window_capturer_linux.cc", + ] +- +- if (build_with_mozilla && (is_linux || is_chromeos)) { +- sources += [ +- "app_capturer_linux.cc", +- "linux/x11/app_capturer_x11.cc", +- "linux/x11/desktop_device_info_linux.cc", +- "linux/x11/desktop_device_info_linux.h", +- "linux/x11/shared_x_util.cc", +- "linux/x11/shared_x_util.h", +- ] +- } + } + + if (rtc_use_x11_extensions) { +@@ -536,9 +511,7 @@ rtc_library("desktop_capture") { + deps += [ "../../rtc_base:sanitizer" ] + } + +- if (!build_with_mozilla) { +- deps += [ "//third_party/libyuv" ] +- } ++ deps += [ "//third_party/libyuv" ] + + if (use_desktop_capture_differ_sse2) { + deps += [ ":desktop_capture_differ_sse2" ] +diff --git a/modules/portal/BUILD.gn b/modules/portal/BUILD.gn +index d0756f269b..d7768b2323 100644 +--- a/modules/portal/BUILD.gn ++++ b/modules/portal/BUILD.gn +@@ -11,6 +11,7 @@ import("//tools/generate_stubs/rules.gni") + import("../../webrtc.gni") + + if ((is_linux || is_chromeos) && rtc_use_pipewire) { ++if (!build_with_mozilla) { + pkg_config("gio") { + packages = [ + "gio-2.0", +@@ -88,6 +89,12 @@ if ((is_linux || is_chromeos) && rtc_use_pipewire) { + defines += [ "WEBRTC_USE_GIO" ] + } + } ++} else { ++ config("pipewire_all") { ++ } ++ config("pipewire_config") { ++ } ++} + + rtc_library("portal") { + sources = [ +@@ -120,5 +127,22 @@ if ((is_linux || is_chromeos) && rtc_use_pipewire) { + + deps += [ ":pipewire_stubs" ] + } ++ ++ if (build_with_mozilla) { ++ configs -= [ ++ ":gio", ++ ":pipewire", ++ ":pipewire_config", ++ ] ++ deps -= [ ":pipewire_stubs" ] ++ defines -= [ "WEBRTC_DLOPEN_PIPEWIRE" ] ++ public_deps = [ ++ "//third_party/pipewire", ++ "//third_party/drm", ++ "//third_party/gbm", ++ "//third_party/libepoxy" ++ ] ++ } + } + } ++ +diff --git a/modules/video_capture/BUILD.gn b/modules/video_capture/BUILD.gn +index cfa5184d71..95548906c4 100644 +--- a/modules/video_capture/BUILD.gn ++++ b/modules/video_capture/BUILD.gn +@@ -93,21 +93,12 @@ if (!build_with_chromium) { + "strmiids.lib", + "user32.lib", + ] +- +- if (build_with_mozilla) { +- sources += [ +- "windows/BaseFilter.cpp", +- "windows/BaseInputPin.cpp", +- "windows/BasePin.cpp", +- "windows/MediaType.cpp", +- ] +- } + } + if (is_fuchsia) { + sources = [ "video_capture_factory_null.cc" ] + } + +- if (build_with_mozilla && is_android) { ++ if (!build_with_mozilla && is_android) { + include_dirs = [ + "/config/external/nspr", + "/nsprpub/lib/ds", +diff --git a/modules/video_coding/BUILD.gn b/modules/video_coding/BUILD.gn +index fe63804b19..e1b5e4ba84 100644 +--- a/modules/video_coding/BUILD.gn ++++ b/modules/video_coding/BUILD.gn +@@ -237,6 +237,12 @@ rtc_library("video_coding") { + "video_receiver2.cc", + "video_receiver2.h", + ] ++ if (build_with_mozilla) { ++ sources += [ ++ "event_wrapper.cc", ++ "event_wrapper.h", ++ ] ++ } + + deps = [ + ":codec_globals_headers", +diff --git a/rtc_base/BUILD.gn b/rtc_base/BUILD.gn +index 0d8bd4f759..7e162cecbb 100644 +--- a/rtc_base/BUILD.gn ++++ b/rtc_base/BUILD.gn +@@ -820,7 +820,9 @@ rtc_library("rtc_json") { + "strings/json.h", + ] + deps = [ ":stringutils" ] ++if (!build_with_mozilla) { + all_dependent_configs = [ "//third_party/jsoncpp:jsoncpp_config" ] ++} + if (rtc_build_json) { + deps += [ "//third_party/jsoncpp" ] + } else { +@@ -2073,7 +2075,7 @@ if (rtc_include_tests) { + } + } + +-if (is_android) { ++if (is_android && !build_with_mozilla) { + rtc_android_library("base_java") { + visibility = [ "*" ] + sources = [ +diff --git a/rtc_base/system/BUILD.gn b/rtc_base/system/BUILD.gn +index 77f5139a2f..486b37590c 100644 +--- a/rtc_base/system/BUILD.gn ++++ b/rtc_base/system/BUILD.gn +@@ -101,7 +101,7 @@ if (is_mac || is_ios) { + rtc_source_set("warn_current_thread_is_deadlocked") { + sources = [ "warn_current_thread_is_deadlocked.h" ] + deps = [] +- if (is_android && !build_with_chromium) { ++ if (is_android && (!build_with_chromium && !build_with_mozilla)) { + sources += [ "warn_current_thread_is_deadlocked.cc" ] + deps += [ + "..:logging", +diff --git a/test/BUILD.gn b/test/BUILD.gn +index 0c71115f0d..04a718c411 100644 +--- a/test/BUILD.gn ++++ b/test/BUILD.gn +@@ -156,6 +156,7 @@ rtc_library("audio_test_common") { + absl_deps = [ "//third_party/abseil-cpp/absl/memory" ] + } + ++if (!build_with_mozilla) { + if (!build_with_chromium) { + if (is_mac || is_ios) { + rtc_library("video_test_mac") { +@@ -204,8 +205,12 @@ if (!build_with_chromium) { + } + } + } ++} + + rtc_library("rtp_test_utils") { ++ if (build_with_mozilla) { ++ sources = [] ++ } else { + testonly = true + sources = [ + "rtcp_packet_parser.cc", +@@ -215,6 +220,7 @@ rtc_library("rtp_test_utils") { + "rtp_file_writer.cc", + "rtp_file_writer.h", + ] ++ } + + deps = [ + "../api:array_view", +@@ -467,7 +473,9 @@ rtc_library("video_test_support") { + absl_deps = [ "//third_party/abseil-cpp/absl/types:optional" ] + + if (!is_ios) { ++ if (!build_with_mozilla) { + deps += [ "//third_party:jpeg" ] ++ } + sources += [ "testsupport/jpeg_frame_writer.cc" ] + } else { + sources += [ "testsupport/jpeg_frame_writer_ios.cc" ] +@@ -1036,6 +1044,10 @@ rtc_library("test_common") { + if (!is_android && !build_with_chromium) { + deps += [ "../modules/video_capture:video_capture_internal_impl" ] + } ++ # This, or some form of it should be upstreamed. ++ if (!rtc_include_tests) { ++ deps -= [ "../rtc_base:task_queue_for_test" ] ++ } + } + + rtc_library("mock_transport") { +@@ -1203,6 +1215,7 @@ rtc_library("copy_to_file_audio_capturer_unittest") { + ] + } + ++if (!build_with_mozilla) { + if (!build_with_chromium && is_android) { + rtc_android_library("native_test_java") { + testonly = true +@@ -1216,6 +1229,7 @@ if (!build_with_chromium && is_android) { + ] + } + } ++} + + rtc_library("call_config_utils") { + # TODO(bugs.webrtc.org/10814): Remove rtc_json_suppressions as soon as it +diff --git a/webrtc.gni b/webrtc.gni +index 8dfcc9d244..6ae1b2329c 100644 +--- a/webrtc.gni ++++ b/webrtc.gni +@@ -35,6 +35,11 @@ if (is_mac) { + import("//build/config/mac/rules.gni") + } + ++if (is_android) { ++ import("//build/config/android/config.gni") ++ import("//build/config/android/rules.gni") ++} ++ + if (is_fuchsia) { + import("//build/config/fuchsia/config.gni") + } +@@ -42,6 +47,11 @@ if (is_fuchsia) { + # This declare_args is separated from the next one because args declared + # in this one, can be read from the next one (args defined in the same + # declare_args cannot be referenced in that scope). ++declare_args() { ++ # Enable to use the Mozilla internal settings. ++ build_with_mozilla = true ++} ++ + declare_args() { + # Setting this to true will make RTC_EXPORT (see rtc_base/system/rtc_export.h) + # expand to code that will manage symbols visibility. +@@ -84,7 +94,7 @@ declare_args() { + # will tell the pre-processor to remove the default definition of the + # SystemTimeNanos() which is defined in rtc_base/system_time.cc. In + # that case a new implementation needs to be provided. +- rtc_exclude_system_time = build_with_chromium ++ rtc_exclude_system_time = build_with_chromium || build_with_mozilla + + # Setting this to false will require the API user to pass in their own + # SSLCertificateVerifier to verify the certificates presented from a +@@ -110,7 +120,7 @@ declare_args() { + + # Used to specify an external OpenSSL include path when not compiling the + # library that comes with WebRTC (i.e. rtc_build_ssl == 0). +- rtc_ssl_root = "" ++ rtc_ssl_root = "unused" + + # Enable when an external authentication mechanism is used for performing + # packet authentication for RTP packets instead of libsrtp. +@@ -127,13 +137,13 @@ declare_args() { + rtc_enable_bwe_test_logging = false + + # Set this to false to skip building examples. +- rtc_build_examples = true ++ rtc_build_examples = false + + # Set this to false to skip building tools. +- rtc_build_tools = true ++ rtc_build_tools = false + + # Set this to false to skip building code that requires X11. +- rtc_use_x11 = ozone_platform_x11 ++ rtc_use_x11 = use_x11 + + # Set this to use PipeWire on the Wayland display server. + # By default it's only enabled on desktop Linux (excludes ChromeOS) and +@@ -144,9 +154,6 @@ declare_args() { + # Set this to link PipeWire and required libraries directly instead of using the dlopen. + rtc_link_pipewire = false + +- # Enable to use the Mozilla internal settings. +- build_with_mozilla = false +- + # Experimental: enable use of Android AAudio which requires Android SDK 26 or above + # and NDK r16 or above. + rtc_enable_android_aaudio = false +@@ -227,7 +234,7 @@ declare_args() { + # When set to true, a capturer implementation that uses the + # Windows.Graphics.Capture APIs will be available for use. This introduces a + # dependency on the Win 10 SDK v10.0.17763.0. +- rtc_enable_win_wgc = is_win ++ rtc_enable_win_wgc = false + + # Includes the dav1d decoder in the internal decoder factory when set to true. + rtc_include_dav1d_in_internal_decoder_factory = true +@@ -256,7 +263,7 @@ declare_args() { + rtc_build_json = !build_with_mozilla + rtc_build_libsrtp = !build_with_mozilla + rtc_build_libvpx = !build_with_mozilla +- rtc_libvpx_build_vp9 = !build_with_mozilla ++ rtc_libvpx_build_vp9 = true + rtc_build_opus = !build_with_mozilla + rtc_build_ssl = !build_with_mozilla + +@@ -275,7 +282,7 @@ declare_args() { + + # Chromium uses its own IO handling, so the internal ADM is only built for + # standalone WebRTC. +- rtc_include_internal_audio_device = !build_with_chromium ++ rtc_include_internal_audio_device = !build_with_chromium && !build_with_mozilla + + # Set this to true to enable the avx2 support in webrtc. + # TODO: Make sure that AVX2 works also for non-clang compilers. +@@ -319,6 +326,9 @@ declare_args() { + rtc_enable_grpc = rtc_enable_protobuf && (is_linux || is_mac) + } + ++# Enable liboam only on non-mozilla builds. ++enable_libaom = !build_with_mozilla ++ + # Make it possible to provide custom locations for some libraries (move these + # up into declare_args should we need to actually use them for the GN build). + rtc_libvpx_dir = "//third_party/libvpx" +@@ -1114,7 +1124,7 @@ if (is_mac || is_ios) { + } + } + +-if (is_android) { ++if (is_android && !build_with_mozilla) { + template("rtc_android_library") { + android_library(target_name) { + forward_variables_from(invoker, +-- +2.34.1 + diff --git a/third_party/libwebrtc/moz-patch-stack/0036.patch b/third_party/libwebrtc/moz-patch-stack/0036.patch new file mode 100644 index 0000000000..12d734372b --- /dev/null +++ b/third_party/libwebrtc/moz-patch-stack/0036.patch @@ -0,0 +1,29 @@ +From: Dan Minor <dminor@mozilla.com> +Date: Fri, 13 Nov 2020 14:34:00 -0500 +Subject: Bug 1654112 - Fully quality AudioLevel::kUri in channel_send.cc. r=ng + +Differential Revision: https://phabricator.services.mozilla.com/D130082 +Mercurial Revision: https://hg.mozilla.org/mozilla-central/rev/7163801a480d607005042292ed9e4fbb892f440d +--- + audio/channel_send.cc | 4 ++-- + 1 file changed, 2 insertions(+), 2 deletions(-) + +diff --git a/audio/channel_send.cc b/audio/channel_send.cc +index 5b0858a76a..0bca328846 100644 +--- a/audio/channel_send.cc ++++ b/audio/channel_send.cc +@@ -692,9 +692,9 @@ void ChannelSend::SetSendAudioLevelIndicationStatus(bool enable, int id) { + RTC_DCHECK_RUN_ON(&worker_thread_checker_); + include_audio_level_indication_.store(enable); + if (enable) { +- rtp_rtcp_->RegisterRtpHeaderExtension(AudioLevel::Uri(), id); ++ rtp_rtcp_->RegisterRtpHeaderExtension(webrtc::AudioLevel::Uri(), id); + } else { +- rtp_rtcp_->DeregisterSendRtpHeaderExtension(AudioLevel::Uri()); ++ rtp_rtcp_->DeregisterSendRtpHeaderExtension(webrtc::AudioLevel::Uri()); + } + } + +-- +2.34.1 + diff --git a/third_party/libwebrtc/moz-patch-stack/0037.patch b/third_party/libwebrtc/moz-patch-stack/0037.patch new file mode 100644 index 0000000000..4b14593e98 --- /dev/null +++ b/third_party/libwebrtc/moz-patch-stack/0037.patch @@ -0,0 +1,52 @@ +From: Dan Minor <dminor@mozilla.com> +Date: Wed, 18 Nov 2020 15:19:00 -0500 +Subject: Bug 1654112 - Fully qualify kIvfHeaderSize. r=ng + +Differential Revision: https://phabricator.services.mozilla.com/D130087 +Mercurial Revision: https://hg.mozilla.org/mozilla-central/rev/9452e226943fea4b0f6ff67e0ee85587d9c23a44 +--- + modules/video_coding/utility/ivf_file_writer.cc | 10 +++++----- + 1 file changed, 5 insertions(+), 5 deletions(-) + +diff --git a/modules/video_coding/utility/ivf_file_writer.cc b/modules/video_coding/utility/ivf_file_writer.cc +index 668390a78c..5b27ef3ef7 100644 +--- a/modules/video_coding/utility/ivf_file_writer.cc ++++ b/modules/video_coding/utility/ivf_file_writer.cc +@@ -39,7 +39,7 @@ IvfFileWriter::IvfFileWriter(FileWrapper file, size_t byte_limit) + last_timestamp_(-1), + using_capture_timestamps_(false), + file_(std::move(file)) { +- RTC_DCHECK(byte_limit == 0 || kIvfHeaderSize <= byte_limit) ++ RTC_DCHECK(byte_limit == 0 || webrtc::kIvfHeaderSize <= byte_limit) + << "The byte_limit is too low, not even the header will fit."; + } + +@@ -59,7 +59,7 @@ bool IvfFileWriter::WriteHeader() { + return false; + } + +- uint8_t ivf_header[kIvfHeaderSize] = {0}; ++ uint8_t ivf_header[webrtc::kIvfHeaderSize] = {0}; + ivf_header[0] = 'D'; + ivf_header[1] = 'K'; + ivf_header[2] = 'I'; +@@ -113,13 +113,13 @@ bool IvfFileWriter::WriteHeader() { + static_cast<uint32_t>(num_frames_)); + ByteWriter<uint32_t>::WriteLittleEndian(&ivf_header[28], 0); // Reserved. + +- if (!file_.Write(ivf_header, kIvfHeaderSize)) { ++ if (!file_.Write(ivf_header, webrtc::kIvfHeaderSize)) { + RTC_LOG(LS_ERROR) << "Unable to write IVF header for ivf output file."; + return false; + } + +- if (bytes_written_ < kIvfHeaderSize) { +- bytes_written_ = kIvfHeaderSize; ++ if (bytes_written_ < webrtc::kIvfHeaderSize) { ++ bytes_written_ = webrtc::kIvfHeaderSize; + } + + return true; +-- +2.34.1 + diff --git a/third_party/libwebrtc/moz-patch-stack/0038.patch b/third_party/libwebrtc/moz-patch-stack/0038.patch new file mode 100644 index 0000000000..e7f9b54d36 --- /dev/null +++ b/third_party/libwebrtc/moz-patch-stack/0038.patch @@ -0,0 +1,32 @@ +From: Dan Minor <dminor@mozilla.com> +Date: Tue, 1 Dec 2020 09:36:00 -0500 +Subject: Bug 1654112 - Disable creating av1 encoder and decoder. r=ng + +Differential Revision: https://phabricator.services.mozilla.com/D130089 +Mercurial Revision: https://hg.mozilla.org/mozilla-central/rev/ef548d7758c7de6e78d38af299c2296bf9d20ec9 +--- + media/engine/internal_decoder_factory.cc | 2 ++ + 1 file changed, 2 insertions(+) + +diff --git a/media/engine/internal_decoder_factory.cc b/media/engine/internal_decoder_factory.cc +index e761fd60c8..001c666313 100644 +--- a/media/engine/internal_decoder_factory.cc ++++ b/media/engine/internal_decoder_factory.cc +@@ -49,12 +49,14 @@ std::vector<SdpVideoFormat> InternalDecoderFactory::GetSupportedFormats() + for (const SdpVideoFormat& h264_format : SupportedH264DecoderCodecs()) + formats.push_back(h264_format); + ++#if !defined(WEBRTC_MOZILLA_BUILD) + if (kDav1dIsIncluded) { + formats.push_back(SdpVideoFormat(cricket::kAv1CodecName)); + formats.push_back(SdpVideoFormat( + cricket::kAv1CodecName, + {{kAV1FmtpProfile, AV1ProfileToString(AV1Profile::kProfile1).data()}})); + } ++#endif + + return formats; + } +-- +2.34.1 + diff --git a/third_party/libwebrtc/moz-patch-stack/0039.patch b/third_party/libwebrtc/moz-patch-stack/0039.patch new file mode 100644 index 0000000000..f1d4402928 --- /dev/null +++ b/third_party/libwebrtc/moz-patch-stack/0039.patch @@ -0,0 +1,133 @@ +From: Andreas Pehrson <apehrson@mozilla.com> +Date: Mon, 18 Jan 2021 11:04:00 +0100 +Subject: Bug 1654112 - Include RtcpPacketTypeCounter in audio send stats, to + not regress nackCount. r=ng + +This is similar to how it's already included for video send. + +Differential Revision: https://phabricator.services.mozilla.com/D102273 +Mercurial Revision: https://hg.mozilla.org/mozilla-central/rev/d380a43d59f4f7cbc001f4eab9b63ee993b32cd8 +--- + audio/audio_send_stream.cc | 1 + + audio/channel_send.cc | 31 +++++++++++++++++++++++++++++++ + audio/channel_send.h | 1 + + call/audio_send_stream.h | 2 ++ + 4 files changed, 35 insertions(+) + +diff --git a/audio/audio_send_stream.cc b/audio/audio_send_stream.cc +index 7d6ec794d4..20af3f7722 100644 +--- a/audio/audio_send_stream.cc ++++ b/audio/audio_send_stream.cc +@@ -442,6 +442,7 @@ webrtc::AudioSendStream::Stats AudioSendStream::GetStats( + stats.target_bitrate_bps = channel_send_->GetTargetBitrate(); + + webrtc::CallSendStatistics call_stats = channel_send_->GetRTCPStatistics(); ++ stats.rtcp_packet_type_counts = call_stats.rtcp_packet_type_counts; + stats.payload_bytes_sent = call_stats.payload_bytes_sent; + stats.header_and_padding_bytes_sent = + call_stats.header_and_padding_bytes_sent; +diff --git a/audio/channel_send.cc b/audio/channel_send.cc +index 0bca328846..c84a91770e 100644 +--- a/audio/channel_send.cc ++++ b/audio/channel_send.cc +@@ -56,6 +56,31 @@ class RtpPacketSenderProxy; + class TransportSequenceNumberProxy; + class VoERtcpObserver; + ++class RtcpCounterObserver : public RtcpPacketTypeCounterObserver { ++ public: ++ explicit RtcpCounterObserver(uint32_t ssrc) : ssrc_(ssrc) {} ++ ++ void RtcpPacketTypesCounterUpdated( ++ uint32_t ssrc, const RtcpPacketTypeCounter& packet_counter) override { ++ if (ssrc_ != ssrc) { ++ return; ++ } ++ ++ MutexLock lock(&mutex_); ++ packet_counter_ = packet_counter; ++ } ++ ++ RtcpPacketTypeCounter GetCounts() { ++ MutexLock lock(&mutex_); ++ return packet_counter_; ++ } ++ ++ private: ++ Mutex mutex_; ++ const uint32_t ssrc_; ++ RtcpPacketTypeCounter packet_counter_; ++}; ++ + class ChannelSend : public ChannelSendInterface, + public AudioPacketizationCallback, // receive encoded + // packets from the ACM +@@ -208,6 +233,8 @@ class ChannelSend : public ChannelSendInterface, + // RtcpBandwidthObserver + const std::unique_ptr<VoERtcpObserver> rtcp_observer_; + ++ const std::unique_ptr<RtcpCounterObserver> rtcp_counter_observer_; ++ + PacketRouter* packet_router_ RTC_GUARDED_BY(&worker_thread_checker_) = + nullptr; + TransportFeedbackObserver* const feedback_observer_; +@@ -460,6 +487,7 @@ ChannelSend::ChannelSend( + : ssrc_(ssrc), + event_log_(rtc_event_log), + rtcp_observer_(new VoERtcpObserver(this)), ++ rtcp_counter_observer_(new RtcpCounterObserver(ssrc)), + feedback_observer_(feedback_observer), + rtp_packet_pacer_proxy_(new RtpPacketSenderProxy()), + retransmission_rate_limiter_( +@@ -482,6 +510,8 @@ ChannelSend::ChannelSend( + + configuration.event_log = event_log_; + configuration.rtt_stats = rtcp_rtt_stats; ++ configuration.rtcp_packet_type_counter_observer = ++ rtcp_counter_observer_.get(); + configuration.retransmission_rate_limiter = + retransmission_rate_limiter_.get(); + configuration.extmap_allow_mixed = extmap_allow_mixed; +@@ -759,6 +789,7 @@ CallSendStatistics ChannelSend::GetRTCPStatistics() const { + RTC_DCHECK_RUN_ON(&worker_thread_checker_); + CallSendStatistics stats = {0}; + stats.rttMs = GetRTT(); ++ stats.rtcp_packet_type_counts = rtcp_counter_observer_->GetCounts(); + + StreamDataCounters rtp_stats; + StreamDataCounters rtx_stats; +diff --git a/audio/channel_send.h b/audio/channel_send.h +index cf9a273f70..9b3969161c 100644 +--- a/audio/channel_send.h ++++ b/audio/channel_send.h +@@ -43,6 +43,7 @@ struct CallSendStatistics { + TimeDelta total_packet_send_delay = TimeDelta::Zero(); + // https://w3c.github.io/webrtc-stats/#dom-rtcoutboundrtpstreamstats-retransmittedpacketssent + uint64_t retransmitted_packets_sent; ++ RtcpPacketTypeCounter rtcp_packet_type_counts; + // A snapshot of Report Blocks with additional data of interest to statistics. + // Within this list, the sender-source SSRC pair is unique and per-pair the + // ReportBlockData represents the latest Report Block that was received for +diff --git a/call/audio_send_stream.h b/call/audio_send_stream.h +index 0f42d0fb82..bafa22d312 100644 +--- a/call/audio_send_stream.h ++++ b/call/audio_send_stream.h +@@ -31,6 +31,7 @@ + #include "call/rtp_config.h" + #include "modules/audio_processing/include/audio_processing_statistics.h" + #include "modules/rtp_rtcp/include/report_block_data.h" ++#include "modules/rtp_rtcp/include/rtcp_statistics.h" + + namespace webrtc { + +@@ -65,6 +66,7 @@ class AudioSendStream : public AudioSender { + + ANAStats ana_statistics; + AudioProcessingStats apm_statistics; ++ RtcpPacketTypeCounter rtcp_packet_type_counts; + + int64_t target_bitrate_bps = 0; + // A snapshot of Report Blocks with additional data of interest to +-- +2.34.1 + diff --git a/third_party/libwebrtc/moz-patch-stack/0040.patch b/third_party/libwebrtc/moz-patch-stack/0040.patch new file mode 100644 index 0000000000..bc0246b353 --- /dev/null +++ b/third_party/libwebrtc/moz-patch-stack/0040.patch @@ -0,0 +1,44 @@ +From: Andreas Pehrson <apehrson@mozilla.com> +Date: Fri, 19 Feb 2021 13:45:00 +0100 +Subject: Bug 1654112 - libwebrtc: Add a REMB on/off switch to + VideoReceiveStream. r=ng + +Differential Revision: https://phabricator.services.mozilla.com/D105774 +Mercurial Revision: https://hg.mozilla.org/mozilla-central/rev/7330681cf4de6d8dd868cc661cbdd6679bbc07b3 +--- + call/video_receive_stream.h | 3 +++ + video/rtp_video_stream_receiver2.cc | 3 +-- + 2 files changed, 4 insertions(+), 2 deletions(-) + +diff --git a/call/video_receive_stream.h b/call/video_receive_stream.h +index 3125993d4b..cda8b1f6af 100644 +--- a/call/video_receive_stream.h ++++ b/call/video_receive_stream.h +@@ -203,6 +203,9 @@ class VideoReceiveStreamInterface : public MediaReceiveStreamInterface { + // disabled. + KeyFrameReqMethod keyframe_method = KeyFrameReqMethod::kPliRtcp; + ++ // See draft-alvestrand-rmcat-remb for information. ++ bool remb = false; ++ + bool tmmbr = false; + + // See LntfConfig for description. +diff --git a/video/rtp_video_stream_receiver2.cc b/video/rtp_video_stream_receiver2.cc +index e7f88665b6..eed9770d93 100644 +--- a/video/rtp_video_stream_receiver2.cc ++++ b/video/rtp_video_stream_receiver2.cc +@@ -297,9 +297,8 @@ RtpVideoStreamReceiver2::RtpVideoStreamReceiver2( + frames_decryptable_(false), + absolute_capture_time_interpolator_(clock) { + packet_sequence_checker_.Detach(); +- constexpr bool remb_candidate = true; + if (packet_router_) +- packet_router_->AddReceiveRtpModule(rtp_rtcp_.get(), remb_candidate); ++ packet_router_->AddReceiveRtpModule(rtp_rtcp_.get(), config_.rtp.remb); + + RTC_DCHECK(config_.rtp.rtcp_mode != RtcpMode::kOff) + << "A stream should not be configured with RTCP disabled. This value is " +-- +2.34.1 + diff --git a/third_party/libwebrtc/moz-patch-stack/0041.patch b/third_party/libwebrtc/moz-patch-stack/0041.patch new file mode 100644 index 0000000000..4fe4f7b929 --- /dev/null +++ b/third_party/libwebrtc/moz-patch-stack/0041.patch @@ -0,0 +1,41 @@ +From: Nico Grunbaum <na-g@nostrum.com> +Date: Wed, 10 Feb 2021 12:24:00 -0800 +Subject: Bug 1654112 - Use newer thread run callback, and adapt + PlatformUIThread; r=pehrsons + +Differential Revision: https://phabricator.services.mozilla.com/D107879 +Mercurial Revision: https://hg.mozilla.org/mozilla-central/rev/bb6417a4cfac1416a8e2565bd68b66c40be4827b +--- + rtc_base/platform_thread.h | 7 +++++++ + 1 file changed, 7 insertions(+) + +diff --git a/rtc_base/platform_thread.h b/rtc_base/platform_thread.h +index 3ab2761f43..52497ffd0c 100644 +--- a/rtc_base/platform_thread.h ++++ b/rtc_base/platform_thread.h +@@ -18,8 +18,13 @@ + #include "absl/types/optional.h" + #include "rtc_base/platform_thread_types.h" + ++#include "rtc_base/deprecated/recursive_critical_section.h" ++ + namespace rtc { + ++// Bug 1691641 ++class PlatformUIThread; ++ + enum class ThreadPriority { + kLow = 1, + kNormal, +@@ -110,6 +115,8 @@ class PlatformThread final { + + absl::optional<Handle> handle_; + bool joinable_ = false; ++ // Bug 1691641 ++ friend PlatformUIThread; + }; + + } // namespace rtc +-- +2.34.1 + diff --git a/third_party/libwebrtc/moz-patch-stack/0042.patch b/third_party/libwebrtc/moz-patch-stack/0042.patch new file mode 100644 index 0000000000..a65571742f --- /dev/null +++ b/third_party/libwebrtc/moz-patch-stack/0042.patch @@ -0,0 +1,57 @@ +From: Nico Grunbaum <na-g@nostrum.com> +Date: Thu, 18 Feb 2021 17:23:00 -0800 +Subject: Bug 1654112 - fix device_info_ds pid and Windows constants includes; + r=pehrsons + +Upstreaming bug 1697385 + +Differential Revision: https://phabricator.services.mozilla.com/D107899 +Mercurial Revision: https://hg.mozilla.org/mozilla-central/rev/24809d566449907edea49ea47528065ad0f76910 +--- + modules/video_capture/windows/device_info_ds.cc | 3 ++- + modules/video_capture/windows/device_info_ds.h | 6 +++++- + 2 files changed, 7 insertions(+), 2 deletions(-) + +diff --git a/modules/video_capture/windows/device_info_ds.cc b/modules/video_capture/windows/device_info_ds.cc +index 3ab95837c0..6b4c57d01e 100644 +--- a/modules/video_capture/windows/device_info_ds.cc ++++ b/modules/video_capture/windows/device_info_ds.cc +@@ -172,7 +172,8 @@ int32_t DeviceInfoDS::GetDeviceName(uint32_t deviceNumber, + char* deviceUniqueIdUTF8, + uint32_t deviceUniqueIdUTF8Length, + char* productUniqueIdUTF8, +- uint32_t productUniqueIdUTF8Length) { ++ uint32_t productUniqueIdUTF8Length, ++ pid_t* pid) { + MutexLock lock(&_apiLock); + const int32_t result = GetDeviceInfo( + deviceNumber, deviceNameUTF8, deviceNameLength, deviceUniqueIdUTF8, +diff --git a/modules/video_capture/windows/device_info_ds.h b/modules/video_capture/windows/device_info_ds.h +index ed2a726d6f..e6dfaed366 100644 +--- a/modules/video_capture/windows/device_info_ds.h ++++ b/modules/video_capture/windows/device_info_ds.h +@@ -12,8 +12,11 @@ + #define MODULES_VIDEO_CAPTURE_MAIN_SOURCE_WINDOWS_DEVICE_INFO_DS_H_ + + #include <dshow.h> ++#include <Ks.h> ++#include <dbt.h> + + #include "modules/video_capture/device_info_impl.h" ++#include "modules/video_capture/video_capture.h" + #include "modules/video_capture/video_capture_impl.h" + + namespace webrtc { +@@ -47,7 +50,8 @@ class DeviceInfoDS : public DeviceInfoImpl { + char* deviceUniqueIdUTF8, + uint32_t deviceUniqueIdUTF8Length, + char* productUniqueIdUTF8, +- uint32_t productUniqueIdUTF8Length) override; ++ uint32_t productUniqueIdUTF8Length, ++ pid_t* pid) override; + + /* + * Display OS /capture device specific settings dialog +-- +2.34.1 + diff --git a/third_party/libwebrtc/moz-patch-stack/0043.patch b/third_party/libwebrtc/moz-patch-stack/0043.patch new file mode 100644 index 0000000000..81e50a23e7 --- /dev/null +++ b/third_party/libwebrtc/moz-patch-stack/0043.patch @@ -0,0 +1,50 @@ +From: "Byron Campen [:bwc]" <docfaraday@gmail.com> +Date: Thu, 29 Apr 2021 18:25:00 +0000 +Subject: Bug 1654112 - Work around the old (<1.5) libxrandr headers on our + build machines. r=mjf,ng + +Differential Revision: https://phabricator.services.mozilla.com/D113830 +Mercurial Revision: https://hg.mozilla.org/mozilla-central/rev/c91f12b557a1d23b468c75c4f2fc00eb0f8d541a +--- + X11/extensions/Xrandr.h | 29 +++++++++++++++++++++++++++++ + 1 file changed, 29 insertions(+) + create mode 100644 X11/extensions/Xrandr.h + +diff --git a/X11/extensions/Xrandr.h b/X11/extensions/Xrandr.h +new file mode 100644 +index 0000000000..876e8b4c7f +--- /dev/null ++++ b/X11/extensions/Xrandr.h +@@ -0,0 +1,29 @@ ++/* This Source Code Form is subject to the terms of the Mozilla Public ++ * License, v. 2.0. If a copy of the MPL was not distributed with this file, ++ * You can obtain one at http://mozilla.org/MPL/2.0/. */ ++ ++// Hack to compensate for the old (<1.5) Xrandr development headers on ++// Mozilla's build boxes. ++ ++#ifndef _XRANDR_H_WRAPPER_HACK_ ++#define _XRANDR_H_WRAPPER_HACK_ ++ ++#include_next <X11/extensions/Xrandr.h> ++ ++#if RANDR_MAJOR == 1 && RANDR_MINOR < 5 // defined in randr.h ++typedef struct _XRRMonitorInfo { ++ Atom name; ++ Bool primary; ++ Bool automatic; ++ int noutput; ++ int x; ++ int y; ++ int width; ++ int height; ++ int mwidth; ++ int mheight; ++ RROutput *outputs; ++} XRRMonitorInfo; ++#endif ++ ++#endif // _XRANDR_H_WRAPPER_HACK_ +-- +2.34.1 + diff --git a/third_party/libwebrtc/moz-patch-stack/0044.patch b/third_party/libwebrtc/moz-patch-stack/0044.patch new file mode 100644 index 0000000000..3052fbef40 --- /dev/null +++ b/third_party/libwebrtc/moz-patch-stack/0044.patch @@ -0,0 +1,63 @@ +From: Michael Froman <mfroman@mozilla.com> +Date: Fri, 16 Apr 2021 17:35:00 -0500 +Subject: Bug 1654112 - General build fixes for paths and naming changes. r=ng + +Differential Revision: https://phabricator.services.mozilla.com/D113438 +Mercurial Revision: https://hg.mozilla.org/mozilla-central/rev/99b99cca6b7b2c2ebffb5472457a4f927bda11c4 +--- + modules/video_capture/video_capture_impl.h | 4 +++- + rtc_base/system/warn_current_thread_is_deadlocked.h | 2 +- + sdk/android/api/org/webrtc/VideoCodecInfo.java | 2 +- + 3 files changed, 5 insertions(+), 3 deletions(-) + +diff --git a/modules/video_capture/video_capture_impl.h b/modules/video_capture/video_capture_impl.h +index f874580471..b9af5f2441 100644 +--- a/modules/video_capture/video_capture_impl.h ++++ b/modules/video_capture/video_capture_impl.h +@@ -81,6 +81,9 @@ class VideoCaptureImpl : public VideoCaptureModule { + VideoCaptureImpl(); + ~VideoCaptureImpl() override; + ++ // moved DeliverCapturedFrame to protected for VideoCaptureAndroid (mjf) ++ int32_t DeliverCapturedFrame(VideoFrame& captureFrame); ++ + char* _deviceUniqueId; // current Device unique name; + Mutex api_lock_; + VideoCaptureCapability _requestedCapability; // Should be set by platform +@@ -89,7 +92,6 @@ class VideoCaptureImpl : public VideoCaptureModule { + private: + void UpdateFrameCount(); + uint32_t CalculateFrameRate(int64_t now_ns); +- int32_t DeliverCapturedFrame(VideoFrame& captureFrame); + void DeliverRawFrame(uint8_t* videoFrame, + size_t videoFrameLength, + const VideoCaptureCapability& frameInfo, +diff --git a/rtc_base/system/warn_current_thread_is_deadlocked.h b/rtc_base/system/warn_current_thread_is_deadlocked.h +index 4a0ba9dc09..eac12022ed 100644 +--- a/rtc_base/system/warn_current_thread_is_deadlocked.h ++++ b/rtc_base/system/warn_current_thread_is_deadlocked.h +@@ -13,7 +13,7 @@ + + namespace webrtc { + +-#if defined(WEBRTC_ANDROID) && !defined(WEBRTC_CHROMIUM_BUILD) ++#if defined(WEBRTC_ANDROID) && !defined(WEBRTC_CHROMIUM_BUILD) && !defined(MOZ_WIDGET_ANDROID) + void WarnThatTheCurrentThreadIsProbablyDeadlocked(); + #else + inline void WarnThatTheCurrentThreadIsProbablyDeadlocked() {} +diff --git a/sdk/android/api/org/webrtc/VideoCodecInfo.java b/sdk/android/api/org/webrtc/VideoCodecInfo.java +index 4f97cf74cf..363be347b5 100644 +--- a/sdk/android/api/org/webrtc/VideoCodecInfo.java ++++ b/sdk/android/api/org/webrtc/VideoCodecInfo.java +@@ -80,7 +80,7 @@ public class VideoCodecInfo { + } + + @CalledByNative +- Map getParams() { ++ Map<String, String> getParams() { + return params; + } + } +-- +2.34.1 + diff --git a/third_party/libwebrtc/moz-patch-stack/0045.patch b/third_party/libwebrtc/moz-patch-stack/0045.patch new file mode 100644 index 0000000000..3c9cf8bc0f --- /dev/null +++ b/third_party/libwebrtc/moz-patch-stack/0045.patch @@ -0,0 +1,61 @@ +From: Michael Froman <mfroman@mozilla.com> +Date: Tue, 15 Jun 2021 12:18:00 -0500 +Subject: Bug 1654112 - suppress android lint warnings for WrongConstant in 2 + libwebrtc java files. r=ng + +Differential Revision: https://phabricator.services.mozilla.com/D118050 +Mercurial Revision: https://hg.mozilla.org/mozilla-central/rev/e60e2f295fb722f69a3a9fe9af34219880afe772 +--- + .../java/src/org/webrtc/voiceengine/WebRtcAudioUtils.java | 5 +++++ + sdk/android/src/java/org/webrtc/audio/WebRtcAudioUtils.java | 5 +++++ + 2 files changed, 10 insertions(+) + +diff --git a/modules/audio_device/android/java/src/org/webrtc/voiceengine/WebRtcAudioUtils.java b/modules/audio_device/android/java/src/org/webrtc/voiceengine/WebRtcAudioUtils.java +index 0472114297..afd3d429af 100644 +--- a/modules/audio_device/android/java/src/org/webrtc/voiceengine/WebRtcAudioUtils.java ++++ b/modules/audio_device/android/java/src/org/webrtc/voiceengine/WebRtcAudioUtils.java +@@ -15,6 +15,7 @@ import static android.media.AudioManager.MODE_IN_COMMUNICATION; + import static android.media.AudioManager.MODE_NORMAL; + import static android.media.AudioManager.MODE_RINGTONE; + ++import android.annotation.SuppressLint; + import android.content.Context; + import android.content.pm.PackageManager; + import android.media.AudioDeviceInfo; +@@ -247,6 +248,10 @@ public final class WebRtcAudioUtils { + } + } + ++ // Moz linting complains even though AudioManager.GET_DEVICES_ALL is ++ // listed in the docs here: ++ // https://developer.android.com/reference/android/media/AudioManager#GET_DEVICES_ALL ++ @SuppressLint("WrongConstant") + private static void logAudioDeviceInfo(String tag, AudioManager audioManager) { + if (Build.VERSION.SDK_INT < 23) { + return; +diff --git a/sdk/android/src/java/org/webrtc/audio/WebRtcAudioUtils.java b/sdk/android/src/java/org/webrtc/audio/WebRtcAudioUtils.java +index 7894659926..7b4b809ab1 100644 +--- a/sdk/android/src/java/org/webrtc/audio/WebRtcAudioUtils.java ++++ b/sdk/android/src/java/org/webrtc/audio/WebRtcAudioUtils.java +@@ -15,6 +15,7 @@ import static android.media.AudioManager.MODE_IN_COMMUNICATION; + import static android.media.AudioManager.MODE_NORMAL; + import static android.media.AudioManager.MODE_RINGTONE; + ++import android.annotation.SuppressLint; + import android.annotation.TargetApi; + import android.content.Context; + import android.content.pm.PackageManager; +@@ -229,6 +230,10 @@ final class WebRtcAudioUtils { + } + } + ++ // Moz linting complains even though AudioManager.GET_DEVICES_ALL is ++ // listed in the docs here: ++ // https://developer.android.com/reference/android/media/AudioManager#GET_DEVICES_ALL ++ @SuppressLint("WrongConstant") + private static void logAudioDeviceInfo(String tag, AudioManager audioManager) { + if (Build.VERSION.SDK_INT < 23) { + return; +-- +2.34.1 + diff --git a/third_party/libwebrtc/moz-patch-stack/0046.patch b/third_party/libwebrtc/moz-patch-stack/0046.patch new file mode 100644 index 0000000000..98fb22f170 --- /dev/null +++ b/third_party/libwebrtc/moz-patch-stack/0046.patch @@ -0,0 +1,38 @@ +From: Michael Froman <mfroman@mozilla.com> +Date: Fri, 25 Jun 2021 15:12:00 -0500 +Subject: Bug 1654112 - Mirror Bug 1714577 - Part 3 - Register WebRTC threads + with the Gecko Profiler. r=ng + +Differential Revision: https://phabricator.services.mozilla.com/D119412 +Mercurial Revision: https://hg.mozilla.org/mozilla-central/rev/d881b16dd8a6813feb5ce1516c2a7ebe0270e72d +--- + rtc_base/platform_thread.cc | 6 ++++++ + 1 file changed, 6 insertions(+) + +diff --git a/rtc_base/platform_thread.cc b/rtc_base/platform_thread.cc +index 556204ac89..71a9f1b224 100644 +--- a/rtc_base/platform_thread.cc ++++ b/rtc_base/platform_thread.cc +@@ -19,6 +19,8 @@ + + #include "rtc_base/checks.h" + ++#include "MicroGeckoProfiler.h" ++ + namespace rtc { + namespace { + +@@ -181,6 +183,10 @@ PlatformThread PlatformThread::SpawnThread( + new std::function<void()>([thread_function = std::move(thread_function), + name = std::string(name), attributes] { + rtc::SetCurrentThreadName(name.c_str()); ++ ++ char stacktop; ++ AutoRegisterProfiler profiler(name.c_str(), &stacktop); ++ + SetPriority(attributes.priority); + thread_function(); + }); +-- +2.34.1 + diff --git a/third_party/libwebrtc/moz-patch-stack/0047.patch b/third_party/libwebrtc/moz-patch-stack/0047.patch new file mode 100644 index 0000000000..fb5bcf0065 --- /dev/null +++ b/third_party/libwebrtc/moz-patch-stack/0047.patch @@ -0,0 +1,300 @@ +From: Nico Grunbaum <na-g@nostrum.com> +Date: Wed, 14 Jul 2021 22:26:00 +0000 +Subject: Bug 1654112 - deconflate the target and host architectures in + libwebrtc build files; r=mjf + +Differential Revision: https://phabricator.services.mozilla.com/D119707 +Mercurial Revision: https://hg.mozilla.org/mozilla-central/rev/58f47eacaf10d12e21dff7362743b6f4cdd1696b +--- + BUILD.gn | 6 +++--- + common_audio/BUILD.gn | 16 ++++++++-------- + common_audio/third_party/ooura/BUILD.gn | 6 +++--- + common_audio/third_party/spl_sqrt_floor/BUILD.gn | 4 ++-- + modules/audio_processing/aec3/BUILD.gn | 6 +++--- + modules/audio_processing/aecm/BUILD.gn | 4 ++-- + modules/audio_processing/agc/BUILD.gn | 2 +- + modules/audio_processing/agc2/rnn_vad/BUILD.gn | 2 +- + modules/audio_processing/ns/BUILD.gn | 2 +- + modules/desktop_capture/BUILD.gn | 2 +- + webrtc.gni | 4 ++-- + 11 files changed, 27 insertions(+), 27 deletions(-) + +diff --git a/BUILD.gn b/BUILD.gn +index 8b2648a306..f32e632ff6 100644 +--- a/BUILD.gn ++++ b/BUILD.gn +@@ -407,12 +407,12 @@ config("common_config") { + } + } + +- if (current_cpu == "arm64") { ++ if (target_cpu == "arm64") { + defines += [ "WEBRTC_ARCH_ARM64" ] + defines += [ "WEBRTC_HAS_NEON" ] + } + +- if (current_cpu == "arm") { ++ if (target_cpu == "arm") { + defines += [ "WEBRTC_ARCH_ARM" ] + if (arm_version >= 7) { + defines += [ "WEBRTC_ARCH_ARM_V7" ] +@@ -422,7 +422,7 @@ config("common_config") { + } + } + +- if (current_cpu == "mipsel") { ++ if (target_cpu == "mipsel") { + defines += [ "MIPS32_LE" ] + if (mips_float_abi == "hard") { + defines += [ "MIPS_FPU_LE" ] +diff --git a/common_audio/BUILD.gn b/common_audio/BUILD.gn +index a45214f754..79d9321bbd 100644 +--- a/common_audio/BUILD.gn ++++ b/common_audio/BUILD.gn +@@ -66,7 +66,7 @@ rtc_library("common_audio") { + deps += [ ":common_audio_neon" ] + } + +- if (current_cpu == "x86" || current_cpu == "x64") { ++ if (target_cpu == "x86" || target_cpu == "x64") { + deps += [ ":common_audio_sse2" ] + deps += [ ":common_audio_avx2" ] + } +@@ -88,7 +88,7 @@ rtc_source_set("mock_common_audio") { + rtc_source_set("common_audio_c_arm_asm") { + sources = [] + deps = [] +- if (current_cpu == "arm") { ++ if (target_cpu == "arm") { + sources += [ "signal_processing/complex_bit_reverse_arm.S" ] + + if (arm_version >= 7) { +@@ -152,7 +152,7 @@ rtc_library("common_audio_c") { + "vad/webrtc_vad.c", + ] + +- if (current_cpu == "mipsel") { ++ if (target_cpu == "mipsel") { + sources += [ + "signal_processing/complex_bit_reverse_mips.c", + "signal_processing/complex_fft_mips.c", +@@ -170,7 +170,7 @@ rtc_library("common_audio_c") { + sources += [ "signal_processing/complex_fft.c" ] + } + +- if (current_cpu != "arm" && current_cpu != "mipsel") { ++ if (target_cpu != "arm" && target_cpu != "mipsel") { + sources += [ + "signal_processing/complex_bit_reverse.c", + "signal_processing/filter_ar_fast_q12.c", +@@ -231,7 +231,7 @@ rtc_library("fir_filter_factory") { + "../rtc_base/system:arch", + "../system_wrappers", + ] +- if (current_cpu == "x86" || current_cpu == "x64") { ++ if (target_cpu == "x86" || target_cpu == "x64") { + deps += [ ":common_audio_sse2" ] + deps += [ ":common_audio_avx2" ] + } +@@ -240,7 +240,7 @@ rtc_library("fir_filter_factory") { + } + } + +-if (current_cpu == "x86" || current_cpu == "x64") { ++if (target_cpu == "x86" || target_cpu == "x64") { + rtc_library("common_audio_sse2") { + sources = [ + "fir_filter_sse.cc", +@@ -289,7 +289,7 @@ if (rtc_build_with_neon) { + "resampler/sinc_resampler_neon.cc", + ] + +- if (current_cpu != "arm64") { ++ if (target_cpu != "arm64") { + # Enable compilation for the NEON instruction set. + suppressed_configs += [ "//build/config/compiler:compiler_arm_fpu" ] + cflags = [ "-mfpu=neon" ] +@@ -312,7 +312,7 @@ if (rtc_build_with_neon) { + "signal_processing/min_max_operations_neon.c", + ] + +- if (current_cpu != "arm64") { ++ if (target_cpu != "arm64") { + # Enable compilation for the NEON instruction set. + suppressed_configs += [ "//build/config/compiler:compiler_arm_fpu" ] + cflags = [ "-mfpu=neon" ] +diff --git a/common_audio/third_party/ooura/BUILD.gn b/common_audio/third_party/ooura/BUILD.gn +index 0cdf98e591..a0ddf777db 100644 +--- a/common_audio/third_party/ooura/BUILD.gn ++++ b/common_audio/third_party/ooura/BUILD.gn +@@ -20,7 +20,7 @@ rtc_library("fft_size_128") { + ] + cflags = [] + +- if (current_cpu == "x86" || current_cpu == "x64") { ++ if (target_cpu == "x86" || target_cpu == "x64") { + sources += [ + "fft_size_128/ooura_fft_sse2.cc", + "fft_size_128/ooura_fft_tables_neon_sse2.h", +@@ -38,14 +38,14 @@ rtc_library("fft_size_128") { + + deps += [ "../../../common_audio" ] + +- if (current_cpu != "arm64") { ++ if (target_cpu != "arm64") { + # Enable compilation for the NEON instruction set. + suppressed_configs += [ "//build/config/compiler:compiler_arm_fpu" ] + cflags += [ "-mfpu=neon" ] + } + } + +- if (current_cpu == "mipsel" && mips_float_abi == "hard") { ++ if (target_cpu == "mipsel" && mips_float_abi == "hard") { + sources += [ "fft_size_128/ooura_fft_mips.cc" ] + } + } +diff --git a/common_audio/third_party/spl_sqrt_floor/BUILD.gn b/common_audio/third_party/spl_sqrt_floor/BUILD.gn +index ac862c65a8..e66ed2796e 100644 +--- a/common_audio/third_party/spl_sqrt_floor/BUILD.gn ++++ b/common_audio/third_party/spl_sqrt_floor/BUILD.gn +@@ -12,11 +12,11 @@ rtc_library("spl_sqrt_floor") { + visibility = [ "../..:common_audio_c" ] + sources = [ "spl_sqrt_floor.h" ] + deps = [] +- if (current_cpu == "arm") { ++ if (target_cpu == "arm") { + sources += [ "spl_sqrt_floor_arm.S" ] + + deps += [ "../../../rtc_base/system:asm_defines" ] +- } else if (current_cpu == "mipsel") { ++ } else if (target_cpu == "mipsel") { + sources += [ "spl_sqrt_floor_mips.c" ] + } else { + sources += [ "spl_sqrt_floor.c" ] +diff --git a/modules/audio_processing/aec3/BUILD.gn b/modules/audio_processing/aec3/BUILD.gn +index 3e11a245a1..c29b893b7d 100644 +--- a/modules/audio_processing/aec3/BUILD.gn ++++ b/modules/audio_processing/aec3/BUILD.gn +@@ -123,7 +123,7 @@ rtc_library("aec3") { + ] + + defines = [] +- if (rtc_build_with_neon && current_cpu != "arm64") { ++ if (rtc_build_with_neon && target_cpu != "arm64") { + suppressed_configs += [ "//build/config/compiler:compiler_arm_fpu" ] + cflags = [ "-mfpu=neon" ] + } +@@ -162,7 +162,7 @@ rtc_library("aec3") { + "//third_party/abseil-cpp/absl/types:optional", + ] + +- if (current_cpu == "x86" || current_cpu == "x64") { ++ if (target_cpu == "x86" || target_cpu == "x64") { + deps += [ ":aec3_avx2" ] + } + } +@@ -253,7 +253,7 @@ rtc_source_set("fft_data") { + ] + } + +-if (current_cpu == "x86" || current_cpu == "x64") { ++if (target_cpu == "x86" || target_cpu == "x64") { + rtc_library("aec3_avx2") { + configs += [ "..:apm_debug_dump" ] + sources = [ +diff --git a/modules/audio_processing/aecm/BUILD.gn b/modules/audio_processing/aecm/BUILD.gn +index 80f2901049..a77f04aba5 100644 +--- a/modules/audio_processing/aecm/BUILD.gn ++++ b/modules/audio_processing/aecm/BUILD.gn +@@ -29,14 +29,14 @@ rtc_library("aecm_core") { + if (rtc_build_with_neon) { + sources += [ "aecm_core_neon.cc" ] + +- if (current_cpu != "arm64") { ++ if (target_cpu != "arm64") { + # Enable compilation for the NEON instruction set. + suppressed_configs += [ "//build/config/compiler:compiler_arm_fpu" ] + cflags += [ "-mfpu=neon" ] + } + } + +- if (current_cpu == "mipsel") { ++ if (target_cpu == "mipsel") { + sources += [ "aecm_core_mips.cc" ] + } else { + sources += [ "aecm_core_c.cc" ] +diff --git a/modules/audio_processing/agc/BUILD.gn b/modules/audio_processing/agc/BUILD.gn +index 508f901b08..75bef1450f 100644 +--- a/modules/audio_processing/agc/BUILD.gn ++++ b/modules/audio_processing/agc/BUILD.gn +@@ -83,7 +83,7 @@ rtc_library("legacy_agc") { + ] + + if (rtc_build_with_neon) { +- if (current_cpu != "arm64") { ++ if (target_cpu != "arm64") { + # Enable compilation for the NEON instruction set. + suppressed_configs += [ "//build/config/compiler:compiler_arm_fpu" ] + cflags = [ "-mfpu=neon" ] +diff --git a/modules/audio_processing/agc2/rnn_vad/BUILD.gn b/modules/audio_processing/agc2/rnn_vad/BUILD.gn +index 3003a585bd..d709eb3699 100644 +--- a/modules/audio_processing/agc2/rnn_vad/BUILD.gn ++++ b/modules/audio_processing/agc2/rnn_vad/BUILD.gn +@@ -18,7 +18,7 @@ rtc_library("rnn_vad") { + ] + + defines = [] +- if (rtc_build_with_neon && current_cpu != "arm64") { ++ if (rtc_build_with_neon && target_cpu != "arm64") { + suppressed_configs += [ "//build/config/compiler:compiler_arm_fpu" ] + cflags = [ "-mfpu=neon" ] + } +diff --git a/modules/audio_processing/ns/BUILD.gn b/modules/audio_processing/ns/BUILD.gn +index d818e23f3c..8c2e9dba84 100644 +--- a/modules/audio_processing/ns/BUILD.gn ++++ b/modules/audio_processing/ns/BUILD.gn +@@ -43,7 +43,7 @@ rtc_static_library("ns") { + ] + + defines = [] +- if (rtc_build_with_neon && current_cpu != "arm64") { ++ if (rtc_build_with_neon && target_cpu != "arm64") { + suppressed_configs += [ "//build/config/compiler:compiler_arm_fpu" ] + cflags = [ "-mfpu=neon" ] + } +diff --git a/modules/desktop_capture/BUILD.gn b/modules/desktop_capture/BUILD.gn +index 4aa9186527..060d4e8200 100644 +--- a/modules/desktop_capture/BUILD.gn ++++ b/modules/desktop_capture/BUILD.gn +@@ -10,7 +10,7 @@ import("//build/config/linux/gtk/gtk.gni") + import("//build/config/ui.gni") + import("../../webrtc.gni") + +-use_desktop_capture_differ_sse2 = current_cpu == "x86" || current_cpu == "x64" ++use_desktop_capture_differ_sse2 = target_cpu == "x86" || target_cpu == "x64" + + config("x11_config") { + if (rtc_use_x11_extensions) { +diff --git a/webrtc.gni b/webrtc.gni +index 6ae1b2329c..e23c9a1cc4 100644 +--- a/webrtc.gni ++++ b/webrtc.gni +@@ -167,13 +167,13 @@ declare_args() { + + # Selects fixed-point code where possible. + rtc_prefer_fixed_point = false +- if (current_cpu == "arm" || current_cpu == "arm64") { ++ if (target_cpu == "arm" || target_cpu == "arm64") { + rtc_prefer_fixed_point = true + } + + # Determines whether NEON code will be built. + rtc_build_with_neon = +- (current_cpu == "arm" && arm_use_neon) || current_cpu == "arm64" ++ (target_cpu == "arm" && arm_use_neon) || target_cpu == "arm64" + + # Enable this to build OpenH264 encoder/FFmpeg decoder. This is supported on + # all platforms except Android and iOS. Because FFmpeg can be built +-- +2.34.1 + diff --git a/third_party/libwebrtc/moz-patch-stack/0048.patch b/third_party/libwebrtc/moz-patch-stack/0048.patch new file mode 100644 index 0000000000..641a244353 --- /dev/null +++ b/third_party/libwebrtc/moz-patch-stack/0048.patch @@ -0,0 +1,297 @@ +From: "Byron Campen [:bwc]" <docfaraday@gmail.com> +Date: Fri, 19 Feb 2021 15:56:00 -0600 +Subject: Bug 1654112 - Get RTCP BYE and RTP timeout handling working again + (from Bug 1595479) r=mjf,dminor + +Differential Revision: https://phabricator.services.mozilla.com/D106145 +Mercurial Revision: https://hg.mozilla.org/mozilla-central/rev/d0b311007c033e83824f5f6996a70ab9e870f31f +--- + audio/audio_receive_stream.cc | 5 ++++- + audio/channel_receive.cc | 13 +++++++++---- + audio/channel_receive.h | 3 ++- + call/audio_receive_stream.h | 3 +++ + call/video_receive_stream.cc | 2 ++ + call/video_receive_stream.h | 3 +++ + modules/rtp_rtcp/include/rtp_rtcp_defines.h | 8 ++++++++ + modules/rtp_rtcp/source/rtcp_receiver.cc | 18 ++++++++++++++++-- + modules/rtp_rtcp/source/rtcp_receiver.h | 1 + + modules/rtp_rtcp/source/rtp_rtcp_interface.h | 3 +++ + video/rtp_video_stream_receiver2.cc | 7 +++++-- + 11 files changed, 56 insertions(+), 10 deletions(-) + +diff --git a/audio/audio_receive_stream.cc b/audio/audio_receive_stream.cc +index 0bb1168384..7063f40186 100644 +--- a/audio/audio_receive_stream.cc ++++ b/audio/audio_receive_stream.cc +@@ -47,6 +47,8 @@ std::string AudioReceiveStreamInterface::Config::Rtp::ToString() const { + } + } + ss << ']'; ++ ss << ", rtcp_event_observer: " ++ << (rtcp_event_observer ? "(rtcp_event_observer)" : "nullptr"); + ss << '}'; + return ss.str(); + } +@@ -81,7 +83,8 @@ std::unique_ptr<voe::ChannelReceiveInterface> CreateChannelReceive( + config.jitter_buffer_fast_accelerate, config.jitter_buffer_min_delay_ms, + config.enable_non_sender_rtt, config.decoder_factory, + config.codec_pair_id, std::move(config.frame_decryptor), +- config.crypto_options, std::move(config.frame_transformer)); ++ config.crypto_options, std::move(config.frame_transformer), ++ config.rtp.rtcp_event_observer); + } + } // namespace + +diff --git a/audio/channel_receive.cc b/audio/channel_receive.cc +index b95d98c20c..50bc94fe1f 100644 +--- a/audio/channel_receive.cc ++++ b/audio/channel_receive.cc +@@ -102,7 +102,8 @@ class ChannelReceive : public ChannelReceiveInterface, + absl::optional<AudioCodecPairId> codec_pair_id, + rtc::scoped_refptr<FrameDecryptorInterface> frame_decryptor, + const webrtc::CryptoOptions& crypto_options, +- rtc::scoped_refptr<FrameTransformerInterface> frame_transformer); ++ rtc::scoped_refptr<FrameTransformerInterface> frame_transformer, ++ RtcpEventObserver* rtcp_event_observer); + ~ChannelReceive() override; + + void SetSink(AudioSinkInterface* sink) override; +@@ -541,7 +542,8 @@ ChannelReceive::ChannelReceive( + absl::optional<AudioCodecPairId> codec_pair_id, + rtc::scoped_refptr<FrameDecryptorInterface> frame_decryptor, + const webrtc::CryptoOptions& crypto_options, +- rtc::scoped_refptr<FrameTransformerInterface> frame_transformer) ++ rtc::scoped_refptr<FrameTransformerInterface> frame_transformer, ++ RtcpEventObserver* rtcp_event_observer) + : worker_thread_(TaskQueueBase::Current()), + event_log_(rtc_event_log), + rtp_receive_statistics_(ReceiveStatistics::Create(clock)), +@@ -586,6 +588,7 @@ ChannelReceive::ChannelReceive( + configuration.local_media_ssrc = local_ssrc; + configuration.rtcp_packet_type_counter_observer = this; + configuration.non_sender_rtt_measurement = enable_non_sender_rtt; ++ configuration.rtcp_event_observer = rtcp_event_observer; + + if (frame_transformer) + InitFrameTransformerDelegate(std::move(frame_transformer)); +@@ -1119,13 +1122,15 @@ std::unique_ptr<ChannelReceiveInterface> CreateChannelReceive( + absl::optional<AudioCodecPairId> codec_pair_id, + rtc::scoped_refptr<FrameDecryptorInterface> frame_decryptor, + const webrtc::CryptoOptions& crypto_options, +- rtc::scoped_refptr<FrameTransformerInterface> frame_transformer) { ++ rtc::scoped_refptr<FrameTransformerInterface> frame_transformer, ++ RtcpEventObserver* rtcp_event_observer) { + return std::make_unique<ChannelReceive>( + clock, neteq_factory, audio_device_module, rtcp_send_transport, + rtc_event_log, local_ssrc, remote_ssrc, jitter_buffer_max_packets, + jitter_buffer_fast_playout, jitter_buffer_min_delay_ms, + enable_non_sender_rtt, decoder_factory, codec_pair_id, +- std::move(frame_decryptor), crypto_options, std::move(frame_transformer)); ++ std::move(frame_decryptor), crypto_options, std::move(frame_transformer), ++ rtcp_event_observer); + } + + } // namespace voe +diff --git a/audio/channel_receive.h b/audio/channel_receive.h +index b47a4b5b97..dd3ca1af83 100644 +--- a/audio/channel_receive.h ++++ b/audio/channel_receive.h +@@ -186,7 +186,8 @@ std::unique_ptr<ChannelReceiveInterface> CreateChannelReceive( + absl::optional<AudioCodecPairId> codec_pair_id, + rtc::scoped_refptr<FrameDecryptorInterface> frame_decryptor, + const webrtc::CryptoOptions& crypto_options, +- rtc::scoped_refptr<FrameTransformerInterface> frame_transformer); ++ rtc::scoped_refptr<FrameTransformerInterface> frame_transformer, ++ RtcpEventObserver* rtcp_event_observer); + + } // namespace voe + } // namespace webrtc +diff --git a/call/audio_receive_stream.h b/call/audio_receive_stream.h +index 1228861c42..6fc93b2d9a 100644 +--- a/call/audio_receive_stream.h ++++ b/call/audio_receive_stream.h +@@ -19,6 +19,7 @@ + #include "absl/types/optional.h" + #include "api/audio_codecs/audio_decoder_factory.h" + #include "api/call/transport.h" ++#include "modules/rtp_rtcp/include/rtp_rtcp_defines.h" + #include "api/crypto/crypto_options.h" + #include "api/rtp_parameters.h" + #include "call/receive_stream.h" +@@ -117,6 +118,8 @@ class AudioReceiveStreamInterface : public MediaReceiveStreamInterface { + + // See NackConfig for description. + NackConfig nack; ++ ++ RtcpEventObserver* rtcp_event_observer = nullptr; + } rtp; + + // Receive-side RTT. +diff --git a/call/video_receive_stream.cc b/call/video_receive_stream.cc +index 87df97cbdd..838dfcf135 100644 +--- a/call/video_receive_stream.cc ++++ b/call/video_receive_stream.cc +@@ -153,6 +153,8 @@ std::string VideoReceiveStreamInterface::Config::Rtp::ToString() const { + ss << ", "; + } + ss << ']'; ++ ss << ", rtcp_event_observer: " ++ << (rtcp_event_observer ? "(rtcp_event_observer)" : "nullptr"); + ss << '}'; + return ss.str(); + } +diff --git a/call/video_receive_stream.h b/call/video_receive_stream.h +index cda8b1f6af..eeb7d14cc3 100644 +--- a/call/video_receive_stream.h ++++ b/call/video_receive_stream.h +@@ -19,6 +19,7 @@ + #include <vector> + + #include "api/call/transport.h" ++#include "modules/rtp_rtcp/include/rtp_rtcp_defines.h" + #include "api/crypto/crypto_options.h" + #include "api/rtp_headers.h" + #include "api/rtp_parameters.h" +@@ -234,6 +235,8 @@ class VideoReceiveStreamInterface : public MediaReceiveStreamInterface { + // meta data is expected to be present in generic frame descriptor + // RTP header extension). + std::set<int> raw_payload_types; ++ ++ RtcpEventObserver* rtcp_event_observer = nullptr; + } rtp; + + // Transport for outgoing packets (RTCP). +diff --git a/modules/rtp_rtcp/include/rtp_rtcp_defines.h b/modules/rtp_rtcp/include/rtp_rtcp_defines.h +index 43bba3e57a..882f861d0b 100644 +--- a/modules/rtp_rtcp/include/rtp_rtcp_defines.h ++++ b/modules/rtp_rtcp/include/rtp_rtcp_defines.h +@@ -211,6 +211,14 @@ class RtcpBandwidthObserver { + virtual ~RtcpBandwidthObserver() {} + }; + ++class RtcpEventObserver { ++ public: ++ virtual void OnRtcpBye() = 0; ++ virtual void OnRtcpTimeout() = 0; ++ ++ virtual ~RtcpEventObserver() {} ++}; ++ + // NOTE! `kNumMediaTypes` must be kept in sync with RtpPacketMediaType! + static constexpr size_t kNumMediaTypes = 5; + enum class RtpPacketMediaType : size_t { +diff --git a/modules/rtp_rtcp/source/rtcp_receiver.cc b/modules/rtp_rtcp/source/rtcp_receiver.cc +index 68171d1c2a..69d62ead5a 100644 +--- a/modules/rtp_rtcp/source/rtcp_receiver.cc ++++ b/modules/rtp_rtcp/source/rtcp_receiver.cc +@@ -145,6 +145,7 @@ RTCPReceiver::RTCPReceiver(const RtpRtcpInterface::Configuration& config, + rtp_rtcp_(owner), + registered_ssrcs_(false, config), + rtcp_bandwidth_observer_(config.bandwidth_callback), ++ rtcp_event_observer_(config.rtcp_event_observer), + rtcp_intra_frame_observer_(config.intra_frame_callback), + rtcp_loss_notification_observer_(config.rtcp_loss_notification_observer), + network_state_estimate_observer_(config.network_state_estimate_observer), +@@ -178,6 +179,7 @@ RTCPReceiver::RTCPReceiver(const RtpRtcpInterface::Configuration& config, + rtp_rtcp_(owner), + registered_ssrcs_(true, config), + rtcp_bandwidth_observer_(config.bandwidth_callback), ++ rtcp_event_observer_(config.rtcp_event_observer), + rtcp_intra_frame_observer_(config.intra_frame_callback), + rtcp_loss_notification_observer_(config.rtcp_loss_notification_observer), + network_state_estimate_observer_(config.network_state_estimate_observer), +@@ -848,6 +850,10 @@ void RTCPReceiver::HandleBye(const CommonHeader& rtcp_block) { + return; + } + ++ if (rtcp_event_observer_) { ++ rtcp_event_observer_->OnRtcpBye(); ++ } ++ + // Clear our lists. + rtts_.erase(bye.sender_ssrc()); + EraseIf(received_report_blocks_, [&](const auto& elem) { +@@ -1265,12 +1271,20 @@ std::vector<rtcp::TmmbItem> RTCPReceiver::TmmbrReceived() { + } + + bool RTCPReceiver::RtcpRrTimeoutLocked(Timestamp now) { +- return ResetTimestampIfExpired(now, last_received_rb_, report_interval_); ++ bool result = ResetTimestampIfExpired(now, last_received_rb_, report_interval_); ++ if (result && rtcp_event_observer_) { ++ rtcp_event_observer_->OnRtcpTimeout(); ++ } ++ return result; + } + + bool RTCPReceiver::RtcpRrSequenceNumberTimeoutLocked(Timestamp now) { +- return ResetTimestampIfExpired(now, last_increased_sequence_number_, ++ bool result = ResetTimestampIfExpired(now, last_increased_sequence_number_, + report_interval_); ++ if (result && rtcp_event_observer_) { ++ rtcp_event_observer_->OnRtcpTimeout(); ++ } ++ return result; + } + + } // namespace webrtc +diff --git a/modules/rtp_rtcp/source/rtcp_receiver.h b/modules/rtp_rtcp/source/rtcp_receiver.h +index 6912912cfc..a05a69059a 100644 +--- a/modules/rtp_rtcp/source/rtcp_receiver.h ++++ b/modules/rtp_rtcp/source/rtcp_receiver.h +@@ -385,6 +385,7 @@ class RTCPReceiver final { + RegisteredSsrcs registered_ssrcs_; + + RtcpBandwidthObserver* const rtcp_bandwidth_observer_; ++ RtcpEventObserver* const rtcp_event_observer_; + RtcpIntraFrameObserver* const rtcp_intra_frame_observer_; + RtcpLossNotificationObserver* const rtcp_loss_notification_observer_; + NetworkStateEstimateObserver* const network_state_estimate_observer_; +diff --git a/modules/rtp_rtcp/source/rtp_rtcp_interface.h b/modules/rtp_rtcp/source/rtp_rtcp_interface.h +index c6854937cb..b988c7805d 100644 +--- a/modules/rtp_rtcp/source/rtp_rtcp_interface.h ++++ b/modules/rtp_rtcp/source/rtp_rtcp_interface.h +@@ -73,6 +73,9 @@ class RtpRtcpInterface : public RtcpFeedbackSenderInterface { + // stream. + RtcpBandwidthObserver* bandwidth_callback = nullptr; + ++ // Called when we receive a RTCP bye or timeout ++ RtcpEventObserver* rtcp_event_observer = nullptr; ++ + NetworkStateEstimateObserver* network_state_estimate_observer = nullptr; + TransportFeedbackObserver* transport_feedback_callback = nullptr; + VideoBitrateAllocationObserver* bitrate_allocation_observer = nullptr; +diff --git a/video/rtp_video_stream_receiver2.cc b/video/rtp_video_stream_receiver2.cc +index eed9770d93..c7b5e7bc7c 100644 +--- a/video/rtp_video_stream_receiver2.cc ++++ b/video/rtp_video_stream_receiver2.cc +@@ -83,7 +83,8 @@ std::unique_ptr<ModuleRtpRtcpImpl2> CreateRtpRtcpModule( + RtcpCnameCallback* rtcp_cname_callback, + bool non_sender_rtt_measurement, + uint32_t local_ssrc, +- RtcEventLog* rtc_event_log) { ++ RtcEventLog* rtc_event_log, ++ RtcpEventObserver* rtcp_event_observer) { + RtpRtcpInterface::Configuration configuration; + configuration.clock = clock; + configuration.audio = false; +@@ -95,6 +96,7 @@ std::unique_ptr<ModuleRtpRtcpImpl2> CreateRtpRtcpModule( + rtcp_packet_type_counter_observer; + configuration.rtcp_cname_callback = rtcp_cname_callback; + configuration.local_media_ssrc = local_ssrc; ++ configuration.rtcp_event_observer = rtcp_event_observer; + configuration.non_sender_rtt_measurement = non_sender_rtt_measurement; + configuration.event_log = rtc_event_log; + +@@ -276,7 +278,8 @@ RtpVideoStreamReceiver2::RtpVideoStreamReceiver2( + rtcp_cname_callback, + config_.rtp.rtcp_xr.receiver_reference_time_report, + config_.rtp.local_ssrc, +- event_log)), ++ event_log, ++ config_.rtp.rtcp_event_observer)), + nack_periodic_processor_(nack_periodic_processor), + complete_frame_callback_(complete_frame_callback), + keyframe_request_method_(config_.rtp.keyframe_method), +-- +2.34.1 + diff --git a/third_party/libwebrtc/moz-patch-stack/0049.patch b/third_party/libwebrtc/moz-patch-stack/0049.patch new file mode 100644 index 0000000000..e804188328 --- /dev/null +++ b/third_party/libwebrtc/moz-patch-stack/0049.patch @@ -0,0 +1,32 @@ +From: "Byron Campen [:bwc]" <docfaraday@gmail.com> +Date: Fri, 12 Mar 2021 08:53:00 -0600 +Subject: Bug 1654112 - libwebrtc modification: Init some stats that were being + passed to us uninitialized. r=ng + +Differential Revision: https://phabricator.services.mozilla.com/D108673 +Mercurial Revision: https://hg.mozilla.org/mozilla-central/rev/c385bb870413b925af48df97aa1f2b80a26e78d2 +--- + call/video_receive_stream.h | 7 ++++--- + 1 file changed, 4 insertions(+), 3 deletions(-) + +diff --git a/call/video_receive_stream.h b/call/video_receive_stream.h +index eeb7d14cc3..31c9bff7b0 100644 +--- a/call/video_receive_stream.h ++++ b/call/video_receive_stream.h +@@ -145,9 +145,10 @@ class VideoReceiveStreamInterface : public MediaReceiveStreamInterface { + RtpReceiveStats rtp_stats; + RtcpPacketTypeCounter rtcp_packet_type_counts; + +- uint32_t rtcp_sender_packets_sent; +- uint32_t rtcp_sender_octets_sent; +- int64_t rtcp_sender_ntp_timestamp_ms; ++ // Mozilla modification: Init these three. ++ uint32_t rtcp_sender_packets_sent = 0; ++ uint32_t rtcp_sender_octets_sent = 0; ++ int64_t rtcp_sender_ntp_timestamp_ms = 0; + + // Timing frame info: all important timestamps for a full lifetime of a + // single 'timing frame'. +-- +2.34.1 + diff --git a/third_party/libwebrtc/moz-patch-stack/0050.patch b/third_party/libwebrtc/moz-patch-stack/0050.patch new file mode 100644 index 0000000000..9eb5a245aa --- /dev/null +++ b/third_party/libwebrtc/moz-patch-stack/0050.patch @@ -0,0 +1,76 @@ +From: "Byron Campen [:bwc]" <docfaraday@gmail.com> +Date: Fri, 12 Mar 2021 08:55:00 -0600 +Subject: Bug 1654112 - libwebrtc modification: Surface video RTCP SR stats + again. r=ng + +libwebrtc has stopped surfacing these, and Chromium does not support +these stats at all. + +Differential Revision: https://phabricator.services.mozilla.com/D108674 +Mercurial Revision: https://hg.mozilla.org/mozilla-central/rev/edac9d01a9ac7594f4b22708a4690753638ca32c +--- + video/rtp_video_stream_receiver2.cc | 10 ++++++++++ + video/rtp_video_stream_receiver2.h | 6 ++++++ + video/video_receive_stream2.cc | 8 ++++++++ + 3 files changed, 24 insertions(+) + +diff --git a/video/rtp_video_stream_receiver2.cc b/video/rtp_video_stream_receiver2.cc +index c7b5e7bc7c..05447ac3bd 100644 +--- a/video/rtp_video_stream_receiver2.cc ++++ b/video/rtp_video_stream_receiver2.cc +@@ -1062,6 +1062,16 @@ absl::optional<int64_t> RtpVideoStreamReceiver2::LastReceivedKeyframePacketMs() + return absl::nullopt; + } + ++// Mozilla modification: VideoReceiveStream2 and friends do not surface RTCP ++// stats at all, and even on the most recent libwebrtc code there does not ++// seem to be any support for these stats right now. So, we hack this in. ++void RtpVideoStreamReceiver2::RemoteRTCPSenderInfo( ++ uint32_t* packet_count, uint32_t* octet_count, ++ int64_t* ntp_timestamp_ms) const { ++ RTC_DCHECK_RUN_ON(&worker_task_checker_); ++ rtp_rtcp_->RemoteRTCPSenderInfo(packet_count, octet_count, ntp_timestamp_ms); ++} ++ + void RtpVideoStreamReceiver2::ManageFrame( + std::unique_ptr<RtpFrameObject> frame) { + RTC_DCHECK_RUN_ON(&packet_sequence_checker_); +diff --git a/video/rtp_video_stream_receiver2.h b/video/rtp_video_stream_receiver2.h +index 48bcab157e..21f125ae2f 100644 +--- a/video/rtp_video_stream_receiver2.h ++++ b/video/rtp_video_stream_receiver2.h +@@ -211,6 +211,12 @@ class RtpVideoStreamReceiver2 : public LossNotificationSender, + absl::optional<int64_t> LastReceivedPacketMs() const; + absl::optional<int64_t> LastReceivedKeyframePacketMs() const; + ++ // Mozilla modification: VideoReceiveStream2 and friends do not surface RTCP ++ // stats at all, and even on the most recent libwebrtc code there does not ++ // seem to be any support for these stats right now. So, we hack this in. ++ void RemoteRTCPSenderInfo(uint32_t* packet_count, uint32_t* octet_count, ++ int64_t* ntp_timestamp_ms) const; ++ + private: + // Implements RtpVideoFrameReceiver. + void ManageFrame(std::unique_ptr<RtpFrameObject> frame) override; +diff --git a/video/video_receive_stream2.cc b/video/video_receive_stream2.cc +index ce96512795..be850834d6 100644 +--- a/video/video_receive_stream2.cc ++++ b/video/video_receive_stream2.cc +@@ -597,6 +597,14 @@ VideoReceiveStreamInterface::Stats VideoReceiveStream2::GetStats() const { + if (rtx_statistician) + stats.total_bitrate_bps += rtx_statistician->BitrateReceived(); + } ++ ++ // Mozilla modification: VideoReceiveStream2 and friends do not surface RTCP ++ // stats at all, and even on the most recent libwebrtc code there does not ++ // seem to be any support for these stats right now. So, we hack this in. ++ rtp_video_stream_receiver_.RemoteRTCPSenderInfo( ++ &stats.rtcp_sender_packets_sent, &stats.rtcp_sender_octets_sent, ++ &stats.rtcp_sender_ntp_timestamp_ms); ++ + return stats; + } + +-- +2.34.1 + diff --git a/third_party/libwebrtc/moz-patch-stack/0051.patch b/third_party/libwebrtc/moz-patch-stack/0051.patch new file mode 100644 index 0000000000..5a487f124f --- /dev/null +++ b/third_party/libwebrtc/moz-patch-stack/0051.patch @@ -0,0 +1,38 @@ +From: Nico Grunbaum <na-g@nostrum.com> +Date: Mon, 26 Jul 2021 22:51:00 -0700 +Subject: Bug 1654112 - fix timestamp issues with RTP sources; r=mjf + +Differential Revision: https://phabricator.services.mozilla.com/D120930 +Mercurial Revision: https://hg.mozilla.org/mozilla-central/rev/de8c14e4972f717bf937b6f2fffcd08c35e21ced +--- + modules/rtp_rtcp/source/source_tracker.cc | 7 ++++++- + 1 file changed, 6 insertions(+), 1 deletion(-) + +diff --git a/modules/rtp_rtcp/source/source_tracker.cc b/modules/rtp_rtcp/source/source_tracker.cc +index 7a5cbac77d..65f21700d0 100644 +--- a/modules/rtp_rtcp/source/source_tracker.cc ++++ b/modules/rtp_rtcp/source/source_tracker.cc +@@ -36,7 +36,8 @@ void SourceTracker::OnFrameDelivered(const RtpPacketInfos& packet_infos) { + SourceKey key(RtpSourceType::CSRC, csrc); + SourceEntry& entry = UpdateEntry(key); + +- entry.timestamp_ms = now_ms; ++ const auto packet_time = packet_info.receive_time().ms(); ++ entry.timestamp_ms = packet_time ? packet_time : now_ms; + entry.audio_level = packet_info.audio_level(); + entry.absolute_capture_time = packet_info.absolute_capture_time(); + entry.local_capture_clock_offset = +@@ -77,6 +78,10 @@ std::vector<RtpSource> SourceTracker::GetSources() const { + .local_capture_clock_offset = entry.local_capture_clock_offset}); + } + ++ std::sort(sources.begin(), sources.end(), [](const auto &a, const auto &b){ ++ return a.timestamp_ms() > b.timestamp_ms(); ++ }); ++ + return sources; + } + +-- +2.34.1 + diff --git a/third_party/libwebrtc/moz-patch-stack/0052.patch b/third_party/libwebrtc/moz-patch-stack/0052.patch new file mode 100644 index 0000000000..1d8546d68d --- /dev/null +++ b/third_party/libwebrtc/moz-patch-stack/0052.patch @@ -0,0 +1,28 @@ +From: Michael Froman <mfroman@mozilla.com> +Date: Thu, 2 Sep 2021 13:22:00 -0500 +Subject: Bug 1654112 - fixes for hybrid build. r=ng + +- adds missing includes in several places +- makes dom/media/webrtc/jsapi unified-only + +Differential Revision: https://phabricator.services.mozilla.com/D124499 +Mercurial Revision: https://hg.mozilla.org/mozilla-central/rev/1965757ee924d49c32eab76d1a3dfa77f9eadcf4 +--- + audio/channel_receive.h | 1 + + 1 file changed, 1 insertion(+) + +diff --git a/audio/channel_receive.h b/audio/channel_receive.h +index dd3ca1af83..1ad3be781b 100644 +--- a/audio/channel_receive.h ++++ b/audio/channel_receive.h +@@ -28,6 +28,7 @@ + #include "call/rtp_packet_sink_interface.h" + #include "call/syncable.h" + #include "modules/audio_coding/include/audio_coding_module_typedefs.h" ++#include "modules/rtp_rtcp/include/rtp_rtcp_defines.h" + #include "modules/rtp_rtcp/source/source_tracker.h" + #include "system_wrappers/include/clock.h" + +-- +2.34.1 + diff --git a/third_party/libwebrtc/moz-patch-stack/0053.patch b/third_party/libwebrtc/moz-patch-stack/0053.patch new file mode 100644 index 0000000000..7bfefa4af5 --- /dev/null +++ b/third_party/libwebrtc/moz-patch-stack/0053.patch @@ -0,0 +1,32 @@ +From: Andreas Pehrson <apehrson@mozilla.com> +Date: Fri, 20 Aug 2021 13:52:00 +0200 +Subject: Bug 1654112 - Don't check the calling thread in + webrtc::AudioReceiveStream::GetSources. r=ng + +source_tracker_ is thread safe with its own internal mutex, so this call is safe +as long as the caller has a guarantee for the lifetime of the +AudioReceiveStream. This is similar to webrtc::VideoReceiveStream. + +Upliftable. + +Differential Revision: https://phabricator.services.mozilla.com/D123226 +Mercurial Revision: https://hg.mozilla.org/mozilla-central/rev/c186df8a088e46285a15e40149182daa34cc6805 +--- + audio/audio_receive_stream.cc | 1 - + 1 file changed, 1 deletion(-) + +diff --git a/audio/audio_receive_stream.cc b/audio/audio_receive_stream.cc +index 7063f40186..20133e6dfe 100644 +--- a/audio/audio_receive_stream.cc ++++ b/audio/audio_receive_stream.cc +@@ -389,7 +389,6 @@ int AudioReceiveStreamImpl::GetBaseMinimumPlayoutDelayMs() const { + } + + std::vector<RtpSource> AudioReceiveStreamImpl::GetSources() const { +- RTC_DCHECK_RUN_ON(&worker_thread_checker_); + return source_tracker_.GetSources(); + } + +-- +2.34.1 + diff --git a/third_party/libwebrtc/moz-patch-stack/0054.patch b/third_party/libwebrtc/moz-patch-stack/0054.patch new file mode 100644 index 0000000000..85bdc9b645 --- /dev/null +++ b/third_party/libwebrtc/moz-patch-stack/0054.patch @@ -0,0 +1,30 @@ +From: Nico Grunbaum <na-g@nostrum.com> +Date: Thu, 28 Oct 2021 18:13:00 +0000 +Subject: Bug 1729367 - P6 - Restore PID recording post cherry-pick;r=mjf + +This restores the code from P0, which was removed to make cherry-picking 439ffe462a66ad9fa9a251b265e4ab28c2647d25 and 449a78b1e20ea85b11f967cf3a184ee610ce21c3 easier. + +Differential Revision: https://phabricator.services.mozilla.com/D129714 +Mercurial Revision: https://hg.mozilla.org/mozilla-central/rev/72a83cb2e571023cd4150bbdef5be5455ce851f4 +--- + modules/desktop_capture/win/window_capture_utils.cc | 4 ++++ + 1 file changed, 4 insertions(+) + +diff --git a/modules/desktop_capture/win/window_capture_utils.cc b/modules/desktop_capture/win/window_capture_utils.cc +index ccfef49bc5..d58c02e17c 100644 +--- a/modules/desktop_capture/win/window_capture_utils.cc ++++ b/modules/desktop_capture/win/window_capture_utils.cc +@@ -79,6 +79,10 @@ BOOL CALLBACK GetWindowListHandler(HWND hwnd, LPARAM param) { + DesktopCapturer::Source window; + window.id = reinterpret_cast<WindowId>(hwnd); + ++ DWORD pid; ++ GetWindowThreadProcessId(hwnd, &pid); ++ window.pid = static_cast<pid_t>(pid); ++ + // GetWindowText* are potentially blocking operations if `hwnd` is + // owned by the current process. The APIs will send messages to the window's + // message loop, and if the message loop is waiting on this operation we will +-- +2.34.1 + diff --git a/third_party/libwebrtc/moz-patch-stack/0055.patch b/third_party/libwebrtc/moz-patch-stack/0055.patch new file mode 100644 index 0000000000..ee530423b1 --- /dev/null +++ b/third_party/libwebrtc/moz-patch-stack/0055.patch @@ -0,0 +1,29 @@ +From: Nico Grunbaum <na-g@nostrum.com> +Date: Thu, 28 Oct 2021 18:13:00 +0000 +Subject: Bug 1729367 - P7 - restore mac PID tracking using new API;r=mjf + a=webrtc-update + +Differential Revision: https://phabricator.services.mozilla.com/D129721 +Mercurial Revision: https://hg.mozilla.org/mozilla-central/rev/1495ca5ef535f8ad692a3a579ca42eddc14f39a8 +--- + modules/desktop_capture/window_capturer_mac.mm | 3 ++- + 1 file changed, 2 insertions(+), 1 deletion(-) + +diff --git a/modules/desktop_capture/window_capturer_mac.mm b/modules/desktop_capture/window_capturer_mac.mm +index f99b4a74d1..10f6a74650 100644 +--- a/modules/desktop_capture/window_capturer_mac.mm ++++ b/modules/desktop_capture/window_capturer_mac.mm +@@ -170,8 +170,9 @@ void WindowCapturerMac::CaptureFrame() { + return webrtc::GetWindowList( + [sources](CFDictionaryRef window) { + WindowId window_id = GetWindowId(window); ++ int pid = GetWindowOwnerPid(window); + if (window_id != kNullWindowId) { +- sources->push_back(DesktopCapturer::Source{window_id, GetWindowTitle(window)}); ++ sources->push_back(DesktopCapturer::Source{window_id, pid, GetWindowTitle(window)}); + } + return true; + }, +-- +2.34.1 + diff --git a/third_party/libwebrtc/moz-patch-stack/0056.patch b/third_party/libwebrtc/moz-patch-stack/0056.patch new file mode 100644 index 0000000000..27b9c65715 --- /dev/null +++ b/third_party/libwebrtc/moz-patch-stack/0056.patch @@ -0,0 +1,208 @@ +From: Andreas Pehrson <apehrson@mozilla.com> +Date: Tue, 2 Nov 2021 14:35:00 +0000 +Subject: Bug 1729455 - Add to stats the local receive time for receiving video + Sender Reports. r=ng + +Differential Revision: https://phabricator.services.mozilla.com/D125712 +Mercurial Revision: https://hg.mozilla.org/mozilla-central/rev/99267b6d193fbcb3e4c845c5e80770424d6d06e2 +--- + call/video_receive_stream.h | 3 ++- + modules/rtp_rtcp/source/rtcp_receiver.cc | 6 ++++-- + modules/rtp_rtcp/source/rtcp_receiver.h | 3 ++- + modules/rtp_rtcp/source/rtp_rtcp_impl.cc | 10 +++++----- + modules/rtp_rtcp/source/rtp_rtcp_impl.h | 3 ++- + modules/rtp_rtcp/source/rtp_rtcp_impl2.cc | 10 +++++----- + modules/rtp_rtcp/source/rtp_rtcp_impl2.h | 3 ++- + modules/rtp_rtcp/source/rtp_rtcp_interface.h | 5 +++-- + video/rtp_video_stream_receiver2.cc | 5 +++-- + video/rtp_video_stream_receiver2.h | 3 ++- + video/video_receive_stream2.cc | 3 ++- + 11 files changed, 32 insertions(+), 22 deletions(-) + +diff --git a/call/video_receive_stream.h b/call/video_receive_stream.h +index 31c9bff7b0..25c294a2a6 100644 +--- a/call/video_receive_stream.h ++++ b/call/video_receive_stream.h +@@ -145,10 +145,11 @@ class VideoReceiveStreamInterface : public MediaReceiveStreamInterface { + RtpReceiveStats rtp_stats; + RtcpPacketTypeCounter rtcp_packet_type_counts; + +- // Mozilla modification: Init these three. ++ // Mozilla modification: Init these. + uint32_t rtcp_sender_packets_sent = 0; + uint32_t rtcp_sender_octets_sent = 0; + int64_t rtcp_sender_ntp_timestamp_ms = 0; ++ int64_t rtcp_sender_remote_ntp_timestamp_ms = 0; + + // Timing frame info: all important timestamps for a full lifetime of a + // single 'timing frame'. +diff --git a/modules/rtp_rtcp/source/rtcp_receiver.cc b/modules/rtp_rtcp/source/rtcp_receiver.cc +index 69d62ead5a..936750c263 100644 +--- a/modules/rtp_rtcp/source/rtcp_receiver.cc ++++ b/modules/rtp_rtcp/source/rtcp_receiver.cc +@@ -432,11 +432,13 @@ RTCPReceiver::ConsumeReceivedXrReferenceTimeInfo() { + + void RTCPReceiver::RemoteRTCPSenderInfo(uint32_t* packet_count, + uint32_t* octet_count, +- int64_t* ntp_timestamp_ms) const { ++ int64_t* ntp_timestamp_ms, ++ int64_t* remote_ntp_timestamp_ms) const { + MutexLock lock(&rtcp_receiver_lock_); + *packet_count = remote_sender_packet_count_; + *octet_count = remote_sender_octet_count_; +- *ntp_timestamp_ms = remote_sender_ntp_time_.ToMs(); ++ *ntp_timestamp_ms = last_received_sr_ntp_.ToMs(); ++ *remote_ntp_timestamp_ms = remote_sender_ntp_time_.ToMs(); + } + + std::vector<ReportBlockData> RTCPReceiver::GetLatestReportBlockData() const { +diff --git a/modules/rtp_rtcp/source/rtcp_receiver.h b/modules/rtp_rtcp/source/rtcp_receiver.h +index a05a69059a..e3f5bc765c 100644 +--- a/modules/rtp_rtcp/source/rtcp_receiver.h ++++ b/modules/rtp_rtcp/source/rtcp_receiver.h +@@ -135,7 +135,8 @@ class RTCPReceiver final { + // Get received sender packet and octet counts + void RemoteRTCPSenderInfo(uint32_t* packet_count, + uint32_t* octet_count, +- int64_t* ntp_timestamp_ms) const; ++ int64_t* ntp_timestamp_ms, ++ int64_t* remote_ntp_timestamp_ms) const; + + // Get rtt. + int32_t RTT(uint32_t remote_ssrc, +diff --git a/modules/rtp_rtcp/source/rtp_rtcp_impl.cc b/modules/rtp_rtcp/source/rtp_rtcp_impl.cc +index bf9e2b3bf9..1c31611409 100644 +--- a/modules/rtp_rtcp/source/rtp_rtcp_impl.cc ++++ b/modules/rtp_rtcp/source/rtp_rtcp_impl.cc +@@ -527,11 +527,11 @@ void ModuleRtpRtcpImpl::GetSendStreamDataCounters( + } + + // Received RTCP report. +-void ModuleRtpRtcpImpl::RemoteRTCPSenderInfo(uint32_t* packet_count, +- uint32_t* octet_count, +- int64_t* ntp_timestamp_ms) const { +- return rtcp_receiver_.RemoteRTCPSenderInfo(packet_count, octet_count, +- ntp_timestamp_ms); ++void ModuleRtpRtcpImpl::RemoteRTCPSenderInfo( ++ uint32_t* packet_count, uint32_t* octet_count, int64_t* ntp_timestamp_ms, ++ int64_t* remote_ntp_timestamp_ms) const { ++ return rtcp_receiver_.RemoteRTCPSenderInfo( ++ packet_count, octet_count, ntp_timestamp_ms, remote_ntp_timestamp_ms); + } + + std::vector<ReportBlockData> ModuleRtpRtcpImpl::GetLatestReportBlockData() +diff --git a/modules/rtp_rtcp/source/rtp_rtcp_impl.h b/modules/rtp_rtcp/source/rtp_rtcp_impl.h +index 5cf558717e..6070b67d44 100644 +--- a/modules/rtp_rtcp/source/rtp_rtcp_impl.h ++++ b/modules/rtp_rtcp/source/rtp_rtcp_impl.h +@@ -194,7 +194,8 @@ class ABSL_DEPRECATED("") ModuleRtpRtcpImpl + + void RemoteRTCPSenderInfo(uint32_t* packet_count, + uint32_t* octet_count, +- int64_t* ntp_timestamp_ms) const override; ++ int64_t* ntp_timestamp_ms, ++ int64_t* remote_ntp_timestamp_ms) const override; + + // A snapshot of the most recent Report Block with additional data of + // interest to statistics. Used to implement RTCRemoteInboundRtpStreamStats. +diff --git a/modules/rtp_rtcp/source/rtp_rtcp_impl2.cc b/modules/rtp_rtcp/source/rtp_rtcp_impl2.cc +index 8378a76133..66d2e7a44e 100644 +--- a/modules/rtp_rtcp/source/rtp_rtcp_impl2.cc ++++ b/modules/rtp_rtcp/source/rtp_rtcp_impl2.cc +@@ -508,11 +508,11 @@ void ModuleRtpRtcpImpl2::GetSendStreamDataCounters( + } + + // Received RTCP report. +-void ModuleRtpRtcpImpl2::RemoteRTCPSenderInfo(uint32_t* packet_count, +- uint32_t* octet_count, +- int64_t* ntp_timestamp_ms) const { +- return rtcp_receiver_.RemoteRTCPSenderInfo(packet_count, octet_count, +- ntp_timestamp_ms); ++void ModuleRtpRtcpImpl2::RemoteRTCPSenderInfo( ++ uint32_t* packet_count, uint32_t* octet_count, int64_t* ntp_timestamp_ms, ++ int64_t* remote_ntp_timestamp_ms) const { ++ return rtcp_receiver_.RemoteRTCPSenderInfo( ++ packet_count, octet_count, ntp_timestamp_ms, remote_ntp_timestamp_ms); + } + + std::vector<ReportBlockData> ModuleRtpRtcpImpl2::GetLatestReportBlockData() +diff --git a/modules/rtp_rtcp/source/rtp_rtcp_impl2.h b/modules/rtp_rtcp/source/rtp_rtcp_impl2.h +index 4ef67d4647..c43d0c34ba 100644 +--- a/modules/rtp_rtcp/source/rtp_rtcp_impl2.h ++++ b/modules/rtp_rtcp/source/rtp_rtcp_impl2.h +@@ -206,7 +206,8 @@ class ModuleRtpRtcpImpl2 final : public RtpRtcpInterface, + + void RemoteRTCPSenderInfo(uint32_t* packet_count, + uint32_t* octet_count, +- int64_t* ntp_timestamp_ms) const override; ++ int64_t* ntp_timestamp_ms, ++ int64_t* remote_ntp_timestamp_ms) const override; + + // A snapshot of the most recent Report Block with additional data of + // interest to statistics. Used to implement RTCRemoteInboundRtpStreamStats. +diff --git a/modules/rtp_rtcp/source/rtp_rtcp_interface.h b/modules/rtp_rtcp/source/rtp_rtcp_interface.h +index b988c7805d..cb4a0a427f 100644 +--- a/modules/rtp_rtcp/source/rtp_rtcp_interface.h ++++ b/modules/rtp_rtcp/source/rtp_rtcp_interface.h +@@ -403,10 +403,11 @@ class RtpRtcpInterface : public RtcpFeedbackSenderInterface { + StreamDataCounters* rtx_counters) const = 0; + + +- // Returns packet count, octet count, and timestamp from RTCP sender report. ++ // Returns packet count, octet count, and timestamps from RTCP sender report. + virtual void RemoteRTCPSenderInfo(uint32_t* packet_count, + uint32_t* octet_count, +- int64_t* ntp_timestamp_ms) const = 0; ++ int64_t* ntp_timestamp_ms, ++ int64_t* remote_ntp_timestamp_ms) const = 0; + // A snapshot of Report Blocks with additional data of interest to statistics. + // Within this list, the sender-source SSRC pair is unique and per-pair the + // ReportBlockData represents the latest Report Block that was received for +diff --git a/video/rtp_video_stream_receiver2.cc b/video/rtp_video_stream_receiver2.cc +index 05447ac3bd..094f8f4a54 100644 +--- a/video/rtp_video_stream_receiver2.cc ++++ b/video/rtp_video_stream_receiver2.cc +@@ -1067,9 +1067,10 @@ absl::optional<int64_t> RtpVideoStreamReceiver2::LastReceivedKeyframePacketMs() + // seem to be any support for these stats right now. So, we hack this in. + void RtpVideoStreamReceiver2::RemoteRTCPSenderInfo( + uint32_t* packet_count, uint32_t* octet_count, +- int64_t* ntp_timestamp_ms) const { ++ int64_t* ntp_timestamp_ms, int64_t* remote_ntp_timestamp_ms) const { + RTC_DCHECK_RUN_ON(&worker_task_checker_); +- rtp_rtcp_->RemoteRTCPSenderInfo(packet_count, octet_count, ntp_timestamp_ms); ++ rtp_rtcp_->RemoteRTCPSenderInfo(packet_count, octet_count, ntp_timestamp_ms, ++ remote_ntp_timestamp_ms); + } + + void RtpVideoStreamReceiver2::ManageFrame( +diff --git a/video/rtp_video_stream_receiver2.h b/video/rtp_video_stream_receiver2.h +index 21f125ae2f..6bf4bf8453 100644 +--- a/video/rtp_video_stream_receiver2.h ++++ b/video/rtp_video_stream_receiver2.h +@@ -215,7 +215,8 @@ class RtpVideoStreamReceiver2 : public LossNotificationSender, + // stats at all, and even on the most recent libwebrtc code there does not + // seem to be any support for these stats right now. So, we hack this in. + void RemoteRTCPSenderInfo(uint32_t* packet_count, uint32_t* octet_count, +- int64_t* ntp_timestamp_ms) const; ++ int64_t* ntp_timestamp_ms, ++ int64_t* remote_ntp_timestamp_ms) const; + + private: + // Implements RtpVideoFrameReceiver. +diff --git a/video/video_receive_stream2.cc b/video/video_receive_stream2.cc +index be850834d6..7cbd49d322 100644 +--- a/video/video_receive_stream2.cc ++++ b/video/video_receive_stream2.cc +@@ -603,7 +603,8 @@ VideoReceiveStreamInterface::Stats VideoReceiveStream2::GetStats() const { + // seem to be any support for these stats right now. So, we hack this in. + rtp_video_stream_receiver_.RemoteRTCPSenderInfo( + &stats.rtcp_sender_packets_sent, &stats.rtcp_sender_octets_sent, +- &stats.rtcp_sender_ntp_timestamp_ms); ++ &stats.rtcp_sender_ntp_timestamp_ms, ++ &stats.rtcp_sender_remote_ntp_timestamp_ms); + + return stats; + } +-- +2.34.1 + diff --git a/third_party/libwebrtc/moz-patch-stack/0057.patch b/third_party/libwebrtc/moz-patch-stack/0057.patch new file mode 100644 index 0000000000..cdb32e990a --- /dev/null +++ b/third_party/libwebrtc/moz-patch-stack/0057.patch @@ -0,0 +1,26 @@ +From: Andreas Pehrson <apehrson@mozilla.com> +Date: Tue, 2 Nov 2021 14:35:00 +0000 +Subject: Bug 1729455 - Ensure the libwebrtc system clock is not used. r=bwc + +Differential Revision: https://phabricator.services.mozilla.com/D128244 +Mercurial Revision: https://hg.mozilla.org/mozilla-central/rev/3e8ac168ee3db089dd892bf140df53e15d6f0918 +--- + rtc_base/system_time.cc | 2 ++ + 1 file changed, 2 insertions(+) + +diff --git a/rtc_base/system_time.cc b/rtc_base/system_time.cc +index 058e6c2990..1a5e447916 100644 +--- a/rtc_base/system_time.cc ++++ b/rtc_base/system_time.cc +@@ -12,6 +12,8 @@ + // rtc::SystemTimeNanos() must be provided externally. + #ifndef WEBRTC_EXCLUDE_SYSTEM_TIME + ++#error Mozilla: Must not use the built-in libwebrtc clock ++ + #include <stdint.h> + + #include <limits> +-- +2.34.1 + diff --git a/third_party/libwebrtc/moz-patch-stack/0058.patch b/third_party/libwebrtc/moz-patch-stack/0058.patch new file mode 100644 index 0000000000..fce52d078f --- /dev/null +++ b/third_party/libwebrtc/moz-patch-stack/0058.patch @@ -0,0 +1,199 @@ +From: Andreas Pehrson <apehrson@mozilla.com> +Date: Tue, 2 Nov 2021 14:35:00 +0000 +Subject: Bug 1729455 - Inject RTCStatsTimestampMakerRealtimeClock into Call + instances. r=bwc + +This patch makes libwebrtc use our clock for timestamps. +It also makes sure there's no use of the libwebrtc realtime clock, other than +for relative time tracking (like timeouts), and that future libwebrtc updates +don't introduce unaudited use of it. + +Differential Revision: https://phabricator.services.mozilla.com/D127714 +Mercurial Revision: https://hg.mozilla.org/mozilla-central/rev/0744d68b8c944e69945de4ac5c4ca71332e78ad8 +--- + audio/channel_send.cc | 2 +- + call/call.cc | 2 ++ + call/call_factory.cc | 4 ++++ + call/degraded_call.cc | 2 ++ + modules/audio_coding/acm2/audio_coding_module.cc | 2 +- + modules/rtp_rtcp/include/flexfec_receiver.h | 2 ++ + modules/rtp_rtcp/source/flexfec_receiver.cc | 2 ++ + rtc_base/task_utils/repeating_task.h | 4 ++-- + system_wrappers/include/clock.h | 2 +- + system_wrappers/source/clock.cc | 2 +- + 10 files changed, 18 insertions(+), 6 deletions(-) + +diff --git a/audio/channel_send.cc b/audio/channel_send.cc +index c84a91770e..9609ac8a31 100644 +--- a/audio/channel_send.cc ++++ b/audio/channel_send.cc +@@ -502,7 +502,7 @@ ChannelSend::ChannelSend( + RtpRtcpInterface::Configuration configuration; + configuration.bandwidth_callback = rtcp_observer_.get(); + configuration.transport_feedback_callback = feedback_observer_; +- configuration.clock = (clock ? clock : Clock::GetRealTimeClock()); ++ configuration.clock = clock; + configuration.audio = true; + configuration.outgoing_transport = rtp_transport; + +diff --git a/call/call.cc b/call/call.cc +index e676d7a30a..a63087f5c1 100644 +--- a/call/call.cc ++++ b/call/call.cc +@@ -485,12 +485,14 @@ std::string Call::Stats::ToString(int64_t time_ms) const { + return ss.str(); + } + ++/* Mozilla: Avoid this since it could use GetRealTimeClock(). + Call* Call::Create(const Call::Config& config) { + Clock* clock = Clock::GetRealTimeClock(); + return Create(config, clock, + RtpTransportControllerSendFactory().Create( + config.ExtractTransportConfig(), clock)); + } ++ */ + + Call* Call::Create(const Call::Config& config, + Clock* clock, +diff --git a/call/call_factory.cc b/call/call_factory.cc +index 380e80ce12..253f8cd7de 100644 +--- a/call/call_factory.cc ++++ b/call/call_factory.cc +@@ -95,6 +95,9 @@ Call* CallFactory::CreateCall(const Call::Config& config) { + + RtpTransportConfig transportConfig = config.ExtractTransportConfig(); + ++ RTC_CHECK(false); ++ return nullptr; ++ /* Mozilla: Avoid this since it could use GetRealTimeClock(). + Call* call = + Call::Create(config, Clock::GetRealTimeClock(), + config.rtp_transport_controller_send_factory->Create( +@@ -107,6 +110,7 @@ Call* CallFactory::CreateCall(const Call::Config& config) { + } + + return call; ++ */ + } + + std::unique_ptr<CallFactoryInterface> CreateCallFactory() { +diff --git a/call/degraded_call.cc b/call/degraded_call.cc +index fc76c7be5c..99c32296c3 100644 +--- a/call/degraded_call.cc ++++ b/call/degraded_call.cc +@@ -129,6 +129,7 @@ bool DegradedCall::FakeNetworkPipeTransportAdapter::SendRtcp( + return true; + } + ++/* Mozilla: Avoid this since it could use GetRealTimeClock(). + DegradedCall::DegradedCall( + std::unique_ptr<Call> call, + const std::vector<TimeScopedNetworkConfig>& send_configs, +@@ -165,6 +166,7 @@ DegradedCall::DegradedCall( + } + } + } ++*/ + + DegradedCall::~DegradedCall() { + RTC_DCHECK_RUN_ON(call_->worker_thread()); +diff --git a/modules/audio_coding/acm2/audio_coding_module.cc b/modules/audio_coding/acm2/audio_coding_module.cc +index 4367ab08fa..2c186273b6 100644 +--- a/modules/audio_coding/acm2/audio_coding_module.cc ++++ b/modules/audio_coding/acm2/audio_coding_module.cc +@@ -620,7 +620,7 @@ int AudioCodingModuleImpl::GetTargetBitrate() const { + AudioCodingModule::Config::Config( + rtc::scoped_refptr<AudioDecoderFactory> decoder_factory) + : neteq_config(), +- clock(Clock::GetRealTimeClock()), ++ clock(Clock::GetRealTimeClockRaw()), + decoder_factory(decoder_factory) { + // Post-decode VAD is disabled by default in NetEq, however, Audio + // Conference Mixer relies on VAD decisions and fails without them. +diff --git a/modules/rtp_rtcp/include/flexfec_receiver.h b/modules/rtp_rtcp/include/flexfec_receiver.h +index 3cf4c3845e..29d9e72786 100644 +--- a/modules/rtp_rtcp/include/flexfec_receiver.h ++++ b/modules/rtp_rtcp/include/flexfec_receiver.h +@@ -29,9 +29,11 @@ class Clock; + + class FlexfecReceiver { + public: ++ /* Mozilla: Avoid this since it could use GetRealTimeClock(). + FlexfecReceiver(uint32_t ssrc, + uint32_t protected_media_ssrc, + RecoveredPacketReceiver* recovered_packet_receiver); ++ */ + FlexfecReceiver(Clock* clock, + uint32_t ssrc, + uint32_t protected_media_ssrc, +diff --git a/modules/rtp_rtcp/source/flexfec_receiver.cc b/modules/rtp_rtcp/source/flexfec_receiver.cc +index 3f345cd6d2..bd67de1f83 100644 +--- a/modules/rtp_rtcp/source/flexfec_receiver.cc ++++ b/modules/rtp_rtcp/source/flexfec_receiver.cc +@@ -29,6 +29,7 @@ constexpr int kPacketLogIntervalMs = 10000; + + } // namespace + ++/* Mozilla: Avoid this since it could use GetRealTimeClock(). + FlexfecReceiver::FlexfecReceiver( + uint32_t ssrc, + uint32_t protected_media_ssrc, +@@ -37,6 +38,7 @@ FlexfecReceiver::FlexfecReceiver( + ssrc, + protected_media_ssrc, + recovered_packet_receiver) {} ++ */ + + FlexfecReceiver::FlexfecReceiver( + Clock* clock, +diff --git a/rtc_base/task_utils/repeating_task.h b/rtc_base/task_utils/repeating_task.h +index e5ea3d8174..c06bac2247 100644 +--- a/rtc_base/task_utils/repeating_task.h ++++ b/rtc_base/task_utils/repeating_task.h +@@ -56,7 +56,7 @@ class RepeatingTaskHandle { + absl::AnyInvocable<TimeDelta()> closure, + TaskQueueBase::DelayPrecision precision = + TaskQueueBase::DelayPrecision::kLow, +- Clock* clock = Clock::GetRealTimeClock()); ++ Clock* clock = Clock::GetRealTimeClockRaw()); + + // DelayedStart is equivalent to Start except that the first invocation of the + // closure will be delayed by the given amount. +@@ -66,7 +66,7 @@ class RepeatingTaskHandle { + absl::AnyInvocable<TimeDelta()> closure, + TaskQueueBase::DelayPrecision precision = + TaskQueueBase::DelayPrecision::kLow, +- Clock* clock = Clock::GetRealTimeClock()); ++ Clock* clock = Clock::GetRealTimeClockRaw()); + + // Stops future invocations of the repeating task closure. Can only be called + // from the TaskQueue where the task is running. The closure is guaranteed to +diff --git a/system_wrappers/include/clock.h b/system_wrappers/include/clock.h +index 60296070cc..214b34c970 100644 +--- a/system_wrappers/include/clock.h ++++ b/system_wrappers/include/clock.h +@@ -49,7 +49,7 @@ class RTC_EXPORT Clock { + } + + // Returns an instance of the real-time system clock implementation. +- static Clock* GetRealTimeClock(); ++ static Clock* GetRealTimeClockRaw(); + }; + + class SimulatedClock : public Clock { +diff --git a/system_wrappers/source/clock.cc b/system_wrappers/source/clock.cc +index 88c99d6a68..f7460b831c 100644 +--- a/system_wrappers/source/clock.cc ++++ b/system_wrappers/source/clock.cc +@@ -57,7 +57,7 @@ class RealTimeClock : public Clock { + } + }; + +-Clock* Clock::GetRealTimeClock() { ++Clock* Clock::GetRealTimeClockRaw() { + static Clock* const clock = new RealTimeClock(); + return clock; + } +-- +2.34.1 + diff --git a/third_party/libwebrtc/moz-patch-stack/0059.patch b/third_party/libwebrtc/moz-patch-stack/0059.patch new file mode 100644 index 0000000000..225dc2a8f2 --- /dev/null +++ b/third_party/libwebrtc/moz-patch-stack/0059.patch @@ -0,0 +1,91 @@ +From: Landry Breuil <landry@openbsd.org> +Date: Wed, 22 Dec 2021 00:09:00 +0000 +Subject: Bug 1654448 - P2 - readd partial support for BSD to webrtc + build;r=mjf + +only OpenBSD/amd64 is supported for now + +Depends on D134432 + +Differential Revision: https://phabricator.services.mozilla.com/D134433 +Mercurial Revision: https://hg.mozilla.org/mozilla-central/rev/0300b32b7de70fb8976dc82d7d3bb3adb9685857 +--- + BUILD.gn | 3 +++ + modules/video_capture/BUILD.gn | 2 +- + modules/video_capture/linux/device_info_v4l2.h | 2 ++ + rtc_base/platform_thread_types.cc | 4 +++- + webrtc.gni | 2 +- + 5 files changed, 10 insertions(+), 3 deletions(-) + +diff --git a/BUILD.gn b/BUILD.gn +index f32e632ff6..0272f6a8fe 100644 +--- a/BUILD.gn ++++ b/BUILD.gn +@@ -215,6 +215,9 @@ config("common_inherited_config") { + if (is_linux || is_chromeos) { + defines += [ "WEBRTC_LINUX" ] + } ++ if (is_bsd) { ++ defines += [ "WEBRTC_BSD" ] ++ } + if (is_mac) { + defines += [ "WEBRTC_MAC" ] + } +diff --git a/modules/video_capture/BUILD.gn b/modules/video_capture/BUILD.gn +index 95548906c4..4a5bf62433 100644 +--- a/modules/video_capture/BUILD.gn ++++ b/modules/video_capture/BUILD.gn +@@ -63,7 +63,7 @@ if (!build_with_chromium) { + "../../system_wrappers", + ] + +- if (is_linux || is_chromeos) { ++ if (is_linux || is_bsd || is_chromeos) { + sources = [ + "linux/device_info_linux.cc", + "linux/device_info_v4l2.cc", +diff --git a/modules/video_capture/linux/device_info_v4l2.h b/modules/video_capture/linux/device_info_v4l2.h +index e3c2395f49..119cb07ab8 100644 +--- a/modules/video_capture/linux/device_info_v4l2.h ++++ b/modules/video_capture/linux/device_info_v4l2.h +@@ -16,7 +16,9 @@ + #include "modules/video_capture/device_info_impl.h" + + #include "rtc_base/platform_thread.h" ++#ifdef WEBRTC_LINUX + #include <sys/inotify.h> ++#endif + + struct v4l2_capability; + +diff --git a/rtc_base/platform_thread_types.cc b/rtc_base/platform_thread_types.cc +index d64ea689bb..c3c6955a7b 100644 +--- a/rtc_base/platform_thread_types.cc ++++ b/rtc_base/platform_thread_types.cc +@@ -50,7 +50,9 @@ PlatformThreadId CurrentThreadId() { + return static_cast<PlatformThreadId>(pthread_self()); + #else + // Default implementation for nacl and solaris. +- return reinterpret_cast<PlatformThreadId>(pthread_self()); ++ // WEBRTC_BSD: pthread_t is a pointer, so cannot be casted to pid_t ++ // (aka int32_t) on 64-bit archs. Required on OpenBSD. ++ return reinterpret_cast<long>(pthread_self()); + #endif + #endif // defined(WEBRTC_POSIX) + } +diff --git a/webrtc.gni b/webrtc.gni +index e23c9a1cc4..1b21d329b2 100644 +--- a/webrtc.gni ++++ b/webrtc.gni +@@ -336,7 +336,7 @@ rtc_opus_dir = "//third_party/opus" + + # Desktop capturer is supported only on Windows, OSX and Linux. + rtc_desktop_capture_supported = +- (is_win && current_os != "winuwp") || is_mac || ++ (is_win && current_os != "winuwp") || is_mac || is_bsd || + ((is_linux || is_chromeos) && (rtc_use_x11_extensions || rtc_use_pipewire)) + + ############################################################################### +-- +2.34.1 + diff --git a/third_party/libwebrtc/moz-patch-stack/0060.patch b/third_party/libwebrtc/moz-patch-stack/0060.patch new file mode 100644 index 0000000000..81458c04df --- /dev/null +++ b/third_party/libwebrtc/moz-patch-stack/0060.patch @@ -0,0 +1,163 @@ +From: Michael Froman <mjfroman@mac.com> +Date: Mon, 4 Apr 2022 12:25:26 -0500 +Subject: Bug 1766646 - (fix) breakout Call::Stats and SharedModuleThread into + seperate files + +--- + call/BUILD.gn | 6 ++++++ + call/call.cc | 13 ------------- + call/call.h | 13 ++----------- + call/call_basic_stats.cc | 20 ++++++++++++++++++++ + call/call_basic_stats.h | 21 +++++++++++++++++++++ + video/video_send_stream.h | 1 - + 6 files changed, 49 insertions(+), 25 deletions(-) + create mode 100644 call/call_basic_stats.cc + create mode 100644 call/call_basic_stats.h + +diff --git a/call/BUILD.gn b/call/BUILD.gn +index 0e52e8fb3f..26618aee80 100644 +--- a/call/BUILD.gn ++++ b/call/BUILD.gn +@@ -33,6 +33,12 @@ rtc_library("call_interfaces") { + "syncable.cc", + "syncable.h", + ] ++ if (build_with_mozilla) { ++ sources += [ ++ "call_basic_stats.cc", ++ "call_basic_stats.h", ++ ] ++ } + + deps = [ + ":audio_sender_interface", +diff --git a/call/call.cc b/call/call.cc +index a63087f5c1..4c3f4b63fc 100644 +--- a/call/call.cc ++++ b/call/call.cc +@@ -472,19 +472,6 @@ class Call final : public webrtc::Call, + }; + } // namespace internal + +-std::string Call::Stats::ToString(int64_t time_ms) const { +- char buf[1024]; +- rtc::SimpleStringBuilder ss(buf); +- ss << "Call stats: " << time_ms << ", {"; +- ss << "send_bw_bps: " << send_bandwidth_bps << ", "; +- ss << "recv_bw_bps: " << recv_bandwidth_bps << ", "; +- ss << "max_pad_bps: " << max_padding_bitrate_bps << ", "; +- ss << "pacer_delay_ms: " << pacer_delay_ms << ", "; +- ss << "rtt_ms: " << rtt_ms; +- ss << '}'; +- return ss.str(); +-} +- + /* Mozilla: Avoid this since it could use GetRealTimeClock(). + Call* Call::Create(const Call::Config& config) { + Clock* clock = Clock::GetRealTimeClock(); +diff --git a/call/call.h b/call/call.h +index 366978392e..42daa95a6c 100644 +--- a/call/call.h ++++ b/call/call.h +@@ -21,6 +21,7 @@ + #include "api/task_queue/task_queue_base.h" + #include "call/audio_receive_stream.h" + #include "call/audio_send_stream.h" ++#include "call/call_basic_stats.h" + #include "call/call_config.h" + #include "call/flexfec_receive_stream.h" + #include "call/packet_receiver.h" +@@ -30,7 +31,6 @@ + #include "rtc_base/copy_on_write_buffer.h" + #include "rtc_base/network/sent_packet.h" + #include "rtc_base/network_route.h" +-#include "rtc_base/ref_count.h" + + namespace webrtc { + +@@ -47,16 +47,7 @@ namespace webrtc { + class Call { + public: + using Config = CallConfig; +- +- struct Stats { +- std::string ToString(int64_t time_ms) const; +- +- int send_bandwidth_bps = 0; // Estimated available send bandwidth. +- int max_padding_bitrate_bps = 0; // Cumulative configured max padding. +- int recv_bandwidth_bps = 0; // Estimated available receive bandwidth. +- int64_t pacer_delay_ms = 0; +- int64_t rtt_ms = -1; +- }; ++ using Stats = CallBasicStats; + + static Call* Create(const Call::Config& config); + static Call* Create(const Call::Config& config, +diff --git a/call/call_basic_stats.cc b/call/call_basic_stats.cc +new file mode 100644 +index 0000000000..74333a663b +--- /dev/null ++++ b/call/call_basic_stats.cc +@@ -0,0 +1,20 @@ ++#include "call/call_basic_stats.h" ++ ++#include "rtc_base/strings/string_builder.h" ++ ++namespace webrtc { ++ ++std::string CallBasicStats::ToString(int64_t time_ms) const { ++ char buf[1024]; ++ rtc::SimpleStringBuilder ss(buf); ++ ss << "Call stats: " << time_ms << ", {"; ++ ss << "send_bw_bps: " << send_bandwidth_bps << ", "; ++ ss << "recv_bw_bps: " << recv_bandwidth_bps << ", "; ++ ss << "max_pad_bps: " << max_padding_bitrate_bps << ", "; ++ ss << "pacer_delay_ms: " << pacer_delay_ms << ", "; ++ ss << "rtt_ms: " << rtt_ms; ++ ss << '}'; ++ return ss.str(); ++} ++ ++} // namespace webrtc +diff --git a/call/call_basic_stats.h b/call/call_basic_stats.h +new file mode 100644 +index 0000000000..98febe9405 +--- /dev/null ++++ b/call/call_basic_stats.h +@@ -0,0 +1,21 @@ ++#ifndef CALL_CALL_BASIC_STATS_H_ ++#define CALL_CALL_BASIC_STATS_H_ ++ ++#include <string> ++ ++namespace webrtc { ++ ++// named to avoid conflicts with video/call_stats.h ++struct CallBasicStats { ++ std::string ToString(int64_t time_ms) const; ++ ++ int send_bandwidth_bps = 0; // Estimated available send bandwidth. ++ int max_padding_bitrate_bps = 0; // Cumulative configured max padding. ++ int recv_bandwidth_bps = 0; // Estimated available receive bandwidth. ++ int64_t pacer_delay_ms = 0; ++ int64_t rtt_ms = -1; ++}; ++ ++} // namespace webrtc ++ ++#endif // CALL_CALL_BASIC_STATS_H_ +diff --git a/video/video_send_stream.h b/video/video_send_stream.h +index a7ce112b21..404873fd39 100644 +--- a/video/video_send_stream.h ++++ b/video/video_send_stream.h +@@ -37,7 +37,6 @@ namespace test { + class VideoSendStreamPeer; + } // namespace test + +-class CallStats; + class IvfFileWriter; + class RateLimiter; + class RtpRtcp; +-- +2.34.1 + diff --git a/third_party/libwebrtc/moz-patch-stack/0061.patch b/third_party/libwebrtc/moz-patch-stack/0061.patch new file mode 100644 index 0000000000..3f39a08eb9 --- /dev/null +++ b/third_party/libwebrtc/moz-patch-stack/0061.patch @@ -0,0 +1,32 @@ +From: Michael Froman <mjfroman@mac.com> +Date: Fri, 8 Apr 2022 11:36:36 -0500 +Subject: Bug 1766646 - (fix-b556b08668) avoid InlinedVector method that can + throw exception + +--- + api/video_codecs/video_encoder.cc | 8 ++++++++ + 1 file changed, 8 insertions(+) + +diff --git a/api/video_codecs/video_encoder.cc b/api/video_codecs/video_encoder.cc +index b85b9328cf..deb4fdc637 100644 +--- a/api/video_codecs/video_encoder.cc ++++ b/api/video_codecs/video_encoder.cc +@@ -179,7 +179,15 @@ std::string VideoEncoder::EncoderInfo::ToString() const { + for (size_t i = 0; i < preferred_pixel_formats.size(); ++i) { + if (i > 0) + oss << ", "; ++#if defined(WEBRTC_MOZILLA_BUILD) ++ // This could assert, as opposed to throw using the form in the ++ // else, but since we're in a for loop that uses .size() we can ++ // be fairly sure that this is safe without doing a further ++ // check to make sure 'i' is in-range. ++ oss << VideoFrameBufferTypeToString(preferred_pixel_formats[i]); ++#else + oss << VideoFrameBufferTypeToString(preferred_pixel_formats.at(i)); ++#endif + } + oss << "]"; + if (is_qp_trusted.has_value()) { +-- +2.34.1 + diff --git a/third_party/libwebrtc/moz-patch-stack/0062.patch b/third_party/libwebrtc/moz-patch-stack/0062.patch new file mode 100644 index 0000000000..fe3ebd6116 --- /dev/null +++ b/third_party/libwebrtc/moz-patch-stack/0062.patch @@ -0,0 +1,81 @@ +From: Michael Froman <mjfroman@mac.com> +Date: Thu, 28 Apr 2022 10:53:43 -0500 +Subject: Bug 1766646 - (fix-a0bb2ef2dc) add back VideoType enum values + +--- + common_video/libyuv/include/webrtc_libyuv.h | 4 ++++ + common_video/libyuv/webrtc_libyuv.cc | 11 +++++++++++ + 2 files changed, 15 insertions(+) + +diff --git a/common_video/libyuv/include/webrtc_libyuv.h b/common_video/libyuv/include/webrtc_libyuv.h +index 08a035a8d7..6d9071bcd5 100644 +--- a/common_video/libyuv/include/webrtc_libyuv.h ++++ b/common_video/libyuv/include/webrtc_libyuv.h +@@ -32,12 +32,16 @@ enum class VideoType { + kI420, + kIYUV, + kRGB24, ++ kABGR, + kARGB, ++ kARGB4444, + kRGB565, ++ kARGB1555, + kYUY2, + kYV12, + kUYVY, + kMJPEG, ++ kNV21, + kBGRA, + kNV12, + }; +diff --git a/common_video/libyuv/webrtc_libyuv.cc b/common_video/libyuv/webrtc_libyuv.cc +index 14e2d22612..8998af191d 100644 +--- a/common_video/libyuv/webrtc_libyuv.cc ++++ b/common_video/libyuv/webrtc_libyuv.cc +@@ -25,6 +25,7 @@ size_t CalcBufferSize(VideoType type, int width, int height) { + size_t buffer_size = 0; + switch (type) { + case VideoType::kI420: ++ case VideoType::kNV21: + case VideoType::kIYUV: + case VideoType::kYV12: + case VideoType::kNV12: { +@@ -33,7 +34,9 @@ size_t CalcBufferSize(VideoType type, int width, int height) { + buffer_size = width * height + half_width * half_height * 2; + break; + } ++ case VideoType::kARGB4444: + case VideoType::kRGB565: ++ case VideoType::kARGB1555: + case VideoType::kYUY2: + case VideoType::kUYVY: + buffer_size = width * height * 2; +@@ -94,6 +97,8 @@ int ConvertVideoType(VideoType video_type) { + return libyuv::FOURCC_YV12; + case VideoType::kRGB24: + return libyuv::FOURCC_24BG; ++ case VideoType::kABGR: ++ return libyuv::FOURCC_ABGR; + case VideoType::kRGB565: + return libyuv::FOURCC_RGBP; + case VideoType::kYUY2: +@@ -102,10 +107,16 @@ int ConvertVideoType(VideoType video_type) { + return libyuv::FOURCC_UYVY; + case VideoType::kMJPEG: + return libyuv::FOURCC_MJPG; ++ case VideoType::kNV21: ++ return libyuv::FOURCC_NV21; + case VideoType::kARGB: + return libyuv::FOURCC_ARGB; + case VideoType::kBGRA: + return libyuv::FOURCC_BGRA; ++ case VideoType::kARGB4444: ++ return libyuv::FOURCC_R444; ++ case VideoType::kARGB1555: ++ return libyuv::FOURCC_RGBO; + case VideoType::kNV12: + return libyuv::FOURCC_NV12; + } +-- +2.34.1 + diff --git a/third_party/libwebrtc/moz-patch-stack/0063.patch b/third_party/libwebrtc/moz-patch-stack/0063.patch new file mode 100644 index 0000000000..37a746e145 --- /dev/null +++ b/third_party/libwebrtc/moz-patch-stack/0063.patch @@ -0,0 +1,76 @@ +From: Michael Froman <mjfroman@mac.com> +Date: Thu, 19 May 2022 15:32:32 -0500 +Subject: Bug 1772380 - Build 1766646 - (fix-c89fdd716c) fixes for the + refactored PlatformThread API;r?mjf + +--- + .../video_capture/linux/device_info_v4l2.cc | 20 ++++++------------- + .../video_capture/linux/device_info_v4l2.h | 3 +-- + 2 files changed, 7 insertions(+), 16 deletions(-) + +diff --git a/modules/video_capture/linux/device_info_v4l2.cc b/modules/video_capture/linux/device_info_v4l2.cc +index e8abcdda78..7651dd6651 100644 +--- a/modules/video_capture/linux/device_info_v4l2.cc ++++ b/modules/video_capture/linux/device_info_v4l2.cc +@@ -133,11 +133,6 @@ int DeviceInfoV4l2::ProcessInotifyEvents() + return 0; + } + +-void DeviceInfoV4l2::InotifyEventThread(void* obj) +-{ +- static_cast<DeviceInfoLinux*> (obj)->InotifyProcess(); +-} +- + void DeviceInfoV4l2::InotifyProcess() + { + _fd_v4l = inotify_init(); +@@ -163,16 +158,14 @@ void DeviceInfoV4l2::InotifyProcess() + + DeviceInfoV4l2::DeviceInfoV4l2() : DeviceInfoImpl() + #ifdef WEBRTC_LINUX +- , _inotifyEventThread(new rtc::PlatformThread( +- InotifyEventThread, this, "InotifyEventThread")) + , _isShutdown(false) + #endif + { + #ifdef WEBRTC_LINUX +- if (_inotifyEventThread) +- { +- _inotifyEventThread->Start(); +- } ++ _inotifyEventThread = rtc::PlatformThread::SpawnJoinable( ++ [this] { ++ InotifyProcess(); ++ }, "InotifyEventThread"); + #endif + } + +@@ -184,9 +177,8 @@ DeviceInfoV4l2::~DeviceInfoV4l2() { + #ifdef WEBRTC_LINUX + _isShutdown = true; + +- if (_inotifyEventThread) { +- _inotifyEventThread->Stop(); +- _inotifyEventThread = nullptr; ++ if (!_inotifyEventThread.empty()) { ++ _inotifyEventThread.Finalize(); + } + #endif + } +diff --git a/modules/video_capture/linux/device_info_v4l2.h b/modules/video_capture/linux/device_info_v4l2.h +index 119cb07ab8..0bec3eb765 100644 +--- a/modules/video_capture/linux/device_info_v4l2.h ++++ b/modules/video_capture/linux/device_info_v4l2.h +@@ -60,8 +60,7 @@ class DeviceInfoV4l2 : public DeviceInfoImpl { + int EventCheck(int fd); + int HandleEvents(int fd); + int ProcessInotifyEvents(); +- std::unique_ptr<rtc::PlatformThread> _inotifyEventThread; +- static void InotifyEventThread(void*); ++ rtc::PlatformThread _inotifyEventThread; + void InotifyProcess(); + int _fd_v4l, _fd_dev, _wd_v4l, _wd_dev; /* accessed on InotifyEventThread thread */ + std::atomic<bool> _isShutdown; +-- +2.34.1 + diff --git a/third_party/libwebrtc/moz-patch-stack/0064.patch b/third_party/libwebrtc/moz-patch-stack/0064.patch new file mode 100644 index 0000000000..34ab2beb03 --- /dev/null +++ b/third_party/libwebrtc/moz-patch-stack/0064.patch @@ -0,0 +1,174 @@ +From: Andreas Pehrson <apehrson@mozilla.com> +Date: Mon, 18 Jan 2021 11:07:00 +0100 +Subject: Bug 1766646 - (fix-ae0d117d51) ifdef our Csrc impl vs upstream's + impl, see Bug 1771332. + +--- + modules/rtp_rtcp/source/rtp_header_extensions.cc | 4 ++++ + modules/rtp_rtcp/source/rtp_header_extensions.h | 4 ++++ + modules/rtp_rtcp/source/rtp_packet.cc | 4 ++++ + modules/rtp_rtcp/source/rtp_sender.cc | 4 ++++ + test/fuzzers/rtp_packet_fuzzer.cc | 4 ++++ + 5 files changed, 20 insertions(+) + +diff --git a/modules/rtp_rtcp/source/rtp_header_extensions.cc b/modules/rtp_rtcp/source/rtp_header_extensions.cc +index a57d9e7f62..de29fd2075 100644 +--- a/modules/rtp_rtcp/source/rtp_header_extensions.cc ++++ b/modules/rtp_rtcp/source/rtp_header_extensions.cc +@@ -185,6 +185,7 @@ bool AudioLevel::Write(rtc::ArrayView<uint8_t> data, + return true; + } + ++#if !defined(WEBRTC_MOZILLA_BUILD) + // An RTP Header Extension for Mixer-to-Client Audio Level Indication + // + // https://tools.ietf.org/html/rfc6465 +@@ -237,6 +238,7 @@ bool CsrcAudioLevel::Write(rtc::ArrayView<uint8_t> data, + } + return true; + } ++#endif + + // From RFC 5450: Transmission Time Offsets in RTP Streams. + // +@@ -446,6 +448,7 @@ bool PlayoutDelayLimits::Write(rtc::ArrayView<uint8_t> data, + return true; + } + ++#if defined(WEBRTC_MOZILLA_BUILD) + // CSRCAudioLevel + // Sample Audio Level Encoding Using the One-Byte Header Format + // Note that the range of len is 1 to 15 which is encoded as 0 to 14 +@@ -484,6 +487,7 @@ bool CsrcAudioLevel::Write(rtc::ArrayView<uint8_t> data, + // This extension if used must have at least one audio level + return csrcAudioLevels.numAudioLevels; + } ++#endif + + // Video Content Type. + // +diff --git a/modules/rtp_rtcp/source/rtp_header_extensions.h b/modules/rtp_rtcp/source/rtp_header_extensions.h +index 89c73955a2..4b4984bf6d 100644 +--- a/modules/rtp_rtcp/source/rtp_header_extensions.h ++++ b/modules/rtp_rtcp/source/rtp_header_extensions.h +@@ -88,6 +88,7 @@ class AudioLevel { + uint8_t audio_level); + }; + ++#if !defined(WEBRTC_MOZILLA_BUILD) + class CsrcAudioLevel { + public: + static constexpr RTPExtensionType kId = kRtpExtensionCsrcAudioLevel; +@@ -102,6 +103,7 @@ class CsrcAudioLevel { + static bool Write(rtc::ArrayView<uint8_t> data, + rtc::ArrayView<const uint8_t> csrc_audio_levels); + }; ++#endif + + class TransmissionOffset { + public: +@@ -292,6 +294,7 @@ class ColorSpaceExtension { + static size_t WriteLuminance(uint8_t* data, float f, int denominator); + }; + ++#if defined(WEBRTC_MOZILLA_BUILD) + class CsrcAudioLevel { + public: + static constexpr RTPExtensionType kId = kRtpExtensionCsrcAudioLevel; +@@ -306,6 +309,7 @@ class CsrcAudioLevel { + static size_t ValueSize(const CsrcAudioLevelList& csrcAudioLevels); + static bool Write(rtc::ArrayView<uint8_t> data, const CsrcAudioLevelList& csrcAudioLevels); + }; ++#endif + + // Base extension class for RTP header extensions which are strings. + // Subclasses must defined kId and kUri static constexpr members. +diff --git a/modules/rtp_rtcp/source/rtp_packet.cc b/modules/rtp_rtcp/source/rtp_packet.cc +index 9495841984..fd2f5c5ae8 100644 +--- a/modules/rtp_rtcp/source/rtp_packet.cc ++++ b/modules/rtp_rtcp/source/rtp_packet.cc +@@ -187,7 +187,9 @@ void RtpPacket::ZeroMutableExtensions() { + break; + } + case RTPExtensionType::kRtpExtensionAudioLevel: ++#if !defined(WEBRTC_MOZILLA_BUILD) + case RTPExtensionType::kRtpExtensionCsrcAudioLevel: ++#endif + case RTPExtensionType::kRtpExtensionAbsoluteCaptureTime: + case RTPExtensionType::kRtpExtensionColorSpace: + case RTPExtensionType::kRtpExtensionGenericFrameDescriptor: +@@ -205,10 +207,12 @@ void RtpPacket::ZeroMutableExtensions() { + // Non-mutable extension. Don't change it. + break; + } ++#if defined(WEBRTC_MOZILLA_BUILD) + case RTPExtensionType::kRtpExtensionCsrcAudioLevel: { + // TODO: This is a Mozilla addition, we need to add a handler for this. + RTC_CHECK(false); + } ++#endif + } + } + } +diff --git a/modules/rtp_rtcp/source/rtp_sender.cc b/modules/rtp_rtcp/source/rtp_sender.cc +index 0ed7243d0a..336a117f4e 100644 +--- a/modules/rtp_rtcp/source/rtp_sender.cc ++++ b/modules/rtp_rtcp/source/rtp_sender.cc +@@ -108,7 +108,9 @@ bool IsNonVolatile(RTPExtensionType type) { + switch (type) { + case kRtpExtensionTransmissionTimeOffset: + case kRtpExtensionAudioLevel: ++#if !defined(WEBRTC_MOZILLA_BUILD) + case kRtpExtensionCsrcAudioLevel: ++#endif + case kRtpExtensionAbsoluteSendTime: + case kRtpExtensionTransportSequenceNumber: + case kRtpExtensionTransportSequenceNumber02: +@@ -132,10 +134,12 @@ bool IsNonVolatile(RTPExtensionType type) { + case kRtpExtensionNumberOfExtensions: + RTC_DCHECK_NOTREACHED(); + return false; ++#if defined(WEBRTC_MOZILLA_BUILD) + case kRtpExtensionCsrcAudioLevel: + // TODO: Mozilla implement for CsrcAudioLevel + RTC_CHECK(false); + return false; ++#endif + } + RTC_CHECK_NOTREACHED(); + } +diff --git a/test/fuzzers/rtp_packet_fuzzer.cc b/test/fuzzers/rtp_packet_fuzzer.cc +index 0e10a8fa3a..5d117529bb 100644 +--- a/test/fuzzers/rtp_packet_fuzzer.cc ++++ b/test/fuzzers/rtp_packet_fuzzer.cc +@@ -77,11 +77,13 @@ void FuzzOneInput(const uint8_t* data, size_t size) { + uint8_t audio_level; + packet.GetExtension<AudioLevel>(&voice_activity, &audio_level); + break; ++#if !defined(WEBRTC_MOZILLA_BUILD) + case kRtpExtensionCsrcAudioLevel: { + std::vector<uint8_t> audio_levels; + packet.GetExtension<CsrcAudioLevel>(&audio_levels); + break; + } ++#endif + case kRtpExtensionAbsoluteSendTime: + uint32_t sendtime; + packet.GetExtension<AbsoluteSendTime>(&sendtime); +@@ -164,11 +166,13 @@ void FuzzOneInput(const uint8_t* data, size_t size) { + // This extension requires state to read and so complicated that + // deserves own fuzzer. + break; ++#if defined(WEBRTC_MOZILLA_BUILD) + case kRtpExtensionCsrcAudioLevel: { + CsrcAudioLevelList levels; + packet.GetExtension<CsrcAudioLevel>(&levels); + break; + } ++#endif + } + } + +-- +2.34.1 + diff --git a/third_party/libwebrtc/moz-patch-stack/0065.patch b/third_party/libwebrtc/moz-patch-stack/0065.patch new file mode 100644 index 0000000000..a26476d3cc --- /dev/null +++ b/third_party/libwebrtc/moz-patch-stack/0065.patch @@ -0,0 +1,25 @@ +From: Michael Froman <mjfroman@mac.com> +Date: Wed, 1 Jun 2022 12:47:00 -0500 +Subject: Bug 1766646 - (fix-f137b75a4d) specify default constructor on + config.emplace(...) + +--- + modules/congestion_controller/goog_cc/loss_based_bwe_v2.cc | 2 +- + 1 file changed, 1 insertion(+), 1 deletion(-) + +diff --git a/modules/congestion_controller/goog_cc/loss_based_bwe_v2.cc b/modules/congestion_controller/goog_cc/loss_based_bwe_v2.cc +index b4d3ae8c1f..b6efdeee9e 100644 +--- a/modules/congestion_controller/goog_cc/loss_based_bwe_v2.cc ++++ b/modules/congestion_controller/goog_cc/loss_based_bwe_v2.cc +@@ -457,7 +457,7 @@ absl::optional<LossBasedBweV2::Config> LossBasedBweV2::CreateConfig( + if (!enabled.Get()) { + return config; + } +- config.emplace(); ++ config.emplace(Config()); + config->bandwidth_rampup_upper_bound_factor = + bandwidth_rampup_upper_bound_factor.Get(); + config->rampup_acceleration_max_factor = rampup_acceleration_max_factor.Get(); +-- +2.34.1 + diff --git a/third_party/libwebrtc/moz-patch-stack/0066.patch b/third_party/libwebrtc/moz-patch-stack/0066.patch new file mode 100644 index 0000000000..72b69625c5 --- /dev/null +++ b/third_party/libwebrtc/moz-patch-stack/0066.patch @@ -0,0 +1,27 @@ +From: Michael Froman <mjfroman@mac.com> +Date: Tue, 21 Jun 2022 11:11:09 -0500 +Subject: Bug 1773223 - Generate webrtc moz.builds for all platforms at once. + r=mjf,firefox-build-system-reviewers,ahochheiden + +--- + build_overrides/build.gni | 4 ++++ + 1 file changed, 4 insertions(+) + +diff --git a/build_overrides/build.gni b/build_overrides/build.gni +index 137b6a40b2..e662c94ece 100644 +--- a/build_overrides/build.gni ++++ b/build_overrides/build.gni +@@ -45,6 +45,10 @@ if (host_os == "mac" || host_os == "linux") { + use_system_xcode = _result == 0 + } + ++use_system_xcode = false ++xcode_version = "10.15" ++mac_xcode_version = "default" ++ + declare_args() { + # WebRTC doesn't depend on //base from production code but only for testing + # purposes. In any case, it doesn't depend on //third_party/perfetto which +-- +2.34.1 + diff --git a/third_party/libwebrtc/moz-patch-stack/0067.patch b/third_party/libwebrtc/moz-patch-stack/0067.patch new file mode 100644 index 0000000000..59954224eb --- /dev/null +++ b/third_party/libwebrtc/moz-patch-stack/0067.patch @@ -0,0 +1,26 @@ +From: Michael Froman <mjfroman@mac.com> +Date: Tue, 21 Jun 2022 11:17:46 -0500 +Subject: Bug 1772380 - to upstream - ref count this in lambda capture + +--- + modules/video_capture/linux/video_capture_v4l2.cc | 4 ++-- + 1 file changed, 2 insertions(+), 2 deletions(-) + +diff --git a/modules/video_capture/linux/video_capture_v4l2.cc b/modules/video_capture/linux/video_capture_v4l2.cc +index b527a331e4..c7dcb722bc 100644 +--- a/modules/video_capture/linux/video_capture_v4l2.cc ++++ b/modules/video_capture/linux/video_capture_v4l2.cc +@@ -250,8 +250,8 @@ int32_t VideoCaptureModuleV4L2::StartCapture( + if (_captureThread.empty()) { + quit_ = false; + _captureThread = rtc::PlatformThread::SpawnJoinable( +- [this] { +- while (CaptureProcess()) { ++ [self = rtc::scoped_refptr(this)] { ++ while (self->CaptureProcess()) { + } + }, + "CaptureThread", +-- +2.34.1 + diff --git a/third_party/libwebrtc/moz-patch-stack/0068.patch b/third_party/libwebrtc/moz-patch-stack/0068.patch new file mode 100644 index 0000000000..a92343b1d3 --- /dev/null +++ b/third_party/libwebrtc/moz-patch-stack/0068.patch @@ -0,0 +1,23 @@ +From: Michael Froman <mjfroman@mac.com> +Date: Wed, 13 Jul 2022 12:50:36 -0500 +Subject: Bug 1766646 - (fix-4d12174ca5) add missing include to fix build + +--- + modules/audio_processing/aec3/multi_channel_content_detector.h | 1 + + 1 file changed, 1 insertion(+) + +diff --git a/modules/audio_processing/aec3/multi_channel_content_detector.h b/modules/audio_processing/aec3/multi_channel_content_detector.h +index be8717f3af..1742c5fc17 100644 +--- a/modules/audio_processing/aec3/multi_channel_content_detector.h ++++ b/modules/audio_processing/aec3/multi_channel_content_detector.h +@@ -11,6 +11,7 @@ + #ifndef MODULES_AUDIO_PROCESSING_AEC3_MULTI_CHANNEL_CONTENT_DETECTOR_H_ + #define MODULES_AUDIO_PROCESSING_AEC3_MULTI_CHANNEL_CONTENT_DETECTOR_H_ + ++#include <memory> + #include <stddef.h> + + #include <memory> +-- +2.34.1 + diff --git a/third_party/libwebrtc/moz-patch-stack/0069.patch b/third_party/libwebrtc/moz-patch-stack/0069.patch new file mode 100644 index 0000000000..05d3466847 --- /dev/null +++ b/third_party/libwebrtc/moz-patch-stack/0069.patch @@ -0,0 +1,51 @@ +From: Michael Froman <mjfroman@mac.com> +Date: Wed, 3 Aug 2022 20:21:25 -0500 +Subject: Bug 1780582 - work around generating VideoFrameBufferType;r=mjf + +--- + .../api/org/webrtc/VideoFrameBufferType.java | 33 +++++++++++++++++++ + 1 file changed, 33 insertions(+) + create mode 100644 sdk/android/api/org/webrtc/VideoFrameBufferType.java + +diff --git a/sdk/android/api/org/webrtc/VideoFrameBufferType.java b/sdk/android/api/org/webrtc/VideoFrameBufferType.java +new file mode 100644 +index 0000000000..7b05b88cba +--- /dev/null ++++ b/sdk/android/api/org/webrtc/VideoFrameBufferType.java +@@ -0,0 +1,33 @@ ++ ++// Copyright 2022 The Chromium Authors. All rights reserved. ++// Use of this source code is governed by a BSD-style license that can be ++// found in the LICENSE file. ++ ++// This file is autogenerated by ++// java_cpp_enum.py ++// From ++// ../../api/video/video_frame_buffer.h ++ ++package org.webrtc; ++ ++import androidx.annotation.IntDef; ++ ++import java.lang.annotation.Retention; ++import java.lang.annotation.RetentionPolicy; ++ ++@IntDef({ ++ VideoFrameBufferType.NATIVE, VideoFrameBufferType.I420, VideoFrameBufferType.I420A, ++ VideoFrameBufferType.I422, VideoFrameBufferType.I444, VideoFrameBufferType.I010, ++ VideoFrameBufferType.I210, VideoFrameBufferType.NV12 ++}) ++@Retention(RetentionPolicy.SOURCE) ++public @interface VideoFrameBufferType { ++ int NATIVE = 0; ++ int I420 = 1; ++ int I420A = 2; ++ int I422 = 3; ++ int I444 = 4; ++ int I010 = 5; ++ int I210 = 6; ++ int NV12 = 7; ++} +-- +2.34.1 + diff --git a/third_party/libwebrtc/moz-patch-stack/0070.patch b/third_party/libwebrtc/moz-patch-stack/0070.patch new file mode 100644 index 0000000000..39c7c3af5f --- /dev/null +++ b/third_party/libwebrtc/moz-patch-stack/0070.patch @@ -0,0 +1,56 @@ +From: Andreas Pehrson <apehrson@mozilla.com> +Date: Mon, 5 Sep 2022 13:56:00 +0000 +Subject: Bug 1786502 - Lock access to DeviceInfo devicechange callbacks. + r=webrtc-reviewers,jib + +Differential Revision: https://phabricator.services.mozilla.com/D155365 +Mercurial Revision: https://hg.mozilla.org/mozilla-central/rev/e826dfadfe1264c59d9b13e3c17d6f75a40f5c33 +--- + modules/video_capture/video_capture.h | 8 +++++++- + 1 file changed, 7 insertions(+), 1 deletion(-) + +diff --git a/modules/video_capture/video_capture.h b/modules/video_capture/video_capture.h +index 258bc7f810..ad1b341b62 100644 +--- a/modules/video_capture/video_capture.h ++++ b/modules/video_capture/video_capture.h +@@ -16,6 +16,8 @@ + #include "modules/desktop_capture/desktop_capture_types.h" + #include "modules/video_capture/raw_video_sink_interface.h" + #include "modules/video_capture/video_capture_defines.h" ++#include "rtc_base/synchronization/mutex.h" ++#include "rtc_base/thread_annotations.h" + #include <set> + + #if defined(ANDROID) +@@ -44,15 +46,18 @@ class VideoCaptureModule : public rtc::RefCountInterface { + virtual uint32_t NumberOfDevices() = 0; + virtual int32_t Refresh() = 0; + virtual void DeviceChange() { ++ MutexLock lock(&_inputCallbacksMutex); + for (auto inputCallBack : _inputCallBacks) { + inputCallBack->OnDeviceChange(); + } + } + virtual void RegisterVideoInputFeedBack(VideoInputFeedBack* callBack) { ++ MutexLock lock(&_inputCallbacksMutex); + _inputCallBacks.insert(callBack); + } + + virtual void DeRegisterVideoInputFeedBack(VideoInputFeedBack* callBack) { ++ MutexLock lock(&_inputCallbacksMutex); + auto it = _inputCallBacks.find(callBack); + if (it != _inputCallBacks.end()) { + _inputCallBacks.erase(it); +@@ -106,7 +111,8 @@ class VideoCaptureModule : public rtc::RefCountInterface { + + virtual ~DeviceInfo() {} + private: +- std::set<VideoInputFeedBack*> _inputCallBacks; ++ Mutex _inputCallbacksMutex; ++ std::set<VideoInputFeedBack*> _inputCallBacks RTC_GUARDED_BY(_inputCallbacksMutex); + }; + + // Register capture data callback +-- +2.34.1 + diff --git a/third_party/libwebrtc/moz-patch-stack/0071.patch b/third_party/libwebrtc/moz-patch-stack/0071.patch new file mode 100644 index 0000000000..27ceba2306 --- /dev/null +++ b/third_party/libwebrtc/moz-patch-stack/0071.patch @@ -0,0 +1,52 @@ +From: Michael Froman <mfroman@mozilla.com> +Date: Mon, 24 Oct 2022 13:00:00 -0500 +Subject: Bug 1797161 - pt1 - tweak BUILD.gn around task_queue_win usage. r?ng! + +Add assurance that we will not build task_queue_win.cc to avoid +possible win32k API usage. + +Differential Revision: https://phabricator.services.mozilla.com/D160115 +Mercurial Revision: https://hg.mozilla.org/mozilla-central/rev/f097eb8cbd8b7686ce306a46a4db691194fd39c1 +--- + api/task_queue/BUILD.gn | 5 +++++ + rtc_base/BUILD.gn | 4 ++++ + 2 files changed, 9 insertions(+) + +diff --git a/api/task_queue/BUILD.gn b/api/task_queue/BUILD.gn +index c9b4a5d0ec..1c342cb57e 100644 +--- a/api/task_queue/BUILD.gn ++++ b/api/task_queue/BUILD.gn +@@ -30,6 +30,11 @@ rtc_library("task_queue") { + ] + } + ++# Mozilla - we want to ensure that rtc_include_tests is set to false ++# to guarantee that default_task_queue_factory is not used so we ++# know that remaining win32k code in task_queue_win.cc is not built. ++# See Bug 1797161 for more info. ++assert(!rtc_include_tests, "Mozilla - verify rtc_include_tests is off") + if (rtc_include_tests) { + rtc_library("task_queue_test") { + visibility = [ "*" ] +diff --git a/rtc_base/BUILD.gn b/rtc_base/BUILD.gn +index 7e162cecbb..3cd0bfff06 100644 +--- a/rtc_base/BUILD.gn ++++ b/rtc_base/BUILD.gn +@@ -686,10 +686,14 @@ if (is_mac || is_ios) { + if (is_win) { + rtc_library("rtc_task_queue_win") { + visibility = [ "../api/task_queue:default_task_queue_factory" ] ++# See Bug 1797161 for more info. Remove from build until win32k ++# usage is removed. ++if (!build_with_mozilla) { + sources = [ + "task_queue_win.cc", + "task_queue_win.h", + ] ++} + deps = [ + ":checks", + ":logging", +-- +2.34.1 + diff --git a/third_party/libwebrtc/moz-patch-stack/0072.patch b/third_party/libwebrtc/moz-patch-stack/0072.patch new file mode 100644 index 0000000000..6ed95e48b9 --- /dev/null +++ b/third_party/libwebrtc/moz-patch-stack/0072.patch @@ -0,0 +1,34 @@ +From: Michael Froman <mfroman@mozilla.com> +Date: Mon, 24 Oct 2022 14:03:00 -0500 +Subject: Bug 1797161 - pt3 - add static_assert to ensure we don't include + task_queue_win.cc in Mozilla builds. r?ng! + +Differential Revision: https://phabricator.services.mozilla.com/D160117 +Mercurial Revision: https://hg.mozilla.org/mozilla-central/rev/50b15e036924203147e34ec20e2689fe4a847645 +--- + rtc_base/task_queue_win.cc | 9 +++++++++ + 1 file changed, 9 insertions(+) + +diff --git a/rtc_base/task_queue_win.cc b/rtc_base/task_queue_win.cc +index 9ea7fc60ae..6da3094548 100644 +--- a/rtc_base/task_queue_win.cc ++++ b/rtc_base/task_queue_win.cc +@@ -8,6 +8,15 @@ + * be found in the AUTHORS file in the root of the source tree. + */ + ++// Mozilla - this file should not be included in Mozilla builds until ++// win32k API usage is removed. This was once done in Bug 1395259, but ++// the upstreaming attempt stalled. Until win32k usage is officially ++// removed upstream, we have reverted to upstream's version of the file ++// (to reduce or elminate merge conflicts), and a static assert is ++// placed here to ensure this file isn't accidentally included in the ++// Mozilla build. ++static_assert(false, "This file should not be built, see Bug 1797161."); ++ + #include "rtc_base/task_queue_win.h" + + // clang-format off +-- +2.34.1 + diff --git a/third_party/libwebrtc/moz-patch-stack/0073.patch b/third_party/libwebrtc/moz-patch-stack/0073.patch new file mode 100644 index 0000000000..08c123508b --- /dev/null +++ b/third_party/libwebrtc/moz-patch-stack/0073.patch @@ -0,0 +1,81 @@ +From: Andreas Pehrson <apehrson@mozilla.com> +Date: Mon, 12 Dec 2022 15:47:00 +0000 +Subject: Bug 1451394 - Expose mac camera capture backend in .gn and switch it + to gecko libyuv. r=webrtc-reviewers,mjf + +Differential Revision: https://phabricator.services.mozilla.com/D163682 +Mercurial Revision: https://hg.mozilla.org/mozilla-central/rev/b0658888969395dca938597783c8a377b9bea209 +--- + BUILD.gn | 4 ++++ + sdk/BUILD.gn | 6 ++++++ + 2 files changed, 10 insertions(+) + +diff --git a/BUILD.gn b/BUILD.gn +index 0272f6a8fe..6515866c2d 100644 +--- a/BUILD.gn ++++ b/BUILD.gn +@@ -559,6 +559,10 @@ if (!build_with_chromium) { + ] + } + ++ if (build_with_mozilla && is_mac) { ++ deps += [ "sdk:videocapture_objc" ] ++ } ++ + if (rtc_enable_protobuf) { + deps += [ "logging:rtc_event_log_proto" ] + } +diff --git a/sdk/BUILD.gn b/sdk/BUILD.gn +index a361656a59..65d1b10124 100644 +--- a/sdk/BUILD.gn ++++ b/sdk/BUILD.gn +@@ -449,6 +449,7 @@ if (is_ios || is_mac) { + ] + } + ++ if (!build_with_mozilla) { + rtc_library("videosource_objc") { + sources = [ + "objc/api/peerconnection/RTCVideoSource+Private.h", +@@ -478,6 +479,7 @@ if (is_ios || is_mac) { + ":used_from_extension", + ] + } ++ } + + rtc_library("videoframebuffer_objc") { + visibility = [ "*" ] +@@ -510,6 +512,7 @@ if (is_ios || is_mac) { + ] + } + ++ if (!build_with_mozilla) { + rtc_library("opengl_objc") { + sources = [ + "objc/components/renderer/opengl/RTCDefaultShader.h", +@@ -662,6 +665,7 @@ if (is_ios || is_mac) { + ":videoframebuffer_objc", + ] + } ++ } + + rtc_library("videocapture_objc") { + visibility = [ "*" ] +@@ -690,6 +694,7 @@ if (is_ios || is_mac) { + ] + } + ++ if (!build_with_mozilla) { + rtc_library("videocodec_objc") { + visibility = [ "*" ] + configs += [ "..:no_global_constructors" ] +@@ -1747,5 +1752,6 @@ if (is_ios || is_mac) { + "VideoToolbox.framework", + ] + } ++ } + } + } +-- +2.34.1 + diff --git a/third_party/libwebrtc/moz-patch-stack/0074.patch b/third_party/libwebrtc/moz-patch-stack/0074.patch new file mode 100644 index 0000000000..dba2e9bd7f --- /dev/null +++ b/third_party/libwebrtc/moz-patch-stack/0074.patch @@ -0,0 +1,31 @@ +From: Andreas Pehrson <apehrson@mozilla.com> +Date: Mon, 12 Dec 2022 15:47:00 +0000 +Subject: Bug 1451394 - Record video frame captures with PerformanceRecorder in + the new mac camera backend. r=padenot + +Also includes: +Bug 1806605 - Pass TrackingId instead of nsCString to CaptureStage. + +Differential Revision: https://phabricator.services.mozilla.com/D163687 +Mercurial Revision: https://hg.mozilla.org/mozilla-central/rev/a7362238c9e6fbe0d28200f6b41fc40a0c9a2158 +--- + modules/video_capture/video_capture.h | 3 +++ + 1 file changed, 3 insertions(+) + +diff --git a/modules/video_capture/video_capture.h b/modules/video_capture/video_capture.h +index ad1b341b62..7e181c538e 100644 +--- a/modules/video_capture/video_capture.h ++++ b/modules/video_capture/video_capture.h +@@ -158,6 +158,9 @@ class VideoCaptureModule : public rtc::RefCountInterface { + // Return whether the rotation is applied or left pending. + virtual bool GetApplyRotation() = 0; + ++ // Mozilla: TrackingId setter for use in profiler markers. ++ virtual void SetTrackingId(uint32_t aTrackingIdProcId) {} ++ + protected: + ~VideoCaptureModule() override {} + }; +-- +2.34.1 + diff --git a/third_party/libwebrtc/moz-patch-stack/0075.patch b/third_party/libwebrtc/moz-patch-stack/0075.patch new file mode 100644 index 0000000000..65398822d0 --- /dev/null +++ b/third_party/libwebrtc/moz-patch-stack/0075.patch @@ -0,0 +1,346 @@ +From: Andreas Pehrson <apehrson@mozilla.com> +Date: Tue, 23 Nov 2021 14:11:00 +0000 +Subject: Bug 1742181 - libwebrtc: Implement packetsDiscarded bookkeeping for + received video. r=ng + +Depends on D131707 + +Differential Revision: https://phabricator.services.mozilla.com/D131708 +Mercurial Revision: https://hg.mozilla.org/mozilla-central/rev/d0196a45a1f449874fc2a759e85e403c45c25575 + +Also includes: + +Bug 1804288 - (fix-de7ae5755b) reimplement Bug 1742181 - libwebrtc: Implement packetsDiscarded bookkeeping for received video. r=pehrsons + +Differential Revision: https://phabricator.services.mozilla.com/D163959 +Mercurial Revision: https://hg.mozilla.org/mozilla-central/rev/ee566d1bfb654d36e5d58dce637fb0580b989ac1 +--- + api/video/frame_buffer.cc | 25 ++++++++++++++++--- + api/video/frame_buffer.h | 4 +++ + call/video_receive_stream.h | 2 ++ + .../include/video_coding_defines.h | 2 ++ + modules/video_coding/packet_buffer.cc | 10 +++++--- + modules/video_coding/packet_buffer.h | 5 +++- + video/receive_statistics_proxy2.cc | 5 ++++ + video/receive_statistics_proxy2.h | 1 + + video/rtp_video_stream_receiver2.cc | 5 +++- + video/rtp_video_stream_receiver2.h | 2 ++ + video/video_receive_stream2.cc | 1 + + video/video_stream_buffer_controller.cc | 12 +++++++++ + video/video_stream_buffer_controller.h | 3 +++ + 13 files changed, 69 insertions(+), 8 deletions(-) + +diff --git a/api/video/frame_buffer.cc b/api/video/frame_buffer.cc +index 4cdf2212a6..8267b8e6cb 100644 +--- a/api/video/frame_buffer.cc ++++ b/api/video/frame_buffer.cc +@@ -140,14 +140,29 @@ void FrameBuffer::DropNextDecodableTemporalUnit() { + } + + auto end_it = std::next(next_decodable_temporal_unit_->last_frame); +- num_dropped_frames_ += std::count_if( +- frames_.begin(), end_it, +- [](const auto& f) { return f.second.encoded_frame != nullptr; }); ++ ++ UpdateDroppedFramesAndDiscardedPackets(frames_.begin(), end_it); + + frames_.erase(frames_.begin(), end_it); + FindNextAndLastDecodableTemporalUnit(); + } + ++void FrameBuffer::UpdateDroppedFramesAndDiscardedPackets(FrameIterator begin_it, ++ FrameIterator end_it) { ++ unsigned int num_discarded_packets = 0; ++ unsigned int num_dropped_frames = ++ std::count_if(begin_it, end_it, [&](const auto& f) { ++ if (f.second.encoded_frame) { ++ const auto& packetInfos = f.second.encoded_frame->PacketInfos(); ++ num_discarded_packets += packetInfos.size(); ++ } ++ return f.second.encoded_frame != nullptr; ++ }); ++ ++ num_dropped_frames_ += num_dropped_frames; ++ num_discarded_packets_ += num_discarded_packets; ++} ++ + absl::optional<int64_t> FrameBuffer::LastContinuousFrameId() const { + return last_continuous_frame_id_; + } +@@ -167,6 +182,9 @@ int FrameBuffer::GetTotalNumberOfContinuousTemporalUnits() const { + int FrameBuffer::GetTotalNumberOfDroppedFrames() const { + return num_dropped_frames_; + } ++int FrameBuffer::GetTotalNumberOfDiscardedPackets() const { ++ return num_discarded_packets_; ++} + + size_t FrameBuffer::CurrentSize() const { + return frames_.size(); +@@ -269,6 +287,7 @@ void FrameBuffer::FindNextAndLastDecodableTemporalUnit() { + } + + void FrameBuffer::Clear() { ++ UpdateDroppedFramesAndDiscardedPackets(frames_.begin(), frames_.end()); + frames_.clear(); + next_decodable_temporal_unit_.reset(); + decodable_temporal_units_info_.reset(); +diff --git a/api/video/frame_buffer.h b/api/video/frame_buffer.h +index 94edf64d5a..81fd12da58 100644 +--- a/api/video/frame_buffer.h ++++ b/api/video/frame_buffer.h +@@ -66,6 +66,7 @@ class FrameBuffer { + + int GetTotalNumberOfContinuousTemporalUnits() const; + int GetTotalNumberOfDroppedFrames() const; ++ int GetTotalNumberOfDiscardedPackets() const; + size_t CurrentSize() const; + + private: +@@ -87,6 +88,8 @@ class FrameBuffer { + void PropagateContinuity(const FrameIterator& frame_it); + void FindNextAndLastDecodableTemporalUnit(); + void Clear(); ++ void UpdateDroppedFramesAndDiscardedPackets(FrameIterator begin_it, ++ FrameIterator end_it); + + const bool legacy_frame_id_jump_behavior_; + const size_t max_size_; +@@ -99,6 +102,7 @@ class FrameBuffer { + + int num_continuous_temporal_units_ = 0; + int num_dropped_frames_ = 0; ++ int num_discarded_packets_ = 0; + }; + + } // namespace webrtc +diff --git a/call/video_receive_stream.h b/call/video_receive_stream.h +index 25c294a2a6..1ab4a2a85b 100644 +--- a/call/video_receive_stream.h ++++ b/call/video_receive_stream.h +@@ -106,6 +106,8 @@ class VideoReceiveStreamInterface : public MediaReceiveStreamInterface { + // https://www.w3.org/TR/webrtc-stats/#dom-rtcvideoreceiverstats-framesdropped + uint32_t frames_dropped = 0; + uint32_t frames_decoded = 0; ++ // https://w3c.github.io/webrtc-stats/#dom-rtcreceivedrtpstreamstats-packetsdiscarded ++ uint64_t packets_discarded = 0; + // https://w3c.github.io/webrtc-stats/#dom-rtcinboundrtpstreamstats-totaldecodetime + TimeDelta total_decode_time = TimeDelta::Zero(); + // https://w3c.github.io/webrtc-stats/#dom-rtcinboundrtpstreamstats-totalprocessingdelay +diff --git a/modules/video_coding/include/video_coding_defines.h b/modules/video_coding/include/video_coding_defines.h +index 8f70e0298d..bf98d5e668 100644 +--- a/modules/video_coding/include/video_coding_defines.h ++++ b/modules/video_coding/include/video_coding_defines.h +@@ -76,6 +76,8 @@ class VCMReceiveStatisticsCallback { + + virtual void OnDroppedFrames(uint32_t frames_dropped) = 0; + ++ virtual void OnDiscardedPackets(uint32_t packets_discarded) = 0; ++ + virtual void OnFrameBufferTimingsUpdated(int max_decode_ms, + int current_delay_ms, + int target_delay_ms, +diff --git a/modules/video_coding/packet_buffer.cc b/modules/video_coding/packet_buffer.cc +index 3dcfc48213..04f02fce97 100644 +--- a/modules/video_coding/packet_buffer.cc ++++ b/modules/video_coding/packet_buffer.cc +@@ -115,25 +115,27 @@ PacketBuffer::InsertResult PacketBuffer::InsertPacket( + return result; + } + +-void PacketBuffer::ClearTo(uint16_t seq_num) { ++uint32_t PacketBuffer::ClearTo(uint16_t seq_num) { + // We have already cleared past this sequence number, no need to do anything. + if (is_cleared_to_first_seq_num_ && + AheadOf<uint16_t>(first_seq_num_, seq_num)) { +- return; ++ return 0; + } + + // If the packet buffer was cleared between a frame was created and returned. + if (!first_packet_received_) +- return; ++ return 0; + + // Avoid iterating over the buffer more than once by capping the number of + // iterations to the `size_` of the buffer. + ++seq_num; ++ uint32_t num_cleared_packets = 0; + size_t diff = ForwardDiff<uint16_t>(first_seq_num_, seq_num); + size_t iterations = std::min(diff, buffer_.size()); + for (size_t i = 0; i < iterations; ++i) { + auto& stored = buffer_[first_seq_num_ % buffer_.size()]; + if (stored != nullptr && AheadOf<uint16_t>(seq_num, stored->seq_num)) { ++ ++num_cleared_packets; + stored = nullptr; + } + ++first_seq_num_; +@@ -149,6 +151,8 @@ void PacketBuffer::ClearTo(uint16_t seq_num) { + + received_padding_.erase(received_padding_.begin(), + received_padding_.lower_bound(seq_num)); ++ ++ return num_cleared_packets; + } + + void PacketBuffer::Clear() { +diff --git a/modules/video_coding/packet_buffer.h b/modules/video_coding/packet_buffer.h +index 53e08c95a1..47b2ffe199 100644 +--- a/modules/video_coding/packet_buffer.h ++++ b/modules/video_coding/packet_buffer.h +@@ -78,7 +78,10 @@ class PacketBuffer { + ABSL_MUST_USE_RESULT InsertResult + InsertPacket(std::unique_ptr<Packet> packet); + ABSL_MUST_USE_RESULT InsertResult InsertPadding(uint16_t seq_num); +- void ClearTo(uint16_t seq_num); ++ ++ // Clear all packets older than |seq_num|. Returns the number of packets ++ // cleared. ++ uint32_t ClearTo(uint16_t seq_num); + void Clear(); + + void ForceSpsPpsIdrIsH264Keyframe(); +diff --git a/video/receive_statistics_proxy2.cc b/video/receive_statistics_proxy2.cc +index 4f208a1d5e..020e4bb0ae 100644 +--- a/video/receive_statistics_proxy2.cc ++++ b/video/receive_statistics_proxy2.cc +@@ -959,6 +959,11 @@ void ReceiveStatisticsProxy::OnDroppedFrames(uint32_t frames_dropped) { + })); + } + ++void ReceiveStatisticsProxy::OnDiscardedPackets(uint32_t packets_discarded) { ++ RTC_DCHECK_RUN_ON(&main_thread_); ++ stats_.packets_discarded += packets_discarded; ++} ++ + void ReceiveStatisticsProxy::OnPreDecode(VideoCodecType codec_type, int qp) { + RTC_DCHECK_RUN_ON(&main_thread_); + last_codec_type_ = codec_type; +diff --git a/video/receive_statistics_proxy2.h b/video/receive_statistics_proxy2.h +index 1a2bb77fa6..20139b45e5 100644 +--- a/video/receive_statistics_proxy2.h ++++ b/video/receive_statistics_proxy2.h +@@ -90,6 +90,7 @@ class ReceiveStatisticsProxy : public VCMReceiveStatisticsCallback, + size_t size_bytes, + VideoContentType content_type) override; + void OnDroppedFrames(uint32_t frames_dropped) override; ++ void OnDiscardedPackets(uint32_t packets_discarded) override; + void OnFrameBufferTimingsUpdated(int max_decode_ms, + int current_delay_ms, + int target_delay_ms, +diff --git a/video/rtp_video_stream_receiver2.cc b/video/rtp_video_stream_receiver2.cc +index 094f8f4a54..46998b6d7c 100644 +--- a/video/rtp_video_stream_receiver2.cc ++++ b/video/rtp_video_stream_receiver2.cc +@@ -244,6 +244,7 @@ RtpVideoStreamReceiver2::RtpVideoStreamReceiver2( + RtcpPacketTypeCounterObserver* rtcp_packet_type_counter_observer, + RtcpCnameCallback* rtcp_cname_callback, + NackPeriodicProcessor* nack_periodic_processor, ++ VCMReceiveStatisticsCallback* vcm_receive_statistics, + OnCompleteFrameCallback* complete_frame_callback, + rtc::scoped_refptr<FrameDecryptorInterface> frame_decryptor, + rtc::scoped_refptr<FrameTransformerInterface> frame_transformer, +@@ -293,6 +294,7 @@ RtpVideoStreamReceiver2::RtpVideoStreamReceiver2( + &rtcp_feedback_buffer_, + &rtcp_feedback_buffer_, + field_trials_)), ++ vcm_receive_statistics_(vcm_receive_statistics), + packet_buffer_(kPacketBufferStartSize, + PacketBufferMaxSize(field_trials_)), + reference_finder_(std::make_unique<RtpFrameReferenceFinder>()), +@@ -1219,7 +1221,8 @@ void RtpVideoStreamReceiver2::FrameDecoded(int64_t picture_id) { + int64_t unwrapped_rtp_seq_num = rtp_seq_num_unwrapper_.Unwrap(seq_num); + packet_infos_.erase(packet_infos_.begin(), + packet_infos_.upper_bound(unwrapped_rtp_seq_num)); +- packet_buffer_.ClearTo(seq_num); ++ uint32_t num_packets_cleared = packet_buffer_.ClearTo(seq_num); ++ vcm_receive_statistics_->OnDiscardedPackets(num_packets_cleared); + reference_finder_->ClearTo(seq_num); + } + } +diff --git a/video/rtp_video_stream_receiver2.h b/video/rtp_video_stream_receiver2.h +index 6bf4bf8453..931525a054 100644 +--- a/video/rtp_video_stream_receiver2.h ++++ b/video/rtp_video_stream_receiver2.h +@@ -91,6 +91,7 @@ class RtpVideoStreamReceiver2 : public LossNotificationSender, + RtcpPacketTypeCounterObserver* rtcp_packet_type_counter_observer, + RtcpCnameCallback* rtcp_cname_callback, + NackPeriodicProcessor* nack_periodic_processor, ++ VCMReceiveStatisticsCallback* vcm_receive_statistics, + // The KeyFrameRequestSender is optional; if not provided, key frame + // requests are sent via the internal RtpRtcp module. + OnCompleteFrameCallback* complete_frame_callback, +@@ -368,6 +369,7 @@ class RtpVideoStreamReceiver2 : public LossNotificationSender, + std::unique_ptr<LossNotificationController> loss_notification_controller_ + RTC_GUARDED_BY(packet_sequence_checker_); + ++ VCMReceiveStatisticsCallback* const vcm_receive_statistics_; + video_coding::PacketBuffer packet_buffer_ + RTC_GUARDED_BY(packet_sequence_checker_); + UniqueTimestampCounter frame_counter_ +diff --git a/video/video_receive_stream2.cc b/video/video_receive_stream2.cc +index 7cbd49d322..beb894e139 100644 +--- a/video/video_receive_stream2.cc ++++ b/video/video_receive_stream2.cc +@@ -211,6 +211,7 @@ VideoReceiveStream2::VideoReceiveStream2( + &stats_proxy_, + &stats_proxy_, + nack_periodic_processor, ++ &stats_proxy_, + this, // OnCompleteFrameCallback + std::move(config_.frame_decryptor), + std::move(config_.frame_transformer), +diff --git a/video/video_stream_buffer_controller.cc b/video/video_stream_buffer_controller.cc +index f7d3acdaf6..7e44eff39a 100644 +--- a/video/video_stream_buffer_controller.cc ++++ b/video/video_stream_buffer_controller.cc +@@ -247,6 +247,7 @@ void VideoStreamBufferController::OnFrameReady( + + // Update stats. + UpdateDroppedFrames(); ++ UpdateDiscardedPackets(); + UpdateJitterDelay(); + UpdateTimingFrameInfo(); + +@@ -312,6 +313,17 @@ void VideoStreamBufferController::UpdateDroppedFrames() + buffer_->GetTotalNumberOfDroppedFrames(); + } + ++void VideoStreamBufferController::UpdateDiscardedPackets() ++ RTC_RUN_ON(&worker_sequence_checker_) { ++ const int discarded_packets = buffer_->GetTotalNumberOfDiscardedPackets() - ++ packets_discarded_before_last_new_frame_; ++ if (discarded_packets > 0) { ++ stats_proxy_->OnDiscardedPackets(discarded_packets); ++ } ++ packets_discarded_before_last_new_frame_ = ++ buffer_->GetTotalNumberOfDiscardedPackets(); ++} ++ + void VideoStreamBufferController::UpdateJitterDelay() { + auto timings = timing_->GetTimings(); + if (timings.num_decoded_frames) { +diff --git a/video/video_stream_buffer_controller.h b/video/video_stream_buffer_controller.h +index ed79b0fa1f..7638c91471 100644 +--- a/video/video_stream_buffer_controller.h ++++ b/video/video_stream_buffer_controller.h +@@ -67,6 +67,7 @@ class VideoStreamBufferController { + void OnTimeout(TimeDelta delay); + void FrameReadyForDecode(uint32_t rtp_timestamp, Timestamp render_time); + void UpdateDroppedFrames() RTC_RUN_ON(&worker_sequence_checker_); ++ void UpdateDiscardedPackets() RTC_RUN_ON(&worker_sequence_checker_); + void UpdateJitterDelay(); + void UpdateTimingFrameInfo(); + bool IsTooManyFramesQueued() const RTC_RUN_ON(&worker_sequence_checker_); +@@ -94,6 +95,8 @@ class VideoStreamBufferController { + RTC_GUARDED_BY(&worker_sequence_checker_); + int frames_dropped_before_last_new_frame_ + RTC_GUARDED_BY(&worker_sequence_checker_) = 0; ++ int packets_discarded_before_last_new_frame_ ++ RTC_GUARDED_BY(&worker_sequence_checker_) = 0; + VCMVideoProtection protection_mode_ + RTC_GUARDED_BY(&worker_sequence_checker_) = kProtectionNack; + +-- +2.34.1 + diff --git a/third_party/libwebrtc/moz-patch-stack/0076.patch b/third_party/libwebrtc/moz-patch-stack/0076.patch new file mode 100644 index 0000000000..af81c02c28 --- /dev/null +++ b/third_party/libwebrtc/moz-patch-stack/0076.patch @@ -0,0 +1,49 @@ +From: Andreas Pehrson <apehrson@mozilla.com> +Date: Thu, 6 Jan 2022 00:16:00 +0000 +Subject: Bug 1748478 - Propagate calculated discarded packets to stats. r=bwc + +Differential Revision: https://phabricator.services.mozilla.com/D135061 +Mercurial Revision: https://hg.mozilla.org/mozilla-central/rev/56fbf0469e25fa0d589c51ca112ce534a7c0ab91 +--- + video/receive_statistics_proxy2.cc | 9 +++++++-- + video/rtp_video_stream_receiver2.cc | 4 +++- + 2 files changed, 10 insertions(+), 3 deletions(-) + +diff --git a/video/receive_statistics_proxy2.cc b/video/receive_statistics_proxy2.cc +index 020e4bb0ae..f5011c46ef 100644 +--- a/video/receive_statistics_proxy2.cc ++++ b/video/receive_statistics_proxy2.cc +@@ -960,8 +960,13 @@ void ReceiveStatisticsProxy::OnDroppedFrames(uint32_t frames_dropped) { + } + + void ReceiveStatisticsProxy::OnDiscardedPackets(uint32_t packets_discarded) { +- RTC_DCHECK_RUN_ON(&main_thread_); +- stats_.packets_discarded += packets_discarded; ++ // Can be called on either the decode queue or the worker thread ++ // See FrameBuffer2 for more details. ++ worker_thread_->PostTask( ++ SafeTask(task_safety_.flag(), [packets_discarded, this]() { ++ RTC_DCHECK_RUN_ON(&main_thread_); ++ stats_.packets_discarded += packets_discarded; ++ })); + } + + void ReceiveStatisticsProxy::OnPreDecode(VideoCodecType codec_type, int qp) { +diff --git a/video/rtp_video_stream_receiver2.cc b/video/rtp_video_stream_receiver2.cc +index 46998b6d7c..a5d5f637e5 100644 +--- a/video/rtp_video_stream_receiver2.cc ++++ b/video/rtp_video_stream_receiver2.cc +@@ -1222,7 +1222,9 @@ void RtpVideoStreamReceiver2::FrameDecoded(int64_t picture_id) { + packet_infos_.erase(packet_infos_.begin(), + packet_infos_.upper_bound(unwrapped_rtp_seq_num)); + uint32_t num_packets_cleared = packet_buffer_.ClearTo(seq_num); +- vcm_receive_statistics_->OnDiscardedPackets(num_packets_cleared); ++ if (num_packets_cleared > 0) { ++ vcm_receive_statistics_->OnDiscardedPackets(num_packets_cleared); ++ } + reference_finder_->ClearTo(seq_num); + } + } +-- +2.34.1 + diff --git a/third_party/libwebrtc/moz-patch-stack/0077.patch b/third_party/libwebrtc/moz-patch-stack/0077.patch new file mode 100644 index 0000000000..a8cd829474 --- /dev/null +++ b/third_party/libwebrtc/moz-patch-stack/0077.patch @@ -0,0 +1,295 @@ +From: Andreas Pehrson <apehrson@mozilla.com> +Date: Thu, 6 Jan 2022 00:16:00 +0000 +Subject: Bug 1748458 - Add TRACE_EVENTs for dropped frames and packets for + received video. r=bwc + +This lets us see in the profiler how many received frames and packets we decide +to drop and the reasons why. + +Differential Revision: https://phabricator.services.mozilla.com/D135062 +Mercurial Revision: https://hg.mozilla.org/mozilla-central/rev/08e252da94c4752eccfd845eef13d8517953cc6a + +Also includes: + +Bug 1804288 - (fix-de7ae5755b) reimplement Bug 1748458 - Add TRACE_EVENTs for dropped frames and packets for received video. r=pehrsons + +Differential Revision: https://phabricator.services.mozilla.com/D163960 +Mercurial Revision: https://hg.mozilla.org/mozilla-central/rev/8e9a326a99cd5eaa6e447ff57c01ad9d79a09744 +--- + api/video/frame_buffer.cc | 33 +++++++++++++++++++++++++ + modules/video_coding/frame_buffer2.cc | 25 +++++++++++++++++++ + video/receive_statistics_proxy2.cc | 11 +++++++++ + video/rtp_video_stream_receiver2.cc | 4 +++ + video/video_stream_buffer_controller.cc | 7 ++++++ + 5 files changed, 80 insertions(+) + +diff --git a/api/video/frame_buffer.cc b/api/video/frame_buffer.cc +index 8267b8e6cb..f5d93f5f76 100644 +--- a/api/video/frame_buffer.cc ++++ b/api/video/frame_buffer.cc +@@ -16,6 +16,7 @@ + #include "absl/container/inlined_vector.h" + #include "rtc_base/logging.h" + #include "rtc_base/numerics/sequence_number_util.h" ++#include "rtc_base/trace_event.h" + + namespace webrtc { + namespace { +@@ -68,7 +69,12 @@ FrameBuffer::FrameBuffer(int max_size, + decoded_frame_history_(max_decode_history) {} + + bool FrameBuffer::InsertFrame(std::unique_ptr<EncodedFrame> frame) { ++ const uint32_t ssrc = ++ frame->PacketInfos().empty() ? 0 : frame->PacketInfos()[0].ssrc(); + if (!ValidReferences(*frame)) { ++ TRACE_EVENT2("webrtc", ++ "FrameBuffer::InsertFrame Frame dropped (Invalid references)", ++ "remote_ssrc", ssrc, "frame_id", frame->Id()); + RTC_DLOG(LS_WARNING) << "Frame " << frame->Id() + << " has invalid references, dropping frame."; + return false; +@@ -78,23 +84,35 @@ bool FrameBuffer::InsertFrame(std::unique_ptr<EncodedFrame> frame) { + if (legacy_frame_id_jump_behavior_ && frame->is_keyframe() && + AheadOf(frame->Timestamp(), + *decoded_frame_history_.GetLastDecodedFrameTimestamp())) { ++ TRACE_EVENT2("webrtc", ++ "FrameBuffer::InsertFrame Frames dropped (OOO + PicId jump)", ++ "remote_ssrc", ssrc, "frame_id", frame->Id()); + RTC_DLOG(LS_WARNING) + << "Keyframe " << frame->Id() + << " has newer timestamp but older picture id, clearing buffer."; + Clear(); + } else { + // Already decoded past this frame. ++ TRACE_EVENT2("webrtc", ++ "FrameBuffer::InsertFrame Frame dropped (Out of order)", ++ "remote_ssrc", ssrc, "frame_id", frame->Id()); + return false; + } + } + + if (frames_.size() == max_size_) { + if (frame->is_keyframe()) { ++ TRACE_EVENT2("webrtc", ++ "FrameBuffer::InsertFrame Frames dropped (KF + Full buffer)", ++ "remote_ssrc", ssrc, "frame_id", frame->Id()); + RTC_DLOG(LS_WARNING) << "Keyframe " << frame->Id() + << " inserted into full buffer, clearing buffer."; + Clear(); + } else { + // No space for this frame. ++ TRACE_EVENT2("webrtc", ++ "FrameBuffer::InsertFrame Frame dropped (Full buffer)", ++ "remote_ssrc", ssrc, "frame_id", frame->Id()); + return false; + } + } +@@ -149,16 +167,31 @@ void FrameBuffer::DropNextDecodableTemporalUnit() { + + void FrameBuffer::UpdateDroppedFramesAndDiscardedPackets(FrameIterator begin_it, + FrameIterator end_it) { ++ uint32_t dropped_ssrc = 0; ++ int64_t dropped_frame_id = 0; + unsigned int num_discarded_packets = 0; + unsigned int num_dropped_frames = + std::count_if(begin_it, end_it, [&](const auto& f) { + if (f.second.encoded_frame) { + const auto& packetInfos = f.second.encoded_frame->PacketInfos(); ++ dropped_frame_id = f.first; ++ if (!packetInfos.empty()) { ++ dropped_ssrc = packetInfos[0].ssrc(); ++ } + num_discarded_packets += packetInfos.size(); + } + return f.second.encoded_frame != nullptr; + }); + ++ if (num_dropped_frames > 0) { ++ TRACE_EVENT2("webrtc", "FrameBuffer Dropping Old Frames", "remote_ssrc", ++ dropped_ssrc, "frame_id", dropped_frame_id); ++ } ++ if (num_discarded_packets > 0) { ++ TRACE_EVENT2("webrtc", "FrameBuffer Discarding Old Packets", "remote_ssrc", ++ dropped_ssrc, "frame_id", dropped_frame_id); ++ } ++ + num_dropped_frames_ += num_dropped_frames; + num_discarded_packets_ += num_discarded_packets; + } +diff --git a/modules/video_coding/frame_buffer2.cc b/modules/video_coding/frame_buffer2.cc +index b289663eec..a70b143a29 100644 +--- a/modules/video_coding/frame_buffer2.cc ++++ b/modules/video_coding/frame_buffer2.cc +@@ -371,9 +371,14 @@ int64_t FrameBuffer::InsertFrame(std::unique_ptr<EncodedFrame> frame) { + + MutexLock lock(&mutex_); + ++ const auto& pis = frame->PacketInfos(); + int64_t last_continuous_frame_id = last_continuous_frame_.value_or(-1); + + if (!ValidReferences(*frame)) { ++ TRACE_EVENT2("webrtc", ++ "FrameBuffer::InsertFrame Frame dropped (Invalid references)", ++ "remote_ssrc", pis.empty() ? 0 : pis[0].ssrc(), "picture_id", ++ frame->Id()); + RTC_LOG(LS_WARNING) << "Frame " << frame->Id() + << " has invalid frame references, dropping frame."; + return last_continuous_frame_id; +@@ -381,11 +386,19 @@ int64_t FrameBuffer::InsertFrame(std::unique_ptr<EncodedFrame> frame) { + + if (frames_.size() >= kMaxFramesBuffered) { + if (frame->is_keyframe()) { ++ TRACE_EVENT2("webrtc", ++ "FrameBuffer::InsertFrame Frames dropped (KF + Full buffer)", ++ "remote_ssrc", pis.empty() ? 0 : pis[0].ssrc(), "picture_id", ++ frame->Id()); + RTC_LOG(LS_WARNING) << "Inserting keyframe " << frame->Id() + << " but buffer is full, clearing" + " buffer and inserting the frame."; + ClearFramesAndHistory(); + } else { ++ TRACE_EVENT2("webrtc", ++ "FrameBuffer::InsertFrame Frame dropped (Full buffer)", ++ "remote_ssrc", pis.empty() ? 0 : pis[0].ssrc(), "picture_id", ++ frame->Id()); + RTC_LOG(LS_WARNING) << "Frame " << frame->Id() + << " could not be inserted due to the frame " + "buffer being full, dropping frame."; +@@ -404,11 +417,19 @@ int64_t FrameBuffer::InsertFrame(std::unique_ptr<EncodedFrame> frame) { + // reconfiguration or some other reason. Even though this is not according + // to spec we can still continue to decode from this frame if it is a + // keyframe. ++ TRACE_EVENT2("webrtc", ++ "FrameBuffer::InsertFrame Frames dropped (OOO + PicId jump)", ++ "remote_ssrc", pis.empty() ? 0 : pis[0].ssrc(), "picture_id", ++ frame->Id()); + RTC_LOG(LS_WARNING) + << "A jump in frame id was detected, clearing buffer."; + ClearFramesAndHistory(); + last_continuous_frame_id = -1; + } else { ++ TRACE_EVENT2("webrtc", ++ "FrameBuffer::InsertFrame Frame dropped (Out of order)", ++ "remote_ssrc", pis.empty() ? 0 : pis[0].ssrc(), "picture_id", ++ frame->Id()); + RTC_LOG(LS_WARNING) << "Frame " << frame->Id() << " inserted after frame " + << *last_decoded_frame + << " was handed off for decoding, dropping frame."; +@@ -421,6 +442,10 @@ int64_t FrameBuffer::InsertFrame(std::unique_ptr<EncodedFrame> frame) { + // when the frame id make large jumps mid stream. + if (!frames_.empty() && frame->Id() < frames_.begin()->first && + frames_.rbegin()->first < frame->Id()) { ++ TRACE_EVENT2("webrtc", ++ "FrameBuffer::InsertFrame Frames dropped (PicId big-jump)", ++ "remote_ssrc", pis.empty() ? 0 : pis[0].ssrc(), "picture_id", ++ frame->Id()); + RTC_LOG(LS_WARNING) << "A jump in frame id was detected, clearing buffer."; + ClearFramesAndHistory(); + last_continuous_frame_id = -1; +diff --git a/video/receive_statistics_proxy2.cc b/video/receive_statistics_proxy2.cc +index f5011c46ef..508c36eaaf 100644 +--- a/video/receive_statistics_proxy2.cc ++++ b/video/receive_statistics_proxy2.cc +@@ -20,6 +20,7 @@ + #include "rtc_base/strings/string_builder.h" + #include "rtc_base/thread.h" + #include "rtc_base/time_utils.h" ++#include "rtc_base/trace_event.h" + #include "system_wrappers/include/clock.h" + #include "system_wrappers/include/metrics.h" + #include "video/video_receive_stream2.h" +@@ -921,6 +922,9 @@ void ReceiveStatisticsProxy::OnCompleteFrame(bool is_keyframe, + VideoContentType content_type) { + RTC_DCHECK_RUN_ON(&main_thread_); + ++ TRACE_EVENT2("webrtc", "ReceiveStatisticsProxy::OnCompleteFrame", ++ "remote_ssrc", remote_ssrc_, "is_keyframe", is_keyframe); ++ + if (is_keyframe) { + ++stats_.frame_counts.key_frames; + } else { +@@ -952,6 +956,8 @@ void ReceiveStatisticsProxy::OnCompleteFrame(bool is_keyframe, + void ReceiveStatisticsProxy::OnDroppedFrames(uint32_t frames_dropped) { + // Can be called on either the decode queue or the worker thread + // See FrameBuffer2 for more details. ++ TRACE_EVENT2("webrtc", "ReceiveStatisticsProxy::OnDroppedFrames", ++ "remote_ssrc", remote_ssrc_, "frames_dropped", frames_dropped); + worker_thread_->PostTask( + SafeTask(task_safety_.flag(), [frames_dropped, this]() { + RTC_DCHECK_RUN_ON(&main_thread_); +@@ -962,6 +968,9 @@ void ReceiveStatisticsProxy::OnDroppedFrames(uint32_t frames_dropped) { + void ReceiveStatisticsProxy::OnDiscardedPackets(uint32_t packets_discarded) { + // Can be called on either the decode queue or the worker thread + // See FrameBuffer2 for more details. ++ TRACE_EVENT2("webrtc", "ReceiveStatisticsProxy::OnDiscardedPackets", ++ "remote_ssrc", remote_ssrc_, "packets_discarded", ++ packets_discarded); + worker_thread_->PostTask( + SafeTask(task_safety_.flag(), [packets_discarded, this]() { + RTC_DCHECK_RUN_ON(&main_thread_); +@@ -991,6 +1000,8 @@ void ReceiveStatisticsProxy::OnStreamInactive() { + + void ReceiveStatisticsProxy::OnRttUpdate(int64_t avg_rtt_ms) { + RTC_DCHECK_RUN_ON(&main_thread_); ++ TRACE_EVENT2("webrtc", "ReceiveStatisticsProxy::OnRttUpdate", ++ "remote_ssrc", remote_ssrc_, "avg_rtt_ms", avg_rtt_ms); + avg_rtt_ms_ = avg_rtt_ms; + } + +diff --git a/video/rtp_video_stream_receiver2.cc b/video/rtp_video_stream_receiver2.cc +index a5d5f637e5..eb0a7787ac 100644 +--- a/video/rtp_video_stream_receiver2.cc ++++ b/video/rtp_video_stream_receiver2.cc +@@ -44,6 +44,7 @@ + #include "rtc_base/checks.h" + #include "rtc_base/logging.h" + #include "rtc_base/strings/string_builder.h" ++#include "rtc_base/trace_event.h" + #include "system_wrappers/include/metrics.h" + #include "system_wrappers/include/ntp_time.h" + +@@ -1223,6 +1224,9 @@ void RtpVideoStreamReceiver2::FrameDecoded(int64_t picture_id) { + packet_infos_.upper_bound(unwrapped_rtp_seq_num)); + uint32_t num_packets_cleared = packet_buffer_.ClearTo(seq_num); + if (num_packets_cleared > 0) { ++ TRACE_EVENT2("webrtc", ++ "RtpVideoStreamReceiver2::FrameDecoded Cleared Old Packets", ++ "remote_ssrc", config_.rtp.remote_ssrc, "seq_num", seq_num); + vcm_receive_statistics_->OnDiscardedPackets(num_packets_cleared); + } + reference_finder_->ClearTo(seq_num); +diff --git a/video/video_stream_buffer_controller.cc b/video/video_stream_buffer_controller.cc +index 7e44eff39a..37724a8338 100644 +--- a/video/video_stream_buffer_controller.cc ++++ b/video/video_stream_buffer_controller.cc +@@ -28,6 +28,7 @@ + #include "rtc_base/checks.h" + #include "rtc_base/logging.h" + #include "rtc_base/thread_annotations.h" ++#include "rtc_base/trace_event.h" + #include "video/frame_decode_scheduler.h" + #include "video/frame_decode_timing.h" + #include "video/task_queue_frame_decode_scheduler.h" +@@ -139,6 +140,9 @@ absl::optional<int64_t> VideoStreamBufferController::InsertFrame( + std::unique_ptr<EncodedFrame> frame) { + RTC_DCHECK_RUN_ON(&worker_sequence_checker_); + FrameMetadata metadata(*frame); ++ const uint32_t ssrc = ++ frame->PacketInfos().empty() ? 0 : frame->PacketInfos()[0].ssrc(); ++ const int64_t frameId = frame->Id(); + int complete_units = buffer_->GetTotalNumberOfContinuousTemporalUnits(); + if (buffer_->InsertFrame(std::move(frame))) { + RTC_DCHECK(metadata.receive_time) << "Frame receive time must be set!"; +@@ -149,6 +153,9 @@ absl::optional<int64_t> VideoStreamBufferController::InsertFrame( + *metadata.receive_time); + } + if (complete_units < buffer_->GetTotalNumberOfContinuousTemporalUnits()) { ++ TRACE_EVENT2("webrtc", ++ "VideoStreamBufferController::InsertFrame Frame Complete", ++ "remote_ssrc", ssrc, "frame_id", frameId); + stats_proxy_->OnCompleteFrame(metadata.is_keyframe, metadata.size, + metadata.contentType); + MaybeScheduleFrameForRelease(); +-- +2.34.1 + diff --git a/third_party/libwebrtc/moz-patch-stack/0078.patch b/third_party/libwebrtc/moz-patch-stack/0078.patch new file mode 100644 index 0000000000..4fadab007e --- /dev/null +++ b/third_party/libwebrtc/moz-patch-stack/0078.patch @@ -0,0 +1,30 @@ +From: Andreas Pehrson <apehrson@mozilla.com> +Date: Thu, 6 Jan 2022 00:16:00 +0000 +Subject: Bug 1748458 - Add TRACE_EVENT for keyframe requests. r=bwc + +Differential Revision: https://phabricator.services.mozilla.com/D135113 +Mercurial Revision: https://hg.mozilla.org/mozilla-central/rev/5b2a7894ef1cf096d0e8977754507c0820e757fc +--- + video/rtp_video_stream_receiver2.cc | 6 ++++++ + 1 file changed, 6 insertions(+) + +diff --git a/video/rtp_video_stream_receiver2.cc b/video/rtp_video_stream_receiver2.cc +index eb0a7787ac..8055ac0e0f 100644 +--- a/video/rtp_video_stream_receiver2.cc ++++ b/video/rtp_video_stream_receiver2.cc +@@ -735,6 +735,12 @@ void RtpVideoStreamReceiver2::OnRtpPacket(const RtpPacketReceived& packet) { + + void RtpVideoStreamReceiver2::RequestKeyFrame() { + RTC_DCHECK_RUN_ON(&worker_task_checker_); ++ TRACE_EVENT2("webrtc", "RtpVideoStreamReceiver2::RequestKeyFrame", ++ "remote_ssrc", config_.rtp.remote_ssrc, "method", ++ keyframe_request_method_ == KeyFrameReqMethod::kPliRtcp ? "PLI" ++ : keyframe_request_method_ == KeyFrameReqMethod::kFirRtcp ? "FIR" ++ : keyframe_request_method_ == KeyFrameReqMethod::kNone ? "None" ++ : "Other"); + // TODO(bugs.webrtc.org/10336): Allow the sender to ignore key frame requests + // issued by anything other than the LossNotificationController if it (the + // sender) is relying on LNTF alone. +-- +2.34.1 + diff --git a/third_party/libwebrtc/moz-patch-stack/0079.patch b/third_party/libwebrtc/moz-patch-stack/0079.patch new file mode 100644 index 0000000000..39abbd9995 --- /dev/null +++ b/third_party/libwebrtc/moz-patch-stack/0079.patch @@ -0,0 +1,45 @@ +From: Andreas Pehrson <apehrson@mozilla.com> +Date: Wed, 11 Jan 2023 22:42:00 +0000 +Subject: Bug 1800942 - Add DCHECKs to + TimestampExtrapolator::ExtrapolateLocalTime. r=mjf + +Differential Revision: https://phabricator.services.mozilla.com/D166536 +Mercurial Revision: https://hg.mozilla.org/mozilla-central/rev/c5df7f40392464ffc63f44a53ddcaab2091741e0 +--- + modules/video_coding/timing/timestamp_extrapolator.cc | 7 +++++++ + 1 file changed, 7 insertions(+) + +diff --git a/modules/video_coding/timing/timestamp_extrapolator.cc b/modules/video_coding/timing/timestamp_extrapolator.cc +index c91aa1a362..dc62ac674a 100644 +--- a/modules/video_coding/timing/timestamp_extrapolator.cc ++++ b/modules/video_coding/timing/timestamp_extrapolator.cc +@@ -125,6 +125,7 @@ void TimestampExtrapolator::Update(Timestamp now, uint32_t ts90khz) { + absl::optional<Timestamp> TimestampExtrapolator::ExtrapolateLocalTime( + uint32_t timestamp90khz) const { + int64_t unwrapped_ts90khz = unwrapper_.PeekUnwrap(timestamp90khz); ++ RTC_DCHECK_GE(unwrapped_ts90khz, 0); + + if (!first_unwrapped_timestamp_) { + return absl::nullopt; +@@ -132,12 +133,18 @@ absl::optional<Timestamp> TimestampExtrapolator::ExtrapolateLocalTime( + constexpr double kRtpTicksPerMs = 90; + TimeDelta diff = TimeDelta::Millis( + (unwrapped_ts90khz - *prev_unwrapped_timestamp_) / kRtpTicksPerMs); ++ if (diff.ms() < 0) { ++ RTC_DCHECK_GE(prev_.ms(), -diff.ms()); ++ } + return prev_ + diff; + } else if (w_[0] < 1e-3) { + return start_; + } else { + double timestampDiff = unwrapped_ts90khz - *first_unwrapped_timestamp_; + auto diff_ms = static_cast<int64_t>((timestampDiff - w_[1]) / w_[0] + 0.5); ++ if (diff_ms < 0) { ++ RTC_DCHECK_GE(start_.ms(), -diff_ms); ++ } + return start_ + TimeDelta::Millis(diff_ms); + } + } +-- +2.34.1 + diff --git a/third_party/libwebrtc/moz-patch-stack/0080.patch b/third_party/libwebrtc/moz-patch-stack/0080.patch new file mode 100644 index 0000000000..69a6d4978f --- /dev/null +++ b/third_party/libwebrtc/moz-patch-stack/0080.patch @@ -0,0 +1,28 @@ +From: Andreas Pehrson <apehrson@mozilla.com> +Date: Wed, 8 Feb 2023 08:01:00 +0000 +Subject: Bug 1814692 - Don't attempt realtime scheduling rtc::PlatformThreads. + r=webrtc-reviewers,bwc + +Differential Revision: https://phabricator.services.mozilla.com/D169036 +Mercurial Revision: https://hg.mozilla.org/mozilla-central/rev/9e64a965e26c8379261466e5273c3b383164b2c7 +--- + rtc_base/platform_thread.cc | 3 +++ + 1 file changed, 3 insertions(+) + +diff --git a/rtc_base/platform_thread.cc b/rtc_base/platform_thread.cc +index 71a9f1b224..bcbb784b97 100644 +--- a/rtc_base/platform_thread.cc ++++ b/rtc_base/platform_thread.cc +@@ -50,6 +50,9 @@ bool SetPriority(ThreadPriority priority) { + // TODO(tommi): Switch to the same mechanism as Chromium uses for changing + // thread priorities. + return true; ++#elif defined(WEBRTC_MOZILLA_BUILD) && defined(WEBRTC_LINUX) ++ // Only realtime audio uses realtime scheduling in Firefox. ++ return true; + #else + const int policy = SCHED_FIFO; + const int min_prio = sched_get_priority_min(policy); +-- +2.34.1 + diff --git a/third_party/libwebrtc/moz-patch-stack/0081.patch b/third_party/libwebrtc/moz-patch-stack/0081.patch new file mode 100644 index 0000000000..7ed521fb6b --- /dev/null +++ b/third_party/libwebrtc/moz-patch-stack/0081.patch @@ -0,0 +1,34 @@ +From: Andreas Pehrson <apehrson@mozilla.com> +Date: Fri, 24 Feb 2023 15:01:00 +0100 +Subject: Bug 1817024 - (fix-0e2cf6cc01) Skip library + create_peer_connection_quality_test_frame_generator. r?mjf! + +Differential Revision: https://phabricator.services.mozilla.com/D170887 +Mercurial Revision: https://hg.mozilla.org/mozilla-central/rev/fbbc1bf963fda30bca26ae6aac0c3459b8ebea6f +--- + api/BUILD.gn | 2 ++ + 1 file changed, 2 insertions(+) + +diff --git a/api/BUILD.gn b/api/BUILD.gn +index ab5d6c91ce..3f313e2743 100644 +--- a/api/BUILD.gn ++++ b/api/BUILD.gn +@@ -702,6 +702,7 @@ rtc_library("create_frame_generator") { + absl_deps = [ "//third_party/abseil-cpp/absl/types:optional" ] + } + ++if (!build_with_mozilla) { + rtc_library("create_peer_connection_quality_test_frame_generator") { + visibility = [ "*" ] + testonly = true +@@ -718,6 +719,7 @@ rtc_library("create_peer_connection_quality_test_frame_generator") { + ] + absl_deps = [ "//third_party/abseil-cpp/absl/types:optional" ] + } ++} + + rtc_source_set("libjingle_logging_api") { + visibility = [ "*" ] +-- +2.34.1 + diff --git a/third_party/libwebrtc/moz-patch-stack/0082.patch b/third_party/libwebrtc/moz-patch-stack/0082.patch new file mode 100644 index 0000000000..ad98ccfed2 --- /dev/null +++ b/third_party/libwebrtc/moz-patch-stack/0082.patch @@ -0,0 +1,189 @@ +From: Michael Froman <mfroman@mozilla.com> +Date: Wed, 8 Mar 2023 00:26:00 +0000 +Subject: Bug 1820869 - avoid building unreachable files. r=ng,webrtc-reviewers + +Differential Revision: https://phabricator.services.mozilla.com/D171922 +Mercurial Revision: https://hg.mozilla.org/mozilla-central/rev/88b3cc6bbece7c53d00e124713330f3d34d2789d +--- + BUILD.gn | 9 +++++++++ + call/BUILD.gn | 10 ++++++++++ + media/BUILD.gn | 7 ++++++- + modules/audio_device/BUILD.gn | 11 ++++++++++- + rtc_base/BUILD.gn | 2 ++ + webrtc.gni | 2 +- + 6 files changed, 38 insertions(+), 3 deletions(-) + +diff --git a/BUILD.gn b/BUILD.gn +index 6515866c2d..465c4d9bfd 100644 +--- a/BUILD.gn ++++ b/BUILD.gn +@@ -549,6 +549,15 @@ if (!build_with_chromium) { + "api/video:video_rtp_headers", + "test:rtp_test_utils", + ] ++ # Added when we removed deps in other places to avoid building ++ # unreachable sources. See Bug 1820869. ++ deps += [ ++ "api/video_codecs:video_codecs_api", ++ "api/video_codecs:rtc_software_fallback_wrappers", ++ "media:rtc_encoder_simulcast_proxy", ++ "modules/video_coding:webrtc_vp8", ++ "modules/video_coding:webrtc_vp9", ++ ] + } else { + deps += [ + "api", +diff --git a/call/BUILD.gn b/call/BUILD.gn +index 26618aee80..fb23b7ef39 100644 +--- a/call/BUILD.gn ++++ b/call/BUILD.gn +@@ -352,6 +352,16 @@ rtc_library("call") { + "//third_party/abseil-cpp/absl/strings", + "//third_party/abseil-cpp/absl/types:optional", + ] ++ if (build_with_mozilla) { # See Bug 1820869. ++ sources -= [ ++ "call_factory.cc", ++ "degraded_call.cc", ++ ] ++ deps -= [ ++ ":fake_network", ++ ":simulated_network", ++ ] ++ } + } + + rtc_source_set("receive_stream_interface") { +diff --git a/media/BUILD.gn b/media/BUILD.gn +index 4ddc8349a8..daca67e033 100644 +--- a/media/BUILD.gn ++++ b/media/BUILD.gn +@@ -442,7 +442,10 @@ rtc_library("rtc_internal_video_codecs") { + "../test:fake_video_codecs", + ] + if (build_with_mozilla) { +- deps -= [ "../test:fake_video_codecs" ] ++ deps -= [ ++ "../modules/video_coding:webrtc_multiplex", # See Bug 1820869. ++ "../test:fake_video_codecs", ++ ] + } + + if (enable_libaom) { +@@ -477,6 +480,8 @@ rtc_library("rtc_internal_video_codecs") { + sources -= [ + "engine/fake_video_codec_factory.cc", + "engine/fake_video_codec_factory.h", ++ "engine/internal_encoder_factory.cc", # See Bug 1820869. ++ "engine/multiplex_codec_factory.cc", # See Bug 1820869. + ] + } + } +diff --git a/modules/audio_device/BUILD.gn b/modules/audio_device/BUILD.gn +index e35a442025..61cd531edd 100644 +--- a/modules/audio_device/BUILD.gn ++++ b/modules/audio_device/BUILD.gn +@@ -30,6 +30,7 @@ rtc_source_set("audio_device_default") { + } + + rtc_source_set("audio_device") { ++if (!build_with_mozilla) { # See Bug 1820869. + visibility = [ "*" ] + public_deps = [ + ":audio_device_api", +@@ -40,6 +41,7 @@ rtc_source_set("audio_device") { + ":audio_device_impl", + ] + } ++} + + rtc_source_set("audio_device_api") { + visibility = [ "*" ] +@@ -58,6 +60,7 @@ rtc_source_set("audio_device_api") { + } + + rtc_library("audio_device_buffer") { ++if (!build_with_mozilla) { # See Bug 1820869. + sources = [ + "audio_device_buffer.cc", + "audio_device_buffer.h", +@@ -85,6 +88,7 @@ rtc_library("audio_device_buffer") { + "../../system_wrappers:metrics", + ] + } ++} + + rtc_library("audio_device_generic") { + sources = [ +@@ -180,6 +184,7 @@ rtc_source_set("audio_device_module_from_input_and_output") { + # Contains default implementations of webrtc::AudioDeviceModule for Windows, + # Linux, Mac, iOS and Android. + rtc_library("audio_device_impl") { ++if (!build_with_mozilla) { # See Bug 1820869. + visibility = [ "*" ] + deps = [ + ":audio_device_api", +@@ -373,6 +378,7 @@ rtc_library("audio_device_impl") { + ] + } + } ++} + + if (is_mac) { + rtc_source_set("audio_device_impl_frameworks") { +@@ -390,6 +396,7 @@ if (is_mac) { + } + } + ++if (!build_with_mozilla) { # See Bug 1820869. + rtc_source_set("mock_audio_device") { + visibility = [ "*" ] + testonly = true +@@ -406,8 +413,10 @@ rtc_source_set("mock_audio_device") { + "../../test:test_support", + ] + } ++} + +-if (rtc_include_tests && !build_with_chromium) { ++# See Bug 1820869 for !build_with_mozilla. ++if (rtc_include_tests && !build_with_chromium && !build_with_mozilla) { + rtc_library("audio_device_unittests") { + testonly = true + +diff --git a/rtc_base/BUILD.gn b/rtc_base/BUILD.gn +index 3cd0bfff06..0b1e2a6208 100644 +--- a/rtc_base/BUILD.gn ++++ b/rtc_base/BUILD.gn +@@ -283,6 +283,7 @@ rtc_library("sample_counter") { + absl_deps = [ "//third_party/abseil-cpp/absl/types:optional" ] + } + ++if (!build_with_mozilla) { # See Bug 1820869. + rtc_library("timestamp_aligner") { + visibility = [ "*" ] + sources = [ +@@ -296,6 +297,7 @@ rtc_library("timestamp_aligner") { + "system:rtc_export", + ] + } ++} + + rtc_library("zero_memory") { + visibility = [ "*" ] +diff --git a/webrtc.gni b/webrtc.gni +index 1b21d329b2..46a9433141 100644 +--- a/webrtc.gni ++++ b/webrtc.gni +@@ -221,7 +221,7 @@ declare_args() { + # video codecs they depends on will not be included in libwebrtc.{a|lib} + # (they will still be included in libjingle_peerconnection_so.so and + # WebRTC.framework) +- rtc_include_builtin_video_codecs = true ++ rtc_include_builtin_video_codecs = !build_with_mozilla # See Bug 1820869. + + # When set to true and in a standalone build, it will undefine UNICODE and + # _UNICODE (which are always defined globally by the Chromium Windows +-- +2.34.1 + diff --git a/third_party/libwebrtc/moz-patch-stack/0083.patch b/third_party/libwebrtc/moz-patch-stack/0083.patch new file mode 100644 index 0000000000..1e5dcbce09 --- /dev/null +++ b/third_party/libwebrtc/moz-patch-stack/0083.patch @@ -0,0 +1,34 @@ +From: Michael Froman <mfroman@mozilla.com> +Date: Wed, 12 Apr 2023 16:03:00 +0000 +Subject: Bug 1826428 - remove libwebrtc's jvm_android.cc from build + r=ng,webrtc-reviewers + +Based on info from John Lin and previous try runs, we're almost +certainly not using this. Let's try removing it from the build +and landing it. If no problems emerge, we'll be able to remove +our custom changes to upstream code in jvm_android.cc. + +Differential Revision: https://phabricator.services.mozilla.com/D174793 +Mercurial Revision: https://hg.mozilla.org/mozilla-central/rev/dca1b97525487ae57d43ced1ebdb4a2d9c9dae89 +--- + modules/utility/BUILD.gn | 4 ++++ + 1 file changed, 4 insertions(+) + +diff --git a/modules/utility/BUILD.gn b/modules/utility/BUILD.gn +index 3fe4ca8c92..46bca17f02 100644 +--- a/modules/utility/BUILD.gn ++++ b/modules/utility/BUILD.gn +@@ -47,6 +47,10 @@ rtc_source_set("utility") { + "../../rtc_base:platform_thread", + "../../rtc_base/system:arch", + ] ++ ++ if (build_with_mozilla) { ++ sources -= [ "source/jvm_android.cc" ] ++ } + } + } + +-- +2.34.1 + diff --git a/third_party/libwebrtc/moz-patch-stack/0084.patch b/third_party/libwebrtc/moz-patch-stack/0084.patch new file mode 100644 index 0000000000..c51b935b44 --- /dev/null +++ b/third_party/libwebrtc/moz-patch-stack/0084.patch @@ -0,0 +1,169 @@ +From: Jan Grulich <jgrulich@redhat.com> +Date: Mon, 20 Feb 2023 21:25:00 +0000 +Subject: Bug 1817263 - fix OS picker behavior under Wayland r=ng,jib,stransky + +Recent WebRTC backports and changes that are about to be backported from +upstream to Firefox breaks and will break how we work with PipWire based +desktop capturer. Currently when constructing device list, a fallback to +ScreenCapturerX11 is used, as we don't call set_allow_pipewire(), which +wouldn't make a difference anyway. In such case the only thing we need +is a placeholder for a screen that will request OS level prompt. We also +need a way to request both screens and windows in one xdg-desktop-portal +call as recent WebRTC made each type be called separately, therefore the +introduction of GenericCapturer. Lastly we need to make sure when there +is a MediaDevice requesting the OS prompt, that it will be checked as +first. + +In order to use unmodified libwebrtc, Firefox would need to rework the +OS picker to request each type (screens and windows) separately so we +can just use regular ScreenCapturer and WindowCapturer. This should be +done ideally the way Chromium does it, where users can actually see +even the preview of what they picked over xdg-desktop-portal before it +is actually shared with requesting web page and they also have option +to make the request again in case they picked a wrong window or screen. + +Differential Revision: https://phabricator.services.mozilla.com/D169627 +Mercurial Revision: https://hg.mozilla.org/mozilla-central/rev/acd6266642951aacf8915a56777c780cae9e9af3 +--- + .../desktop_capture/desktop_capture_types.h | 2 +- + modules/desktop_capture/desktop_capturer.cc | 28 +++++++++++++++++++ + modules/desktop_capture/desktop_capturer.h | 13 +++++++++ + .../linux/wayland/base_capturer_pipewire.cc | 11 +------- + .../linux/wayland/screencast_portal.cc | 2 ++ + 5 files changed, 45 insertions(+), 11 deletions(-) + +diff --git a/modules/desktop_capture/desktop_capture_types.h b/modules/desktop_capture/desktop_capture_types.h +index 381d1021c4..e777a45f92 100644 +--- a/modules/desktop_capture/desktop_capture_types.h ++++ b/modules/desktop_capture/desktop_capture_types.h +@@ -19,7 +19,7 @@ typedef int pid_t; // matching what used to be in + + namespace webrtc { + +-enum class CaptureType { kWindow, kScreen }; ++enum class CaptureType { kWindow, kScreen, kAnyScreenContent }; + + // Type used to identify windows on the desktop. Values are platform-specific: + // - On Windows: HWND cast to intptr_t. +diff --git a/modules/desktop_capture/desktop_capturer.cc b/modules/desktop_capture/desktop_capturer.cc +index 4baa93cab9..7df6becb4e 100644 +--- a/modules/desktop_capture/desktop_capturer.cc ++++ b/modules/desktop_capture/desktop_capturer.cc +@@ -26,6 +26,10 @@ + #include "rtc_base/win/windows_version.h" + #endif // defined(RTC_ENABLE_WIN_WGC) + ++#if defined(WEBRTC_USE_PIPEWIRE) || defined(WEBRTC_USE_X11) ++#include "modules/desktop_capture/linux/wayland/base_capturer_pipewire.h" ++#endif ++ + namespace webrtc { + + void LogDesktopCapturerFullscreenDetectorUsage() { +@@ -84,6 +88,30 @@ std::unique_ptr<DesktopCapturer> DesktopCapturer::CreateWindowCapturer( + return capturer; + } + ++#if defined(WEBRTC_USE_PIPEWIRE) || defined(WEBRTC_USE_X11) ++// static ++std::unique_ptr<DesktopCapturer> DesktopCapturer::CreateGenericCapturer( ++ const DesktopCaptureOptions& options) { ++ std::unique_ptr<DesktopCapturer> capturer = CreateRawGenericCapturer(options); ++ if (capturer && options.detect_updated_region()) { ++ capturer.reset(new DesktopCapturerDifferWrapper(std::move(capturer))); ++ } ++ ++ return capturer; ++} ++ ++// static ++std::unique_ptr<DesktopCapturer> DesktopCapturer::CreateRawGenericCapturer( ++ const DesktopCaptureOptions& options) { ++ if (options.allow_pipewire() && DesktopCapturer::IsRunningUnderWayland()) { ++ return std::make_unique<BaseCapturerPipeWire>(options, ++ CaptureType::kAnyScreenContent); ++ } ++ ++ return nullptr; ++} ++#endif // defined(WEBRTC_USE_PIPEWIRE) || defined(WEBRTC_USE_X11) ++ + // static + std::unique_ptr<DesktopCapturer> DesktopCapturer::CreateScreenCapturer( + const DesktopCaptureOptions& options) { +diff --git a/modules/desktop_capture/desktop_capturer.h b/modules/desktop_capture/desktop_capturer.h +index cf75004af5..951c4a9b10 100644 +--- a/modules/desktop_capture/desktop_capturer.h ++++ b/modules/desktop_capture/desktop_capturer.h +@@ -170,6 +170,12 @@ class RTC_EXPORT DesktopCapturer { + // The return value if `pos` is out of the scope of the source is undefined. + virtual bool IsOccluded(const DesktopVector& pos); + ++#if defined(WEBRTC_USE_PIPEWIRE) || defined(WEBRTC_USE_X11) ++ // Creates a DesktopCapturer instance which targets to capture windows and screens. ++ static std::unique_ptr<DesktopCapturer> CreateGenericCapturer( ++ const DesktopCaptureOptions& options); ++#endif ++ + // Creates a DesktopCapturer instance which targets to capture windows. + static std::unique_ptr<DesktopCapturer> CreateWindowCapturer( + const DesktopCaptureOptions& options); +@@ -198,6 +204,13 @@ class RTC_EXPORT DesktopCapturer { + // CroppingWindowCapturer needs to create raw capturers without wrappers, so + // the following two functions are protected. + ++#if defined(WEBRTC_USE_PIPEWIRE) || defined(WEBRTC_USE_X11) ++ // Creates a platform specific DesktopCapturer instance which targets to ++ // capture windows and screens. ++ static std::unique_ptr<DesktopCapturer> CreateRawGenericCapturer( ++ const DesktopCaptureOptions& options); ++#endif ++ + // Creates a platform specific DesktopCapturer instance which targets to + // capture windows. + static std::unique_ptr<DesktopCapturer> CreateRawWindowCapturer( +diff --git a/modules/desktop_capture/linux/wayland/base_capturer_pipewire.cc b/modules/desktop_capture/linux/wayland/base_capturer_pipewire.cc +index dae2b70510..cf4f7dc9aa 100644 +--- a/modules/desktop_capture/linux/wayland/base_capturer_pipewire.cc ++++ b/modules/desktop_capture/linux/wayland/base_capturer_pipewire.cc +@@ -165,15 +165,6 @@ void BaseCapturerPipeWire::CaptureFrame() { + callback_->OnCaptureResult(Result::SUCCESS, std::move(frame)); + } + +-// Keep in sync with defines at browser/actors/WebRTCParent.jsm +-// With PipeWire we can't select which system resource is shared so +-// we don't create a window/screen list. Instead we place these constants +-// as window name/id so frontend code can identify PipeWire backend +-// and does not try to create screen/window preview. +- +-#define PIPEWIRE_ID 0xaffffff +-#define PIPEWIRE_NAME "####_PIPEWIRE_PORTAL_####" +- + bool BaseCapturerPipeWire::GetSourceList(SourceList* sources) { + RTC_DCHECK(sources->size() == 0); + // List of available screens is already presented by the xdg-desktop-portal, +@@ -190,7 +181,7 @@ bool BaseCapturerPipeWire::GetSourceList(SourceList* sources) { + bool BaseCapturerPipeWire::SelectSource(SourceId id) { + // Screen selection is handled by the xdg-desktop-portal. + selected_source_id_ = id; +- return id == PIPEWIRE_ID; ++ return true; + } + + DelegatedSourceListController* +diff --git a/modules/desktop_capture/linux/wayland/screencast_portal.cc b/modules/desktop_capture/linux/wayland/screencast_portal.cc +index abfade56e7..e7aaee001b 100644 +--- a/modules/desktop_capture/linux/wayland/screencast_portal.cc ++++ b/modules/desktop_capture/linux/wayland/screencast_portal.cc +@@ -41,6 +41,8 @@ ScreenCastPortal::CaptureSourceType ScreenCastPortal::ToCaptureSourceType( + return ScreenCastPortal::CaptureSourceType::kScreen; + case CaptureType::kWindow: + return ScreenCastPortal::CaptureSourceType::kWindow; ++ case CaptureType::kAnyScreenContent: ++ return ScreenCastPortal::CaptureSourceType::kAnyScreenContent; + } + } + +-- +2.34.1 + diff --git a/third_party/libwebrtc/moz-patch-stack/0085.patch b/third_party/libwebrtc/moz-patch-stack/0085.patch new file mode 100644 index 0000000000..bac4cf6c9c --- /dev/null +++ b/third_party/libwebrtc/moz-patch-stack/0085.patch @@ -0,0 +1,100 @@ +From: Jan Grulich <jgrulich@redhat.com> +Date: Mon, 27 Feb 2023 13:57:00 +0000 +Subject: Bug 1819044 - fix build non-pipewire builds + r=webrtc-reviewers,pehrsons + +We should check only for PipeWire presence when building code specific +to PipeWire. + +Differential Revision: https://phabricator.services.mozilla.com/D171071 +Mercurial Revision: https://hg.mozilla.org/mozilla-central/rev/7a879ad084a6e9768479c73cc5c3f4e9d95a2ab9 + +Also includes: + + Bug 1819044 - fix build non-pipewire builds (attempt #2) r=webrtc-reviewers,pehrsons + + Make the new API available to everyone and just return an empty capturer + in case when building without PipeWire. It will not make any difference + because using X11 based capturers on Wayland is useless anyway so if we + fail for missing PipeWire on Wayland, it will have the same outcome. + + Differential Revision: https://phabricator.services.mozilla.com/D171192 + Mercurial Revision: https://hg.mozilla.org/mozilla-central/rev/ad247b0aac896d884eba5e40f0ec8a9f50d8b85b +--- + modules/desktop_capture/desktop_capturer.cc | 7 +++---- + modules/desktop_capture/desktop_capturer.h | 4 ---- + 2 files changed, 3 insertions(+), 8 deletions(-) + +diff --git a/modules/desktop_capture/desktop_capturer.cc b/modules/desktop_capture/desktop_capturer.cc +index 7df6becb4e..1af19a1fd2 100644 +--- a/modules/desktop_capture/desktop_capturer.cc ++++ b/modules/desktop_capture/desktop_capturer.cc +@@ -26,7 +26,7 @@ + #include "rtc_base/win/windows_version.h" + #endif // defined(RTC_ENABLE_WIN_WGC) + +-#if defined(WEBRTC_USE_PIPEWIRE) || defined(WEBRTC_USE_X11) ++#if defined(WEBRTC_USE_PIPEWIRE) + #include "modules/desktop_capture/linux/wayland/base_capturer_pipewire.h" + #endif + +@@ -88,7 +88,6 @@ std::unique_ptr<DesktopCapturer> DesktopCapturer::CreateWindowCapturer( + return capturer; + } + +-#if defined(WEBRTC_USE_PIPEWIRE) || defined(WEBRTC_USE_X11) + // static + std::unique_ptr<DesktopCapturer> DesktopCapturer::CreateGenericCapturer( + const DesktopCaptureOptions& options) { +@@ -100,17 +99,17 @@ std::unique_ptr<DesktopCapturer> DesktopCapturer::CreateGenericCapturer( + return capturer; + } + +-// static + std::unique_ptr<DesktopCapturer> DesktopCapturer::CreateRawGenericCapturer( + const DesktopCaptureOptions& options) { ++#if defined(WEBRTC_USE_PIPEWIRE) + if (options.allow_pipewire() && DesktopCapturer::IsRunningUnderWayland()) { + return std::make_unique<BaseCapturerPipeWire>(options, + CaptureType::kAnyScreenContent); + } ++#endif // defined(WEBRTC_USE_PIPEWIRE) + + return nullptr; + } +-#endif // defined(WEBRTC_USE_PIPEWIRE) || defined(WEBRTC_USE_X11) + + // static + std::unique_ptr<DesktopCapturer> DesktopCapturer::CreateScreenCapturer( +diff --git a/modules/desktop_capture/desktop_capturer.h b/modules/desktop_capture/desktop_capturer.h +index 951c4a9b10..12ef57ba26 100644 +--- a/modules/desktop_capture/desktop_capturer.h ++++ b/modules/desktop_capture/desktop_capturer.h +@@ -170,11 +170,9 @@ class RTC_EXPORT DesktopCapturer { + // The return value if `pos` is out of the scope of the source is undefined. + virtual bool IsOccluded(const DesktopVector& pos); + +-#if defined(WEBRTC_USE_PIPEWIRE) || defined(WEBRTC_USE_X11) + // Creates a DesktopCapturer instance which targets to capture windows and screens. + static std::unique_ptr<DesktopCapturer> CreateGenericCapturer( + const DesktopCaptureOptions& options); +-#endif + + // Creates a DesktopCapturer instance which targets to capture windows. + static std::unique_ptr<DesktopCapturer> CreateWindowCapturer( +@@ -204,12 +202,10 @@ class RTC_EXPORT DesktopCapturer { + // CroppingWindowCapturer needs to create raw capturers without wrappers, so + // the following two functions are protected. + +-#if defined(WEBRTC_USE_PIPEWIRE) || defined(WEBRTC_USE_X11) + // Creates a platform specific DesktopCapturer instance which targets to + // capture windows and screens. + static std::unique_ptr<DesktopCapturer> CreateRawGenericCapturer( + const DesktopCaptureOptions& options); +-#endif + + // Creates a platform specific DesktopCapturer instance which targets to + // capture windows. +-- +2.34.1 + diff --git a/third_party/libwebrtc/moz-patch-stack/0086.patch b/third_party/libwebrtc/moz-patch-stack/0086.patch new file mode 100644 index 0000000000..b43f8bb151 --- /dev/null +++ b/third_party/libwebrtc/moz-patch-stack/0086.patch @@ -0,0 +1,77 @@ +From: Jan Grulich <jgrulich@redhat.com> +Date: Fri, 10 Mar 2023 09:21:00 +0000 +Subject: Bug 1819035 - get EGL display based on the used platform in the + browser r=webrtc-reviewers,ng + +Because of a possible misconfiguration or a possible driver issue it +might happen that the browser will use a different driver on X11 and +end up using yet another one for wayland/gbm, which might lead to not +working screen sharing in the better case, but also to a crash in the +other driver (Nvidia). This adds a check for platform the browser runs +on, if it's XWayland or Wayland and based on that query EGL display for +that specific platform, rather than going for the Wayland one only. + +Differential Revision: https://phabricator.services.mozilla.com/D171858 +Mercurial Revision: https://hg.mozilla.org/mozilla-central/rev/c8606497de1f461a6352456e0e511c2ae498d526 +--- + .../linux/wayland/egl_dmabuf.cc | 30 +++++++++++++++++-- + 1 file changed, 28 insertions(+), 2 deletions(-) + +diff --git a/modules/desktop_capture/linux/wayland/egl_dmabuf.cc b/modules/desktop_capture/linux/wayland/egl_dmabuf.cc +index 5bbd5d7aba..80da597e29 100644 +--- a/modules/desktop_capture/linux/wayland/egl_dmabuf.cc ++++ b/modules/desktop_capture/linux/wayland/egl_dmabuf.cc +@@ -13,6 +13,7 @@ + #include <asm/ioctl.h> + #include <dlfcn.h> + #include <fcntl.h> ++#include <gdk/gdk.h> + #include <libdrm/drm_fourcc.h> + #include <linux/types.h> + #include <spa/param/video/format-utils.h> +@@ -200,6 +201,26 @@ static void CloseLibrary(void* library) { + } + } + ++static bool IsWaylandDisplay() { ++ static auto sGdkWaylandDisplayGetType = ++ (GType (*)(void))dlsym(RTLD_DEFAULT, "gdk_wayland_display_get_type"); ++ if (!sGdkWaylandDisplayGetType) { ++ return false; ++ } ++ return (G_TYPE_CHECK_INSTANCE_TYPE ((gdk_display_get_default()), ++ sGdkWaylandDisplayGetType())); ++} ++ ++static bool IsX11Display() { ++ static auto sGdkX11DisplayGetType = ++ (GType (*)(void))dlsym(RTLD_DEFAULT, "gdk_x11_display_get_type"); ++ if (!sGdkX11DisplayGetType) { ++ return false; ++ } ++ return (G_TYPE_CHECK_INSTANCE_TYPE ((gdk_display_get_default()), ++ sGdkX11DisplayGetType())); ++} ++ + static void* g_lib_egl = nullptr; + + RTC_NO_SANITIZE("cfi-icall") +@@ -331,8 +352,13 @@ EglDmaBuf::EglDmaBuf() { + return; + } + +- egl_.display = EglGetPlatformDisplay(EGL_PLATFORM_WAYLAND_KHR, +- (void*)EGL_DEFAULT_DISPLAY, nullptr); ++ if (IsWaylandDisplay()) { ++ egl_.display = EglGetPlatformDisplay(EGL_PLATFORM_WAYLAND_KHR, ++ (void*)EGL_DEFAULT_DISPLAY, nullptr); ++ } else if (IsX11Display()) { ++ egl_.display = EglGetPlatformDisplay(EGL_PLATFORM_X11_KHR, ++ (void*)EGL_DEFAULT_DISPLAY, nullptr); ++ } + + if (egl_.display == EGL_NO_DISPLAY) { + RTC_LOG(LS_ERROR) << "Failed to obtain default EGL display: " +-- +2.34.1 + diff --git a/third_party/libwebrtc/moz-patch-stack/0087.patch b/third_party/libwebrtc/moz-patch-stack/0087.patch new file mode 100644 index 0000000000..67cea03cd6 --- /dev/null +++ b/third_party/libwebrtc/moz-patch-stack/0087.patch @@ -0,0 +1,34 @@ +From: Andreas Pehrson <apehrson@mozilla.com> +Date: Mon, 27 Feb 2023 16:22:00 +0000 +Subject: Bug 1817024 - (fix-fdcfefa708) In PhysicalSocket avoid a non-trivial + designated initializer. r=mjf,webrtc-reviewers + +This fixes a build failure in the base-toolchain job with GCC 7.5.0: + In file included from Unified_cpp_threading_gn0.cpp:38:0: + .../third_party/libwebrtc/rtc_base/physical_socket_server.cc: In member function 'int rtc::PhysicalSocket::DoReadFromSocket(void*, size_t, rtc::SocketAddress*, int64_t*)': + .../third_party/libwebrtc/rtc_base/physical_socket_server.cc:463:51: sorry, unimplemented: non-trivial designated initializers not supported + msghdr msg = {.msg_iov = &iov, .msg_iovlen = 1}; + ^ + +Differential Revision: https://phabricator.services.mozilla.com/D171057 +Mercurial Revision: https://hg.mozilla.org/mozilla-central/rev/a3447f709befd84a282ca40f29b7a5ea76d5b68d +--- + rtc_base/physical_socket_server.cc | 2 +- + 1 file changed, 1 insertion(+), 1 deletion(-) + +diff --git a/rtc_base/physical_socket_server.cc b/rtc_base/physical_socket_server.cc +index 60d024c769..3b112e6188 100644 +--- a/rtc_base/physical_socket_server.cc ++++ b/rtc_base/physical_socket_server.cc +@@ -460,7 +460,7 @@ int PhysicalSocket::DoReadFromSocket(void* buffer, + int received = 0; + if (read_scm_timestamp_experiment_) { + iovec iov = {.iov_base = buffer, .iov_len = length}; +- msghdr msg = {.msg_iov = &iov, .msg_iovlen = 1}; ++ msghdr msg = {.msg_name = nullptr, .msg_namelen = 0, .msg_iov = &iov, .msg_iovlen = 1}; + if (out_addr) { + out_addr->Clear(); + msg.msg_name = addr; +-- +2.34.1 + diff --git a/third_party/libwebrtc/moz-patch-stack/0088.patch b/third_party/libwebrtc/moz-patch-stack/0088.patch new file mode 100644 index 0000000000..667d9da427 --- /dev/null +++ b/third_party/libwebrtc/moz-patch-stack/0088.patch @@ -0,0 +1,526 @@ +From: Jan Grulich <jgrulich@redhat.com> +Date: Tue, 28 Mar 2023 14:41:00 +0000 +Subject: Bug 1823404 - PipeWire capturer: import DMABufs directly into desktop + frame r=webrtc-reviewers,stransky,bwc + +Originally DMABufs were imported into a temporary buffer followed by a +copy operation into the desktop frame itself. This is not needed as we +can import them directly into desktop frames and avoid this overhead. + +Also drop support for MemPtr buffers as both Mutter and KWin don't seem +to support them and they are going to be too slow anyway. + +Testing with latest Chromium, I could see two processes with usage +around 20% and 40% without this change going down to 10% and 20% with +this change applied. + +Also drop old DmaBuf support. + +Differential Revision: https://phabricator.services.mozilla.com/D173021 +Mercurial Revision: https://hg.mozilla.org/mozilla-central/rev/581fe5ce66f9f3c725f5345b3e57407d1ec1e312 +--- + .../linux/wayland/egl_dmabuf.cc | 114 +++++++--- + .../linux/wayland/egl_dmabuf.h | 16 +- + .../linux/wayland/shared_screencast_stream.cc | 209 +++++++++--------- + 3 files changed, 201 insertions(+), 138 deletions(-) + +diff --git a/modules/desktop_capture/linux/wayland/egl_dmabuf.cc b/modules/desktop_capture/linux/wayland/egl_dmabuf.cc +index 80da597e29..6a019c64b4 100644 +--- a/modules/desktop_capture/linux/wayland/egl_dmabuf.cc ++++ b/modules/desktop_capture/linux/wayland/egl_dmabuf.cc +@@ -102,11 +102,23 @@ typedef void (*glDeleteTextures_func)(GLsizei n, const GLuint* textures); + typedef void (*glGenTextures_func)(GLsizei n, GLuint* textures); + typedef GLenum (*glGetError_func)(void); + typedef const GLubyte* (*glGetString_func)(GLenum name); +-typedef void (*glGetTexImage_func)(GLenum target, +- GLint level, +- GLenum format, +- GLenum type, +- void* pixels); ++typedef void (*glReadPixels_func)(GLint x, ++ GLint y, ++ GLsizei width, ++ GLsizei height, ++ GLenum format, ++ GLenum type, ++ void* data); ++typedef void (*glGenFramebuffers_func)(GLsizei n, GLuint* ids); ++typedef void (*glDeleteFramebuffers_func)(GLsizei n, ++ const GLuint* framebuffers); ++typedef void (*glBindFramebuffer_func)(GLenum target, GLuint framebuffer); ++typedef void (*glFramebufferTexture2D_func)(GLenum target, ++ GLenum attachment, ++ GLenum textarget, ++ GLuint texture, ++ GLint level); ++typedef GLenum (*glCheckFramebufferStatus_func)(GLenum target); + typedef void (*glTexParameteri_func)(GLenum target, GLenum pname, GLint param); + typedef void* (*glXGetProcAddressARB_func)(const char*); + +@@ -119,7 +131,12 @@ glDeleteTextures_func GlDeleteTextures = nullptr; + glGenTextures_func GlGenTextures = nullptr; + glGetError_func GlGetError = nullptr; + glGetString_func GlGetString = nullptr; +-glGetTexImage_func GlGetTexImage = nullptr; ++glReadPixels_func GlReadPixels = nullptr; ++glGenFramebuffers_func GlGenFramebuffers = nullptr; ++glDeleteFramebuffers_func GlDeleteFramebuffers = nullptr; ++glBindFramebuffer_func GlBindFramebuffer = nullptr; ++glFramebufferTexture2D_func GlFramebufferTexture2D = nullptr; ++glCheckFramebufferStatus_func GlCheckFramebufferStatus = nullptr; + glTexParameteri_func GlTexParameteri = nullptr; + glXGetProcAddressARB_func GlXGetProcAddressARB = nullptr; + +@@ -300,12 +317,26 @@ static bool LoadGL() { + (glDeleteTextures_func)GlXGetProcAddressARB("glDeleteTextures"); + GlGenTextures = (glGenTextures_func)GlXGetProcAddressARB("glGenTextures"); + GlGetError = (glGetError_func)GlXGetProcAddressARB("glGetError"); +- GlGetTexImage = (glGetTexImage_func)GlXGetProcAddressARB("glGetTexImage"); ++ GlReadPixels = (glReadPixels_func)GlXGetProcAddressARB("glReadPixels"); ++ GlGenFramebuffers = ++ (glGenFramebuffers_func)GlXGetProcAddressARB("glGenFramebuffers"); ++ GlDeleteFramebuffers = ++ (glDeleteFramebuffers_func)GlXGetProcAddressARB("glDeleteFramebuffers"); ++ GlBindFramebuffer = ++ (glBindFramebuffer_func)GlXGetProcAddressARB("glBindFramebuffer"); ++ GlFramebufferTexture2D = (glFramebufferTexture2D_func)GlXGetProcAddressARB( ++ "glFramebufferTexture2D"); ++ GlCheckFramebufferStatus = ++ (glCheckFramebufferStatus_func)GlXGetProcAddressARB( ++ "glCheckFramebufferStatus"); ++ + GlTexParameteri = + (glTexParameteri_func)GlXGetProcAddressARB("glTexParameteri"); + + return GlBindTexture && GlDeleteTextures && GlGenTextures && GlGetError && +- GlGetTexImage && GlTexParameteri; ++ GlReadPixels && GlGenFramebuffers && GlDeleteFramebuffers && ++ GlBindFramebuffer && GlFramebufferTexture2D && ++ GlCheckFramebufferStatus && GlTexParameteri; + } + + return false; +@@ -461,6 +492,14 @@ EglDmaBuf::~EglDmaBuf() { + EglTerminate(egl_.display); + } + ++ if (fbo_) { ++ GlDeleteFramebuffers(1, &fbo_); ++ } ++ ++ if (texture_) { ++ GlDeleteTextures(1, &texture_); ++ } ++ + // BUG: crbug.com/1290566 + // Closing libEGL.so.1 when using NVidia drivers causes a crash + // when EglGetPlatformDisplayEXT() is used, at least this one is enough +@@ -492,20 +531,20 @@ bool EglDmaBuf::GetClientExtensions(EGLDisplay dpy, EGLint name) { + } + + RTC_NO_SANITIZE("cfi-icall") +-std::unique_ptr<uint8_t[]> EglDmaBuf::ImageFromDmaBuf( +- const DesktopSize& size, +- uint32_t format, +- const std::vector<PlaneData>& plane_datas, +- uint64_t modifier) { +- std::unique_ptr<uint8_t[]> src; +- ++bool EglDmaBuf::ImageFromDmaBuf(const DesktopSize& size, ++ uint32_t format, ++ const std::vector<PlaneData>& plane_datas, ++ uint64_t modifier, ++ const DesktopVector& offset, ++ const DesktopSize& buffer_size, ++ uint8_t* data) { + if (!egl_initialized_) { +- return src; ++ return false; + } + + if (plane_datas.size() <= 0) { + RTC_LOG(LS_ERROR) << "Failed to process buffer: invalid number of planes"; +- return src; ++ return false; + } + + EGLint attribs[47]; +@@ -594,20 +633,32 @@ std::unique_ptr<uint8_t[]> EglDmaBuf::ImageFromDmaBuf( + if (image == EGL_NO_IMAGE) { + RTC_LOG(LS_ERROR) << "Failed to record frame: Error creating EGLImage - " + << FormatEGLError(EglGetError()); +- return src; ++ return false; + } + + // create GL 2D texture for framebuffer +- GLuint texture; +- GlGenTextures(1, &texture); +- GlTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_NEAREST); +- GlTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_NEAREST); +- GlTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_S, GL_CLAMP_TO_EDGE); +- GlTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_T, GL_CLAMP_TO_EDGE); +- GlBindTexture(GL_TEXTURE_2D, texture); ++ if (!texture_) { ++ GlGenTextures(1, &texture_); ++ GlTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_NEAREST); ++ GlTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_NEAREST); ++ GlTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_S, GL_CLAMP_TO_EDGE); ++ GlTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_T, GL_CLAMP_TO_EDGE); ++ } ++ GlBindTexture(GL_TEXTURE_2D, texture_); + GlEGLImageTargetTexture2DOES(GL_TEXTURE_2D, image); + +- src = std::make_unique<uint8_t[]>(plane_datas[0].stride * size.height()); ++ if (!fbo_) { ++ GlGenFramebuffers(1, &fbo_); ++ } ++ ++ GlBindFramebuffer(GL_FRAMEBUFFER, fbo_); ++ GlFramebufferTexture2D(GL_FRAMEBUFFER, GL_COLOR_ATTACHMENT0, GL_TEXTURE_2D, ++ texture_, 0); ++ if (GlCheckFramebufferStatus(GL_FRAMEBUFFER) != GL_FRAMEBUFFER_COMPLETE) { ++ RTC_LOG(LS_ERROR) << "Failed to bind DMA buf framebuffer"; ++ EglDestroyImageKHR(egl_.display, image); ++ return false; ++ } + + GLenum gl_format = GL_BGRA; + switch (format) { +@@ -624,17 +675,18 @@ std::unique_ptr<uint8_t[]> EglDmaBuf::ImageFromDmaBuf( + gl_format = GL_BGRA; + break; + } +- GlGetTexImage(GL_TEXTURE_2D, 0, gl_format, GL_UNSIGNED_BYTE, src.get()); + +- if (GlGetError()) { ++ GlReadPixels(offset.x(), offset.y(), buffer_size.width(), ++ buffer_size.height(), gl_format, GL_UNSIGNED_BYTE, data); ++ ++ const GLenum error = GlGetError(); ++ if (error) { + RTC_LOG(LS_ERROR) << "Failed to get image from DMA buffer."; +- return src; + } + +- GlDeleteTextures(1, &texture); + EglDestroyImageKHR(egl_.display, image); + +- return src; ++ return !error; + } + + RTC_NO_SANITIZE("cfi-icall") +diff --git a/modules/desktop_capture/linux/wayland/egl_dmabuf.h b/modules/desktop_capture/linux/wayland/egl_dmabuf.h +index f1d96b2f80..22a8f5ab52 100644 +--- a/modules/desktop_capture/linux/wayland/egl_dmabuf.h ++++ b/modules/desktop_capture/linux/wayland/egl_dmabuf.h +@@ -41,11 +41,15 @@ class EglDmaBuf { + EglDmaBuf(); + ~EglDmaBuf(); + +- std::unique_ptr<uint8_t[]> ImageFromDmaBuf( +- const DesktopSize& size, +- uint32_t format, +- const std::vector<PlaneData>& plane_datas, +- uint64_t modifiers); ++ // Returns whether the image was successfully imported from ++ // given DmaBuf and its parameters ++ bool ImageFromDmaBuf(const DesktopSize& size, ++ uint32_t format, ++ const std::vector<PlaneData>& plane_datas, ++ uint64_t modifiers, ++ const DesktopVector& offset, ++ const DesktopSize& buffer_size, ++ uint8_t* data); + std::vector<uint64_t> QueryDmaBufModifiers(uint32_t format); + + bool IsEglInitialized() const { return egl_initialized_; } +@@ -58,6 +62,8 @@ class EglDmaBuf { + int32_t drm_fd_ = -1; // for GBM buffer mmap + gbm_device* gbm_device_ = nullptr; // for passed GBM buffer retrieval + ++ GLuint fbo_ = 0; ++ GLuint texture_ = 0; + EGLStruct egl_; + + absl::optional<std::string> GetRenderNode(); +diff --git a/modules/desktop_capture/linux/wayland/shared_screencast_stream.cc b/modules/desktop_capture/linux/wayland/shared_screencast_stream.cc +index 71bde9b212..bcd7e3a33c 100644 +--- a/modules/desktop_capture/linux/wayland/shared_screencast_stream.cc ++++ b/modules/desktop_capture/linux/wayland/shared_screencast_stream.cc +@@ -38,7 +38,6 @@ constexpr int CursorMetaSize(int w, int h) { + w * h * kCursorBpp); + } + +-constexpr PipeWireVersion kDmaBufMinVersion = {0, 3, 24}; + constexpr PipeWireVersion kDmaBufModifierMinVersion = {0, 3, 33}; + constexpr PipeWireVersion kDropSingleModifierMinVersion = {0, 3, 40}; + +@@ -155,6 +154,12 @@ class SharedScreenCastStreamPrivate { + struct spa_video_info_raw spa_video_format_; + + void ProcessBuffer(pw_buffer* buffer); ++ bool ProcessMemFDBuffer(pw_buffer* buffer, ++ DesktopFrame& frame, ++ const DesktopVector& offset); ++ bool ProcessDMABuffer(pw_buffer* buffer, ++ DesktopFrame& frame, ++ const DesktopVector& offset); + void ConvertRGBxToBGRx(uint8_t* frame, uint32_t size); + + // PipeWire callbacks +@@ -277,10 +282,9 @@ void SharedScreenCastStreamPrivate::OnStreamParamChanged( + has_modifier ? that->spa_video_format_.modifier : DRM_FORMAT_MOD_INVALID; + std::vector<const spa_pod*> params; + const int buffer_types = +- has_modifier || (that->pw_server_version_ >= kDmaBufMinVersion) +- ? (1 << SPA_DATA_DmaBuf) | (1 << SPA_DATA_MemFd) | +- (1 << SPA_DATA_MemPtr) +- : (1 << SPA_DATA_MemFd) | (1 << SPA_DATA_MemPtr); ++ has_modifier ++ ? (1 << SPA_DATA_DmaBuf) | (1 << SPA_DATA_MemFd) ++ : (1 << SPA_DATA_MemFd); + + params.push_back(reinterpret_cast<spa_pod*>(spa_pod_builder_add_object( + &builder, SPA_TYPE_OBJECT_ParamBuffers, SPA_PARAM_Buffers, +@@ -613,9 +617,6 @@ DesktopVector SharedScreenCastStreamPrivate::CaptureCursorPosition() { + RTC_NO_SANITIZE("cfi-icall") + void SharedScreenCastStreamPrivate::ProcessBuffer(pw_buffer* buffer) { + spa_buffer* spa_buffer = buffer->buffer; +- ScopedBuf map; +- std::unique_ptr<uint8_t[]> src_unique_ptr; +- uint8_t* src = nullptr; + + // Try to update the mouse cursor first, because it can be the only + // information carried by the buffer +@@ -657,79 +658,6 @@ void SharedScreenCastStreamPrivate::ProcessBuffer(pw_buffer* buffer) { + return; + } + +- if (spa_buffer->datas[0].type == SPA_DATA_MemFd) { +- map.initialize( +- static_cast<uint8_t*>( +- mmap(nullptr, +- spa_buffer->datas[0].maxsize + spa_buffer->datas[0].mapoffset, +- PROT_READ, MAP_PRIVATE, spa_buffer->datas[0].fd, 0)), +- spa_buffer->datas[0].maxsize + spa_buffer->datas[0].mapoffset, +- spa_buffer->datas[0].fd); +- +- if (!map) { +- RTC_LOG(LS_ERROR) << "Failed to mmap the memory: " +- << std::strerror(errno); +- return; +- } +- +- src = SPA_MEMBER(map.get(), spa_buffer->datas[0].mapoffset, uint8_t); +- } else if (spa_buffer->datas[0].type == SPA_DATA_DmaBuf) { +- const uint n_planes = spa_buffer->n_datas; +- +- if (!n_planes) { +- return; +- } +- +- std::vector<EglDmaBuf::PlaneData> plane_datas; +- for (uint32_t i = 0; i < n_planes; ++i) { +- EglDmaBuf::PlaneData data = { +- static_cast<int32_t>(spa_buffer->datas[i].fd), +- static_cast<uint32_t>(spa_buffer->datas[i].chunk->stride), +- static_cast<uint32_t>(spa_buffer->datas[i].chunk->offset)}; +- plane_datas.push_back(data); +- } +- +- // When importing DMA-BUFs, we use the stride (number of bytes from one row +- // of pixels in the buffer) provided by PipeWire. The stride from PipeWire +- // is given by the graphics driver and some drivers might add some +- // additional padding for memory layout optimizations so not everytime the +- // stride is equal to BYTES_PER_PIXEL x WIDTH. This is fine, because during +- // the import we will use OpenGL and same graphics driver so it will be able +- // to work with the stride it provided, but later on when we work with +- // images we get from DMA-BUFs we will need to update the stride to be equal +- // to BYTES_PER_PIXEL x WIDTH as that's the size of the DesktopFrame we +- // allocate for each captured frame. +- src_unique_ptr = egl_dmabuf_->ImageFromDmaBuf( +- stream_size_, spa_video_format_.format, plane_datas, modifier_); +- if (src_unique_ptr) { +- src = src_unique_ptr.get(); +- } else { +- RTC_LOG(LS_ERROR) << "Dropping DMA-BUF modifier: " << modifier_ +- << " and trying to renegotiate stream parameters"; +- +- if (pw_server_version_ >= kDropSingleModifierMinVersion) { +- modifiers_.erase( +- std::remove(modifiers_.begin(), modifiers_.end(), modifier_), +- modifiers_.end()); +- } else { +- modifiers_.clear(); +- } +- +- pw_loop_signal_event(pw_thread_loop_get_loop(pw_main_loop_), +- renegotiate_); +- return; +- } +- } else if (spa_buffer->datas[0].type == SPA_DATA_MemPtr) { +- src = static_cast<uint8_t*>(spa_buffer->datas[0].data); +- } +- +- if (!src) { +- if (observer_) { +- observer_->OnFailedToProcessBuffer(); +- } +- return; +- } +- + // Use SPA_META_VideoCrop metadata to get the frame size. KDE and GNOME do + // handle screen/window sharing differently. KDE/KWin doesn't use + // SPA_META_VideoCrop metadata and when sharing a window, it always sets +@@ -787,8 +715,8 @@ void SharedScreenCastStreamPrivate::ProcessBuffer(pw_buffer* buffer) { + } + + // Get the position of the video crop within the stream. Just double-check +- // that the position doesn't exceed the size of the stream itself. NOTE: +- // Currently it looks there is no implementation using this. ++ // that the position doesn't exceed the size of the stream itself. ++ // NOTE: Currently it looks there is no implementation using this. + uint32_t y_offset = + videocrop_metadata_use && + (videocrop_metadata->region.position.y + frame_size_.height() <= +@@ -801,22 +729,7 @@ void SharedScreenCastStreamPrivate::ProcessBuffer(pw_buffer* buffer) { + stream_size_.width()) + ? videocrop_metadata->region.position.x + : 0; +- +- const uint32_t stream_stride = kBytesPerPixel * stream_size_.width(); +- uint32_t buffer_stride = spa_buffer->datas[0].chunk->stride; +- uint32_t src_stride = buffer_stride; +- +- if (spa_buffer->datas[0].type == SPA_DATA_DmaBuf && +- buffer_stride > stream_stride) { +- // When DMA-BUFs are used, sometimes spa_buffer->stride we get might +- // contain additional padding, but after we import the buffer, the stride +- // we used is no longer relevant and we should just calculate it based on +- // the stream width. For more context see https://crbug.com/1333304. +- src_stride = stream_stride; +- } +- +- uint8_t* updated_src = +- src + (src_stride * y_offset) + (kBytesPerPixel * x_offset); ++ DesktopVector offset = DesktopVector(x_offset, y_offset); + + webrtc::MutexLock lock(&queue_lock_); + +@@ -836,9 +749,20 @@ void SharedScreenCastStreamPrivate::ProcessBuffer(pw_buffer* buffer) { + queue_.ReplaceCurrentFrame(SharedDesktopFrame::Wrap(std::move(frame))); + } + +- queue_.current_frame()->CopyPixelsFrom( +- updated_src, (src_stride - (kBytesPerPixel * x_offset)), +- DesktopRect::MakeWH(frame_size_.width(), frame_size_.height())); ++ bool bufferProcessed = false; ++ if (spa_buffer->datas[0].type == SPA_DATA_MemFd) { ++ bufferProcessed = ++ ProcessMemFDBuffer(buffer, *queue_.current_frame(), offset); ++ } else if (spa_buffer->datas[0].type == SPA_DATA_DmaBuf) { ++ bufferProcessed = ProcessDMABuffer(buffer, *queue_.current_frame(), offset); ++ } ++ ++ if (!bufferProcessed) { ++ if (observer_) { ++ observer_->OnFailedToProcessBuffer(); ++ } ++ return; ++ } + + if (spa_video_format_.format == SPA_VIDEO_FORMAT_RGBx || + spa_video_format_.format == SPA_VIDEO_FORMAT_RGBA) { +@@ -885,6 +809,87 @@ void SharedScreenCastStreamPrivate::ProcessBuffer(pw_buffer* buffer) { + queue_.current_frame()->set_may_contain_cursor(is_cursor_embedded_); + } + ++RTC_NO_SANITIZE("cfi-icall") ++bool SharedScreenCastStreamPrivate::ProcessMemFDBuffer( ++ pw_buffer* buffer, ++ DesktopFrame& frame, ++ const DesktopVector& offset) { ++ spa_buffer* spa_buffer = buffer->buffer; ++ ScopedBuf map; ++ uint8_t* src = nullptr; ++ ++ map.initialize( ++ static_cast<uint8_t*>( ++ mmap(nullptr, ++ spa_buffer->datas[0].maxsize + spa_buffer->datas[0].mapoffset, ++ PROT_READ, MAP_PRIVATE, spa_buffer->datas[0].fd, 0)), ++ spa_buffer->datas[0].maxsize + spa_buffer->datas[0].mapoffset, ++ spa_buffer->datas[0].fd); ++ ++ if (!map) { ++ RTC_LOG(LS_ERROR) << "Failed to mmap the memory: " << std::strerror(errno); ++ return false; ++ } ++ ++ src = SPA_MEMBER(map.get(), spa_buffer->datas[0].mapoffset, uint8_t); ++ ++ uint32_t buffer_stride = spa_buffer->datas[0].chunk->stride; ++ uint32_t src_stride = buffer_stride; ++ ++ uint8_t* updated_src = ++ src + (src_stride * offset.y()) + (kBytesPerPixel * offset.x()); ++ ++ frame.CopyPixelsFrom( ++ updated_src, (src_stride - (kBytesPerPixel * offset.x())), ++ DesktopRect::MakeWH(frame.size().width(), frame.size().height())); ++ ++ return true; ++} ++ ++RTC_NO_SANITIZE("cfi-icall") ++bool SharedScreenCastStreamPrivate::ProcessDMABuffer( ++ pw_buffer* buffer, ++ DesktopFrame& frame, ++ const DesktopVector& offset) { ++ spa_buffer* spa_buffer = buffer->buffer; ++ ++ const uint n_planes = spa_buffer->n_datas; ++ ++ if (!n_planes) { ++ return false; ++ } ++ ++ std::vector<EglDmaBuf::PlaneData> plane_datas; ++ for (uint32_t i = 0; i < n_planes; ++i) { ++ EglDmaBuf::PlaneData data = { ++ static_cast<int32_t>(spa_buffer->datas[i].fd), ++ static_cast<uint32_t>(spa_buffer->datas[i].chunk->stride), ++ static_cast<uint32_t>(spa_buffer->datas[i].chunk->offset)}; ++ plane_datas.push_back(data); ++ } ++ ++ const bool imported = egl_dmabuf_->ImageFromDmaBuf( ++ stream_size_, spa_video_format_.format, plane_datas, modifier_, offset, ++ frame.size(), frame.data()); ++ if (!imported) { ++ RTC_LOG(LS_ERROR) << "Dropping DMA-BUF modifier: " << modifier_ ++ << " and trying to renegotiate stream parameters"; ++ ++ if (pw_server_version_ >= kDropSingleModifierMinVersion) { ++ modifiers_.erase( ++ std::remove(modifiers_.begin(), modifiers_.end(), modifier_), ++ modifiers_.end()); ++ } else { ++ modifiers_.clear(); ++ } ++ ++ pw_loop_signal_event(pw_thread_loop_get_loop(pw_main_loop_), renegotiate_); ++ return false; ++ } ++ ++ return true; ++} ++ + void SharedScreenCastStreamPrivate::ConvertRGBxToBGRx(uint8_t* frame, + uint32_t size) { + for (uint32_t i = 0; i < size; i += 4) { +-- +2.34.1 + diff --git a/third_party/libwebrtc/moz-patch-stack/0089.patch b/third_party/libwebrtc/moz-patch-stack/0089.patch new file mode 100644 index 0000000000..409649736b --- /dev/null +++ b/third_party/libwebrtc/moz-patch-stack/0089.patch @@ -0,0 +1,269 @@ +From: Byron Campen <docfaraday@gmail.com> +Date: Fri, 31 Mar 2023 16:21:00 -0500 +Subject: Bug 1822194 - (fix-acabb3641b) Break the new SetParametersCallback + stuff into stand-alone files. + +acabb3641b from upstream added a callback mechanism to allow failures to be +propagated back to RTCRtpSender.setParameters. Unfortunately, this callback +mechanism was (needlessly) tightly coupled to libwebrtc's implementation of +RTCRtpSender, and also their media channel code. This introduced a lot of +unnecessary dependencies throughout libwebrtc, that spilled into our code as +well. +Mercurial Revision: https://hg.mozilla.org/mozilla-central/rev/59232687efa00e5f7b7bd3d6befca129149e2bf5 +--- + api/BUILD.gn | 14 +++++++++++- + api/rtp_sender_interface.h | 4 ++-- + api/rtp_sender_setparameters_callback.cc | 27 +++++++++++++++++++++++ + api/rtp_sender_setparameters_callback.h | 28 ++++++++++++++++++++++++ + call/BUILD.gn | 4 ++-- + call/audio_send_stream.h | 2 +- + call/video_send_stream.h | 2 +- + media/BUILD.gn | 2 +- + media/base/media_channel.h | 4 ---- + media/base/media_channel_impl.cc | 13 ----------- + video/BUILD.gn | 4 ++-- + 11 files changed, 77 insertions(+), 27 deletions(-) + create mode 100644 api/rtp_sender_setparameters_callback.cc + create mode 100644 api/rtp_sender_setparameters_callback.h + +diff --git a/api/BUILD.gn b/api/BUILD.gn +index 3f313e2743..56afc5efce 100644 +--- a/api/BUILD.gn ++++ b/api/BUILD.gn +@@ -214,8 +214,8 @@ rtc_library("rtp_sender_interface") { + ":dtmf_sender_interface", + ":frame_transformer_interface", + ":media_stream_interface", +- ":rtc_error", + ":rtp_parameters", ++ ":rtp_sender_setparameters_callback", + ":scoped_refptr", + "../rtc_base:checks", + "../rtc_base:refcount", +@@ -223,6 +223,18 @@ rtc_library("rtp_sender_interface") { + "crypto:frame_encryptor_interface", + "video_codecs:video_codecs_api", + ] ++} ++ ++rtc_library("rtp_sender_setparameters_callback") { ++ visibility = [ "*" ] ++ ++ sources = [ ++ "rtp_sender_setparameters_callback.cc", ++ "rtp_sender_setparameters_callback.h", ++ ] ++ deps = [ ++ ":rtc_error", ++ ] + absl_deps = [ "//third_party/abseil-cpp/absl/functional:any_invocable" ] + } + +diff --git a/api/rtp_sender_interface.h b/api/rtp_sender_interface.h +index 2786a2ac19..98ee91b1cc 100644 +--- a/api/rtp_sender_interface.h ++++ b/api/rtp_sender_interface.h +@@ -32,9 +32,9 @@ + #include "rtc_base/ref_count.h" + #include "rtc_base/system/rtc_export.h" + +-namespace webrtc { ++#include "api/rtp_sender_setparameters_callback.h" + +-using SetParametersCallback = absl::AnyInvocable<void(RTCError) &&>; ++namespace webrtc { + + class RTC_EXPORT RtpSenderInterface : public rtc::RefCountInterface { + public: +diff --git a/api/rtp_sender_setparameters_callback.cc b/api/rtp_sender_setparameters_callback.cc +new file mode 100644 +index 0000000000..99728ef95e +--- /dev/null ++++ b/api/rtp_sender_setparameters_callback.cc +@@ -0,0 +1,27 @@ ++/* ++ * Copyright 2015 The WebRTC project authors. All Rights Reserved. ++ * ++ * Use of this source code is governed by a BSD-style license ++ * that can be found in the LICENSE file in the root of the source ++ * tree. An additional intellectual property rights grant can be found ++ * in the file PATENTS. All contributing project authors may ++ * be found in the AUTHORS file in the root of the source tree. ++ */ ++ ++// File added by mozilla, to decouple this from libwebrtc's implementation of ++// RTCRtpSender. ++ ++#include "api/rtp_sender_setparameters_callback.h" ++ ++namespace webrtc { ++ ++webrtc::RTCError InvokeSetParametersCallback(SetParametersCallback& callback, ++ RTCError error) { ++ if (callback) { ++ std::move(callback)(error); ++ callback = nullptr; ++ } ++ return error; ++} ++ ++} // namespace webrtc +diff --git a/api/rtp_sender_setparameters_callback.h b/api/rtp_sender_setparameters_callback.h +new file mode 100644 +index 0000000000..45194f5ace +--- /dev/null ++++ b/api/rtp_sender_setparameters_callback.h +@@ -0,0 +1,28 @@ ++/* ++ * Copyright 2015 The WebRTC project authors. All Rights Reserved. ++ * ++ * Use of this source code is governed by a BSD-style license ++ * that can be found in the LICENSE file in the root of the source ++ * tree. An additional intellectual property rights grant can be found ++ * in the file PATENTS. All contributing project authors may ++ * be found in the AUTHORS file in the root of the source tree. ++ */ ++ ++// File added by mozilla, to decouple this from libwebrtc's implementation of ++// RTCRtpSender. ++ ++#ifndef API_RTP_SENDER_SETPARAMETERS_CALLBACK_H_ ++#define API_RTP_SENDER_SETPARAMETERS_CALLBACK_H_ ++ ++#include "api/rtc_error.h" ++#include "absl/functional/any_invocable.h" ++ ++namespace webrtc { ++ ++using SetParametersCallback = absl::AnyInvocable<void(RTCError) &&>; ++ ++webrtc::RTCError InvokeSetParametersCallback(SetParametersCallback& callback, ++ RTCError error); ++} // namespace webrtc ++ ++#endif // API_RTP_SENDER_SETPARAMETERS_CALLBACK_H_ +diff --git a/call/BUILD.gn b/call/BUILD.gn +index fb23b7ef39..2bc7aaec92 100644 +--- a/call/BUILD.gn ++++ b/call/BUILD.gn +@@ -52,7 +52,7 @@ rtc_library("call_interfaces") { + "../api:rtc_error", + "../api:rtp_headers", + "../api:rtp_parameters", +- "../api:rtp_sender_interface", ++ "../api:rtp_sender_setparameters_callback", + "../api:scoped_refptr", + "../api:transport_api", + "../api/adaptation:resource_adaptation_api", +@@ -389,7 +389,7 @@ rtc_library("video_stream_api") { + "../api:frame_transformer_interface", + "../api:rtp_headers", + "../api:rtp_parameters", +- "../api:rtp_sender_interface", ++ "../api:rtp_sender_setparameters_callback", + "../api:scoped_refptr", + "../api:transport_api", + "../api/adaptation:resource_adaptation_api", +diff --git a/call/audio_send_stream.h b/call/audio_send_stream.h +index bafa22d312..187ec65ed8 100644 +--- a/call/audio_send_stream.h ++++ b/call/audio_send_stream.h +@@ -25,7 +25,7 @@ + #include "api/crypto/frame_encryptor_interface.h" + #include "api/frame_transformer_interface.h" + #include "api/rtp_parameters.h" +-#include "api/rtp_sender_interface.h" ++#include "api/rtp_sender_setparameters_callback.h" + #include "api/scoped_refptr.h" + #include "call/audio_sender.h" + #include "call/rtp_config.h" +diff --git a/call/video_send_stream.h b/call/video_send_stream.h +index 431c267e1e..de18fc7b92 100644 +--- a/call/video_send_stream.h ++++ b/call/video_send_stream.h +@@ -23,7 +23,7 @@ + #include "api/crypto/crypto_options.h" + #include "api/frame_transformer_interface.h" + #include "api/rtp_parameters.h" +-#include "api/rtp_sender_interface.h" ++#include "api/rtp_sender_setparameters_callback.h" + #include "api/scoped_refptr.h" + #include "api/video/video_content_type.h" + #include "api/video/video_frame.h" +diff --git a/media/BUILD.gn b/media/BUILD.gn +index daca67e033..80662563b0 100644 +--- a/media/BUILD.gn ++++ b/media/BUILD.gn +@@ -64,7 +64,7 @@ rtc_library("rtc_media_base") { + "../api:media_stream_interface", + "../api:rtc_error", + "../api:rtp_parameters", +- "../api:rtp_sender_interface", ++ "../api:rtp_sender_setparameters_callback", + "../api:scoped_refptr", + "../api:sequence_checker", + "../api:transport_api", +diff --git a/media/base/media_channel.h b/media/base/media_channel.h +index 43e09290bd..138d28ae4c 100644 +--- a/media/base/media_channel.h ++++ b/media/base/media_channel.h +@@ -64,10 +64,6 @@ class Timing; + + namespace webrtc { + class VideoFrame; +- +-webrtc::RTCError InvokeSetParametersCallback(SetParametersCallback& callback, +- RTCError error); +- + } // namespace webrtc + + namespace cricket { +diff --git a/media/base/media_channel_impl.cc b/media/base/media_channel_impl.cc +index a72b97413d..0e72f47d6d 100644 +--- a/media/base/media_channel_impl.cc ++++ b/media/base/media_channel_impl.cc +@@ -31,19 +31,6 @@ + #include "modules/rtp_rtcp/include/report_block_data.h" + #include "rtc_base/checks.h" + +-namespace webrtc { +- +-webrtc::RTCError InvokeSetParametersCallback(SetParametersCallback& callback, +- RTCError error) { +- if (callback) { +- std::move(callback)(error); +- callback = nullptr; +- } +- return error; +-} +- +-} // namespace webrtc +- + namespace cricket { + using webrtc::FrameDecryptorInterface; + using webrtc::FrameEncryptorInterface; +diff --git a/video/BUILD.gn b/video/BUILD.gn +index e21e1c7ea8..2c0a411e35 100644 +--- a/video/BUILD.gn ++++ b/video/BUILD.gn +@@ -17,7 +17,7 @@ rtc_library("video_stream_encoder_interface") { + "../api:fec_controller_api", + "../api:rtc_error", + "../api:rtp_parameters", +- "../api:rtp_sender_interface", ++ "../api:rtp_sender_setparameters_callback", + "../api:scoped_refptr", + "../api/adaptation:resource_adaptation_api", + "../api/units:data_rate", +@@ -410,7 +410,7 @@ rtc_library("video_stream_encoder_impl") { + ":video_stream_encoder_interface", + "../api:field_trials_view", + "../api:rtp_parameters", +- "../api:rtp_sender_interface", ++ "../api:rtp_sender_setparameters_callback", + "../api:sequence_checker", + "../api/adaptation:resource_adaptation_api", + "../api/task_queue:pending_task_safety_flag", +-- +2.34.1 + diff --git a/third_party/libwebrtc/moz-patch-stack/0090.patch b/third_party/libwebrtc/moz-patch-stack/0090.patch new file mode 100644 index 0000000000..affbb277e2 --- /dev/null +++ b/third_party/libwebrtc/moz-patch-stack/0090.patch @@ -0,0 +1,25 @@ +From: Byron Campen <docfaraday@gmail.com> +Date: Tue, 4 Apr 2023 16:34:00 -0500 +Subject: Bug 1822194 - (fix-3b51cd328e) - Add missing designated initializer + that gcc is sad about. + +Mercurial Revision: https://hg.mozilla.org/mozilla-central/rev/108046c7cbb21c6cf19320c0804e9aee1a3eb4bf +--- + modules/audio_processing/audio_processing_impl.cc | 1 + + 1 file changed, 1 insertion(+) + +diff --git a/modules/audio_processing/audio_processing_impl.cc b/modules/audio_processing/audio_processing_impl.cc +index c80cc76a3d..c304453388 100644 +--- a/modules/audio_processing/audio_processing_impl.cc ++++ b/modules/audio_processing/audio_processing_impl.cc +@@ -450,6 +450,7 @@ AudioProcessingImpl::GetGainController2ExperimentParams() { + }, + .adaptive_digital_controller = + { ++ .enabled = false, + .headroom_db = static_cast<float>(headroom_db.Get()), + .max_gain_db = static_cast<float>(max_gain_db.Get()), + .initial_gain_db = +-- +2.34.1 + diff --git a/third_party/libwebrtc/moz-patch-stack/0091.patch b/third_party/libwebrtc/moz-patch-stack/0091.patch new file mode 100644 index 0000000000..95d608bbaa --- /dev/null +++ b/third_party/libwebrtc/moz-patch-stack/0091.patch @@ -0,0 +1,27 @@ +From: Byron Campen <docfaraday@gmail.com> +Date: Fri, 7 Apr 2023 20:28:00 +0000 +Subject: Bug 1819048: Remove this bad assertion. r=webrtc-reviewers,jib + +Differential Revision: https://phabricator.services.mozilla.com/D174978 +Mercurial Revision: https://hg.mozilla.org/mozilla-central/rev/5a52e1b0c808edfda82f0abea668699eb68098dc +--- + video/task_queue_frame_decode_scheduler.cc | 3 ++- + 1 file changed, 2 insertions(+), 1 deletion(-) + +diff --git a/video/task_queue_frame_decode_scheduler.cc b/video/task_queue_frame_decode_scheduler.cc +index cd109c2932..6dd7b47f17 100644 +--- a/video/task_queue_frame_decode_scheduler.cc ++++ b/video/task_queue_frame_decode_scheduler.cc +@@ -37,7 +37,8 @@ void TaskQueueFrameDecodeScheduler::ScheduleFrame( + uint32_t rtp, + FrameDecodeTiming::FrameSchedule schedule, + FrameReleaseCallback cb) { +- RTC_DCHECK(!stopped_) << "Can not schedule frames after stopped."; ++ // Mozilla modification, until https://bugs.webrtc.org/14944 is fixed ++ //RTC_DCHECK(!stopped_) << "Can not schedule frames after stopped."; + RTC_DCHECK(!scheduled_rtp_.has_value()) + << "Can not schedule two frames for release at the same time."; + RTC_DCHECK(cb); +-- +2.34.1 + diff --git a/third_party/libwebrtc/moz-patch-stack/0092.patch b/third_party/libwebrtc/moz-patch-stack/0092.patch new file mode 100644 index 0000000000..110b8d56ce --- /dev/null +++ b/third_party/libwebrtc/moz-patch-stack/0092.patch @@ -0,0 +1,146 @@ +From: Michael Froman <mfroman@mozilla.com> +Date: Thu, 20 Apr 2023 09:59:00 -0500 +Subject: Bug 1828517 - (fix-794d599741) account for moved files in BUILD.gn + that we don't want to build. + +Mercurial Revision: https://hg.mozilla.org/mozilla-central/rev/4a969f6709183d4f55215adaffb8a52b790a8492 +--- + api/BUILD.gn | 10 ++++++++++ + media/BUILD.gn | 20 ++++++++++---------- + 2 files changed, 20 insertions(+), 10 deletions(-) + +diff --git a/api/BUILD.gn b/api/BUILD.gn +index 56afc5efce..7c16b45e05 100644 +--- a/api/BUILD.gn ++++ b/api/BUILD.gn +@@ -175,6 +175,10 @@ rtc_source_set("ice_transport_interface") { + } + + rtc_library("dtls_transport_interface") { ++# Previously, Mozilla has tried to limit including this dep, but as ++# upstream changes, it requires whack-a-mole. Making it an empty ++# definition has the same effect, but only requires one change. ++if (!build_with_mozilla) { + visibility = [ "*" ] + + sources = [ +@@ -191,6 +195,7 @@ rtc_library("dtls_transport_interface") { + ] + absl_deps = [ "//third_party/abseil-cpp/absl/types:optional" ] + } ++} + + rtc_library("dtmf_sender_interface") { + visibility = [ "*" ] +@@ -203,6 +208,10 @@ rtc_library("dtmf_sender_interface") { + } + + rtc_library("rtp_sender_interface") { ++# Previously, Mozilla has tried to limit including this dep, but as ++# upstream changes, it requires whack-a-mole. Making it an empty ++# definition has the same effect, but only requires one change. ++if (!build_with_mozilla) { + visibility = [ "*" ] + + sources = [ +@@ -224,6 +233,7 @@ rtc_library("rtp_sender_interface") { + "video_codecs:video_codecs_api", + ] + } ++} + + rtc_library("rtp_sender_setparameters_callback") { + visibility = [ "*" ] +diff --git a/media/BUILD.gn b/media/BUILD.gn +index 80662563b0..36c3aa9d1c 100644 +--- a/media/BUILD.gn ++++ b/media/BUILD.gn +@@ -154,23 +154,14 @@ rtc_library("rtc_media_base") { + "base/audio_source.h", + "base/delayable.h", + "base/media_channel.h", +- "base/media_channel_impl.cc", +- "base/media_channel_impl.h", + "base/media_engine.cc", + "base/media_engine.h", +- "base/rid_description.cc", +- "base/rid_description.h", +- "base/rtp_utils.cc", +- "base/rtp_utils.h", +- "base/stream_params.cc", +- "base/stream_params.h", +- "base/turn_utils.cc", +- "base/turn_utils.h", + ] + } + } + + rtc_library("media_channel_impl") { ++if (!build_with_mozilla) { + sources = [ + "base/media_channel_impl.cc", + "base/media_channel_impl.h", +@@ -219,6 +210,7 @@ rtc_library("media_channel_impl") { + "//third_party/abseil-cpp/absl/types:optional", + ] + } ++} + + rtc_source_set("media_channel") { + sources = [ "base/media_channel.h" ] +@@ -292,6 +284,7 @@ rtc_library("codec") { + } + + rtc_library("rtp_utils") { ++if (!build_with_mozilla) { + sources = [ + "base/rtp_utils.cc", + "base/rtp_utils.h", +@@ -308,8 +301,10 @@ rtc_library("rtp_utils") { + ] + absl_deps = [ "//third_party/abseil-cpp/absl/strings" ] + } ++} + + rtc_library("stream_params") { ++if (!build_with_mozilla) { + sources = [ + "base/stream_params.cc", + "base/stream_params.h", +@@ -322,6 +317,7 @@ rtc_library("stream_params") { + ] + absl_deps = [ "//third_party/abseil-cpp/absl/algorithm:container" ] + } ++} + + rtc_library("media_constants") { + sources = [ +@@ -332,6 +328,7 @@ rtc_library("media_constants") { + } + + rtc_library("turn_utils") { ++if (!build_with_mozilla) { + sources = [ + "base/turn_utils.cc", + "base/turn_utils.h", +@@ -342,14 +339,17 @@ rtc_library("turn_utils") { + "../rtc_base/system:rtc_export", + ] + } ++} + + rtc_library("rid_description") { ++if (!build_with_mozilla) { + sources = [ + "base/rid_description.cc", + "base/rid_description.h", + ] + deps = [] + } ++} + + rtc_library("rtc_simulcast_encoder_adapter") { + visibility = [ "*" ] +-- +2.34.1 + diff --git a/third_party/libwebrtc/moz-patch-stack/0093.patch b/third_party/libwebrtc/moz-patch-stack/0093.patch new file mode 100644 index 0000000000..76bb463ca7 --- /dev/null +++ b/third_party/libwebrtc/moz-patch-stack/0093.patch @@ -0,0 +1,177 @@ +From: Michael Froman <mfroman@mozilla.com> +Date: Thu, 20 Apr 2023 14:52:00 -0500 +Subject: Bug 1828517 - (fix-a138c6c8a5) handle file moves in BUILD.gn + +Mercurial Revision: https://hg.mozilla.org/mozilla-central/rev/cf7e333da17689b3c115a6ffd07fab042bc5f086 +--- + rtc_base/BUILD.gn | 24 ++++++++++++++++++++++++ + 1 file changed, 24 insertions(+) + +diff --git a/rtc_base/BUILD.gn b/rtc_base/BUILD.gn +index 0b1e2a6208..7573a137ab 100644 +--- a/rtc_base/BUILD.gn ++++ b/rtc_base/BUILD.gn +@@ -1147,6 +1147,7 @@ if (!build_with_chromium) { + } + + rtc_library("network") { ++if (!build_with_mozilla) { + visibility = [ "*" ] + sources = [ + "network.cc", +@@ -1185,16 +1186,20 @@ rtc_library("network") { + deps += [ ":win32" ] + } + } ++} + + rtc_library("socket_address_pair") { ++if (!build_with_mozilla) { + sources = [ + "socket_address_pair.cc", + "socket_address_pair.h", + ] + deps = [ ":socket_address" ] + } ++} + + rtc_library("net_helper") { ++if (!build_with_mozilla) { + visibility = [ "*" ] + sources = [ + "net_helper.cc", +@@ -1203,8 +1208,10 @@ rtc_library("net_helper") { + absl_deps = [ "//third_party/abseil-cpp/absl/strings" ] + deps = [ "system:rtc_export" ] + } ++} + + rtc_library("socket_adapters") { ++if (!build_with_mozilla) { + visibility = [ "*" ] + sources = [ + "socket_adapters.cc", +@@ -1224,6 +1231,7 @@ rtc_library("socket_adapters") { + ] + absl_deps = [ "//third_party/abseil-cpp/absl/strings" ] + } ++} + + rtc_library("network_route") { + sources = [ +@@ -1238,6 +1246,7 @@ rtc_library("network_route") { + } + + rtc_library("async_tcp_socket") { ++if (!build_with_mozilla) { + sources = [ + "async_tcp_socket.cc", + "async_tcp_socket.h", +@@ -1256,8 +1265,10 @@ rtc_library("async_tcp_socket") { + "third_party/sigslot", + ] + } ++} + + rtc_library("async_udp_socket") { ++if (!build_with_mozilla) { + visibility = [ "*" ] + sources = [ + "async_udp_socket.cc", +@@ -1279,8 +1290,10 @@ rtc_library("async_udp_socket") { + ] + absl_deps = [ "//third_party/abseil-cpp/absl/types:optional" ] + } ++} + + rtc_library("async_packet_socket") { ++if (!build_with_mozilla) { + visibility = [ "*" ] + sources = [ + "async_packet_socket.cc", +@@ -1298,6 +1311,7 @@ rtc_library("async_packet_socket") { + "third_party/sigslot", + ] + } ++} + + rtc_library("mdns_responder_interface") { + sources = [ "mdns_responder_interface.h" ] +@@ -1310,6 +1324,7 @@ rtc_library("dscp") { + } + + rtc_library("proxy_info") { ++if (!build_with_mozilla) { + visibility = [ "*" ] + sources = [ + "proxy_info.cc", +@@ -1320,6 +1335,7 @@ rtc_library("proxy_info") { + ":socket_address", + ] + } ++} + + rtc_library("file_rotating_stream") { + sources = [ +@@ -1348,6 +1364,7 @@ rtc_library("data_rate_limiter") { + } + + rtc_library("unique_id_generator") { ++if (!build_with_mozilla) { + sources = [ + "unique_id_generator.cc", + "unique_id_generator.h", +@@ -1362,6 +1379,7 @@ rtc_library("unique_id_generator") { + ] + absl_deps = [ "//third_party/abseil-cpp/absl/strings" ] + } ++} + + rtc_library("crc32") { + sources = [ +@@ -1389,6 +1407,7 @@ rtc_library("stream") { + } + + rtc_library("rtc_certificate_generator") { ++if (!build_with_mozilla) { + visibility = [ "*" ] + sources = [ + "rtc_certificate_generator.cc", +@@ -1406,8 +1425,10 @@ rtc_library("rtc_certificate_generator") { + "//third_party/abseil-cpp/absl/types:optional", + ] + } ++} + + rtc_library("ssl") { ++if (!build_with_mozilla) { + visibility = [ "*" ] + sources = [ + "helpers.cc", +@@ -1506,6 +1527,7 @@ rtc_library("ssl") { + deps += [ ":win32" ] + } + } ++} + + rtc_library("crypt_string") { + sources = [ +@@ -1515,6 +1537,7 @@ rtc_library("crypt_string") { + } + + rtc_library("http_common") { ++if (!build_with_mozilla) { + sources = [ + "http_common.cc", + "http_common.h", +@@ -1531,6 +1554,7 @@ rtc_library("http_common") { + + absl_deps = [ "//third_party/abseil-cpp/absl/strings" ] + } ++} + + rtc_source_set("gtest_prod") { + sources = [ "gtest_prod_util.h" ] +-- +2.34.1 + diff --git a/third_party/libwebrtc/moz-patch-stack/0094.patch b/third_party/libwebrtc/moz-patch-stack/0094.patch new file mode 100644 index 0000000000..ce92900d98 --- /dev/null +++ b/third_party/libwebrtc/moz-patch-stack/0094.patch @@ -0,0 +1,86 @@ +From: Andreas Pehrson <apehrson@mozilla.com> +Date: Wed, 10 May 2023 07:06:00 +0000 +Subject: Bug 1810949 - cherry-pick upstream libwebrtc commit 7e5d9edfdf. + r=webrtc-reviewers,mjf +MIME-Version: 1.0 +Content-Type: text/plain; charset=UTF-8 +Content-Transfer-Encoding: 8bit + +Upstream commit: https://webrtc.googlesource.com/src/+/7e5d9edfdfe82e06182b790afe848cd0da179a87 + webrtc_libyuv: Raise warnings for unhandled types at compile time + + Bug: webrtc:14830 + Change-Id: Ib5141e585f673098bbedd2872dbd6e6ed9df4864 + Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/291528 + Reviewed-by: Erik SprÃ¥ng <sprang@webrtc.org> + Reviewed-by: Stefan Holmer <stefan@webrtc.org> + Commit-Queue: Erik SprÃ¥ng <sprang@webrtc.org> + Cr-Commit-Position: refs/heads/main@{#39408} + +Differential Revision: https://phabricator.services.mozilla.com/D177228 +Mercurial Revision: https://hg.mozilla.org/mozilla-central/rev/2bc179dfe25391e4b621ce407239beaebbe09be8 +--- + common_video/libyuv/webrtc_libyuv.cc | 23 +++++++++-------------- + 1 file changed, 9 insertions(+), 14 deletions(-) + +diff --git a/common_video/libyuv/webrtc_libyuv.cc b/common_video/libyuv/webrtc_libyuv.cc +index 8998af191d..d07d739119 100644 +--- a/common_video/libyuv/webrtc_libyuv.cc ++++ b/common_video/libyuv/webrtc_libyuv.cc +@@ -22,7 +22,6 @@ namespace webrtc { + size_t CalcBufferSize(VideoType type, int width, int height) { + RTC_DCHECK_GE(width, 0); + RTC_DCHECK_GE(height, 0); +- size_t buffer_size = 0; + switch (type) { + case VideoType::kI420: + case VideoType::kNV21: +@@ -31,28 +30,24 @@ size_t CalcBufferSize(VideoType type, int width, int height) { + case VideoType::kNV12: { + int half_width = (width + 1) >> 1; + int half_height = (height + 1) >> 1; +- buffer_size = width * height + half_width * half_height * 2; +- break; ++ return width * height + half_width * half_height * 2; + } + case VideoType::kARGB4444: + case VideoType::kRGB565: + case VideoType::kARGB1555: + case VideoType::kYUY2: + case VideoType::kUYVY: +- buffer_size = width * height * 2; +- break; ++ return width * height * 2; + case VideoType::kRGB24: +- buffer_size = width * height * 3; +- break; ++ return width * height * 3; + case VideoType::kBGRA: + case VideoType::kARGB: +- buffer_size = width * height * 4; +- break; +- default: +- RTC_DCHECK_NOTREACHED(); +- break; ++ return width * height * 4; ++ case VideoType::kMJPEG: ++ case VideoType::kUnknown: + } +- return buffer_size; ++ RTC_DCHECK_NOTREACHED() << "Unexpected pixel format " << type; ++ return 0; + } + + int ExtractBuffer(const rtc::scoped_refptr<I420BufferInterface>& input_frame, +@@ -120,7 +115,7 @@ int ConvertVideoType(VideoType video_type) { + case VideoType::kNV12: + return libyuv::FOURCC_NV12; + } +- RTC_DCHECK_NOTREACHED(); ++ RTC_DCHECK_NOTREACHED() << "Unexpected pixel format " << video_type; + return libyuv::FOURCC_ANY; + } + +-- +2.34.1 + diff --git a/third_party/libwebrtc/moz-patch-stack/0095.patch b/third_party/libwebrtc/moz-patch-stack/0095.patch new file mode 100644 index 0000000000..560b9137ab --- /dev/null +++ b/third_party/libwebrtc/moz-patch-stack/0095.patch @@ -0,0 +1,69 @@ +From: Andreas Pehrson <apehrson@mozilla.com> +Date: Wed, 10 May 2023 07:06:00 +0000 +Subject: Bug 1810949 - cherry-pick upstream libwebrtc commit ba41b40461. + r=webrtc-reviewers,mjf +MIME-Version: 1.0 +Content-Type: text/plain; charset=UTF-8 +Content-Transfer-Encoding: 8bit + +Upstream commit: https://webrtc.googlesource.com/src/+/ba41b40461df191624c61f0c98ae76e69fe1d46b + webrtc_libyuv: Add support for more video types for consistency + + Bug: webrtc:14830 + Change-Id: I0998fb04a03745131f9f5cca878b0cdb46f3b62b + Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/291529 + Reviewed-by: Erik SprÃ¥ng <sprang@webrtc.org> + Commit-Queue: Ilya Nikolaevskiy <ilnik@webrtc.org> + Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org> + Cr-Commit-Position: refs/heads/main@{#39940} + +Differential Revision: https://phabricator.services.mozilla.com/D177229 +Mercurial Revision: https://hg.mozilla.org/mozilla-central/rev/b6eaef560895caa06e11cce40083eb87fd8724a8 +--- + common_video/libyuv/include/webrtc_libyuv.h | 3 ++- + common_video/libyuv/webrtc_libyuv.cc | 4 ++++ + 2 files changed, 6 insertions(+), 1 deletion(-) + +diff --git a/common_video/libyuv/include/webrtc_libyuv.h b/common_video/libyuv/include/webrtc_libyuv.h +index 6d9071bcd5..253a33294d 100644 +--- a/common_video/libyuv/include/webrtc_libyuv.h ++++ b/common_video/libyuv/include/webrtc_libyuv.h +@@ -32,8 +32,9 @@ enum class VideoType { + kI420, + kIYUV, + kRGB24, +- kABGR, ++ kBGR24, + kARGB, ++ kABGR, + kARGB4444, + kRGB565, + kARGB1555, +diff --git a/common_video/libyuv/webrtc_libyuv.cc b/common_video/libyuv/webrtc_libyuv.cc +index d07d739119..8c68162624 100644 +--- a/common_video/libyuv/webrtc_libyuv.cc ++++ b/common_video/libyuv/webrtc_libyuv.cc +@@ -39,9 +39,11 @@ size_t CalcBufferSize(VideoType type, int width, int height) { + case VideoType::kUYVY: + return width * height * 2; + case VideoType::kRGB24: ++ case VideoType::kBGR24: + return width * height * 3; + case VideoType::kBGRA: + case VideoType::kARGB: ++ case VideoType::kABGR: + return width * height * 4; + case VideoType::kMJPEG: + case VideoType::kUnknown: +@@ -92,6 +94,8 @@ int ConvertVideoType(VideoType video_type) { + return libyuv::FOURCC_YV12; + case VideoType::kRGB24: + return libyuv::FOURCC_24BG; ++ case VideoType::kBGR24: ++ return libyuv::FOURCC_RAW; + case VideoType::kABGR: + return libyuv::FOURCC_ABGR; + case VideoType::kRGB565: +-- +2.34.1 + diff --git a/third_party/libwebrtc/moz-patch-stack/0096.patch b/third_party/libwebrtc/moz-patch-stack/0096.patch new file mode 100644 index 0000000000..fe8d05110a --- /dev/null +++ b/third_party/libwebrtc/moz-patch-stack/0096.patch @@ -0,0 +1,56 @@ +From: Michael Froman <mjfroman@mac.com> +Date: Thu, 11 May 2023 12:02:36 -0500 +Subject: Bug 1810949 - cherry-pick upstream libwebrtc commit b1a174041d. + r=webrtc-reviewers,mjf + +Upstream commit: https://webrtc.googlesource.com/src/+/b1a174041ddf3057f5d6d2f87affa0f11f9413df + Relax VideoCaptureImpl::IncomingFrame size check + + When testing manually with gstreamer and v4l2loopback, the incoming + buffer is often larger than the expected size. This change allows + such frames, while still logging the error. + + Bug: webrtc:14830 + Change-Id: I399aa55af6437d75b50830166a667547f6d144d4 + Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/291530 + Commit-Queue: Stefan Holmer <stefan@webrtc.org> + Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org> + Reviewed-by: Stefan Holmer <stefan@webrtc.org> + Cr-Commit-Position: refs/heads/main@{#39972} + +Differential Revision: https://phabricator.services.mozilla.com/D177230 +Mercurial Revision: https://hg.mozilla.org/mozilla-central/rev/19b5723ad94d55e52d89aad38d5219b72ab0473e +--- + modules/video_capture/video_capture_impl.cc | 16 +++++++++++----- + 1 file changed, 11 insertions(+), 5 deletions(-) + +diff --git a/modules/video_capture/video_capture_impl.cc b/modules/video_capture/video_capture_impl.cc +index d227d41c34..9d9a1471e8 100644 +--- a/modules/video_capture/video_capture_impl.cc ++++ b/modules/video_capture/video_capture_impl.cc +@@ -160,11 +160,17 @@ int32_t VideoCaptureImpl::IncomingFrame(uint8_t* videoFrame, + } + + // Not encoded, convert to I420. +- if (frameInfo.videoType != VideoType::kMJPEG && +- CalcBufferSize(frameInfo.videoType, width, abs(height)) != +- videoFrameLength) { +- RTC_LOG(LS_ERROR) << "Wrong incoming frame length."; +- return -1; ++ if (frameInfo.videoType != VideoType::kMJPEG) { ++ // Allow buffers larger than expected. On linux gstreamer allocates buffers ++ // page-aligned and v4l2loopback passes us the buffer size verbatim which ++ // for most cases is larger than expected. ++ // See https://github.com/umlaeute/v4l2loopback/issues/190. ++ if (auto size = CalcBufferSize(frameInfo.videoType, width, abs(height)); ++ videoFrameLength < size) { ++ RTC_LOG(LS_ERROR) << "Wrong incoming frame length. Expected " << size ++ << ", Got " << videoFrameLength << "."; ++ return -1; ++ } + } + + int target_width = width; +-- +2.34.1 + diff --git a/third_party/libwebrtc/moz-patch-stack/0097.patch b/third_party/libwebrtc/moz-patch-stack/0097.patch new file mode 100644 index 0000000000..c113b33563 --- /dev/null +++ b/third_party/libwebrtc/moz-patch-stack/0097.patch @@ -0,0 +1,100 @@ +From: Andreas Pehrson <apehrson@mozilla.com> +Date: Wed, 10 May 2023 07:06:00 +0000 +Subject: Bug 1810949 - cherry-pick upstream libwebrtc commit 32b64e895c. + r=webrtc-reviewers,mjf + +Upstream commit: https://webrtc.googlesource.com/src/+/32b64e895c0231fe6891a8f6335d66f1dad4f1f5 + Improve ergonomics of dealing with pixel formats in v4l2 camera backend + + Bug: webrtc:14830 + Change-Id: Ib49bf65895fe008e75223abb03867d412c1b5a60 + Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/291531 + Commit-Queue: Ilya Nikolaevskiy <ilnik@webrtc.org> + Reviewed-by: Stefan Holmer <stefan@webrtc.org> + Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org> + Cr-Commit-Position: refs/heads/main@{#39976} + +Differential Revision: https://phabricator.services.mozilla.com/D177231 +Mercurial Revision: https://hg.mozilla.org/mozilla-central/rev/f0afe1a1097f3e02d59fd06d5e83d00a0f4d69ae +--- + .../video_capture/linux/device_info_v4l2.cc | 4 ++- + .../video_capture/linux/video_capture_v4l2.cc | 31 +++++++++---------- + 2 files changed, 17 insertions(+), 18 deletions(-) + +diff --git a/modules/video_capture/linux/device_info_v4l2.cc b/modules/video_capture/linux/device_info_v4l2.cc +index 7651dd6651..f854c2ccc7 100644 +--- a/modules/video_capture/linux/device_info_v4l2.cc ++++ b/modules/video_capture/linux/device_info_v4l2.cc +@@ -391,10 +391,10 @@ int32_t DeviceInfoV4l2::FillCapabilities(int fd) { + video_fmt.type = V4L2_BUF_TYPE_VIDEO_CAPTURE; + video_fmt.fmt.pix.sizeimage = 0; + +- int totalFmts = 5; + unsigned int videoFormats[] = {V4L2_PIX_FMT_MJPEG, V4L2_PIX_FMT_YUV420, + V4L2_PIX_FMT_YUYV, V4L2_PIX_FMT_UYVY, + V4L2_PIX_FMT_NV12}; ++ constexpr int totalFmts = sizeof(videoFormats) / sizeof(unsigned int); + + int sizes = 13; + unsigned int size[][2] = {{128, 96}, {160, 120}, {176, 144}, {320, 240}, +@@ -424,6 +424,8 @@ int32_t DeviceInfoV4l2::FillCapabilities(int fd) { + cap.videoType = VideoType::kUYVY; + } else if (videoFormats[fmts] == V4L2_PIX_FMT_NV12) { + cap.videoType = VideoType::kNV12; ++ } else { ++ RTC_DCHECK_NOTREACHED(); + } + + // get fps of current camera mode +diff --git a/modules/video_capture/linux/video_capture_v4l2.cc b/modules/video_capture/linux/video_capture_v4l2.cc +index c7dcb722bc..baf1916331 100644 +--- a/modules/video_capture/linux/video_capture_v4l2.cc ++++ b/modules/video_capture/linux/video_capture_v4l2.cc +@@ -130,23 +130,18 @@ int32_t VideoCaptureModuleV4L2::StartCapture( + // Supported video formats in preferred order. + // If the requested resolution is larger than VGA, we prefer MJPEG. Go for + // I420 otherwise. +- const int nFormats = 6; +- unsigned int fmts[nFormats]; +- if (capability.width > 640 || capability.height > 480) { +- fmts[0] = V4L2_PIX_FMT_MJPEG; +- fmts[1] = V4L2_PIX_FMT_YUV420; +- fmts[2] = V4L2_PIX_FMT_YUYV; +- fmts[3] = V4L2_PIX_FMT_UYVY; +- fmts[4] = V4L2_PIX_FMT_NV12; +- fmts[5] = V4L2_PIX_FMT_JPEG; +- } else { +- fmts[0] = V4L2_PIX_FMT_YUV420; +- fmts[1] = V4L2_PIX_FMT_YUYV; +- fmts[2] = V4L2_PIX_FMT_UYVY; +- fmts[3] = V4L2_PIX_FMT_NV12; +- fmts[4] = V4L2_PIX_FMT_MJPEG; +- fmts[5] = V4L2_PIX_FMT_JPEG; +- } ++ unsigned int hdFmts[] = { ++ V4L2_PIX_FMT_MJPEG, V4L2_PIX_FMT_YUV420, V4L2_PIX_FMT_YUYV, ++ V4L2_PIX_FMT_UYVY, V4L2_PIX_FMT_NV12, V4L2_PIX_FMT_JPEG, ++ }; ++ unsigned int sdFmts[] = { ++ V4L2_PIX_FMT_YUV420, V4L2_PIX_FMT_YUYV, V4L2_PIX_FMT_UYVY, ++ V4L2_PIX_FMT_NV12, V4L2_PIX_FMT_MJPEG, V4L2_PIX_FMT_JPEG, ++ }; ++ const bool isHd = capability.width > 640 || capability.height > 480; ++ unsigned int* fmts = isHd ? hdFmts : sdFmts; ++ static_assert(sizeof(hdFmts) == sizeof(sdFmts)); ++ constexpr int nFormats = sizeof(hdFmts) / sizeof(unsigned int); + + // Enumerate image formats. + struct v4l2_fmtdesc fmt; +@@ -195,6 +190,8 @@ int32_t VideoCaptureModuleV4L2::StartCapture( + else if (video_fmt.fmt.pix.pixelformat == V4L2_PIX_FMT_MJPEG || + video_fmt.fmt.pix.pixelformat == V4L2_PIX_FMT_JPEG) + _captureVideoType = VideoType::kMJPEG; ++ else ++ RTC_DCHECK_NOTREACHED(); + + // set format and frame size now + if (ioctl(_deviceFd, VIDIOC_S_FMT, &video_fmt) < 0) { +-- +2.34.1 + diff --git a/third_party/libwebrtc/moz-patch-stack/0098.patch b/third_party/libwebrtc/moz-patch-stack/0098.patch new file mode 100644 index 0000000000..9eca10032f --- /dev/null +++ b/third_party/libwebrtc/moz-patch-stack/0098.patch @@ -0,0 +1,238 @@ +From: Andreas Pehrson <apehrson@mozilla.com> +Date: Wed, 10 May 2023 07:06:00 +0000 +Subject: Bug 1810949 - cherry-pick upstream libwebrtc commit 91d5fc2ed6. + r=webrtc-reviewers,mjf + +Upstream commit: https://webrtc.googlesource.com/src/+/91d5fc2ed6ef347d90182868320267d45cf9525b + Support more pixel formats in v4l2 camera backend + + These were tested with gstreamer and v4l2loopback, example setup: + $ sudo v4l2loopback-ctl add -n BGRA 10 + $ gst-launch-1.0 videotestsrc pattern=smpte-rp-219 ! \ + video/x-raw,format=BGRA ! v4l2sink device=/dev/video10 > /dev/null & + + Then conversion was confirmed with video_loopback: + $ ./video_loopback --capture_device_index=3 --logs 2>&1 | grep -i \ + capture + + Bug: webrtc:14830 + Change-Id: I35c8e453cf7f9a2923935b0ad82477a3144e8c12 + Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/291532 + Commit-Queue: Stefan Holmer <stefan@webrtc.org> + Reviewed-by: Mirko Bonadei <mbonadei@webrtc.org> + Reviewed-by: Stefan Holmer <stefan@webrtc.org> + Cr-Commit-Position: refs/heads/main@{#39979} + +Differential Revision: https://phabricator.services.mozilla.com/D177232 +Mercurial Revision: https://hg.mozilla.org/mozilla-central/rev/92dc582fdcf3a2fdb3fcdbcd96080d081de8f8d5 +--- + .../video_capture/linux/device_info_v4l2.cc | 71 +++++++++++++++++-- + .../video_capture/linux/video_capture_v4l2.cc | 63 ++++++++++++++-- + 2 files changed, 126 insertions(+), 8 deletions(-) + +diff --git a/modules/video_capture/linux/device_info_v4l2.cc b/modules/video_capture/linux/device_info_v4l2.cc +index f854c2ccc7..ccd4b2bd2a 100644 +--- a/modules/video_capture/linux/device_info_v4l2.cc ++++ b/modules/video_capture/linux/device_info_v4l2.cc +@@ -39,6 +39,24 @@ + #define BUF_LEN ( 1024 * ( EVENT_SIZE + 16 ) ) + #endif + ++// These defines are here to support building on kernel 3.16 which some ++// downstream projects, e.g. Firefox, use. ++// TODO(apehrson): Remove them and their undefs when no longer needed. ++#ifndef V4L2_PIX_FMT_ABGR32 ++#define ABGR32_OVERRIDE 1 ++#define V4L2_PIX_FMT_ABGR32 v4l2_fourcc('A', 'R', '2', '4') ++#endif ++ ++#ifndef V4L2_PIX_FMT_ARGB32 ++#define ARGB32_OVERRIDE 1 ++#define V4L2_PIX_FMT_ARGB32 v4l2_fourcc('B', 'A', '2', '4') ++#endif ++ ++#ifndef V4L2_PIX_FMT_RGBA32 ++#define RGBA32_OVERRIDE 1 ++#define V4L2_PIX_FMT_RGBA32 v4l2_fourcc('A', 'B', '2', '4') ++#endif ++ + namespace webrtc { + namespace videocapturemodule { + #ifdef WEBRTC_LINUX +@@ -391,9 +409,13 @@ int32_t DeviceInfoV4l2::FillCapabilities(int fd) { + video_fmt.type = V4L2_BUF_TYPE_VIDEO_CAPTURE; + video_fmt.fmt.pix.sizeimage = 0; + +- unsigned int videoFormats[] = {V4L2_PIX_FMT_MJPEG, V4L2_PIX_FMT_YUV420, +- V4L2_PIX_FMT_YUYV, V4L2_PIX_FMT_UYVY, +- V4L2_PIX_FMT_NV12}; ++ unsigned int videoFormats[] = { ++ V4L2_PIX_FMT_MJPEG, V4L2_PIX_FMT_JPEG, V4L2_PIX_FMT_YUV420, ++ V4L2_PIX_FMT_YVU420, V4L2_PIX_FMT_YUYV, V4L2_PIX_FMT_UYVY, ++ V4L2_PIX_FMT_NV12, V4L2_PIX_FMT_BGR24, V4L2_PIX_FMT_RGB24, ++ V4L2_PIX_FMT_RGB565, V4L2_PIX_FMT_ABGR32, V4L2_PIX_FMT_ARGB32, ++ V4L2_PIX_FMT_RGBA32, V4L2_PIX_FMT_BGR32, V4L2_PIX_FMT_RGB32, ++ }; + constexpr int totalFmts = sizeof(videoFormats) / sizeof(unsigned int); + + int sizes = 13; +@@ -418,12 +440,38 @@ int32_t DeviceInfoV4l2::FillCapabilities(int fd) { + cap.videoType = VideoType::kYUY2; + } else if (videoFormats[fmts] == V4L2_PIX_FMT_YUV420) { + cap.videoType = VideoType::kI420; +- } else if (videoFormats[fmts] == V4L2_PIX_FMT_MJPEG) { ++ } else if (videoFormats[fmts] == V4L2_PIX_FMT_YVU420) { ++ cap.videoType = VideoType::kYV12; ++ } else if (videoFormats[fmts] == V4L2_PIX_FMT_MJPEG || ++ videoFormats[fmts] == V4L2_PIX_FMT_JPEG) { + cap.videoType = VideoType::kMJPEG; + } else if (videoFormats[fmts] == V4L2_PIX_FMT_UYVY) { + cap.videoType = VideoType::kUYVY; + } else if (videoFormats[fmts] == V4L2_PIX_FMT_NV12) { + cap.videoType = VideoType::kNV12; ++ } else if (videoFormats[fmts] == V4L2_PIX_FMT_BGR24) { ++ // NB that for RGB formats, `VideoType` follows naming conventions ++ // of libyuv[1], where e.g. the format for FOURCC "ARGB" stores ++ // pixels in BGRA order in memory. V4L2[2] on the other hand names ++ // its formats based on the order of the RGB components as stored in ++ // memory. Applies to all RGB formats below. ++ // [1]https://chromium.googlesource.com/libyuv/libyuv/+/refs/heads/main/docs/formats.md#the-argb-fourcc ++ // [2]https://www.kernel.org/doc/html/v6.2/userspace-api/media/v4l/pixfmt-rgb.html#bits-per-component ++ cap.videoType = VideoType::kRGB24; ++ } else if (videoFormats[fmts] == V4L2_PIX_FMT_RGB24) { ++ cap.videoType = VideoType::kBGR24; ++ } else if (videoFormats[fmts] == V4L2_PIX_FMT_RGB565) { ++ cap.videoType = VideoType::kRGB565; ++ } else if (videoFormats[fmts] == V4L2_PIX_FMT_ABGR32) { ++ cap.videoType = VideoType::kARGB; ++ } else if (videoFormats[fmts] == V4L2_PIX_FMT_ARGB32) { ++ cap.videoType = VideoType::kBGRA; ++ } else if (videoFormats[fmts] == V4L2_PIX_FMT_BGR32) { ++ cap.videoType = VideoType::kARGB; ++ } else if (videoFormats[fmts] == V4L2_PIX_FMT_RGB32) { ++ cap.videoType = VideoType::kBGRA; ++ } else if (videoFormats[fmts] == V4L2_PIX_FMT_RGBA32) { ++ cap.videoType = VideoType::kABGR; + } else { + RTC_DCHECK_NOTREACHED(); + } +@@ -452,3 +500,18 @@ int32_t DeviceInfoV4l2::FillCapabilities(int fd) { + + } // namespace videocapturemodule + } // namespace webrtc ++ ++#ifdef ABGR32_OVERRIDE ++#undef ABGR32_OVERRIDE ++#undef V4L2_PIX_FMT_ABGR32 ++#endif ++ ++#ifdef ARGB32_OVERRIDE ++#undef ARGB32_OVERRIDE ++#undef V4L2_PIX_FMT_ARGB32 ++#endif ++ ++#ifdef RGBA32_OVERRIDE ++#undef RGBA32_OVERRIDE ++#undef V4L2_PIX_FMT_RGBA32 ++#endif +diff --git a/modules/video_capture/linux/video_capture_v4l2.cc b/modules/video_capture/linux/video_capture_v4l2.cc +index baf1916331..2935cd027d 100644 +--- a/modules/video_capture/linux/video_capture_v4l2.cc ++++ b/modules/video_capture/linux/video_capture_v4l2.cc +@@ -37,6 +37,24 @@ + #include "modules/video_capture/video_capture.h" + #include "rtc_base/logging.h" + ++// These defines are here to support building on kernel 3.16 which some ++// downstream projects, e.g. Firefox, use. ++// TODO(apehrson): Remove them and their undefs when no longer needed. ++#ifndef V4L2_PIX_FMT_ABGR32 ++#define ABGR32_OVERRIDE 1 ++#define V4L2_PIX_FMT_ABGR32 v4l2_fourcc('A', 'R', '2', '4') ++#endif ++ ++#ifndef V4L2_PIX_FMT_ARGB32 ++#define ARGB32_OVERRIDE 1 ++#define V4L2_PIX_FMT_ARGB32 v4l2_fourcc('B', 'A', '2', '4') ++#endif ++ ++#ifndef V4L2_PIX_FMT_RGBA32 ++#define RGBA32_OVERRIDE 1 ++#define V4L2_PIX_FMT_RGBA32 v4l2_fourcc('A', 'B', '2', '4') ++#endif ++ + namespace webrtc { + namespace videocapturemodule { + VideoCaptureModuleV4L2::VideoCaptureModuleV4L2() +@@ -131,12 +149,18 @@ int32_t VideoCaptureModuleV4L2::StartCapture( + // If the requested resolution is larger than VGA, we prefer MJPEG. Go for + // I420 otherwise. + unsigned int hdFmts[] = { +- V4L2_PIX_FMT_MJPEG, V4L2_PIX_FMT_YUV420, V4L2_PIX_FMT_YUYV, +- V4L2_PIX_FMT_UYVY, V4L2_PIX_FMT_NV12, V4L2_PIX_FMT_JPEG, ++ V4L2_PIX_FMT_MJPEG, V4L2_PIX_FMT_YUV420, V4L2_PIX_FMT_YVU420, ++ V4L2_PIX_FMT_YUYV, V4L2_PIX_FMT_UYVY, V4L2_PIX_FMT_NV12, ++ V4L2_PIX_FMT_ABGR32, V4L2_PIX_FMT_ARGB32, V4L2_PIX_FMT_RGBA32, ++ V4L2_PIX_FMT_BGR32, V4L2_PIX_FMT_RGB32, V4L2_PIX_FMT_BGR24, ++ V4L2_PIX_FMT_RGB24, V4L2_PIX_FMT_RGB565, V4L2_PIX_FMT_JPEG, + }; + unsigned int sdFmts[] = { +- V4L2_PIX_FMT_YUV420, V4L2_PIX_FMT_YUYV, V4L2_PIX_FMT_UYVY, +- V4L2_PIX_FMT_NV12, V4L2_PIX_FMT_MJPEG, V4L2_PIX_FMT_JPEG, ++ V4L2_PIX_FMT_YUV420, V4L2_PIX_FMT_YVU420, V4L2_PIX_FMT_YUYV, ++ V4L2_PIX_FMT_UYVY, V4L2_PIX_FMT_NV12, V4L2_PIX_FMT_ABGR32, ++ V4L2_PIX_FMT_ARGB32, V4L2_PIX_FMT_RGBA32, V4L2_PIX_FMT_BGR32, ++ V4L2_PIX_FMT_RGB32, V4L2_PIX_FMT_BGR24, V4L2_PIX_FMT_RGB24, ++ V4L2_PIX_FMT_RGB565, V4L2_PIX_FMT_MJPEG, V4L2_PIX_FMT_JPEG, + }; + const bool isHd = capability.width > 640 || capability.height > 480; + unsigned int* fmts = isHd ? hdFmts : sdFmts; +@@ -183,10 +207,26 @@ int32_t VideoCaptureModuleV4L2::StartCapture( + _captureVideoType = VideoType::kYUY2; + else if (video_fmt.fmt.pix.pixelformat == V4L2_PIX_FMT_YUV420) + _captureVideoType = VideoType::kI420; ++ else if (video_fmt.fmt.pix.pixelformat == V4L2_PIX_FMT_YVU420) ++ _captureVideoType = VideoType::kYV12; + else if (video_fmt.fmt.pix.pixelformat == V4L2_PIX_FMT_UYVY) + _captureVideoType = VideoType::kUYVY; + else if (video_fmt.fmt.pix.pixelformat == V4L2_PIX_FMT_NV12) + _captureVideoType = VideoType::kNV12; ++ else if (video_fmt.fmt.pix.pixelformat == V4L2_PIX_FMT_BGR24) ++ _captureVideoType = VideoType::kRGB24; ++ else if (video_fmt.fmt.pix.pixelformat == V4L2_PIX_FMT_RGB24) ++ _captureVideoType = VideoType::kBGR24; ++ else if (video_fmt.fmt.pix.pixelformat == V4L2_PIX_FMT_RGB565) ++ _captureVideoType = VideoType::kRGB565; ++ else if (video_fmt.fmt.pix.pixelformat == V4L2_PIX_FMT_ABGR32 || ++ video_fmt.fmt.pix.pixelformat == V4L2_PIX_FMT_BGR32) ++ _captureVideoType = VideoType::kARGB; ++ else if (video_fmt.fmt.pix.pixelformat == V4L2_PIX_FMT_ARGB32 || ++ video_fmt.fmt.pix.pixelformat == V4L2_PIX_FMT_RGB32) ++ _captureVideoType = VideoType::kBGRA; ++ else if (video_fmt.fmt.pix.pixelformat == V4L2_PIX_FMT_RGBA32) ++ _captureVideoType = VideoType::kABGR; + else if (video_fmt.fmt.pix.pixelformat == V4L2_PIX_FMT_MJPEG || + video_fmt.fmt.pix.pixelformat == V4L2_PIX_FMT_JPEG) + _captureVideoType = VideoType::kMJPEG; +@@ -432,3 +472,18 @@ int32_t VideoCaptureModuleV4L2::CaptureSettings( + } + } // namespace videocapturemodule + } // namespace webrtc ++ ++#ifdef ABGR32_OVERRIDE ++#undef ABGR32_OVERRIDE ++#undef V4L2_PIX_FMT_ABGR32 ++#endif ++ ++#ifdef ARGB32_OVERRIDE ++#undef ARGB32_OVERRIDE ++#undef V4L2_PIX_FMT_ARGB32 ++#endif ++ ++#ifdef RGBA32_OVERRIDE ++#undef RGBA32_OVERRIDE ++#undef V4L2_PIX_FMT_RGBA32 ++#endif +-- +2.34.1 + diff --git a/third_party/libwebrtc/moz-patch-stack/0099.patch b/third_party/libwebrtc/moz-patch-stack/0099.patch new file mode 100644 index 0000000000..70f2f6a79b --- /dev/null +++ b/third_party/libwebrtc/moz-patch-stack/0099.patch @@ -0,0 +1,28 @@ +From: Andreas Pehrson <apehrson@mozilla.com> +Date: Wed, 10 May 2023 07:47:00 +0000 +Subject: Bug 1810949 - In webrtc_libyuv.cc put a label at end of compound + statement. CLOSED TREE + +Differential Revision: https://phabricator.services.mozilla.com/D177600 + +Depends on D177599 +Mercurial Revision: https://hg.mozilla.org/mozilla-central/rev/6ac8147fee0323be3bf1d7f292a2a408a9901159 +--- + common_video/libyuv/webrtc_libyuv.cc | 1 + + 1 file changed, 1 insertion(+) + +diff --git a/common_video/libyuv/webrtc_libyuv.cc b/common_video/libyuv/webrtc_libyuv.cc +index 8c68162624..05a4b184c2 100644 +--- a/common_video/libyuv/webrtc_libyuv.cc ++++ b/common_video/libyuv/webrtc_libyuv.cc +@@ -47,6 +47,7 @@ size_t CalcBufferSize(VideoType type, int width, int height) { + return width * height * 4; + case VideoType::kMJPEG: + case VideoType::kUnknown: ++ break; + } + RTC_DCHECK_NOTREACHED() << "Unexpected pixel format " << type; + return 0; +-- +2.34.1 + diff --git a/third_party/libwebrtc/moz-patch-stack/0100.patch b/third_party/libwebrtc/moz-patch-stack/0100.patch new file mode 100644 index 0000000000..e7fc0e79a8 --- /dev/null +++ b/third_party/libwebrtc/moz-patch-stack/0100.patch @@ -0,0 +1,115 @@ +From: Andreas Pehrson <apehrson@mozilla.com> +Date: Wed, 26 Apr 2023 19:45:00 +0000 +Subject: Bug 1694304 - cherry-pick libwebrtc 28ac56a415. + r=webrtc-reviewers,jib + +Upstream commit: https://webrtc.googlesource.com/src/+/28ac56a415a7513f1ebfb985659bf2012d84df3f + In VideoCaptureDS::Stop() fully stop the device + + This makes the device light turn off when stopped. + + Bug: webrtc:15109 + Change-Id: I1deecbc2463e2e316e01ff1f061ab6b0313c1aa1 + Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/302200 + Commit-Queue: Ilya Nikolaevskiy <ilnik@webrtc.org> + Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org> + Reviewed-by: Per Kjellander <perkj@webrtc.org> + Cr-Commit-Position: refs/heads/main@{#39953} + +Differential Revision: https://phabricator.services.mozilla.com/D176507 +Mercurial Revision: https://hg.mozilla.org/mozilla-central/rev/dabb9e2ec9d3b77e4cc19de0fb98cae4ce88293d +--- + .../test/video_capture_unittest.cc | 33 +++++++++++++++++++ + .../video_capture/windows/video_capture_ds.cc | 18 +++++----- + 2 files changed, 41 insertions(+), 10 deletions(-) + +diff --git a/modules/video_capture/test/video_capture_unittest.cc b/modules/video_capture/test/video_capture_unittest.cc +index 4cf3d5931c..c8af222b57 100644 +--- a/modules/video_capture/test/video_capture_unittest.cc ++++ b/modules/video_capture/test/video_capture_unittest.cc +@@ -341,3 +341,36 @@ TEST_F(VideoCaptureTest, DISABLED_TestTwoCameras) { + EXPECT_EQ(0, module2->StopCapture()); + EXPECT_EQ(0, module1->StopCapture()); + } ++ ++#ifdef WEBRTC_MAC ++// No VideoCaptureImpl on Mac. ++#define MAYBE_ConcurrentAccess DISABLED_ConcurrentAccess ++#else ++#define MAYBE_ConcurrentAccess ConcurrentAccess ++#endif ++TEST_F(VideoCaptureTest, MAYBE_ConcurrentAccess) { ++ TestVideoCaptureCallback capture_observer1; ++ rtc::scoped_refptr<VideoCaptureModule> module1( ++ OpenVideoCaptureDevice(0, &capture_observer1)); ++ ASSERT_TRUE(module1.get() != NULL); ++ VideoCaptureCapability capability; ++ device_info_->GetCapability(module1->CurrentDeviceName(), 0, capability); ++ capture_observer1.SetExpectedCapability(capability); ++ ++ TestVideoCaptureCallback capture_observer2; ++ rtc::scoped_refptr<VideoCaptureModule> module2( ++ OpenVideoCaptureDevice(0, &capture_observer2)); ++ ASSERT_TRUE(module2.get() != NULL); ++ capture_observer2.SetExpectedCapability(capability); ++ ++ // Starting module1 should work. ++ ASSERT_NO_FATAL_FAILURE(StartCapture(module1.get(), capability)); ++ EXPECT_TRUE_WAIT(capture_observer1.incoming_frames() >= 5, kTimeOut); ++ ++ // When module1 is stopped, starting module2 for the same device should work. ++ EXPECT_EQ(0, module1->StopCapture()); ++ ASSERT_NO_FATAL_FAILURE(StartCapture(module2.get(), capability)); ++ EXPECT_TRUE_WAIT(capture_observer2.incoming_frames() >= 5, kTimeOut); ++ ++ EXPECT_EQ(0, module2->StopCapture()); ++} +diff --git a/modules/video_capture/windows/video_capture_ds.cc b/modules/video_capture/windows/video_capture_ds.cc +index 74b31d98be..8695f76245 100644 +--- a/modules/video_capture/windows/video_capture_ds.cc ++++ b/modules/video_capture/windows/video_capture_ds.cc +@@ -113,17 +113,9 @@ int32_t VideoCaptureDS::Init(const char* deviceUniqueIdUTF8) { + return -1; + } + +- // Temporary connect here. +- // This is done so that no one else can use the capture device. + if (SetCameraOutput(_requestedCapability) != 0) { + return -1; + } +- hr = _mediaControl->Pause(); +- if (FAILED(hr)) { +- RTC_LOG(LS_INFO) +- << "Failed to Pause the Capture device. Is it already occupied? " << hr; +- return -1; +- } + RTC_LOG(LS_INFO) << "Capture device '" << deviceUniqueIdUTF8 + << "' initialized."; + return 0; +@@ -139,7 +131,13 @@ int32_t VideoCaptureDS::StartCapture(const VideoCaptureCapability& capability) { + return -1; + } + } +- HRESULT hr = _mediaControl->Run(); ++ HRESULT hr = _mediaControl->Pause(); ++ if (FAILED(hr)) { ++ RTC_LOG(LS_INFO) ++ << "Failed to Pause the Capture device. Is it already occupied? " << hr; ++ return -1; ++ } ++ hr = _mediaControl->Run(); + if (FAILED(hr)) { + RTC_LOG(LS_INFO) << "Failed to start the Capture device."; + return -1; +@@ -150,7 +148,7 @@ int32_t VideoCaptureDS::StartCapture(const VideoCaptureCapability& capability) { + int32_t VideoCaptureDS::StopCapture() { + MutexLock lock(&api_lock_); + +- HRESULT hr = _mediaControl->Pause(); ++ HRESULT hr = _mediaControl->StopWhenReady(); + if (FAILED(hr)) { + RTC_LOG(LS_INFO) << "Failed to stop the capture graph. " << hr; + return -1; +-- +2.34.1 + diff --git a/third_party/libwebrtc/moz-patch-stack/0101.patch b/third_party/libwebrtc/moz-patch-stack/0101.patch new file mode 100644 index 0000000000..95fe9665c6 --- /dev/null +++ b/third_party/libwebrtc/moz-patch-stack/0101.patch @@ -0,0 +1,71 @@ +From: Andreas Pehrson <apehrson@mozilla.com> +Date: Wed, 10 May 2023 08:01:00 +0000 +Subject: Bug 1828065 - cherry-pick upstream libwebrtc commit 6fc1ae58be. + r=webrtc-reviewers,mjf + +Upstream commit: https://webrtc.googlesource.com/src/+/6fc1ae58be7cbb61ad0431d90eb7a7dc333de2f2 + In DeviceInfoDS check that out vars were set + + Bug: chromium:1441804 + Change-Id: Id07cb61519315d77c2d7cdab1053efaaf7473e1a + Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/304060 + Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org> + Reviewed-by: Per Kjellander <perkj@webrtc.org> + Commit-Queue: Ilya Nikolaevskiy <ilnik@webrtc.org> + Cr-Commit-Position: refs/heads/main@{#39982} + +Differential Revision: https://phabricator.services.mozilla.com/D177236 +Mercurial Revision: https://hg.mozilla.org/mozilla-central/rev/60d381b6016ae5af0181b44c6a31b41eab094793 +--- + modules/video_capture/windows/device_info_ds.cc | 13 +++++++------ + modules/video_capture/windows/help_functions_ds.cc | 3 +++ + 2 files changed, 10 insertions(+), 6 deletions(-) + +diff --git a/modules/video_capture/windows/device_info_ds.cc b/modules/video_capture/windows/device_info_ds.cc +index 6b4c57d01e..2c192fe4f6 100644 +--- a/modules/video_capture/windows/device_info_ds.cc ++++ b/modules/video_capture/windows/device_info_ds.cc +@@ -505,9 +505,9 @@ int32_t DeviceInfoDS::CreateCapabilityMap(const char* deviceUniqueIdUTF8) + } + + if (hrVC == S_OK) { +- LONGLONG* frameDurationList; +- LONGLONG maxFPS; +- long listSize; ++ LONGLONG* frameDurationList = NULL; ++ LONGLONG maxFPS = 0; ++ long listSize = 0; + SIZE size; + size.cx = capability.width; + size.cy = capability.height; +@@ -520,9 +520,10 @@ int32_t DeviceInfoDS::CreateCapabilityMap(const char* deviceUniqueIdUTF8) + hrVC = videoControlConfig->GetFrameRateList( + outputCapturePin, tmp, size, &listSize, &frameDurationList); + +- // On some odd cameras, you may get a 0 for duration. +- // GetMaxOfFrameArray returns the lowest duration (highest FPS) +- if (hrVC == S_OK && listSize > 0 && ++ // On some odd cameras, you may get a 0 for duration. Some others may ++ // not update the out vars. GetMaxOfFrameArray returns the lowest ++ // duration (highest FPS), or 0 if there was no list with elements. ++ if (hrVC == S_OK && + 0 != (maxFPS = GetMaxOfFrameArray(frameDurationList, listSize))) { + capability.maxFPS = static_cast<int>(10000000 / maxFPS); + capability.supportFrameRateControl = true; +diff --git a/modules/video_capture/windows/help_functions_ds.cc b/modules/video_capture/windows/help_functions_ds.cc +index b767726107..47fecfe4a1 100644 +--- a/modules/video_capture/windows/help_functions_ds.cc ++++ b/modules/video_capture/windows/help_functions_ds.cc +@@ -21,6 +21,9 @@ namespace webrtc { + namespace videocapturemodule { + // This returns minimum :), which will give max frame rate... + LONGLONG GetMaxOfFrameArray(LONGLONG* maxFps, long size) { ++ if (!maxFps || size <= 0) { ++ return 0; ++ } + LONGLONG maxFPS = maxFps[0]; + for (int i = 0; i < size; i++) { + if (maxFPS > maxFps[i]) +-- +2.34.1 + diff --git a/third_party/libwebrtc/moz-patch-stack/0102.patch b/third_party/libwebrtc/moz-patch-stack/0102.patch new file mode 100644 index 0000000000..65b445cb91 --- /dev/null +++ b/third_party/libwebrtc/moz-patch-stack/0102.patch @@ -0,0 +1,52 @@ +From: Andreas Pehrson <apehrson@mozilla.com> +Date: Wed, 10 May 2023 08:01:00 +0000 +Subject: Bug 1828065 - cherry-pick upstream libwebrtc commit adf55790b6. + r=webrtc-reviewers,mjf + +Upstream commit: https://webrtc.googlesource.com/src/+/adf55790b6ecf50c4bb2b2cf7d58441303b9d946 + In DeviceInfoDS free the frame duration list after use + + Per the docs, the caller is responsible for freeing the memory. + + Bug: chromium:1441804 + Change-Id: I9aaae493a1a86d8ab4f03930715a643a3c9fb61b + Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/304061 + Reviewed-by: Per Kjellander <perkj@webrtc.org> + Commit-Queue: Ilya Nikolaevskiy <ilnik@webrtc.org> + Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org> + Cr-Commit-Position: refs/heads/main@{#39983} + +Differential Revision: https://phabricator.services.mozilla.com/D177237 +Mercurial Revision: https://hg.mozilla.org/mozilla-central/rev/854cc79e99130e6537eebe8433a6a7adf5e1578d +--- + modules/video_capture/windows/device_info_ds.cc | 11 +++++++++-- + 1 file changed, 9 insertions(+), 2 deletions(-) + +diff --git a/modules/video_capture/windows/device_info_ds.cc b/modules/video_capture/windows/device_info_ds.cc +index 2c192fe4f6..2b01fc6930 100644 +--- a/modules/video_capture/windows/device_info_ds.cc ++++ b/modules/video_capture/windows/device_info_ds.cc +@@ -520,11 +520,18 @@ int32_t DeviceInfoDS::CreateCapabilityMap(const char* deviceUniqueIdUTF8) + hrVC = videoControlConfig->GetFrameRateList( + outputCapturePin, tmp, size, &listSize, &frameDurationList); + ++ if (hrVC == S_OK) { ++ maxFPS = GetMaxOfFrameArray(frameDurationList, listSize); ++ } ++ ++ CoTaskMemFree(frameDurationList); ++ frameDurationList = NULL; ++ listSize = 0; ++ + // On some odd cameras, you may get a 0 for duration. Some others may + // not update the out vars. GetMaxOfFrameArray returns the lowest + // duration (highest FPS), or 0 if there was no list with elements. +- if (hrVC == S_OK && +- 0 != (maxFPS = GetMaxOfFrameArray(frameDurationList, listSize))) { ++ if (0 != maxFPS) { + capability.maxFPS = static_cast<int>(10000000 / maxFPS); + capability.supportFrameRateControl = true; + } else // use existing method +-- +2.34.1 + diff --git a/third_party/libwebrtc/moz-patch-stack/0103.patch b/third_party/libwebrtc/moz-patch-stack/0103.patch new file mode 100644 index 0000000000..09b70568a9 --- /dev/null +++ b/third_party/libwebrtc/moz-patch-stack/0103.patch @@ -0,0 +1,54 @@ +From: Andreas Pehrson <apehrson@mozilla.com> +Date: Wed, 10 May 2023 18:06:00 +0000 +Subject: Bug 1831824 - libwebrtc: In PacketSequencer set timestamps of rtx + padding packets when there is no last packet. r=webrtc-reviewers,dbaker + +Prior to this patch timestamps are not adjusted when they are 0, and they are 0 +when the packet sequencer has not yet seen a media packet. Code and comments in +PacketSequencer and RTPSender::GeneratePadding make it clear that rtx padding +packets are allowed prior to seeing a media packet, and therefore the 0 +timestamp case has to be handled. + +For rtx the padding packets do not need to have the same timestamp as any media +packets, like plain padding packets do -- because they can only be part of a media +frame, so a media packet has to be known. + +With this patch both rtp timestamps and capture timestamps are set to current +time when sequencing rtx padding packets without having seen a media packet. + +This fixes a DCHECK failure in TransmissionOffset::Write. + +Differential Revision: https://phabricator.services.mozilla.com/D177306 +Mercurial Revision: https://hg.mozilla.org/mozilla-central/rev/b0339fd77c82c4c54e5aacaeef66d739b1643827 +--- + modules/rtp_rtcp/source/packet_sequencer.cc | 14 ++++++++++++-- + 1 file changed, 12 insertions(+), 2 deletions(-) + +diff --git a/modules/rtp_rtcp/source/packet_sequencer.cc b/modules/rtp_rtcp/source/packet_sequencer.cc +index 55edd768a8..acc2e87aa3 100644 +--- a/modules/rtp_rtcp/source/packet_sequencer.cc ++++ b/modules/rtp_rtcp/source/packet_sequencer.cc +@@ -118,8 +118,18 @@ void PacketSequencer::PopulatePaddingFields(RtpPacketToSend& packet) { + return; + } + +- packet.SetTimestamp(last_rtp_timestamp_); +- packet.set_capture_time(Timestamp::Millis(last_capture_time_ms_)); ++ if (last_timestamp_time_ms_ > 0) { ++ RTC_DCHECK_GT(last_rtp_timestamp_, 0); ++ RTC_DCHECK_GT(last_capture_time_ms_, 0); ++ packet.SetTimestamp(last_rtp_timestamp_); ++ packet.set_capture_time(Timestamp::Millis(last_capture_time_ms_)); ++ } else { ++ // No media packet has been sent yet so timestamps are not known. Set them ++ // now as they will be needed when serializing the packet later on. ++ auto now = clock_->CurrentTime(); ++ packet.SetTimestamp(now.ms() * kTimestampTicksPerMs); ++ packet.set_capture_time(now); ++ } + + // Only change the timestamp of padding packets sent over RTX. + // Padding only packets over RTP has to be sent as part of a media +-- +2.34.1 + diff --git a/third_party/libwebrtc/moz-patch-stack/0104.patch b/third_party/libwebrtc/moz-patch-stack/0104.patch new file mode 100644 index 0000000000..cd71ff5830 --- /dev/null +++ b/third_party/libwebrtc/moz-patch-stack/0104.patch @@ -0,0 +1,237 @@ +From: Michael Froman <mfroman@mozilla.com> +Date: Wed, 3 May 2023 14:41:00 +0000 +Subject: Bug 1685245 - cherry pick upstream libwebrtc commit 6aba07e5fe. r=ng + +Differential Revision: https://phabricator.services.mozilla.com/D176944 +Mercurial Revision: https://hg.mozilla.org/mozilla-central/rev/0a46882336b7e5e97b54492e361f4bd9b33f8a39 +--- + modules/rtp_rtcp/source/rtp_sender.cc | 33 +++++++++ + modules/rtp_rtcp/source/rtp_sender.h | 3 + + modules/rtp_rtcp/source/rtp_sender_video.cc | 26 +++---- + .../source/rtp_sender_video_unittest.cc | 67 +++++++++++++++++++ + 4 files changed, 117 insertions(+), 12 deletions(-) + +diff --git a/modules/rtp_rtcp/source/rtp_sender.cc b/modules/rtp_rtcp/source/rtp_sender.cc +index 336a117f4e..d5e8bdcccb 100644 +--- a/modules/rtp_rtcp/source/rtp_sender.cc ++++ b/modules/rtp_rtcp/source/rtp_sender.cc +@@ -558,6 +558,39 @@ std::unique_ptr<RtpPacketToSend> RTPSender::AllocatePacket() const { + return packet; + } + ++size_t RTPSender::RtxPacketOverhead() const { ++ MutexLock lock(&send_mutex_); ++ if (rtx_ == kRtxOff) { ++ return 0; ++ } ++ size_t overhead = 0; ++ ++ // Count space for the RTP header extensions that might need to be added to ++ // the RTX packet. ++ if (!always_send_mid_and_rid_ && (!rtx_ssrc_has_acked_ && ssrc_has_acked_)) { ++ // Prefer to reserve extra byte in case two byte header rtp header ++ // extensions are used. ++ static constexpr int kRtpExtensionHeaderSize = 2; ++ ++ // Rtx packets hasn't been acked and would need to have mid and rrsid rtp ++ // header extensions, while media packets no longer needs to include mid and ++ // rsid extensions. ++ if (!mid_.empty()) { ++ overhead += (kRtpExtensionHeaderSize + mid_.size()); ++ } ++ if (!rid_.empty()) { ++ overhead += (kRtpExtensionHeaderSize + rid_.size()); ++ } ++ // RTP header extensions are rounded up to 4 bytes. Depending on already ++ // present extensions adding mid & rrsid may add up to 3 bytes of padding. ++ overhead += 3; ++ } ++ ++ // Add two bytes for the original sequence number in the RTP payload. ++ overhead += kRtxHeaderSize; ++ return overhead; ++} ++ + void RTPSender::SetSendingMediaStatus(bool enabled) { + MutexLock lock(&send_mutex_); + sending_media_ = enabled; +diff --git a/modules/rtp_rtcp/source/rtp_sender.h b/modules/rtp_rtcp/source/rtp_sender.h +index 55dee7f219..b49afe0dec 100644 +--- a/modules/rtp_rtcp/source/rtp_sender.h ++++ b/modules/rtp_rtcp/source/rtp_sender.h +@@ -106,6 +106,9 @@ class RTPSender { + absl::optional<uint32_t> RtxSsrc() const RTC_LOCKS_EXCLUDED(send_mutex_) { + return rtx_ssrc_; + } ++ // Returns expected size difference between an RTX packet and media packet ++ // that RTX packet is created from. Returns 0 if RTX is disabled. ++ size_t RtxPacketOverhead() const; + + void SetRtxPayloadType(int payload_type, int associated_payload_type) + RTC_LOCKS_EXCLUDED(send_mutex_); +diff --git a/modules/rtp_rtcp/source/rtp_sender_video.cc b/modules/rtp_rtcp/source/rtp_sender_video.cc +index e1ac4e41c3..99a00025c1 100644 +--- a/modules/rtp_rtcp/source/rtp_sender_video.cc ++++ b/modules/rtp_rtcp/source/rtp_sender_video.cc +@@ -493,6 +493,13 @@ bool RTPSenderVideo::SendVideo( + // Backward compatibility for older receivers without temporal layer logic. + retransmission_settings = kRetransmitBaseLayer | kRetransmitHigherLayers; + } ++ const uint8_t temporal_id = GetTemporalId(video_header); ++ // TODO(bugs.webrtc.org/10714): retransmission_settings_ should generally be ++ // replaced by expected_retransmission_time_ms.has_value(). ++ const bool allow_retransmission = ++ expected_retransmission_time_ms.has_value() && ++ AllowRetransmission(temporal_id, retransmission_settings, ++ *expected_retransmission_time_ms); + + MaybeUpdateCurrentPlayoutDelay(video_header); + if (video_header.frame_type == VideoFrameType::kVideoFrameKey) { +@@ -514,16 +521,19 @@ bool RTPSenderVideo::SendVideo( + video_header.generic->frame_id, video_header.generic->chain_diffs); + } + +- const uint8_t temporal_id = GetTemporalId(video_header); + // No FEC protection for upper temporal layers, if used. + const bool use_fec = fec_type_.has_value() && + (temporal_id == 0 || temporal_id == kNoTemporalIdx); + + // Maximum size of packet including rtp headers. + // Extra space left in case packet will be resent using fec or rtx. +- int packet_capacity = rtp_sender_->MaxRtpPacketSize() - +- (use_fec ? FecPacketOverhead() : 0) - +- (rtp_sender_->RtxStatus() ? kRtxHeaderSize : 0); ++ int packet_capacity = rtp_sender_->MaxRtpPacketSize(); ++ if (use_fec) { ++ packet_capacity -= FecPacketOverhead(); ++ } ++ if (allow_retransmission) { ++ packet_capacity -= rtp_sender_->RtxPacketOverhead(); ++ } + + absl::optional<Timestamp> capture_time; + if (capture_time_ms > 0) { +@@ -652,14 +662,6 @@ bool RTPSenderVideo::SendVideo( + std::unique_ptr<RtpPacketizer> packetizer = + RtpPacketizer::Create(codec_type, payload, limits, video_header); + +- // TODO(bugs.webrtc.org/10714): retransmission_settings_ should generally be +- // replaced by expected_retransmission_time_ms.has_value(). For now, though, +- // only VP8 with an injected frame buffer controller actually controls it. +- const bool allow_retransmission = +- expected_retransmission_time_ms.has_value() +- ? AllowRetransmission(temporal_id, retransmission_settings, +- expected_retransmission_time_ms.value()) +- : false; + const size_t num_packets = packetizer->NumPackets(); + + if (num_packets == 0) +diff --git a/modules/rtp_rtcp/source/rtp_sender_video_unittest.cc b/modules/rtp_rtcp/source/rtp_sender_video_unittest.cc +index 72dfd0238d..d6fbba7bd8 100644 +--- a/modules/rtp_rtcp/source/rtp_sender_video_unittest.cc ++++ b/modules/rtp_rtcp/source/rtp_sender_video_unittest.cc +@@ -29,6 +29,9 @@ + #include "api/video/video_timing.h" + #include "modules/rtp_rtcp/include/rtp_cvo.h" + #include "modules/rtp_rtcp/include/rtp_header_extension_map.h" ++#include "modules/rtp_rtcp/source/rtcp_packet/nack.h" ++#include "modules/rtp_rtcp/source/rtcp_packet/receiver_report.h" ++#include "modules/rtp_rtcp/source/rtcp_packet/report_block.h" + #include "modules/rtp_rtcp/source/rtp_dependency_descriptor_extension.h" + #include "modules/rtp_rtcp/source/rtp_format_video_generic.h" + #include "modules/rtp_rtcp/source/rtp_generic_frame_descriptor.h" +@@ -57,6 +60,7 @@ using ::testing::ElementsAre; + using ::testing::ElementsAreArray; + using ::testing::IsEmpty; + using ::testing::NiceMock; ++using ::testing::Not; + using ::testing::Return; + using ::testing::ReturnArg; + using ::testing::SaveArg; +@@ -81,6 +85,7 @@ constexpr VideoCodecType kType = VideoCodecType::kVideoCodecGeneric; + constexpr uint32_t kTimestamp = 10; + constexpr uint16_t kSeqNum = 33; + constexpr uint32_t kSsrc = 725242; ++constexpr uint32_t kRtxSsrc = 912364; + constexpr int kMaxPacketLength = 1500; + constexpr Timestamp kStartTime = Timestamp::Millis(123456789); + constexpr int64_t kDefaultExpectedRetransmissionTimeMs = 125; +@@ -182,6 +187,8 @@ class RtpSenderVideoTest : public ::testing::Test { + config.retransmission_rate_limiter = &retransmission_rate_limiter_; + config.field_trials = &field_trials_; + config.local_media_ssrc = kSsrc; ++ config.rtx_send_ssrc = kRtxSsrc; ++ config.rid = "rid"; + return config; + }())), + rtp_sender_video_( +@@ -505,6 +512,66 @@ TEST_F(RtpSenderVideoTest, ConditionalRetransmitLimit) { + rtp_sender_video_->AllowRetransmission(header, kSettings, kRttMs)); + } + ++TEST_F(RtpSenderVideoTest, ++ ReservesEnoughSpaceForRtxPacketWhenMidAndRsidAreRegistered) { ++ constexpr int kMediaPayloadId = 100; ++ constexpr int kRtxPayloadId = 101; ++ constexpr size_t kMaxPacketSize = 1'000; ++ ++ rtp_module_->SetMaxRtpPacketSize(kMaxPacketSize); ++ rtp_module_->RegisterRtpHeaderExtension(RtpMid::Uri(), 1); ++ rtp_module_->RegisterRtpHeaderExtension(RtpStreamId::Uri(), 2); ++ rtp_module_->RegisterRtpHeaderExtension(RepairedRtpStreamId::Uri(), 3); ++ rtp_module_->RegisterRtpHeaderExtension(AbsoluteSendTime::Uri(), 4); ++ rtp_module_->SetMid("long_mid"); ++ rtp_module_->SetRtxSendPayloadType(kRtxPayloadId, kMediaPayloadId); ++ rtp_module_->SetStorePacketsStatus(/*enable=*/true, 10); ++ rtp_module_->SetRtxSendStatus(kRtxRetransmitted); ++ ++ RTPVideoHeader header; ++ header.codec = kVideoCodecVP8; ++ header.frame_type = VideoFrameType::kVideoFrameDelta; ++ auto& vp8_header = header.video_type_header.emplace<RTPVideoHeaderVP8>(); ++ vp8_header.temporalIdx = 0; ++ ++ uint8_t kPayload[kMaxPacketSize] = {}; ++ EXPECT_TRUE(rtp_sender_video_->SendVideo( ++ kMediaPayloadId, /*codec_type=*/kVideoCodecVP8, /*rtp_timestamp=*/0, ++ /*capture_time_ms=*/1'000, kPayload, header, ++ /*expected_retransmission_time_ms=*/absl::nullopt, /*csrcs=*/{})); ++ ASSERT_THAT(transport_.sent_packets(), Not(IsEmpty())); ++ // Ack media ssrc, but not rtx ssrc. ++ rtcp::ReceiverReport rr; ++ rtcp::ReportBlock rb; ++ rb.SetMediaSsrc(kSsrc); ++ rb.SetExtHighestSeqNum(transport_.last_sent_packet().SequenceNumber()); ++ rr.AddReportBlock(rb); ++ rtp_module_->IncomingRtcpPacket(rr.Build()); ++ ++ // Test for various frame size close to `kMaxPacketSize` to catch edge cases ++ // when rtx packet barely fit. ++ for (size_t frame_size = 800; frame_size < kMaxPacketSize; ++frame_size) { ++ SCOPED_TRACE(frame_size); ++ rtc::ArrayView<const uint8_t> payload(kPayload, frame_size); ++ ++ EXPECT_TRUE(rtp_sender_video_->SendVideo( ++ kMediaPayloadId, /*codec_type=*/kVideoCodecVP8, /*rtp_timestamp=*/0, ++ /*capture_time_ms=*/1'000, payload, header, ++ /*expected_retransmission_time_ms=*/1'000, /*csrcs=*/{})); ++ const RtpPacketReceived& media_packet = transport_.last_sent_packet(); ++ EXPECT_EQ(media_packet.Ssrc(), kSsrc); ++ ++ rtcp::Nack nack; ++ nack.SetMediaSsrc(kSsrc); ++ nack.SetPacketIds({media_packet.SequenceNumber()}); ++ rtp_module_->IncomingRtcpPacket(nack.Build()); ++ ++ const RtpPacketReceived& rtx_packet = transport_.last_sent_packet(); ++ EXPECT_EQ(rtx_packet.Ssrc(), kRtxSsrc); ++ EXPECT_LE(rtx_packet.size(), kMaxPacketSize); ++ } ++} ++ + TEST_F(RtpSenderVideoTest, SendsDependencyDescriptorWhenVideoStructureIsSet) { + const int64_t kFrameId = 100000; + uint8_t kFrame[100]; +-- +2.34.1 + diff --git a/third_party/libwebrtc/moz-patch-stack/0f87b38535.no-op-cherry-pick-msg b/third_party/libwebrtc/moz-patch-stack/0f87b38535.no-op-cherry-pick-msg new file mode 100644 index 0000000000..157e95cb7c --- /dev/null +++ b/third_party/libwebrtc/moz-patch-stack/0f87b38535.no-op-cherry-pick-msg @@ -0,0 +1 @@ +We cherry-picked this in bug 1832717 diff --git a/third_party/libwebrtc/moz-patch-stack/28ac56a415.no-op-cherry-pick-msg b/third_party/libwebrtc/moz-patch-stack/28ac56a415.no-op-cherry-pick-msg new file mode 100644 index 0000000000..3daebf527d --- /dev/null +++ b/third_party/libwebrtc/moz-patch-stack/28ac56a415.no-op-cherry-pick-msg @@ -0,0 +1 @@ +We cherry-picked this in Bug 1694304. diff --git a/third_party/libwebrtc/moz-patch-stack/301e546a68.no-op-cherry-pick-msg b/third_party/libwebrtc/moz-patch-stack/301e546a68.no-op-cherry-pick-msg new file mode 100644 index 0000000000..347f095596 --- /dev/null +++ b/third_party/libwebrtc/moz-patch-stack/301e546a68.no-op-cherry-pick-msg @@ -0,0 +1 @@ +We cherry-picked this in Bug 1832751 diff --git a/third_party/libwebrtc/moz-patch-stack/318cf28945.no-op-cherry-pick-msg b/third_party/libwebrtc/moz-patch-stack/318cf28945.no-op-cherry-pick-msg new file mode 100644 index 0000000000..d868e53df6 --- /dev/null +++ b/third_party/libwebrtc/moz-patch-stack/318cf28945.no-op-cherry-pick-msg @@ -0,0 +1 @@ +We already cherry-picked this when we vendored 218b56e516. diff --git a/third_party/libwebrtc/moz-patch-stack/32b64e895c.no-op-cherry-pick-msg b/third_party/libwebrtc/moz-patch-stack/32b64e895c.no-op-cherry-pick-msg new file mode 100644 index 0000000000..d5186fa724 --- /dev/null +++ b/third_party/libwebrtc/moz-patch-stack/32b64e895c.no-op-cherry-pick-msg @@ -0,0 +1 @@ +We cherry-picked this in Bug 1810949. diff --git a/third_party/libwebrtc/moz-patch-stack/3da04a93cd.no-op-cherry-pick-msg b/third_party/libwebrtc/moz-patch-stack/3da04a93cd.no-op-cherry-pick-msg new file mode 100644 index 0000000000..347f095596 --- /dev/null +++ b/third_party/libwebrtc/moz-patch-stack/3da04a93cd.no-op-cherry-pick-msg @@ -0,0 +1 @@ +We cherry-picked this in Bug 1832751 diff --git a/third_party/libwebrtc/moz-patch-stack/6aba07e5fe.no-op-cherry-pick-msg b/third_party/libwebrtc/moz-patch-stack/6aba07e5fe.no-op-cherry-pick-msg new file mode 100644 index 0000000000..47dd36c704 --- /dev/null +++ b/third_party/libwebrtc/moz-patch-stack/6aba07e5fe.no-op-cherry-pick-msg @@ -0,0 +1 @@ +We cherry-picked this in Bug 1685245. diff --git a/third_party/libwebrtc/moz-patch-stack/6fc1ae58be.no-op-cherry-pick-msg b/third_party/libwebrtc/moz-patch-stack/6fc1ae58be.no-op-cherry-pick-msg new file mode 100644 index 0000000000..632f6ee687 --- /dev/null +++ b/third_party/libwebrtc/moz-patch-stack/6fc1ae58be.no-op-cherry-pick-msg @@ -0,0 +1 @@ +We cherry-picked this in Bug 1828065. diff --git a/third_party/libwebrtc/moz-patch-stack/7b0d7f48fb.no-op-cherry-pick-msg b/third_party/libwebrtc/moz-patch-stack/7b0d7f48fb.no-op-cherry-pick-msg new file mode 100644 index 0000000000..024b3046d3 --- /dev/null +++ b/third_party/libwebrtc/moz-patch-stack/7b0d7f48fb.no-op-cherry-pick-msg @@ -0,0 +1 @@ +We cherry-picked this commit in bug 1832770 diff --git a/third_party/libwebrtc/moz-patch-stack/7e5d9edfdf.no-op-cherry-pick-msg b/third_party/libwebrtc/moz-patch-stack/7e5d9edfdf.no-op-cherry-pick-msg new file mode 100644 index 0000000000..d5186fa724 --- /dev/null +++ b/third_party/libwebrtc/moz-patch-stack/7e5d9edfdf.no-op-cherry-pick-msg @@ -0,0 +1 @@ +We cherry-picked this in Bug 1810949. diff --git a/third_party/libwebrtc/moz-patch-stack/91d5fc2ed6.no-op-cherry-pick-msg b/third_party/libwebrtc/moz-patch-stack/91d5fc2ed6.no-op-cherry-pick-msg new file mode 100644 index 0000000000..d5186fa724 --- /dev/null +++ b/third_party/libwebrtc/moz-patch-stack/91d5fc2ed6.no-op-cherry-pick-msg @@ -0,0 +1 @@ +We cherry-picked this in Bug 1810949. diff --git a/third_party/libwebrtc/moz-patch-stack/a09331a603.no-op-cherry-pick-msg b/third_party/libwebrtc/moz-patch-stack/a09331a603.no-op-cherry-pick-msg new file mode 100644 index 0000000000..b5bacba288 --- /dev/null +++ b/third_party/libwebrtc/moz-patch-stack/a09331a603.no-op-cherry-pick-msg @@ -0,0 +1 @@ +We cherry-picked this commit in bug 1832804 diff --git a/third_party/libwebrtc/moz-patch-stack/adf55790b6.no-op-cherry-pick-msg b/third_party/libwebrtc/moz-patch-stack/adf55790b6.no-op-cherry-pick-msg new file mode 100644 index 0000000000..632f6ee687 --- /dev/null +++ b/third_party/libwebrtc/moz-patch-stack/adf55790b6.no-op-cherry-pick-msg @@ -0,0 +1 @@ +We cherry-picked this in Bug 1828065. diff --git a/third_party/libwebrtc/moz-patch-stack/b1a174041d.no-op-cherry-pick-msg b/third_party/libwebrtc/moz-patch-stack/b1a174041d.no-op-cherry-pick-msg new file mode 100644 index 0000000000..d5186fa724 --- /dev/null +++ b/third_party/libwebrtc/moz-patch-stack/b1a174041d.no-op-cherry-pick-msg @@ -0,0 +1 @@ +We cherry-picked this in Bug 1810949. diff --git a/third_party/libwebrtc/moz-patch-stack/ba41b40461.no-op-cherry-pick-msg b/third_party/libwebrtc/moz-patch-stack/ba41b40461.no-op-cherry-pick-msg new file mode 100644 index 0000000000..d5186fa724 --- /dev/null +++ b/third_party/libwebrtc/moz-patch-stack/ba41b40461.no-op-cherry-pick-msg @@ -0,0 +1 @@ +We cherry-picked this in Bug 1810949. |