Usually the threshold is reset internally in MediaDataDecoder subclasses
that support the hint in their Flush() implementations so the value
will start fresh after seeking completed. But sometimes when there are
consecutive seek requests, MediaFormatReader::DecoderData::Flush() could
return early because DecoderData::mFlushed stays true when there is no
sample demuxed yet, and the threshold will not be cleared. Also, in
MediaFormatReader::SetVideoDecodeThreshold() we decide not to set the
hint when the seek target is close to EOS by checking the existence of
the next keyframe, and that could fail when there are gaps between MSE
buffered ranges. To make sure the hint is never out of date, we should
clear it rather than leaving it untouched.
Differential Revision: https://phabricator.services.mozilla.com/D15227
--HG--
extra : moz-landing-system : lando
Handle mp4parse-rust providing cbcs data in the track metadata. Explicitly check
the crypto scheme we get in the metadata and error if we encounter something
outside of cenc and cbcs -- catch unexpected data early.
Differential Revision: https://phabricator.services.mozilla.com/D15878
--HG--
extra : moz-landing-system : lando
Rework our mp4 sample iterator to handle cbcs crypto data.
To support this we populate the following new data for samples:
- Crypto pattern information, this is split into a count of encrypted blocks
and a count of clear blocks.
- A constant IV.
This information is available at a track level and a sample group level. The
sample group level supersedes track level information if both a present.
Prior to this patch, some crypto information was written to samples in
the SampleIterator in Index.cpp, and some in the MP4Demuxer (based on if the
SampleIterator had not populated the data). This patch moves all these
operations into the SampleIterator -- the idea being that the sample iterator
should be the component responsible for setting up sample meta data.
Differential Revision: https://phabricator.services.mozilla.com/D15877
--HG--
extra : moz-landing-system : lando
Explicitly store the crypto scheme being used on our crypto structs to let us
differentiate between cenc and cbcs data. In doing so remove mMode and replace
mValid with IsEncrypted() for the following reasons:
- Different modes within the existing schemes are not currently utilized by the
spec: the scheme implies mode. Having a mode and a scheme could lead to confusion
between the two. We can return mMode if ever needed by the spec --
possibly if the isProtected flag which we were tracking with mMode, is
ever changed to be more than a bool in the spec.
- mValid was typically used to check if these structs contained valid crypto
data or not. With only one scheme this was often shorthand for 'IsEncrypted',
but with multiple schemes what is considered valid data for one may not be for
another. Do away with this and just explicitly have an 'IsEncrypted'.
Differential Revision: https://phabricator.services.mozilla.com/D15874
--HG--
extra : moz-landing-system : lando
Handle mp4parse-rust providing cbcs data in the track metadata. Explicitly check
the crypto scheme we get in the metadata and error if we encounter something
outside of cenc and cbcs -- catch unexpected data early.
Differential Revision: https://phabricator.services.mozilla.com/D15878
--HG--
extra : moz-landing-system : lando
Rework our mp4 sample iterator to handle cbcs crypto data.
To support this we populate the following new data for samples:
- Crypto pattern information, this is split into a count of encrypted blocks
and a count of clear blocks.
- A constant IV.
This information is available at a track level and a sample group level. The
sample group level supersedes track level information if both a present.
Prior to this patch, some crypto information was written to samples in
the SampleIterator in Index.cpp, and some in the MP4Demuxer (based on if the
SampleIterator had not populated the data). This patch moves all these
operations into the SampleIterator -- the idea being that the sample iterator
should be the component responsible for setting up sample meta data.
Differential Revision: https://phabricator.services.mozilla.com/D15877
--HG--
extra : moz-landing-system : lando
Explicitly store the crypto scheme being used on our crypto structs to let us
differentiate between cenc and cbcs data. In doing so remove mMode and replace
mValid with IsEncrypted() for the following reasons:
- Different modes within the existing schemes are not currently utilized by the
spec of implementation. Having a mode and a scheme could lead to confusion
between the two. We can return mMode if ever needed by the spec.
- mValid was typically used to check if these structs contained valid crypto
data or not. With only one scheme this was often shorthand for 'IsEncrypted',
but with multiple schemes what is considered valid data for one may not be for
another. Do away with this and just explicitly have an 'IsEncrypted'.
Differential Revision: https://phabricator.services.mozilla.com/D15874
--HG--
extra : moz-landing-system : lando
Avoid resetting volume in every AudioSink restart to stop unexpected volume change during play/pause and seek. Volume changes could occur if the volume has been modified outside firefox (for example pavucontrol in Linux).
Differential Revision: https://phabricator.services.mozilla.com/D15210
--HG--
extra : moz-landing-system : lando
Processing MediaCodec output in Android binder threads while flushing
in task queue could cause race condition and leftover frames. Dispatch
the processing to task queue ensures all frames prior to flushing will
be cleared (by mDecodedData.Clear()) or ignored (by mInputInfos.Clear()).
Also consolidate all flushing operations in one task to avoid frame
insertion between emptying mDecodedData and mInputInfos.
Differential Revision: https://phabricator.services.mozilla.com/D15228
--HG--
extra : moz-landing-system : lando
Usually the threshold is reset internally in MediaDataDecoder subclasses
that support the hint in their Flush() implementations so the value
will start fresh after seeking completed. But sometimes when there are
consecutive seek requests, MediaFormatReader::DecoderData::Flush() could
return early because DecoderData::mFlushed stays true when there is no
sample demuxed yet, and the threshold will not be cleared. Also, in
MediaFormatReader::SetVideoDecodeThreshold() we decide not to set the
hint when the seek target is close to EOS by checking the existence of
the next keyframe, and that could fail when there are gaps between MSE
buffered ranges. To make sure the hint is never out of date, we should
clear it rather than leaving it untouched.
Differential Revision: https://phabricator.services.mozilla.com/D15227
--HG--
extra : moz-landing-system : lando
'mWasAllowedToStart' would be set to false if AudioContext is not allowed to start, and would be set to true if AudioContext has been allowed to start.
Differential Revision: https://phabricator.services.mozilla.com/D14636
--HG--
extra : moz-landing-system : lando
Wrap 'nsContentUtils::ReportToConsole()' to reduce necessary input parameters and call it when we need to log error or warning message. Show the warning when autoplay is blocked.
For web audio, this restores the console messages removed in part4 and also reports the same message when the AudioContext is blocked in the constructor.
Differential Revision: https://phabricator.services.mozilla.com/D14330
--HG--
extra : moz-landing-system : lando
AudioContext won't need to ask permission request anymore, and we will report error message in patch5.
Differential Revision: https://phabricator.services.mozilla.com/D14329
--HG--
extra : moz-landing-system : lando
We're going to remove all autoplay temporary permission related codes, so we don't need to cache it anymore.
Differential Revision: https://phabricator.services.mozilla.com/D14325
--HG--
extra : moz-landing-system : lando
Driveby tidying:
- Tidy and clarify comment impacted by global clang-formatting.
- Update comments on two test cases disabled by bug 1224019. These tests no
longer have log spam issues, and have some value as sanity tests, but remain
disabled due to being resource hogs. Comments indicate this.
Differential Revision: https://phabricator.services.mozilla.com/D15441
--HG--
extra : moz-landing-system : lando
Persisting this box lets us use it later while indexing into metadata.
Importantly, it lets us look up the appropriate sample description entry, which
lets us determine if a fragment is associated with crypto information in the
init segment.
Being able to do this is required for cbcs encryption. This will be done in a
follow up bug: Bug 1487416 (and possibly others).
Differential Revision: https://phabricator.services.mozilla.com/D15440
--HG--
extra : moz-landing-system : lando
This is required to disambiguate if samples should be considered encrypted or
not when parsing certain cbcs encrypted files. Unlike with cenc encryption, cbcs
encrypted media may have fragments that lack characteristics from which we can
infer encryption. Because of this we need to store and trust this information
from the sample description box.
Differential Revision: https://phabricator.services.mozilla.com/D15439
--HG--
extra : moz-landing-system : lando
The following code appears to be dead and is removed:
- The AuxInfo class.
- Moof::ParseSaiz.
- Moof::ParseSaio.
- MoofParser::ParseSinf.
Differential Revision: https://phabricator.services.mozilla.com/D15438
--HG--
extra : moz-landing-system : lando
Otherwise we risk building up a buffer in the microphone source from when
Start() is run until pulling is enabled. This manifests itself as input latency
to the user.
Differential Revision: https://phabricator.services.mozilla.com/D15194
--HG--
extra : moz-landing-system : lando
All subclasses are now returning NS_OK, so there's no point in keeping nsresult
as the return type.
Differential Revision: https://phabricator.services.mozilla.com/D15193
--HG--
extra : moz-landing-system : lando
There's no guarantee a backend exists when a MediaManager does, and
crash-stats shows that shutdown can occur between posting a task and running
it, making it illegal to create a backend anew.
We're safer off avoiding creating a new backend. The cleanup step we're trying
to do is only effective if a backend already exists anyway.
Differential Revision: https://phabricator.services.mozilla.com/D15631
--HG--
extra : moz-landing-system : lando
These tests have always been skipped silently. Now that https tests are
enabled to actually run, we find that some (all?) actually fail on Android:
now explicitly skipped to allow for green runs of tier 1 suites.
The guts of MemoryReportRequestClient's supporting runnables contain
switches on the particular type of process we're running. If you're
bringing up a new process type, having to add extra cases for your
process type here is a bit onerous. These runnables really shouldn't
know anything about the process types that they're running on, either.
The easiest thing to do is modify MemoryReportRequestClient::Start to
take callbacks for what to do when a report is created and when
reporting is finished. Then all process-specific knowledge can be
pushed out to the clients themselves, leaving MemoryReportRequestClient
and friends process-type agnostic. We could even, at some later date,
move this code into xpcom/base/ to sit near nsMemoryReporterManager,
where it belongs.
Summary: Really sorry for the size of the patch. It's mostly automatic
s/nsIDocument/Document/ but I had to fix up in a bunch of places manually to
add the right namespacing and such.
Overall it's not a very interesting patch I think.
nsDocument.cpp turns into Document.cpp, nsIDocument.h into Document.h and
nsIDocumentInlines.h into DocumentInlines.h.
I also changed a bunch of nsCOMPtr usage to RefPtr, but not all of it.
While fixing up some of the bits I also removed some unneeded OwnerDoc() null
checks and such, but I didn't do anything riskier than that.
If we can't get any sample anymore, the resource might have been closed, so we should change state to 'completedState'.
Differential Revision: https://phabricator.services.mozilla.com/D15029
--HG--
extra : moz-landing-system : lando
Otherwise we risk building up a buffer in the microphone source from when
Start() is run until pulling is enabled. This manifests itself as input latency
to the user.
Differential Revision: https://phabricator.services.mozilla.com/D15194
--HG--
extra : moz-landing-system : lando
All subclasses are now returning NS_OK, so there's no point in keeping nsresult
as the return type.
Differential Revision: https://phabricator.services.mozilla.com/D15193
--HG--
extra : moz-landing-system : lando
If video has not been within the potential visible range (which is larger than viewport) yet, its visibility state won't
be updated and would stay in 'UNTRACK'. As those kinds of video are still invisible to users, we don't need to decode
any video frames, we can suspend their video decoding until they're going to be visible.
Differential Revision: https://phabricator.services.mozilla.com/D13804
--HG--
extra : moz-landing-system : lando
Removes RTP stat field isRemote and adds the new types remote-inbound-rtp, and remote-outbound-rtp
Differential Revision: https://phabricator.services.mozilla.com/D15068
--HG--
extra : moz-landing-system : lando
Removes RTP stat field isRemote and adds the new types remote-inbound-rtp, and remote-outbound-rtp
Differential Revision: https://phabricator.services.mozilla.com/D15068
--HG--
extra : moz-landing-system : lando
Adds log statements for all the reasons that GMPVideoEncoderParent::Encode can fail
Differential Revision: https://phabricator.services.mozilla.com/D15051
--HG--
extra : moz-landing-system : lando
This clamps requests for FPS that are below 1 to 1. As far as I can tell,
the getDisplayMedia specification does not provide any guidance on how to
interpret negative or zero FPS constraints, so this seems like a reasonable
limitation.
I'm not adding a test here as there will be a wpt test that covers this
as part of Bug 1321221.
Differential Revision: https://phabricator.services.mozilla.com/D15105
--HG--
extra : moz-landing-system : lando
This initializes a one of the stats that lost its default initialization when it was changed from an Atomic to a plain old int.
Differential Revision: https://phabricator.services.mozilla.com/D14978
--HG--
extra : moz-landing-system : lando
In libavcodec 58 and later, the old avcodec_decode_video2 is broken and only return the first visible frame found after a VP9 super-frame.
This resulted in some YouTube videos for about 10% of the frames to never be returned.
Only the new API properly behaves so we upgrade our code to use it.
Differential Revision: https://phabricator.services.mozilla.com/D14682
--HG--
extra : moz-landing-system : lando
The value for mozAvSyncDelay has been broken since the branch 57 update
(Bug 1341285). We added SetCurrentSyncOffset() but never called it from
anywhere.
In the future we should be getting stats from AudioReceiveStream rather than
modifying the channel code, the delay_estimate_ms field provides almost the
same information.
Since we're attempting to get rid of moz prefixed stats, it makes sense to just
remove this code rather than fix it. The associated telemetry code has been
broken since Bug 1341285 as well so I think it is safe to remove.
Differential Revision: https://phabricator.services.mozilla.com/D14462
--HG--
extra : moz-landing-system : lando
This patch moves all UA Widget calls to helper functions in Element.cpp. The helper function AttachAndSetUAShadowRoot sets the shadow root in a runnable, so that it is in the same order of NotifyUAWidget* runnables.
Differential Revision: https://phabricator.services.mozilla.com/D13479
--HG--
extra : moz-landing-system : lando
The RTCP timestamps have different timebases and reporting sources, this makes the source and timebase the same for all RTCP stats
Differential Revision: https://phabricator.services.mozilla.com/D7354
--HG--
extra : moz-landing-system : lando
This patch move the actual widget construction to a onsetup method, allow UAWidgetsChild to hold the reference of the widget instance even if the actual setup (happens in the onsetup call) throws. With the reference of the widget kept, UAWidgetsChild will finally able to call its destructor later on.
Depends on D13607
Differential Revision: https://phabricator.services.mozilla.com/D13608
--HG--
extra : moz-landing-system : lando
When talking about autoplay, video without audio track should also be controlled by the pref "media.autoplay.allow-muted".
Differential Revision: https://phabricator.services.mozilla.com/D14480
--HG--
extra : moz-landing-system : lando
If video has not been within the potential visible range (which is larger than viewport) yet, its visibility state won't
be updated and would stay in 'UNTRACK'. As those kinds of video are still invisible to users, we don't need to decode
any video frames, we can suspend their video decoding until they're going to be visible.
Differential Revision: https://phabricator.services.mozilla.com/D13804
--HG--
extra : moz-landing-system : lando
To make sure media sink starts from the correct position, otherwise, we would incorrectly estimate the decoded audio
duration when we directly seek looping audio to EOS. That would results in MDSM continually dispatching decoding tasks
even if we've enough data.
Differential Revision: https://phabricator.services.mozilla.com/D13949
--HG--
extra : moz-landing-system : lando
Update mp4parse-rust update script and pull the new version.
This update changes the mp4parse C-API. Specifically, each track can now
have multiple sample descriptions. Previously we'd just exposed the first for
the entire track, and if others were available they were not exposed via the
API. Because of the API change, we update the C++ interface with mp4parse-rust.
We now inspect the sample info to make sure they're consistent with the parsers
expectations:
- Only a single codec is present for a track, multiple codecs in a track will
result in us returning an error.
- Only 0 or 1 crypto info is present for a track, more than one set of info will
result in us returning an error.
We still generalize some of the first sample info to the samples of the track,
as we did before this patch. However, we will now catch the above cases
explicitly.
We now handle crypto information if it is not present on the first sample info.
The parser will iterate through sample infos and use the first set of crypto
info it finds (and fail if it finds 2+).
Differential Revision: https://phabricator.services.mozilla.com/D14107
--HG--
extra : moz-landing-system : lando
The VideoSink shares the AudioSink's own EndedPromise to notify its user that it has ended. As such, the MozPromise used must be non-exclusive.
Using the GenericPromise for such purpose only hid that requirement.
We also remove the MediaSink from the media namespace, and clarify the naming of some arguments and class members to accurately describe what they do.
Differential Revision: https://phabricator.services.mozilla.com/D14024
--HG--
extra : moz-landing-system : lando
MozPromise most common use is to have an single or exclusive listener. By making the MozPromise generated by IPDL exclusive we can also use move semantics.
While at it, we also use move semantics for the ResponseRejectReason and via the callback's reject method so that the lambda used with the MozPromise::Then can be identical to the one used by the IPDL callback.
As it currently is, it provides no advantage over a copy as it's just an enum; however, this will facilitate future changes where it may not be.
Differential Revision: https://phabricator.services.mozilla.com/D13906
--HG--
extra : moz-landing-system : lando
The webrtc.org capturer implementations do not allow Start to be called more
than once. Previously we worked around this by adding a Stop method that was
called from StopCapture. With this change, we instead free the capturer in
StopCapture and create or re-create it as needed from StartCapture or
FocusOnSelectedSource.
Depends on D14066
Differential Revision: https://phabricator.services.mozilla.com/D14067
--HG--
extra : moz-landing-system : lando
Update WebRTC stat deprecation messages which were originally targeting 65, which are now targeting 66.
Differential Revision: https://phabricator.services.mozilla.com/D13950
--HG--
extra : moz-landing-system : lando
Update mp4parse-rust update script and pull the new version.
This update changes the mp4parse C-API. Specifically, each track can now
have multiple sample descriptions. Previously we'd just exposed the first for
the entire track, and if others were available they were not exposed via the
API. Because of the API change, we update the C++ interface with mp4parse-rust.
We now inspect the sample info to make sure they're consistent with the parsers
expectations:
- Only a single codec is present for a track, multiple codecs in a track will
result in us returning an error.
- Only 0 or 1 crypto info is present for a track, more than one set of info will
result in us returning an error.
We still generalize some of the first sample info to the samples of the track,
as we did before this patch. However, we will now catch the above cases
explicitly.
We now handle crypto information if it is not present on the first sample info.
The parser will iterate through sample infos and use the first set of crypto
info it finds (and fail if it finds 2+).
Differential Revision: https://phabricator.services.mozilla.com/D14107
--HG--
extra : moz-landing-system : lando
Latest dav1d version supports to store the timing information in undecoded frame and restore it later from decoded picture. This can provide more accurate timing especially during drain. In additionto that, colorspace information is set according to the size of the image. Finally this patch addresses some style comments.
Differential Revision: https://phabricator.services.mozilla.com/D13428
--HG--
extra : moz-landing-system : lando
PlatformUIThread is only used by the video_engine code, so it makes sense to
move it there rather than maintain it as a diff against upstream webrtc.org.
Differential Revision: https://phabricator.services.mozilla.com/D13531
--HG--
extra : moz-landing-system : lando
The destination stream will provide currentFrame for AudioWorkletGlobalScope on the worklet execution thread.
On the control thread, it supports messages via the MediaStreamGraph and identifies the associated AudioContext.
Differential Revision: https://phabricator.services.mozilla.com/D13219
--HG--
extra : moz-landing-system : lando
Latest dav1d version supports to store the timing information in undecoded frame and restore it later from decoded picture. This can provide more accurate timing especially during drain. In additionto that, colorspace information is set according to the size of the image. Finally this patch addresses some style comments.
Differential Revision: https://phabricator.services.mozilla.com/D13428
--HG--
extra : moz-landing-system : lando
This does what's done in bug 1505547 but in a different way. For some reason, access
shadowRoot.host in the function can trigger an assertion.
this.video is a Proxy so it cannot be compared, but all its methods are proxied.
Differential Revision: https://phabricator.services.mozilla.com/D15767
--HG--
extra : moz-landing-system : lando
- modify line wrap up to 80 chars; (tw=80)
- modify size of tab to 2 chars everywhere; (sts=2, sw=2)
--HG--
extra : rebase_source : 7eedce0311b340c9a5a1265dc42d3121cc0f32a0
extra : amend_source : 9cb4ffdd5005f5c4c14172390dd00b04b2066cd7
Without knownTracksTime, StreamTracks::GetFirstTrackEnd() returns
STREAM_TIME_MAX for an empty StreamTracks, so PullNewData() thinks no new data
is needed.
This circumvents that by always checking whether tracks need data.
Differential Revision: https://phabricator.services.mozilla.com/D12928
--HG--
extra : moz-landing-system : lando
Historically this code was part of webrtc.org but has since been removed
from upstream. Rather than maintaining it as a local diff against upstream,
we should just move it to where it is used.
Differential Revision: https://phabricator.services.mozilla.com/D13092
--HG--
rename : media/webrtc/trunk/webrtc/video_engine/browser_capture_impl.h => dom/media/systemservices/video_engine/browser_capture_impl.h
rename : media/webrtc/trunk/webrtc/video_engine/desktop_capture_impl.cc => dom/media/systemservices/video_engine/desktop_capture_impl.cc
rename : media/webrtc/trunk/webrtc/video_engine/desktop_capture_impl.h => dom/media/systemservices/video_engine/desktop_capture_impl.h
extra : moz-landing-system : lando
DecoderDoctorLifeLogger is used as a base class in many classes but it does need to have a virtual destructor to avoid adding a vtable in the class. Thus the constructor and destructor become protected to avoid being used as base pointer to delete the real object.
Differential Revision: https://phabricator.services.mozilla.com/D13074
--HG--
extra : moz-landing-system : lando