This is inherently large, because modifying these bits of DOMMediaStream and
MediaStreamTrack affects all consumers and producers of all DOMMediaStreams and
MediaStreamTracks.
Things are generally much simpler now.
Producers of tracks now create a MediaStream in the graph, add it to a
MediaStreamTrackSource subclass that takes ownership of it, and add the source
to a MediaStreamTrack. Should the producer need a DOMMediaStream it is now much
simpler to create as the only thing needed is the current window. The stream is
a rather simple wrapper around an array of MediaStreamTracks.
HTMLMediaElement is still not as straight forward as other consumers since it
consumes the DOMMediaStream directly, as opposed to a set of tracks.
The new MediaStreamRenderer helper class helps bridge the gap between this fact
and the new track-based MediaStreamGraph interface, as it needs to juggle
registering multiple audio tracks for audio output. This hooks into existing
HTMLMediaElement logic and brings a welcome simplification to all the glue
previously needed there.
Differential Revision: https://phabricator.services.mozilla.com/D37934
--HG--
extra : moz-landing-system : lando
Normally a track in mUpdateTracks is cleared by ExtractPendingInput, when that
track's ending is processed. However, if the SourceMediaStream is destroyed
before an ended track is processed, the track including it's buffered segment
in mUpdateTracks will leak until the SourceMediaStream is destroyed.
This might not be until late XPCOM Shutdown when the cycle collector shuts down,
which is too late to release graphics resources.
Depends on D37931
Differential Revision: https://phabricator.services.mozilla.com/D37932
--HG--
extra : moz-landing-system : lando
A legit case that fails this assert is:
- CloseAudioInput() on main thread for last non-webaudio MediaStream
- AudioContext closes on main thread
- CloseAudioInputImpl() on graph thread sets next driver to an output-only audio
driver since there are AudioNodeStreams still present
- AudioContext's Close operation is applied on graph thread, first all
AudioNodeStreams are suspended, making the graph consider itself as having no
audio tracks present. Then we check the next driver, which is an audio driver
per above. This fails the assert.
Differential Revision: https://phabricator.services.mozilla.com/D37931
--HG--
extra : moz-landing-system : lando
This requires replacing inclusions of it with inclusions of more specific prefs
files.
The exception is that StaticPrefsAll.h, which is equivalent to StaticPrefs.h,
and is used in `Codegen.py` because doing something smarter is tricky and
suitable for a follow-up. As a result, any change to StaticPrefList.yaml will
still trigger recompilation of all the generated DOM bindings files, but that's
still a big improvement over trigger recompilation of every file that uses
static prefs.
Most of the changes in this commit are very boring. The only changes that are
not boring are modules/libpref/*, Codegen.py, and ServoBindings.toml.
Differential Revision: https://phabricator.services.mozilla.com/D39138
--HG--
extra : moz-landing-system : lando
This will be used to release the AudioWorkletProcessor on the correct thread.
Depends on D34480
Differential Revision: https://phabricator.services.mozilla.com/D34481
--HG--
extra : moz-landing-system : lando
The renaming of MediaWebspeechRecognitionEnable to
media_webspeech_recognition_enable is to follow the StaticPrefs convention,
because bindings are going to assume that convention.
Differential Revision: https://phabricator.services.mozilla.com/D32942
--HG--
extra : moz-landing-system : lando
We don't currently know any ways in which external code can make the
suspendCount negative, but this will keep us safe should a future bug enable it.
Depends on D28805
Differential Revision: https://phabricator.services.mozilla.com/D28806
--HG--
extra : moz-landing-system : lando
`streamCurrentTime` is relative to the start of the stream, as is the track's
segment. We need to account for the track's start time in the stream when
notifying of output.
Differential Revision: https://phabricator.services.mozilla.com/D27268
--HG--
extra : moz-landing-system : lando
This is necessary to get an accurate total track duration in case DecodedStream
does a lot of seeking and pausing.
This also removes previous code for avoiding blocking a video track, as this new
code handles that case too.
Differential Revision: https://phabricator.services.mozilla.com/D27264
--HG--
extra : moz-landing-system : lando
Null frames could still be valid. It's how DecodedStream signals a pause. They
could also be valid and have an intrinsic size and the force-black flag set.
Differential Revision: https://phabricator.services.mozilla.com/D27263
--HG--
extra : moz-landing-system : lando
CloseAudioInut method posts a message, to the graph thread, in order to close the input asynchonously. When CloseAudioInput method was being executed from a thread other than the main thread, a runnable would be posted to main thread in order to post the async message from there. That was a risky path because when the graph was shutting down there were no guarantee that the close-input message would reach the graph thread before destroy takes place. By limiting CloseAudioInput to main thread it is ensured that the close-input message will be executed before destroy.
Differential Revision: https://phabricator.services.mozilla.com/D27751
--HG--
extra : moz-landing-system : lando
`streamCurrentTime` is relative to the start of the stream, as is the track's
segment. We need to account for the track's start time in the stream when
notifying of output.
Differential Revision: https://phabricator.services.mozilla.com/D27268
--HG--
extra : moz-landing-system : lando
This is necessary to get an accurate total track duration in case DecodedStream
does a lot of seeking and pausing.
This also removes previous code for avoiding blocking a video track, as this new
code handles that case too.
Differential Revision: https://phabricator.services.mozilla.com/D27264
--HG--
extra : moz-landing-system : lando
Null frames could still be valid. It's how DecodedStream signals a pause. They
could also be valid and have an intrinsic size and the force-black flag set.
Differential Revision: https://phabricator.services.mozilla.com/D27263
--HG--
extra : moz-landing-system : lando
This allows suspending and resuming the context from the debugger without having
observable side-effects.
Differential Revision: https://phabricator.services.mozilla.com/D24915
--HG--
extra : moz-landing-system : lando
instead of special signaling, which can get out of order with other messages.
Using AppendMessage() provides that control messages sent to the graph before
shutdown are run on the graph thread.
Differential Revision: https://phabricator.services.mozilla.com/D24856
--HG--
extra : moz-landing-system : lando
Disabling a track from a MediaEngineRemoteVideoSource will automatically stop
that source until the track is enabled again. With consumers of this track
getting frames through direct listeners only, there would be no more frames
coming after the source has stopped. This stopping races with making the next
frame that is appended to the track (through the MSG, though direct) black,
sometimes leaving the consumer with no black frame at all.
Specs say to render disabled video tracks as black, so with direct listeners
this needs to be signaled to, and handled by, the consumers explicitly.
Differential Revision: https://phabricator.services.mozilla.com/D22923
--HG--
extra : moz-landing-system : lando
VideoSegments still have durations, and they are still needed by the
MediaStreamGraph as it shuffles MediaSegments around.
They do not have a say in the wall-clock duration of video frames however.
Removing this should prevent any producers starting to add video chunks with
durations in the future.
Differential Revision: https://phabricator.services.mozilla.com/D22914
--HG--
extra : moz-landing-system : lando
Long-term we want to lift durations out of video altogether, and only use
wall-clock timestamps. This patch achieves this in VideoTrackEncoder.
When the MediaStreamGraph is audio-only, the equivalent for video will be
completely duration-less. Until we have that, some pieces around the MSG will
still need durations for track-bookkeeping reasons.
This also integrates the DriftCompensator into VideoTrackEncoder, by
compensating for drift when frames are moved from mIncomingBuffer to
mOutgoingBuffer, i.e., when we recalculate time stamps into durations for the
underlying encoder to use.
Differential Revision: https://phabricator.services.mozilla.com/D22909
--HG--
extra : moz-landing-system : lando
DecodedStream has been basing video timestamps on something called
StreamTracksStartTimeStamp in MediaStreams, which call through all the way
down to the GraphDriver.
This removes the entire timestamp mechanism, except for a bit of legit
usage internally in the SystemClockDriver. Video timestamps are instead
based on the audio clock through GetPosition(), the same way the VideoSink
operates.
Differential Revision: https://phabricator.services.mozilla.com/D22896
--HG--
extra : moz-landing-system : lando
The way it's implemented it only adds plumbing and overhead, no value.
This patch moves it to a thin wrapper around DirectMediaStreamTrackListener,
managed by VideoStreamTrack, instead.
Differential Revision: https://phabricator.services.mozilla.com/D22895
--HG--
extra : moz-landing-system : lando
With the per-track pulling settings we have today it is clearly the intention
of the producer to start consuming a track that has pulling enabled, even if
there are other tracks where pulling is disabled.
This patch fixes that, so that when a stream has at least one pulled track it
will append null data to other tracks (commonly video) if they are lacking
data.
This means that we still block an entire stream if all tracks have pulling
disabled - to maintain the status quo for media element capture, which is
the only push-only producer of data.
Differential Revision: https://phabricator.services.mozilla.com/D17611
--HG--
extra : moz-landing-system : lando
