This change is a continuation of Part 1 (Bug 1570128), where the 2D content rendered by Firefox for Firefox Reality on Desktop is marshalled through VRHost so that it can be presented in a VR environment.
A new class, FxrOutputHandler, is created to manage creating a sharable texture, sharing it through VRShMem, and updating it when content updates. This class updates content with both WebRender and conventional rendering output.
This initial iteration of FxrOutputHandler does not have synchronization between reading and writing this shared texture across processes. A subsequent fix (Bug 1581881) is pending, which will reuse WebVR code to manage writing to and reading from a pool of textures.
This also presents issues with rendering protected media, so an additional class, FxrWindowManager, is created to manage all windows created for Firefox Reality on Desktop so that it can inform whether or not protected media can be presented.
The automated manual tests in vrhosttest.cpp now show the real shared texture handle rather than a fake value, which shows that marshaling succeeded.
Differential Revision: https://phabricator.services.mozilla.com/D46179
--HG--
extra : moz-landing-system : lando
In bug1583978, we are going to remove `AudibleState::eMaybeAudible`. But before that, as `AudibleState::eMaybeAudible` is only used for delaying media playback, so we can remove its usage from media element and move it to the dedicated class to reduce confusion about how and when we should use `AudibleState::eMaybeAudible` or `AudibleState::eNotAudible`.
Differential Revision: https://phabricator.services.mozilla.com/D47153
--HG--
extra : moz-landing-system : lando
Separate the logic of delaying media playback from `HTMLMediaElement`'s `mAudioChannelWrapper` to a new class, because delaying media playback is a special usage of using `AudioChannelAgent`.
It would make the code cleaner and separate the normal usage and the special usage, to reduce confusion.
We usually start `AudioChannelAgent` when media playback starts and stop it when media playback stops, however, in this case, we would start the agent even if media hasn't be started yet, because we would like to use the agent as a connection with `AudioChannelService` in order to receive the resume command when we're able to play delayed playback.
Differential Revision: https://phabricator.services.mozilla.com/D44746
--HG--
extra : moz-landing-system : lando
This change is a continuation of Part 1 (Bug 1570128), where the 2D content rendered by Firefox for Firefox Reality on Desktop is marshalled through VRHost so that it can be presented in a VR environment.
A new class, FxrOutputHandler, is created to manage creating a sharable texture, sharing it through VRShMem, and updating it when content updates. This class updates content with both WebRender and conventional rendering output.
This initial iteration of FxrOutputHandler does not have synchronization between reading and writing this shared texture across processes. A subsequent fix (Bug 1581881) is pending, which will reuse WebVR code to manage writing to and reading from a pool of textures.
This also presents issues with rendering protected media, so an additional class, FxrWindowManager, is created to manage all windows created for Firefox Reality on Desktop so that it can inform whether or not protected media can be presented.
The automated manual tests in vrhosttest.cpp now show the real shared texture handle rather than a fake value, which shows that marshaling succeeded.
Differential Revision: https://phabricator.services.mozilla.com/D46179
--HG--
extra : moz-landing-system : lando
After applying patch4, now we would pull the change from `AudioChannelService` everytime after starting the agent, so we don't need to rely on letting `NotifyStartedPlaying()` to modify the config and use it to update our state.
Differential Revision: https://phabricator.services.mozilla.com/D45753
--HG--
extra : moz-landing-system : lando
As we start audio capturing explicitly, we should also take responsibility to stop audio capturing when we don't need it.
We were hiding too many details on `AudioChannelAgent` before, which allow us hard to know who and where we handle audio capturing.
Differential Revision: https://phabricator.services.mozilla.com/D45752
--HG--
extra : moz-landing-system : lando
Instead of calling those callback functions seperately, we could provide a function to pull those changes at once after starting the agent.
In addition, `WindowXXXChanged` are callback functions of `nsIAudioChannelAgentCallback`, so they should only be called by `AudioChannelAgent`, to indicate receiving the change from `AudioChannelService`. We should not call them directly.
Differential Revision: https://phabricator.services.mozilla.com/D45751
--HG--
extra : moz-landing-system : lando
`nsIAudioChannelAgent` was created a common interface for a usage of in both js and C++ before, now we have no any JS code would use `nsIAudioChannelAgent`, it's only used in C++ code.
Therefore, in a coming refactoring (bug1580662), we will remove `nsIAudioChannelAgent` and use `AudioChannelAgent` as the only interface. Here we can make these classes start to reference `AudioChannelAgent`, instead of `nsIAudioChannelAgent`.
Differential Revision: https://phabricator.services.mozilla.com/D45748
--HG--
extra : moz-landing-system : lando
GetCurrentPhysicalThread and GetCurrentVirtualThread are, in practice,
identical, as the TLS override that GetCurrentVirtualThread depends on
is never actually set. This simply removes that and renames some things/
deletes some comments.
Differential Revision: https://phabricator.services.mozilla.com/D41247
--HG--
extra : moz-landing-system : lando
After applying patch4, now we would pull the change from `AudioChannelService` everytime after starting the agent, so we don't need to rely on letting `NotifyStartedPlaying()` to modify the config and use it to update our state.
Differential Revision: https://phabricator.services.mozilla.com/D45753
--HG--
extra : moz-landing-system : lando
As we start audio capturing explicitly, we should also take responsibility to stop audio capturing when we don't need it.
We were hiding too many details on `AudioChannelAgent` before, which allow us hard to know who and where we handle audio capturing.
Differential Revision: https://phabricator.services.mozilla.com/D45752
--HG--
extra : moz-landing-system : lando
Instead of calling those callback functions seperately, we could provide a function to pull those changes at once after starting the agent.
In addition, `WindowXXXChanged` are callback functions of `nsIAudioChannelAgentCallback`, so they should only be called by `AudioChannelAgent`, to indicate receiving the change from `AudioChannelService`. We should not call them directly.
Differential Revision: https://phabricator.services.mozilla.com/D45751
--HG--
extra : moz-landing-system : lando
`nsIAudioChannelAgent` was created a common interface for a usage of in both js and C++ before, now we have no any JS code would use `nsIAudioChannelAgent`, it's only used in C++ code.
Therefore, in a coming refactoring (bug1580662), we will remove `nsIAudioChannelAgent` and use `AudioChannelAgent` as the only interface. Here we can make these classes start to reference `AudioChannelAgent`, instead of `nsIAudioChannelAgent`.
Differential Revision: https://phabricator.services.mozilla.com/D45748
--HG--
extra : moz-landing-system : lando
Our implementation modified the state, in addition to throwing the exception.
And the tests not only didn't test for this, but wouldn't have reached that
code anyway, due to the harness giving up as soon as anything failed.
Differential Revision: https://phabricator.services.mozilla.com/D46106
--HG--
extra : moz-landing-system : lando
Also while doing it:
* Ensure activity observers get notified after visibility is computed already.
This is how we notify all other activity observers already, and we are
double-notifying in the case we actually get a page show _and_ a visibility
change, but this is a pre-existing problem.
* Remove special-cases for InFrameSwap() from MediaRecorder. Now that pagehide
doesn't mess up with our visibility state the regular check just works. I
ensured I didn't regress bug 1444541.
* Had to fix a UITour test that relied on the visibility changing back and
forth for the detached tab. It seems there's no real place in UITour that
listens to that event so we should be good.
* Added tests, verifying that they both fail without the patch.
After this we can remove nsDocShell::InFrameSwap(), as the only caller is the
assertion, but I wanted to keep it regardless, at least for now, until this
patch has been in for a bit.
Differential Revision: https://phabricator.services.mozilla.com/D45906
--HG--
extra : moz-landing-system : lando
Also while doing it:
* Ensure activity observers get notified after visibility is computed already.
This is how we notify all other activity observers already, and we are
double-notifying in the case we actually get a page show _and_ a visibility
change, but this is a pre-existing problem.
* Remove special-cases for InFrameSwap() from MediaRecorder. Now that pagehide
doesn't mess up with our visibility state the regular check just works. I
ensured I didn't regress bug 1444541.
* Had to fix a UITour test that relied on the visibility changing back and
forth for the detached tab. It seems there's no real place in UITour that
listens to that event so we should be good.
* Added tests, verifying that they both fail without the patch.
After this we can remove nsDocShell::InFrameSwap(), as the only caller is the
assertion, but I wanted to keep it regardless, at least for now, until this
patch has been in for a bit.
Differential Revision: https://phabricator.services.mozilla.com/D45906
--HG--
extra : moz-landing-system : lando
Both of these files obviously use things from the newly-included
headers, but they were not including them prior to this point.
Differential Revision: https://phabricator.services.mozilla.com/D46632
--HG--
extra : moz-landing-system : lando
This replaces the instances where Length was being called on an interval set
to determine if it was empty or not empty. IsEmpty makes the intention in these
cases more immediately obvious, and avoids the need to think about ints as
bools in cases like `if(!mySet.Length())`.
Depends on D46638
Differential Revision: https://phabricator.services.mozilla.com/D46639
--HG--
extra : moz-landing-system : lando
Restyle include guard + includes to conform to new style rules.
Depends on D46637
Differential Revision: https://phabricator.services.mozilla.com/D46638
--HG--
extra : moz-landing-system : lando
This is a quality of life improvement:
- More concise than checking if Length() == 0.
- More readable then if(mySet.Length()) (in my opinion).
- We already have checks that test if IntervalSets are empty by looking at their
length, so there is a use case for this.
Differential Revision: https://phabricator.services.mozilla.com/D46637
--HG--
extra : moz-landing-system : lando
Set a trivial duration (1us) on 0 duration samples before feeding them to the
WMF MFT decoder to prevent it from estimating the duration for such samples.
Differential Revision: https://phabricator.services.mozilla.com/D46516
--HG--
extra : moz-landing-system : lando
This is a back out of changeset c52127007a01, i.e. reverts bug 1560440. That bug
was itself reverting a prior bug where we started setting the duration.
There are two scenarios involved, each with their own issues:
- We don't set the duration when feeding the decoder (what we're doing prior to
this change). In this case the decoder will estimate durations for all
samples. This works in many cases, but leads to issues with files that have
non-uniform durations - as seen in bug 1576990 (this bug).
- We do set the duration when feeding the decoder (what will happen after this
change). In this case the decoder will not estimate durations, except it still
seems to do so when fed 0 duration samples. Since our demuxing pipeline can
create 0 duration samples, this leads to issues with such files (as seen in
bug 1560440).
This patch fixes the former of the above, and the next patch in the chain will
address the latter.
Differential Revision: https://phabricator.services.mozilla.com/D46515
--HG--
extra : moz-landing-system : lando
CubebDeviceEnumerator already knows when an audio device changes. It is enhanced to allow listeners/observers registration and to create notifications when that happens. Also, it is hooked to the existing notification path.
On a minor note, it has been revisited the way the enumerator is touched in MediaEngineWebRTC class.
Differential Revision: https://phabricator.services.mozilla.com/D46271
--HG--
extra : moz-landing-system : lando
DeviceChangeCallback class implements the observer pattern. However, the role of the observer and the subject is integrated into the same class which makes use of virtual methods to allow a separation of the roles. This makes code reading difficult. Also, it does not allow from a class to inherit only the observer role or the subject role. This patch breaks the DeviceChangeCallback class into two classes according to the observer or subject functionality.
Differential Revision: https://phabricator.services.mozilla.com/D46270
--HG--
extra : moz-landing-system : lando
It appears possible for data eviction to take place despite the manager not
currently having any buffered ranges. This shouldn't happen, but if it can if
the buffers have become full with 0 duration data. This patch attempts to
mitigate that case by removing all coded frames. This prevents a bad access on
the buffered ranges and should clear up space in the buffers to append more
data.
Differential Revision: https://phabricator.services.mozilla.com/D44711
--HG--
extra : moz-landing-system : lando
The ideal situation for media control should be incooperated with the audio competition so that we only have to control one controller.
However, if it's multiple controllers at the same time, we would tend to only control the last one in order to reduce confustion.
Ex. If we pause controller A first, and then start controller B. So what should we do when we press `play/pause` hardware key? To resume controller A or to stop controller B, or do both things at the same time?
Differential Revision: https://phabricator.services.mozilla.com/D43315
--HG--
extra : moz-landing-system : lando
In order to receive platform level media hardward keys event, we create a `MediaHardwareKeysEventSource` which is used to implement intercepting those events according to different platforms.
We can add a `MediaHardwareKeysEventListener` onto `MediaHardwareKeysEventSource`, so that we can get notification whenever hardware media keys are being pressed.
`MediaHardwareKeysManager` is used to encapsulate all these details, it would create a source and corresponding listener.
Differential Revision: https://phabricator.services.mozilla.com/D43313
--HG--
extra : moz-landing-system : lando
