We were marking them used even if only a decode was requested.
This can cause us to hold extra decoded copies of the image around because we have a tendency to request decode at the intrinsic size.
Calls to do_QueryInterface to a base class can be replaced by a static
cast, which is faster.
Differential Revision: https://phabricator.services.mozilla.com/D7224
--HG--
extra : moz-landing-system : lando
If class A is derived from class B, then an instance of class A can be
converted to B via a static cast, so a slower QI is not needed.
Differential Revision: https://phabricator.services.mozilla.com/D6861
--HG--
extra : moz-landing-system : lando
The lack of clarity over which functions initiate observing and which don't
is a headache since it makes it hard to reason about what's going on. This
rename makes it explicit in the function names.
Differential Revision: https://phabricator.services.mozilla.com/D7187
--HG--
extra : rebase_source : 1f2f86423a9bee7843533c09b3ea78416b233bcd
extra : amend_source : a89125d6a3b7b75a4056c4d600de74a5386ac4ff
If we do not pass the high quality scaling flag than the resulting surface will be marked as cannot substitute, which is not accurate, so we don't want.
The only place that actually tries to be smart about the size is nsImageFrame::MaybeDecodeForPredictedSize. All other cases just ask for the intrinsic size.
The two most likely cases are that there are no decoded copies of the image, or there is one decoded (or in progress) copy of the image.
In the first case we will request decode at the instrinsic size, and then if we draw at a different size that draw will request the proper size. This doesn't change with this patch.
In the second case there is a decoded copy already available, this is likely from a draw call on the image, and that is the surface size that we want. So we save a decode. If we are actually drawing the image at two different sizes the second size will be slightly delayed, but we have the wrongly sized copy of the image that we can draw until then. This seems like a good tradeoff to avoid always decoding an instrinic size copy of images.
By delegating responsibility for shared surfaces reporting to imagelib,
we can cross reference the GPU shared surfaces cache with the local
surface cache in a content process (or the main process). This will
allow us to identify entries that are in the GPU cache but not in the
content/main process cache, and aid in debugging memory leaks. This
functionality is pref'd off by default behind image.mem.debug-reporting.
Additionally, we want to report every entry that was mapped into the
compositor process, in the compositor process memory report. This will
give us a sense of how much of our resident memory is consumed by mapped
images in absence of the more detailed cross referencing above.
At present, surface providers roll up all of their individual surfaces
into a single reporting unit. Specifically this means animated image
frames are all reported as a block. This patch removes that
consolidation and reports every frame as its own SurfaceMemoryReport.
This is important because each frame may have its own external image ID,
and we want to cross reference that with what we expect from the GPU
shared surfaces cache.
By delegating responsibility for shared surfaces reporting to imagelib,
we can cross reference the GPU shared surfaces cache with the local
surface cache in a content process (or the main process). This will
allow us to identify entries that are in the GPU cache but not in the
content/main process cache, and aid in debugging memory leaks. This
functionality is pref'd off by default behind image.mem.debug-reporting.
Additionally, we want to report every entry that was mapped into the
compositor process, in the compositor process memory report. This will
give us a sense of how much of our resident memory is consumed by mapped
images in absence of the more detailed cross referencing above.
At present, surface providers roll up all of their individual surfaces
into a single reporting unit. Specifically this means animated image
frames are all reported as a block. This patch removes that
consolidation and reports every frame as its own SurfaceMemoryReport.
This is important because each frame may have its own external image ID,
and we want to cross reference that with what we expect from the GPU
shared surfaces cache.
If FLAG_HIGH_QUALITY_SCALING is used, we should use
SurfaceCache::LookupBestMatch just like how it is done in RasterImage.
This may provide an alternative size at which we should rasterize the
SVG instead of the requested size. Since SurfaceCache imposes a maximum
size for which it will permit rasterized SVGs, we should also bypass the
cache entirely if we are well above that and simply draw directly to the
draw target in such cases.
With WebRender, it is somewhat more complicated. We will now return
NOT_SUPPORTED if the size is too big, and this should trigger fallback
to blob images. This should only produce drawing commands for the
relevant region and save us the high cost of rasterized a very large
surface on the main thread, which at the same time, looking as crisp as
a user would expect.
There is one main difference between raster images and vector images
with respect to factor of 2 scaling. Vector images may be scaled
infinitely and so we need to extend factor of 2 scaling to permit
growing instead of just shrinking. Also, we don't want to scale
infinitely, so we should configure a maximum size limit. This size limit
will apply even outside of factor of 2 scaling, and so the caller
(VectorImage) will need to be careful to take this into account.
If FLAG_HIGH_QUALITY_SCALING is used, we should use
SurfaceCache::LookupBestMatch just like how it is done in RasterImage.
This may provide an alternative size at which we should rasterize the
SVG instead of the requested size. Since SurfaceCache imposes a maximum
size for which it will permit rasterized SVGs, we should also bypass the
cache entirely if we are well above that and simply draw directly to the
draw target in such cases.
With WebRender, it is somewhat more complicated. We will now return
NOT_SUPPORTED if the size is too big, and this should trigger fallback
to blob images. This should only produce drawing commands for the
relevant region and save us the high cost of rasterized a very large
surface on the main thread, which at the same time, looking as crisp as
a user would expect.
There is one main difference between raster images and vector images
with respect to factor of 2 scaling. Vector images may be scaled
infinitely and so we need to extend factor of 2 scaling to permit
growing instead of just shrinking. Also, we don't want to scale
infinitely, so we should configure a maximum size limit. This size limit
will apply even outside of factor of 2 scaling, and so the caller
(VectorImage) will need to be careful to take this into account.
DecoderFlags::BLEND_ANIMATION will cause the decoder to inject the
BlendAnimationFilter from the previous patch into the SurfacePipe filter
chain. All frames produced by this decoder will be complete, and
should be equivalent to the result outputted by FrameAnimator.
This new SurfaceFilter can be added to a SurfacePipe to perform the
blending of a previous frame with the current partial frame, for an
animated image. This functionality is currently provided by
FrameAnimator and must be performed each time we want to advance the
displayed frame, all on the main thread. Moving this to SurfacePipe
allows us to do the same operation once per frame decode, and on a
decoder thread.
This should reduce the cost of a refresh tick since advancing animated
images is reduced to merely checking if the frame is available. Also, if
the image is below the discard frames threshold (to save memory), then
we will also save CPU due to only blending once at decode.
DecoderFlags::BLEND_ANIMATION will cause the decoder to inject the
BlendAnimationFilter from the previous patch into the SurfacePipe filter
chain. All frames produced by this decoder will be complete, and
should be equivalent to the result outputted by FrameAnimator.
This new SurfaceFilter can be added to a SurfacePipe to perform the
blending of a previous frame with the current partial frame, for an
animated image. This functionality is currently provided by
FrameAnimator and must be performed each time we want to advance the
displayed frame, all on the main thread. Moving this to SurfacePipe
allows us to do the same operation once per frame decode, and on a
decoder thread.
This should reduce the cost of a refresh tick since advancing animated
images is reduced to merely checking if the frame is available. Also, if
the image is below the discard frames threshold (to save memory), then
we will also save CPU due to only blending once at decode.
When generating display lists for WebRender, we were not caching the
draw result via nsDisplayItemGenericImageGeometry::UpdateDrawResult (or
similar) after completing CreateWebRenderCommands. This is important
because reftests use this to force sync decoding for images; it may be a
reason for image-related intermittent failures on *-qr builds.
Additionally, we may have been requesting fallback in cases where fallback
could not do anything more than WebRender could. For example, if we can't
get an image container yet, there is no point in requesting fallback
because it might just be we haven't started decoding yet. We should just
return the actual draw result in such cases.
In addition to the image container, the draw result can also be useful
for callers to know whether or not the surface(s) in the container are
fully decoded or not. This is used in subsequent parts to avoid
flickering in some cases.
