GFX2: widget/gfx rewrite

"GFX2" provides what is currently provided by both gfx and widget (including timers).

Problems:

Graphics

Windows specific apis, hacks, etc. (make this better)

Performance

Due to some of the current APIs, it is hard for us to use the operating system for many things that it can be used for.

Blending/Alpha compositing

???

Clipping

We currently do far too much clipping. This causes numerous performance problems, especially on X11 based systems. Due to the way X works, changing the clip region makes the X server have to flush its connections to make sure nothing has changed since it received the clip region. (this isn't a completely accurate description, change it)

Images

Memory size
Currently, we hold around at least two copies of the image data around in memory. One compressed and one decompressed. As part of GFX2 and imagelib 1.5, we hope to only hold a single copy of the image data around.
Scaling and dithering
Both image scaling and dithering currently is done in both imagelib and gfx depending on your platform and the code path that layout uses to draw the image. This causes confusion and does not allow the platform to optimize the scaling and dithering operations.
Gamma
??

twips

twips are the devices independent units that we use so that printing, etc can print and look the same, but they are not needed and cause performance hits due to constant float to integer conversions and back. They also provide confusion with some APIs taking twips and others pixels.

SVG (scaleable vector graphics)

The current graphics APIs that we have do not provide support for rendering vector graphic. There is no clean way to add these drawing primitives to the current gfx APIs.

Widgets

Obsolete APIs (designed for when we had native widgets) (make this better)

Events

The way operating systems handle events are often quite different. This creates a problem with the current way we handle events. Engineers are forced to spend large amounts of time tracking down bugs that are caused by event ordering, and with no solid specification on how we should sent events, this problem will continue. (make this better)

Native form controls

There is good bit of code left in the current widget module that is no longer used or needed. These interfaces and code should be removed. Should the need for "native" form controls arise again, XBL should be able to do the job of making native looking and feeling form controls.

Proposal:

Graphics

merge widget and gfx in to a single component

Since both widget and gfx deal with the native OS, drawing, and events, there is no reason for them to be separate components.

floating point pixel coordinate system

Remove twips.
With the CPUs used in modern hardware, we can replace twips with floating point pixel based coordinates and dimensions. Using floating point will account for the rounding problems that using twips aims to fix as well as reduce the need to convert to and from twips constantly. For systems without a FPU (StrongArm, etc), Scott Collins has volunteered to write a fixed-point class that we could simply plug in instead of floating points. Devices such as printers would be able to use a transformation matrix to scale the pixel values so that they match the DPI that you are printing to.

SVG (scaleable vector graphics)

Interfaces

SVG related drawing methods will be put on their own interface, something like nsIVectorDrawable, which you can QI to and from an nsIDrawable. An XP implementation that does its own rasterization and delegates everything else to an underlying GFX implimentation is needed to do this.

Cross-platform Solutions

Fonts
FreeType is a high-quality font rendering engine capible of rendering anti-aliased fonts as well as rotating them.(blah blah blah)
Drawing
libart is a library designed to render curves, vectors, do alpha compositing, etc. Unfortunatly the author no longer has much time to work on it, so we would need someone here to work on it and integrate it with our system. (blah blah blah)

Clipping

Less of it

Clipping should be done only when it is absolutly needed. (more stuff)

Mandatory vs. discretionary clip regions (potentially)

add some of the things roc and I have discussed.

Images

Imglib1.5

Tim Rowley is working on imglib 1.5. The new image library in conjunction with the new imaging APIs in GFX2, we should be able to reduce the number of copies of the image data to one.
Scaling and dithering
As part of imglib 1.5, Tim will be removing imglib's image scaling and dithering code. All image data will come from imglib as 24bit image data.
Each GFX2 implementation must be able to scale an image using either native system APIs or another means. Having each platform do this allows platforms that do support scaling and/or dithering to take advantage of any hardware optimizations that they can.
nsIDrawable provides separate methods for drawing an image and drawing a scaled image to avoid any confusion about how to draw an image.
Platforms that do not support scaling or dithering could probably port gdkrgb and/or gdk_pixbuf to their platform to provide these services in software for them.
RGBA packed image data
???
Gamma
???

Widgets

Events

As part of GFX2, the events generated by the system will have to conform to a specification that will be written up by Stuart Parmenter, Tom Pixley, Chris Saari, and others. This will ensure that all events received by listeners to windows will receive them in a cross platform manner and help to remove numerous problems that we currently have due to this. A test listener suite should be made that validates a platform implementation to ensure that it meets the specification perfectly.

Native form controls

XBL can be used to do this work with less overhead and problems than using real native widgets.
Internet Explorer on widows does not use win32 native form controls.
Currently there is no need for native form controls, but this could change. The APIs have been designed so that native form controls and other native widgets may be added later without disrupting the current set of GFX2 APIs.

System services

Clipboard

The clipboard interfaces remain mostly the same. There has been discussion about making the getData methods asynchronous, but nothing has been finalized at this time.

Drag and Drop

At the time of this writing, the drag and drop interfaces have not been looked at, but they are not expected to change a great deal.

File Picker

The file picker interface has not changed. There has been talk of changing the way filtering works a bit, but this has not been finalized at this time.

System look and feel

Currently, we have two interfaces, nsILookAndFeel and nsIDeviceContext that can both be asked for information about CSS fonts, colors, etc. These have been merged in to a single nsISystemLook interface that contains all of this information.

Interfaces

Documentation

The GFX2 interfaces are full documented using doxygen and JavaDoc style comments.
See the full list of interfaces and information.

Questions

Is GFX2 thread safe?

The implementations are probably not, however an attempt has been made to make the interfaces reentrant (with noted exceptions such as nsIClipboard)

Will GFX2 break existing platforms such as BeOS and OS/2?

No

An adaptation layer will be made that will convert calls made to the new GFX2 APIs in to the old APIs allowing current implementations to continue working until they are ported.

Should GFX2 interfaces use XPCOM?

Pro

Scriptability
Having the interfaces be scriptable makes writing test cases for them much easier. To help people writing new implementations and porting to new platforms, having a test suite that tests all of their code will make a huge difference. We can also use scriptability to do things like check event ordering from windows, etc.
Run time switching
Having GFX2 use XPCOM interfaces allows us to change toolkits at runtime. This may not seem like a big win, but imagine us shipping an Xlib based browser for Unix systems and someone wants to run using NanoX. They could use our Unix builds and simply switch out their GFX2 implementation for their own without having to build the whole product specifically for them.

Con

Speed
Due to the cost of virtual functions, it will be a bit slower than if we did not use XPCOM for GFX2. If the GFX2 library was linked to other libraries, the compiler/linker might be able to further optimize the code paths.

Overall

Should we use it?
After spending time looking at the usage patterns of GFX through layout, using XPCOM should provide only an insignificant speed hit, while providing us with scriptability and run time switching of implementations.
Time would be better spent removing XPCOM interfaces by focusing on interfaces such as nsISupportsArray that are used constantly throughout the entire code base.

Is GFX2 Y2K compliant?

maybe

comments/questions: email Stuart Parmenter