Reproducing the layout test results on the Qt bots

The Szeged hackers did it, once again :)

http://webkit.sed.hu/blog/20101028/qtwebkit-builder-and-tester-virtual-machine

notify-me

I find this script (notify-me) most useful when I build webkit :-)

Just 'make && notify-me'. You will get a tray notification when the build is complete. Code is not mine, it's ripped from some site.

#!/usr/bin/env python
"""This is a python 2.5 script that creates a notification using dbus."""
import dbus
item = ('org.freedesktop.Notifications')
path = ('/org/freedesktop/Notifications')
interface = ('org.freedesktop.Notifications')

icon = ''
array = ''
hint = ''
time = 10000 # Use seconds x 1000
app_name = ('Test Application')
title = ('Whatever it is, is done')
body = ('Its all over.')

bus = dbus.SessionBus()
notif = bus.get_object(item, path)
notify = dbus.Interface(notif, interface)
notify.Notify(app_name, 0, icon, title, body, array, hint, time)

Texture Mapper: Accelerated compositing evolved

In the last year I've been developing and maintaining the Qt implementation of Accelerated Compositing. The accelerated compositing code path in WebKit was originally written by Apple, to allow WebKit ports to accelerate animations, transitions and 3D transforms using the GPU. It was originally using CoreAnimations on Mac and on the iPhone, and something similar on Windows. In January 2010 we introduced the Qt implementation, that uses QGraphicsView and the Qt animation framework, which was then followed by the Chromium implementation, for, as you can guess, Chromium.

The Accelerated Compositing concept comes to optimize for cases where an element would be painted to the screen multiple times without its content changing. For example, a menu sliding into the screen, or a static toolbar on top of a video. It does so by creating a scene graph, a tree of objects (graphics layers), which have properties attached to them - transformation matrix, opacity, position, effects etc., and also a notification when the layer's content needs to be re-rendered.

How Qt Accelerated Compositing Works Today

If you want to know the basics about WebKit's graphics rendering, I recommend reading an entry from the Chromium blog entry from a few months ago: (https://sites.google.com/a/chromium.org/dev/developers/design-documents/gpu-accelerated-compositing-in-chrome).

Our architecture right now is slightly simpler than Chromium, since we don't use their cross process architecture. The idea right now is that each graphics layer is backed by a QGraphicsItem and a QPixmap, and the graphics scene is responsible to know which items to draw, who clips who, apply effects etc. The hardware acceleration is achieved by QtOpenGL, which knows how to accelerate repeated painting of QPixmaps, by caching them into OpenGL textures.

Why this was a good idea, and why it needs to change

Since the Qt APIs are so easy to use, getting Accelerated Compositing running on top of

QGraphicsView

was quite simple. We were up and running with this approach within days, and the feature became stable rather quickly. We learned a lot about how to / how not to do this right during this time, and this wouldn't be possible without the

QGraphicsView

API.

But now, after having more and more people use this feature and push its limits, the

QGraphicsView

approach is showing its limitations. For example, getting 3D transformations to work correctly on top of the 2D graphics view was a pain, and it still doesn't look exactly right because

QGraphicsView

support depth buffering, and isn't as fast as it can be, due to the software 3D->2D projection. Also mask and reflection effects are difficult to optimize, and texture uploads are a bottleneck. But the main problem is that there are too many abstractions - CSS goes to webkit, which goes to

QGraphicsView

, then QPixmap, QPainter, the GL paint engine, and then OpenGL and the driver. Too many assumptions are being made along the way, that are not always correct in the context of the application.

Introducing Texture Mapper

The idea was to remove most of the abstractions, and go from GraphicsLayer directly to OpenGL, making the code that traverses the scene-graph and draws the texture-backed layers a cross-platform WebKit implementation that can be also used by other ports. The original code did exactly that, walked through the tree and issued GL calls, saving a texture per graphics layer. But the need for some abstraction layer came about - some GPUs require different optimizations then others, some have DirectFB and not OpenGL, some can benefit from EGL texture sharing and other proprietary extensions, and we still want this to work on pure software if hardware acceleration is not available.

The current implementation uses an abstraction called TextureMapper: it's a couple of thin abstract classes, one is a BitmapTexture and one is the TextureMapper. A BitmapTexture is equivalent to a GL texture or a QPixmap. Like QPixmap is gives you API to paint into it with software via a GraphicsContext, but it also allows you to use it as the render target for hardware accelerated drawing, like an FBO. This is important when using effects or opacity, for example. To render masks or opacity correctly on an element that has several descendant elements, the sub-tree has to be rendered into an intermediate buffer before the effect is applied. With QGraphicsEffect this is only possible using software, because of the limitations imposed by the many abstraction layers.

The TextureMapper class is equivalent to a QPainter or a GraphicsContext, but it's very limited in scope - you can bind a BitmapTexture to be the current rendering target, set the clip, and draw BitmapTextures.

Abstraction layers are usually a mistake

Abstraction layers are something I'm usually very careful about - people implement things from both sides of the layer, and you become tied to concepts you invented before the scope of the problem was completely clear. TextureMapper is an abstraction layer - which holds the same perils - but I chose to do it here anyway because of the other constraints I had.

To reduce the risks of having an abstraction layer, I limited its scope. The main limitation is that this abstraction layer is not public API, and is not pluggable. That means that, at least for now, the header files for that abstraction might change, as long as all the implementations inside webkit trunk can live with that change. So the abstraction layer just helps us separate parts of the code, it's not an "API".

Acknowledgements

This endeavor became real thanks to Sam Magnuson from Netflix, who did a lot of the work with me, and helped test and optimize it with a real-world application on TV hardware. also it was his idea and I stole it :)

Working across companies in the context of an open source project made a huge difference - they got better user experience for their application, and we got a QtWebkit feature that's functioning much better in a productized context.

Status

This code is now in webkit trunk (as of changeset 70487). To enable it, build webkit trunk with Qt (see http://trac.webkit.org/wiki/QtWebKit), adding the texmap CONFIG option. For example:

WebKit/WebKitTools/Scripts/build-webkit --qt --qmakearg="CONFIG+=texmap"

Most of the code resides under WebCore/platform/graphics/texmap, and the OpenGL implementation is under WebCore/platform/graphics/opengl.

qtwebkit-2.1-week42 status report - changes since qtwebkit-2.1-week41

Release Notes for qtwebkit-2.1-week42 - Changes since qtwebkit-2.1-week41

8 bugs, 12 commits

Bugs fixed:

• #39625: [Qt] Sending a QInputMethodEvent::Selection event forces the Editor to go into Composition mode
• #41480: [Qt][Symbian] Variable max heap size between target/emulator
• #43827: [Qt] Unable to load pages on QtTestBrowser after canceling a page load.
• #45363: [Qt] Crash when showing Flash content at staples.com...
• #46186: [Qt] window.close() doesn't work in qt
• #46616: [Qt] 32-bit Flash crashes on repeated SetWindow calls
• #47545: [Qt] Right clicking on Flash in windowless mode crashes
• #47845: QWebPage::ViewportAttributes class is not exported.

Commits cherry-picked:

• 42e90ac: [Qt] window.close() doesn't work in qt
• 00ea9ca: Winscw build fix
• ec96dfd: Fix winscw cast when returning bool on methods which return Value
• d333d7a: Updated the def file for winscw
• 1be7a0e: Clean up the winscw changes
• 80daa72: [Qt][Symbian] Variable max heap size between target/emulator
• 230afa2: [QT] Unable to load pages on QtTestBrowser after canceling a page load.
• 23549dc: [Qt] Export QWebPage::ViewportAttributes class.
• 1994999: Rubber-stamped by Kenneth Rohde Christiansen.
• 5afae7a: [Qt] Re-enable single-NPP_SetWindow quirk for 64-bit
• e273313: [Qt] Windowless mode 64-bit Flash/X11 freezes when right-click is sent.
• 1d1745e: [Qt] Sending a QInputMethodEvent::Selection event forces the

qtwebkit-2.1-week41 status report - changes since qtwebkit-2.1-week40

Release Notes for qtwebkit-2.1-week41 - Changes since qtwebkit-2.1-week40

5 bugs, 6 commits
(slow week due to Qt Dev Days)

Bugs fixed:

• #43484: [Qt] Setting wmode to "opaque" is not necessary for Symbian
• #46287: [Qt] Enable Netscape plugin metadata caching on Linux should not be set in QWebSettings::enablePersistentStorage
• #47426: [Qt] Editing commands should not be executed on non-editable content.
• #47540: [Qt] Update .def file for Symbian
• #47613: [Qt] QtTestBrowser shows two Url input fields

Commits cherry-picked:

• 42a55d8: 2010-09-16 Antonio Gomes
• 1551146: 2010-10-12 David Leong
• a5266b9: 2010-10-12 Yael Aharon
• e90c2b2: 2010-10-13 Dinu Jacob
• e36bbba: 2010-10-08 Hui Huang
• 995f39c: 2010-10-13 Yi Shen

qtwebkit-2.1-week40 status report - changes since qtwebkit-2.1-week39

Release Notes for qtwebkit-2.1-week40 - Changes since qtwebkit-2.1-week39

13 bugs, 19 commits

Bugs fixed:

• #40598: [Qt] QtWebkit Crashes on loading CelticKane Standard tests
• #46182: [Qt] On Maemo platform, web style is missing for some inputs
• #46347: [Qt] Remove setDeviceSize methods
• #46461: [Qt] New input style for Qt Mobile theme
• #46755: Viewport data change notification
• #46896: Spatial Navigation: select element does not release focus with Spatial Navigation
• #46901: Spatial Navigation: Press key "down" after page load should bring the focus to the 1st focusable element
• #46920: [Qt] New buttons style for Qt Mobile theme
• #46984: [Qt] Segmentation fault during zoom out
• #46993: Spatial Navigation: Add support for
• #47172: [Qt]Windowless flash plugin is not rendered in Symbian
• #47268: Renaming WebCore::ViewportConfiguration to WebCore::ViewportAttributes
• #47325: [Qt] API: ViewportConfiguration => ViewportArguments

Commits cherry-picked:

• 0ddd163: [Qt] Fix platform plugin support after r68128
• 1254672: 2010-10-01 Viatcheslav Ostapenko
• e6e21be: 2010-09-28 Luiz Agostini
• cb60b0d: 2010-10-04 Yael Aharon
• c99c5e7: 2010-10-04 Chang Shu
• 22653f2: QtWebkit API 2.1 cleanup.
• 867aa08: 2010-10-04 Yael Aharon
• 5587a3d: Fix RVCT 2.2 def files on Symbian. viewportConfigurationForSize incorrect.
• 3d45725: [Qt] On Maemo platform, web style is missing for some inputs https://bugs.webkit.org/show_bug.cgi?id=46182
• fd20b0a: 2010-09-27 Ragner Magalhaes
• fb11250: 2010-10-01 Ragner Magalhaes
• 1d5ee2c: 2010-10-01 Ragner Magalhaes
• b2d63cf: [Qt] Remove setDeviceSize methods
• 69b269b: Unreviewed build fix.
• 330d6aa: Build fix due to unsuccessful merge (b2d63cf)
• 88b5ab2: [Qt] Re-enable the web inspector for maemo6 builds
• f188916: 2010-10-06 Luiz Agostini
• 15761d2: 2010-10-07 Andreas Kling
• 009aaea: Windowless flash plugin is not rendered in Symbian https://bugs.webkit.org/show_bug.cgi?id=47172

AC for plugins

I have been away from WebKit development for a while. There were some pressing matters in Maemo6 to attend to.

One of the things I am working on is the Flash support for Maemo6. And I am finding that the abstractions of QGV, meegotouch, Qt/WebKit are "adding up" resulting in much slower fps than using just plain Qt.

Some time back, Simon explained to me the concept of Accelerate Compositing. WebKit has no AC support for NPAPI plugins but by doing so we can remove the Qt/WebKit overhead when updating frames. Also, it allows us to implement Flash transparency correctly in QGV.

So, yesterday evening, I started hacking on Accelerated Composition (AC) for NPAPI plugins. With help from Noam, I am happy to report that Flash is rendering with AC :-) Expect it to land sometime next week. Here's a screenie with the fps.



(And oh, that's the first screenshot in this blog :)

UPDATE: You can find the initial code at http://gitorious.org/~girish/webkit/girishs-webkit/commits/plugins_ac_35524. https://bugs.webkit.org/show_bug.cgi?id=35524 is the bugzilla entry.

Cut your link time in half when building in Scratchbox/arm

... by passing --no-svg to build-webkit :)

I'm down to 1 minute 50 seconds from about 5 minutes. What's your link time? :)

Any other ideas how to improve the linking?

Problems with perl when building WebKit on Windows (even for Symbian)?

Seen this error message before?



python C:/dev/webkit/WebCore/html/parser/create-html-entity-table -o generated/HTMLEntityTable.cpp C:/dev/webkit/WebCore/html/parser/HTMLEntityNames.in
perl C:/dev/webkit/WebCore/make-hash-tools.pl generated C:/dev/webkit/WebCore/html/DocTypeStrings.gperf
syntax error at C:/dev/webkit/WebCore/make-hash-tools.pl line 149, near "} continue"
syntax error at C:/dev/webkit/WebCore/make-hash-tools.pl line 151, near "}"
Execution of C:/dev/webkit/WebCore/make-hash-tools.pl aborted due to compilation errors.
make: *** [generated\DocTypeStrings.cpp] Error 9
make: Leaving directory `C:/dev/webkit/WebCore'
Failed to generate WebCore's derived sources!


The reason is most likely that the files in your working directory - including the perl script(s) - are checked out with Windows line-endings and your perl version doesn't like it. The build of WebKit works most reliably with Unix linefeeds, so here is how you can convert your existing checkout:

git config core.autocrlf false
del .git\index
git read-tree HEAD
git checkout -f

qtwebkit-2.1-week40 status report - changes since qtwebkit-2.1-week38

Release Notes for qtwebkit-2.1-week40 - Changes since qtwebkit-2.1-week38

7 bugs, 9 commits

Bugs fixed:

• #45509: REGRESSION: r61215 broke web views with transparent backgrounds
• #46385: Keep viewport information in Document
• #46617: [Qt] tst_QWebFrame::popupFocus() randomly fail on MeeGo handset because the focus is not set on the window
• #46618: [Qt] : Enable local rendering on Maemo6
• #46706: [Qt] tst_QWebPage::testStopScheduledPageRefresh() fails on MeeGo handset
• #46730: [Qt] Clean up QWebPage::ViewportConfiguration API
• #46812: [Qt] Crash if an scene with accelerated compositing layout during the paint event

Commits cherry-picked:

• cddf898: 2010-09-27 Benjamin Poulain
• eda6059: 2010-09-27 Girish Ramakrishnan
• 90ea290: 2010-09-24 Luiz Agostini
• d407fb8: 2010-08-18 Laszlo Gombos
• 37291a9: 2010-09-28 Benjamin Poulain
• 9615fdd: 2010-09-28 Andreas Kling
• bf85131: 2010-09-09 Simon Fraser
• 13e454d: 2010-09-29 Andreas Kling
• 35e5a3e: 2010-09-30 Benjamin Poulain