Monday, November 29, 2010

qtwebkit-2.1-week47 status report - changes since qtwebkit-2.1-week46

Release Notes for qtwebkit-2.1-week47 - Changes since qtwebkit-2.1-week46
6 bugs, 11 commits

Bugs fixed / Tasks done:
  • #44405: [Qt] Proper workaround for missing Gtk initialization in Adobe's flash plugins...
  • #46810: [Qt] Review the setUserPermission & friends API
  • #49373: [Qt] Add an API for informing the actual visible contents rect to WebCore
  • #49382: Spatial Navigation: issues with the node selection algorithm.
  • #49442: Spatial Navigation: Code cleanup
  • #49787: QtWebKit does not respond properly to QInputMethodEvents with replacement text in them.
Commits added or cherry-picked:
  • e60aee8: WebKit/qt:
  • e645d43: [Qt] Remove synchronous QWebPage::checkPermissions signal
  • 2290254: [Qt] Review the setUserPermission & friends API
  • 68124d7: [Qt] Review the setUserPermission & friends API
  • aa7ca33: [Qt] Review the setUserPermission & friends API
  • df0e5bc: [Qt] Review the setUserPermission & friends API
  • 0f2e0cc: Backport to qtwebkit-2.1 the fixes for #49442 and #49382
  • 85a22e1: Backport fix for #49787 to qtwebkit-2.1
  • 29553ff: Update symbian .def files after recent API changes
  • e922782: Fix regressions introduced by the spatial navigation fixes (#49442 and #49382)
  • 64a2028: Proper workaround for missing Gtk initialization in Adobe's flash plugins.

Monday, November 22, 2010

qtwebkit-2.1-week46 status report - changes since qtwebkit-2.1-week45

Release Notes for qtwebkit-2.1-week46 - Changes since qtwebkit-2.1-week45
5 bugs, 10 commits

Update: the original release notes included bug #49382 in the list of fixes, but the respective commit was reverted before the tag was made and is not part of this release.

Bugs fixed / Tasks done:
  • #34539: Implement NPN_GetValueForURL and NPN_SetValueForURL and provide a stub for NPN_GetAuthenticationInfo
  • #38911: [Qt] Patch to locate Java plugin DLL on Windows via its Registry...
  • #47022: Add proxy server query function proxyServersForURL and change the Mac plug-in code to use it
  • #49718: [Qt] Canvas doesn't animate on OpenVG backend
Commits added or cherry-picked:
  • 4364e90: Fix HTML Canvas not updating properly with VG backend
  • 9d9a127: Revert "Fix HTML Canvas not updating properly with VG backend"
  • 8c6f4ae: [Qt] Search the Windows registry for the location of the Java plugin DLL.
  • 39e824c: Add proxy server query function proxyServersForURL and change the Mac plug-in code to use it
  • a86a9d1: Add Qt ProxyServer stub.
  • 4b6d887: Fix typo.
  • 3c8023c: Implemented NPN_GetValueForURL and NPN_SetValueForURL and NPN_GetAuthenticationInfo.
  • fcda793: Fix for #49718: [Qt] Canvas doesn't animate on OpenVG backend
  • a55b974: Spatial Navigation: issues with the node selection algorithm.
  • 8576340: Revert "Spatial Navigation: issues with the node selection algorithm."

Tuesday, November 16, 2010

qtwebkit-2.1-week45 status report - changes since qtwebkit-2.1-week44

Release Notes for qtwebkit-2.1-week45 - Changes since qtwebkit-2.1-week44
8 bugs, 12 commits

Bugs fixed:
  • #44089: [Qt] Making effective use of Document::nodesFromRect
  • #46336: Make Document::nodesFromRect more flexible
  • #46492: document.nodesFromRect() needs to be removed from Document.idl
  • #46580: [Mac][DRT] Implement LayoutTestController::nodesFromRect
  • #48141: [Qt][QtTestBrowser] Toggle use of QGraphicsView messes up the menu
  • #48198: [Qt] Button's background style from default Qt Mobile Theme is overriding facebook's button style
  • #49372: [Qt] QWebScriptWorld should not be in QtWebKit 2.1
  • #49398: Spatial Navigation: Need better way to control size of content in tests.
Commits cherry-picked:
  • ee16ad6: [Qt][QtTestBrowser] Toggle use of QGraphicsView messes up the menu
  • 24c3199: Build fix (leftover from previous commit)
  • c49afb4: [Mac][DRT] Implement LayoutTestController::nodesFromRect
  • 848c9ed: Make Document::nodesFromRect more flexible
  • 659ae6b: document.nodesFromRect() needs to be removed from Document.idl
  • 3650536: [Qt] Making effective use of nodesFromRect.
  • bf5a319: Revert "[Qt] Prospective build fix for MSVC"
  • f329fc4: Revert "[Qt] Support evaluateScriptInIsolatedWorld()"
  • d625977: Revert "[Qt] Support evaluateScriptInIsolatedWorld()"
  • 9f57391: Tentative update of symbian .def files to remove QWebScriptWorld symbols
  • 1453361: [Qt] Button's background style from default Qt Mobile Theme is overriding facebook's button style
  • 655d83e: Spatial Navigation: Need better way to control size of content in tests.

Monday, November 8, 2010

qtwebkit-2.1-week44 status report - changes since qtwebkit-2.1-week42

Release Notes for qtwebkit-2.1-week44 - Changes since qtwebkit-2.1-week42
14 bugs, 15 commits

Please note that there was no week43 tag/release, exceptionally.

Bugs fixed:
  • #37153: Spatial Navigation: add support to <input type=text> and <textarea>
  • #40641: [Qt] Clicking button input does not give it focus
  • #43987: Downloading using XHR is much slower than before
  • #46814: [Qt] Fix tst_QWebPage::geolocationRequestJS()
  • #47809: [Qt] QtTestBrowser: Prevent calling load() directly from loadFinished() in robot mode.
  • #48079: [Qt] InspectorClientQt crashes when deleting a qwebpage which has an inspector
  • #48319: [Qt] Some unit tests try to use the data:<content-type>,<content> URL syntax with setHtml
  • #48323: [Qt] Assert in the tst_hybridPixmap::hybridPixmap() unit test.
  • #48324: [Qt] Skipping popup focus test for symbian
  • #48432: [Qt] tst_QWebFrame::popupFocus fails.
  • #48450: [Qt] Extend the Platform Plugin to define the padding of HitTestResult
  • #48692: [Qt] QWebPage's InputMethod event handling calls ShortcutOverride handler too
  • #48730: [Qt] QtTestBrowser runs out of memory with most of javascript benchmarks.
  • #48767: [Qt] QtTestBrowser : set mmp rule pageddata in Symbian
Commits cherry-picked:
  • baa0e2d: [Qt] Layout tests for fix focusing of control elements on mouse click.
  • f5d4208: [Qt] Unit tests: setHtml("data:text/html,...") -> load(QUrl("data:text/html,..."))
  • 12362e4: [Qt] Fix an assert with QtPixmapInstance creation.
  • 7e76cf1: [Qt] Fix tst_QWebPage::geolocationRequestJS()
  • f169505: [Qt] Fix tst_QWebFrame::popupFocus().
  • 4d3698b: [Qt] QtTestBrowser: Prevent calling load() directly from loadFinished() in robot mode.
  • a12122e: Fix the gap in the def files that was accidentially introduced in
  • da8bfab: [Qt] InspectorClientQt crashes when deleting a qwebpage which has an inspector
  • dcd6fcd: [Qt][Symbian] QtTestBrowser runs out of memory with most of javascript benchmarks.
  • 0e4d96b: [Qt] QWebPage's InputMethod event handling calls ShortcutOverride handler too
  • 136130c: [Qt] Skipping popup focus test for symbian
  • 68f99d0: [Qt] QtTestBrowser : set mmp rule pageddata in Symbian
  • 6ddf322: Spatial Navigation: add support to <input type=text> and <textarea>
  • 78ff335: Using the Platform Plugin to define the default values for the padding of HitTestResult.
  • 20290f0: Fix for #43874: Downloading using XHR is much slower than before

Thursday, October 28, 2010

notify-me

I find this script (notify-me) most useful when I build webkit :-)

Just 'make && notify-me'. You will get a tray notification when the build is complete. Code is not mine, it's ripped from some site.

#!/usr/bin/env python
"""This is a python 2.5 script that creates a notification using dbus."""
import dbus
item = ('org.freedesktop.Notifications')
path = ('/org/freedesktop/Notifications')
interface = ('org.freedesktop.Notifications')

icon = ''
array = ''
hint = ''
time = 10000 # Use seconds x 1000
app_name = ('Test Application')
title = ('Whatever it is, is done')
body = ('Its all over.')

bus = dbus.SessionBus()
notif = bus.get_object(item, path)
notify = dbus.Interface(notif, interface)
notify.Notify(app_name, 0, icon, title, body, array, hint, time)

Monday, October 25, 2010

Texture Mapper: Accelerated compositing evolved

In the last year I've been developing and maintaining the Qt implementation of Accelerated Compositing. The accelerated compositing code path in WebKit was originally written by Apple, to allow WebKit ports to accelerate animations, transitions and 3D transforms using the GPU. It was originally using CoreAnimations on Mac and on the iPhone, and something similar on Windows. In January 2010 we introduced the Qt implementation, that uses QGraphicsView and the Qt animation framework, which was then followed by the Chromium implementation, for, as you can guess, Chromium.

The Accelerated Compositing concept comes to optimize for cases where an element would be painted to the screen multiple times without its content changing. For example, a menu sliding into the screen, or a static toolbar on top of a video. It does so by creating a scene graph, a tree of objects (graphics layers), which have properties attached to them - transformation matrix, opacity, position, effects etc., and also a notification when the layer's content needs to be re-rendered.

How Qt Accelerated Compositing Works Today
If you want to know the basics about WebKit's graphics rendering, I recommend reading an entry from the Chromium blog entry from a few months ago: (https://sites.google.com/a/chromium.org/dev/developers/design-documents/gpu-accelerated-compositing-in-chrome).

Our architecture right now is slightly simpler than Chromium, since we don't use their cross process architecture. The idea right now is that each graphics layer is backed by a QGraphicsItem and a QPixmap, and the graphics scene is responsible to know which items to draw, who clips who, apply effects etc. The hardware acceleration is achieved by QtOpenGL, which knows how to accelerate repeated painting of QPixmaps, by caching them into OpenGL textures.

Why this was a good idea, and why it needs to change
Since the Qt APIs are so easy to use, getting Accelerated Compositing running on top of was quite simple. We were up and running with this approach within days, and the feature became stable rather quickly. We learned a lot about how to / how not to do this right during this time, and this wouldn't be possible without the API.

But now, after having more and more people use this feature and push its limits, the approach is showing its limitations. For example, getting 3D transformations to work correctly on top of the 2D graphics view was a pain, and it still doesn't look exactly right because support depth buffering, and isn't as fast as it can be, due to the software 3D->2D projection. Also mask and reflection effects are difficult to optimize, and texture uploads are a bottleneck. But the main problem is that there are too many abstractions - CSS goes to webkit, which goes to , then QPixmap, QPainter, the GL paint engine, and then OpenGL and the driver. Too many assumptions are being made along the way, that are not always correct in the context of the application.

Introducing Texture Mapper
The idea was to remove most of the abstractions, and go from GraphicsLayer directly to OpenGL, making the code that traverses the scene-graph and draws the texture-backed layers a cross-platform WebKit implementation that can be also used by other ports. The original code did exactly that, walked through the tree and issued GL calls, saving a texture per graphics layer. But the need for some abstraction layer came about - some GPUs require different optimizations then others, some have DirectFB and not OpenGL, some can benefit from EGL texture sharing and other proprietary extensions, and we still want this to work on pure software if hardware acceleration is not available.

The current implementation uses an abstraction called TextureMapper: it's a couple of thin abstract classes, one is a BitmapTexture and one is the TextureMapper. A BitmapTexture is equivalent to a GL texture or a QPixmap. Like QPixmap is gives you API to paint into it with software via a GraphicsContext, but it also allows you to use it as the render target for hardware accelerated drawing, like an FBO. This is important when using effects or opacity, for example. To render masks or opacity correctly on an element that has several descendant elements, the sub-tree has to be rendered into an intermediate buffer before the effect is applied. With QGraphicsEffect this is only possible using software, because of the limitations imposed by the many abstraction layers.

The TextureMapper class is equivalent to a QPainter or a GraphicsContext, but it's very limited in scope - you can bind a BitmapTexture to be the current rendering target, set the clip, and draw BitmapTextures.

Abstraction layers are usually a mistake
Abstraction layers are something I'm usually very careful about - people implement things from both sides of the layer, and you become tied to concepts you invented before the scope of the problem was completely clear. TextureMapper is an abstraction layer - which holds the same perils - but I chose to do it here anyway because of the other constraints I had.

To reduce the risks of having an abstraction layer, I limited its scope. The main limitation is that this abstraction layer is not public API, and is not pluggable. That means that, at least for now, the header files for that abstraction might change, as long as all the implementations inside webkit trunk can live with that change. So the abstraction layer just helps us separate parts of the code, it's not an "API".

Acknowledgements

This endeavor became real thanks to Sam Magnuson from Netflix, who did a lot of the work with me, and helped test and optimize it with a real-world application on TV hardware. also it was his idea and I stole it :)

Working across companies in the context of an open source project made a huge difference - they got better user experience for their application, and we got a QtWebkit feature that's functioning much better in a productized context.


Status
This code is now in webkit trunk (as of changeset 70487). To enable it, build webkit trunk with Qt (see http://trac.webkit.org/wiki/QtWebKit), adding the texmap CONFIG option. For example:

WebKit/WebKitTools/Scripts/build-webkit --qt --qmakearg="CONFIG+=texmap"

Most of the code resides under WebCore/platform/graphics/texmap, and the OpenGL implementation is under WebCore/platform/graphics/opengl.