Wednesday, 4 February 2009

WebKit-gtk, libsoup, cURL and midori

It seems that webkit-gtk developers would like to see libsoup as the default backend in the gtk port, whereas the current setting (which is also used in the Fedora WebKit package) is the cURL backend. And since I have finally gathered enough free space to be able to build webkit, I decided I give it a go. In the following table I've summarized the differences between the two backends (libsoup backend with r40306 selfbuilt and cURL backend with r40351, grabbed from koji) I've, as a user, noticed (click on the images to get the large version).

Effects on Browser UI ResponsivenesI haven't noticed any effect on UI responsiveness when loading a web page. The spinner is spinning with constant speed, I can freely switch between tabs, create new tabs, ...I noticed significant effects on the UI responsivenes when loading pages. Especially when loading more pages at once, the UI tends to become unresponsive for a while. Also the spinner animation is not smooth and at times it even stops completely.
Web AuthenticationAuthentication seems to work pretty good, still there seems to be no way how to use ssh certificate to authenticate to kojiAuthentication does not work at all.
FTP Protocol Handling If you try to access FTP you're straight away told that the site does not exists. I'd expect either a message that it cannot handle FTP protocol or, even better, automagically opening the requested url in file browser (which should know how to handle ftp)CRASH! Although in the previous built I've been using it was giving, after an unreasonably long wait, listing, without any links or formating, of the requested ftp directory.
AniDB (probably perl handling) As you can see it gives a similar output to what you get by running e.g. cat /bin/ls. I suspect that the cause is that this site is made in perl. Works just as expected.

To me, who uses midori (with webkit-gtk backend) as a secondary browser, the option with libsoup sounds better - I can use it to authenticate to fedorahosted, also the UI is much more responsive; and the things that do not work here, I can do with epiphany (still with gecko backend).

Now, I'd share a few words about Midori. It's a pretty interesting browser developed primarilly for XFCE users that is using WebKit-gtk backend. One of its interesting features (among the almost classical basic session management and saving, configurable search engines and google search implemented in address bar) is an option to reopen closed tab. It also has pretty clean and simple UI and I find working with it to be rather pleasant. And on top of that, thanks to webkit backend, it is pretty fast and from my experience generally faster than gecko based browsers (firefox, epiphany). Finally, I like its name - midori (緑) is a japanese for green/greenery.

And to end this post I'd like to write a few features (some of which are being already in progress) I am still missing and some bugs in webkit-gtk that prevents me using it as a primary browser backend:
  • Reasonable handling of downloads. It just tells you that document cannot be displayed. Right-click, download works in midori (if you set a download manager in midori settings), but it works the same way as with copy link + wget, which means when the link is actually a html page or some script that forwards you to the real source, it fails...
  • As I said earlier, lack to authenticate with ssh certificate
  • Inability to upload files - the UI for it works very well, just like in firefox, but when you hit the upload button, the website complains nothing was set to be uploaded.
  • Occasional crashes when closing a tab.
  • It seems it is not able to keep coockies between browser sessions (i.e. I have to login to web pages every time I restart midori)
  • It does not remember my user name at sites like fedoraforum, even when I check "Remember me"
  • Sometimes sloppy widget focus - at times cursor is blinking in edit fields but nothing can be written until I switch focus elsewhere and back...

And to answer the obvious question you might have, nope, I have not yet filled bug reports about the things I mentioned above, but will definitelly do so sooner or later (though actually for some of the above mentioned things there already are opened bugreports in webkit bugzilla). Anyway, I wish webkit bright future and hope that the gtk port will soon get to the point where it would make sense to use it in epiphany by default.

Also, to end it on a positive note, among the things that get better since the last time I've built webkit there is improved scrolling which is now really smooth (in past it felt snatchy, especially when one tried to scroll fast) and finally working opening new windows via javascript (though midori forces it to open rather in new tabs which is IMHO really cool). Also, I can finally watch youtube videos (among other things) in webkit as well (in past swfdec just failed to load the necessary files, probably thanks to some bug or missing feature in webkit).


npmccallum said...

Don't forget that the libsoup backend will use libproxy, so you get proxy goodness too!

Martin said...

npmccallum, that sounds great!

Yanko said...

There is something awfully wrong with font fallback in current rawhide WebKit-gtk.

$ /usr/libexec/WebKit/GtkLauncher

or repeat that with strace -eopen

Martin said...

Yanko, I cannot try on Rawhide, but the same SVN revision of WebKit (r40351) works on F10 (the package is also built with same configure options). I don't understand chinese, but the page you referenced seems to load fine...

Yanko said...

You should try without chinese fonts installed

Christoph said...

Thanks for this article, Martin. I'm going to meet with Christian (the Midori dev) at FOSDEM tomorrow and I think I will show him your posting. Maybe he can tell me what we can expect from Midori in the future.

Casey said...

Not sure what's involved in getting cert auth to work. ftp isn't webkit's job necessarily. As a programmer you can tell webkit that for certain URIs you'd like to have webkit simply hand them to some other function in your program, so call that a midori bug (though 404 isn't the right error, and that may be webkit).

And AniDB being in perl should affect nothing. Its a server-side language. The content delivered to the browser should not be different for the fact that a perl script wrote it. This is, however, likely indicative of how webkit/libsoup handle some special header that anidb uses.

patrys said...

As for AniDB - I'd suspect it uses inflate or gzip to compress the data during transfer and Webkit either fails to check the header or simply doesn't handle content compression yet.

Christian Hergert said...

I think the UI responsiveness issue with cURL is similar to what I've been dealing with in Marina[1]. It seems that before webkit_web_view_execute_script(), webkit_web_view_open(), or webkit_web_view_load_html_string() return, there is a call to getaddrinfo() via the libcurl backend. And of course, getaddrinfo() tends to block on a dns query. So if you happen to have really shitty DNS (or slow due to bandwidth) there is a good chance you will starve the main loop.

Thanks for the excellent write up.


-- Christian

Lalit Arora said...

Hi Guys,

I am new the webkit world. Need ur help.

I hv successfully complied WebKit-r41944.

But GtkLauncher is not opening default URL: ""

On checking configure file I fould enable http_backend option was not available

please let me know how can I configure libsoup as http_backend

Anonymous said...

Just for reference, F-11 uses libsoup by default.

Anonymous said...

As a heavy-duty libcurl user, I'm confident the libcurl issues could be resolved, if someone were committed to making it work well.

For responsiveness, use the curl multi interface with your choice of I/O multiplexing functions (select, epoll, etc.). If used properly, it will never block on I/O.

libcurl absolutely handles authentication and ftp! Again this must be a webkit issue.

For DNS resolution, build libcurl with c-ares (a truly async resolver library).

Finally, libcurl is much faster than libsoup. If you assume webkit is being used over relatively slow links, you probably wouldn't care. But if you want it to work well for high-throughput LAN and datacenter tasks, choose carefully.