Duetto: A faster and smarter alternative to Emscripten. And more.
Posted by Alessandro Pignotti in Leaningtech on April 16, 2013
We have seen a lot of hype on the Web recently after the announcement of Asm.js by Mozilla: a new “extraordinarily optimizable, low-level subset of JavaScript”. The system builds on the work that has been done on Emscripten: a LLVM-based solution which compiles C++ to JavaScript, allowing for an easy port of applications and games to the Web. The excitement for Asm.js stems from the fact that, by using a special virtual machine integrated in Firefox, it can improve the performance of emscripten-generated code and get it even closer to native performance.
We (Leaning Technologies Ltd) would like to introduce Duetto, our own LLVM-based solution for programming the Web using C++. And by the Web, we mean both the client and server side of it, but let’s talk about the client side first.
Emscripten handles C++ code by emulating a full byte-addressable address space. This is definitely a good solution, but suboptimal. Javascript is not based on a byte-addressable address model, but on an object-addressable model: all the accessible memory is contained in some object. But when you think about it, C++ is not that different.
Our solution integrates with clang and the LLVM toolchain and is able to map C++ object-oriented constructs to native JavaScript objects. It turns out that accessing objects on modern JavaScript engines is faster than accessing arrays. By using this (and a few more) tricks we managed to get the following, preliminary, results on micro benchmarks.

For each benchmark the best time in 10 runs has been selected. The V8 and Spidermonky JavaScript shells has been used. The respective commits are b13921fa78ce7d7a94ce74f6198db79e075a2e03 and b9d56a1e0a61. *The fasta benchmark has been modified by removing a memory
We managed to do this by realizing that by disallowing some unsafe C++ capabilities (such as type unsafe pointer casting and pointer arithmetics inside structures) it’s actually possible to create more efficient, smaller and faster JavaScript code from C++. Interestingly enough, we discovered that, in most cases, the needed limitations on the language are actually specified as undefined behaviour!
So, yes, Duetto does need some minimal porting to bring C++ code to the Web while emscripten makes it mostly free. What you get in exchange for that is faster performance with no need of a special VM and deep integration with the browser. Duetto creates a really seamless C++ programming experience for the Web:
- Seamless integration with the browser environment, complete access to the DOM and HTML5 technologies including WebGL. You can even access and use your favourite JavaScript library or existing JavaScript from C++ by declaring the available interfaces in the C++ code using a simple convention.
- Seamless client/server programming, using transparent RPCs in single codebase. The compiler will split the code automatically in the client part (compiled to JavaScript) and server part (compiled to native code).
The Duetto backend is already in a very advanced state, and we believe it’s already suitable to bring the first applications to the Web. Especially games, which are our primary target. Unfortunately our front end is not yet as polished as we would like, as we want to improve the error reporting to make the porting experience as smooth as possible.
We are not yet ready to release Duetto, but we are eager to start opening collaborations, so if you are interested in bringing your C++ application or game to the web, feel free to contact me (alessandro@leaningtech.com). We believe that in six months or less from now we will be able to release a robust product, most probably capable of generating even faster code. And we want to release it as open source.
For more information please visit our site: http://www.leaningtech.com
A tale of hidden symbols, weakness and gold
Posted by Alessandro Pignotti in Coding tricks on March 8, 2013
A few days ago I was profiling the startup time of the clang compiler. The callgrind report highlighted that a relatively high amount of time was being spent in the _dl_lookop_symbol_x. A quick gdb inspection of the call stack quickly pointed out that the dynamic linker was spending the time doing dynamic relocations. Tons of them.
Using objdump -R clang I found out that a whopping 42% of them actually contained the word “clang” in the mangled symbol names. Demangling the names made it clear that they were mostly definitions of the C++ virtual tables for clang internal classes.
Basically 42% of the relocations happening at run time were actually looking for symbols which are obviously defined inside the clang executable itself. So what’s happening here?
Turns out the problem is a poor interaction between the traditional linker (ld, from the binutils package) and C++‘s ODR rule.
ODR stands for One Definition Rule and says that any entity in C++ code must be defined only once. This is obviously trivial for methods, since if they are defined twice, even in different translation units, there will be an error at link time. There are, though, a few things that are implicity defined by the compiler, for example the virtual tables, which are defined whenever a class using virtual methods is declared. (Here I’m speaking informally, I suspect the terminology is not 100% correct) Since the vtable is potentially defined in more than a translation unit, the symbols for the vtable are flagged as weak. Weak symbols will not conflict with each other and they will be all discarded but one (any one is fine). The surviving one will be used by the end product of the linker.
Unfortunately, the compiler does not know if the compiled object file will be used as part of a dynamic library or as part of a main executable. This means that it has to treat the vtable symbols like any library method, since (if the target is a dynamic library) potentially such symbols may be overridden by another definition in a library or in the main executable. This has to happen since the ODR rule must also apply across the library borders for proper support of a few things, especially exception handling.
So at compile time there is no way around using dynamic relocation on the vtable symbols. A possible workaroud would be to compile all the code with the -fvisibility=hidden flag. Unfortunately this is actually wrong since it may break the ODR rule!
At link time the linker has the chance of eliminating the dynamic relocation, since it does know if the target is a main executable and symbols defined in an executable cannot be overridden by anything (not even by LD_PRELOADed libraries). Unfortunately the traditional ld linker does not apply this optimization.
The new gold linker, originally developed at Google, does tough! It is able to completely eradicate the dynamic relocations to internally defined symbols, effectively reducing the load time.
Moral of the tale: use the gold linker. It should work in most cases (I think kernel code is a notable exception) and generate faster executables while consuming less memory and cpu time during linking.
And please, dear debian/ubuntu maintainers, link clang using gold.
Linux support for Asus Xonar U1 USB audio device
Posted by Alessandro Pignotti in Uncategorized on January 15, 2013
I got as a Christmas gift an external USB audio device: an Asus Xonar U1. It’s a nice device with a decent audio quality. The audio itself, both in and out works perfectly with the standard snd-usb-audio kernel module. Unfortunately the audio controls on the device are not standard, so I developed a simple daemon and accompanying udev and pm-utils script to get the device to work. I’ve published everything under the GPL hoping that they might be useful for someone else. Feel free to clone and fork my repository on github
Lightspark 0.7.0 released
Posted by Alessandro Pignotti in Lightspark on October 28, 2012
Lightspark 0.7.0 has been released, including several months of improvements and bug fixes.
Version 0.7.0:
* Support LZMA compressed SWFs [Requires liblzma]
* Improved BitmapData support
* Improved ActionScript compatibility
* Improved virtual machine performance and memory consumption
* Improved XML support
* Experimental support for bytecode optimization at runtime
* Improved ExternalInterface (browser integration) support
* Improved performance of JPEG loading
* Support for XMLSocket
* Completely redesigned and improved masking support
You can grab a copy of the sources from launchpad as usual.
I would also like to ask for information on a specific problem the project is facing. We are encountering more sites that tries to load library code in the form of SWZ files, which seems to be undocumented (probably obfuscated/encrypted) flash files. Any information on how to parse/load them is appreciated.
Lightspark 0.6.0.1 released
Posted by Alessandro Pignotti in Uncategorized on June 11, 2012
I’m very happy to annouce a new major release for Lightspark, the open source flash player implementation. This release includes quite a lot of fixes, both internal and visible to users. The most important ones from a user point of view are probably the improved support for PulseAudio flat volumes, which makes it impossible for lightspark to manipulate the system volume and the newly added support for the BBC video player. To use BBC site you might need to use AdBlock, which is generally recommended since more often than not flash advertisements are not supported by Lightspark and causes failures in otherwise supported sites.
From the ChangeLog:
* Enable RTMP support by default, requires librtmp
* Fixed support for IEventDispatcher implementation pattern
* Improved serialization robustness
* Improved matrix handling
* Implement string and namespace pooling to reduce memory consumption
* Proper support for private namespaces
* Improved support for fonts
* Support LLVM 3.1
* Fix full volume issue when PulseAudio flat volumes are enabled
* Initial support for AIR desktop applications
* Support for www.bbc.co.uk video player
Source tarball is, as usual available from Launchpad.
Lightspark 100% volume issue is now fixed
Posted by Alessandro Pignotti in Lightspark on May 27, 2012
For some time I’ve received a few reports about lightspark raising the system volume to 100% (and “killing ears”) whenever a YouTube clip was started. It took some time for me to figure out what was going on, since I was not able to reproduce the issue on my system and of course the code was not willingly touching the system volume.
It turns out that the issue was caused by the recently introduced “flat-volume” support in PulseAudio. When flat volume is enabled the volume of a stream is absolute and not relative to the system volume. Since the default volume in flash is 100% by spec then Lightspark was unwillingly setting the system volume to an extremely high value, and I’m really sorry for that.
I’ve just committed a fix in git master that should fix this problem for good by properly virtualizing the volume seen by flash and scaling it by the system volume. I hope this helps people with flat volume enabled.
Lightspark 0.5.7 released
Posted by Alessandro Pignotti in Lightspark on May 16, 2012
A new version of Lightspark has been released yesterday. You can give it a try by getting the source code from launchpad. Ubuntu packages should be available shortly from our PPA
Beside a lot of small improvements this new release improves the graphics capabilities, with a focus on Flash features used by games.
Here is the changelog:
* Fixed a few memory leaks
* Improved support for BitmapData::draw
* Support for BitmapData::copyPixels
* Support for soft masking
* Support for memory usage profiling (massif compatible)
Moreover, there has been some work on implementing fully accelerated Stage3D support, it’s still in early design phase... but stay tuned.
Lightspark 0.5.6 released
Posted by Alessandro Pignotti in Lightspark on April 12, 2012
After a few months of great progresses under the stewardship of Jani Monoses Lightspark, the modern and open source flash player implementation is now again being maintained by me. And I’m very proud to announce the release of lightspark 0.5.6, that brings quite a few new features and a usual load of stability fixes. Source code is available of course from Launchpad. Packages should be shortly available for various distros. I will also try to get an updated Windows build in the next few days.
Here are the new features, from both 0.5.5 and 0.5.6, since I was too lazy to publish the previous release. My bad, it will not happen again.
- Fix YouTube support for several videos
- Support for custom serialization/deserialization
- Support RPC (NetConnection::call)
- Support for PNG images
- Experimental support for Google Street View
- Support for Firefox 10
As you can see experimental support for Google Street View is now available (kudos to Antti Ajanki for this). Moreover, the RPC and serialization support has been written in an ongoing effort by me to support FarmVille and similar highly interactive and complex flash games. We are definitely expanding the scope of what is considered supported, but there are still some unfinished corners in YouTube support, and many other video players do not work yet.
This is way I would like to encourage young hackers, especially students, that are interested in contributing to a complex project like lightspark to choose their favorite site using flash to show videos and drop by the #lightspark IRC channel on Freenode. Me and the other lightspark developers will provide the guidance you may need in the first steps. Most often it just a matter of implementing or improving a few Flash APIs to see a player start working. And you favorite site (and your name) may be posted here in the next Lightspark release!
Gnome shell extension: GTG in the calendar panel
Posted by Luca Invernizzi in gtg, planetgnome on December 24, 2011
from today, you can keep an eye on your tasks with style.
Here's the pic:
Here you can install it, here you can get the source code and report bugs.
It works only with GTG 0.3 and above, that will be released in March. You can also use Trunk, which is quite stable (all you have to do is bzr branch lp:gtg && cd gtg && ./gtg).
One final thing: this is the first time I develop a gnome-shell extension, and I have to say I was impressed by how nice and easy is to code. Kudos Shell people!
Happy holidays!
Lightspark 0.5.0 — Bacchus released
Posted by Alessandro Pignotti in Lightspark on July 26, 2011
After the last round of bug fixing and improvements over the release candidate I’m happy to announce that version 0.5.0 of Lightspark, a FOSS flash player implementation aimed at supporting newer flash files (SWF 9+), is finally released.
Beside stability improvements there is no new great new feature since the last released candidate (see here for more info). You can grab your copy of the sources on Launchpad as usual. Packages for both Ubuntu natty and oneiric are available from our PPA. Packages for your favorite distro will be probably available in a short time. I’d also like to say “thank you” to all downstream packagers for the critical work they have always done.
As always bug reporting and testing is very appreciated. If you need any help you can drop into the #lightspark channel on Freenode.

