Duetto (C++ for the Web) 0.9.2 is out — OpenGL ES implementation in WebGL and toolchain improvements
A lot of progress was made here at Leaningtech during the last few weeks, leading to our latest release of duetto — version 0.9.2.
For this version we have been focusing mainly on usability. The most relevant updates include:
A WebGL-based OpenGL ES implementation
Automatic linking of system libraries;
General toolchain usability improvement, including simplified command line syntax for common operations.
Duetto’s OpenGL ES implementation exposes the standard API available on desktop and mobile to Web developers. We have designed it to make porting of existing GLES apps and games easier, but it is of course possible to use it in new code. Moreover, we expose the underlying WebGL context to make it possible to mix and match between GLES and WebGL code, to take advantage of the native capabilities of the browser like using JPEG and PNG compressed images as textures. This GLES implementation is used in this Nontetris game, developed in C++ using duetto. You can find its source code here.
The last two points are aimed at a general effort of increasing the ease of use of duetto — and reducing the likelihood of the most frequent errors related to missing links to libraries.
We have started an open wiki here. On the wiki you can already find installation instructions for the various platforms, build instructions and the getting started guide. Feel also free to contribute to the wiki if you feel something is missing.
We are also happy to announce that we will be speaking at the mloc.js conference in February in Budapest. We look forward to it as a great chance to meet some awesome people working on JS as a compiler target and discuss about our technology.
Yesterday, Alon Zakai (alias kripken, creator of Emscripten) wrote an excellent post comparing three C++ to JS compilers: Mandreel, Emscripten and Duetto. We would like to provide a few more details regarding Duetto as we feel that, although the comparison was definitely fair, we can add some points which have been missed.
Memory model: Contrarily to Mandreel and Emscripten, which emulate a traditional flat address space using a singleton typed array, Duetto maps C++ objects to JS objects. This naturally implies an overhead both in terms of meta-data that the JS engine manages and in terms of computational work that the Garbage Collector (GC) needs to do. While these may look like major disadvantages, other factors should be taken into account. The computational overhead does not go wasted, as it is functional to keep memory usage in check and make sure that the amount of occupied memory is closer to the amount actually required.
In addition, the typed array based memory model incurs in some overhead as well, as memory areas are still managed using some kind of malloc. And they also suffer a lot from fragmentation as the browser can’t reclaim memory in the middle of the array which is not currently being used.
We strongly believe that the duetto memory model being based on JS objects is more fair to other applications, both native and web-based and to the user’s system as a whole. If the system gets tight on memory, it is possible for the browser to reclaim currently unused objects used in code generated by duetto, on the other hand when a large typed array is used, the system can at best page-out unused or seldom used pages to swap since it has no way to know what memory is actually used and what is not. On the other hand it is definitely true that frequently allocating and deallocating objects may cause an unacceptable slowdown, but this is actually true for native programs as well and the solution is to re-use the same objects over and over (for example by creating pools) at the application level, and this is something that is, of course, possible using duetto as well. Generally we believe that using JS objects also make duetto compiled code more similar to handwritten JS code, which is already handled very well by JS engines, with the added advantage of guaranteed type immutability which can be exploited by the engine to generate faster code. Still, duetto is flexible enough that it will be possible, in the future, to support using an emscripten like memory model on a type-by-type basis. This would be useful to adapt to various workloads and for compatibility with code compiled by emscripten.
APIs: Duetto is built around the concept that the browser is a platform having JS as the machine language. Such platform has APIs like WebGL, HTML5 Canvas, DOM events and so on which are the lowest level accessible functionalities. With Duetto you can use all of them directly from C++ and no JS wrapper code is required.
What about the cross platform APIs used on native systems like SDL and OpenAL? Well, we don’t currently support them, but they can be ported to Duetto by writing a new backend for the browser platform, similarly to what has been already done to port them to Windows, Linux and Mac. And using duetto it is possible to write such porting code directly in C++ because you have direct access to the browser APIs. Once the cross platform APIs support duetto, applications using them will run as well without much (or any) work.
And what about GLES? Well, we received so many requests for this functionality that we are working on providing a WebGL based GLES implementation ourselves. We plan to release it very soon. With that it should be possible to build programs using GLES with duetto by only changing the windowing system initialization code, similarly to porting from windows’ WGL to X11 glX.
C++ compatibility: Alon correctly states that duetto has “partial” C++ compatibility. This is true, but given the incredible amount of capabilities that the C++ language has, it is important to clarify what these limitations are.
Let’s start from the basis: our C++ compiler was of course not written from scratch, but was based on clang. This means that we inherit complete support for all C++ features, including recent and advanced ones like C++11, template support and lambda. And we actually use such features in the C++ code which manages transparent client/server RPCs.
The limitations are more about a few specific unsupported capabilities, currently the major ones are: missing support for virtual inheritance/virtual base classes, some issues with pointer comparison in a couple of corner cases and missing support for some parts of the standard library. But we would like to stress that those limitations are (mostly) non structural and we will do our best to increase the compatibility as much as possible while still be coherent with C++ philosophy of exposing all platform capabilities and limitations to the user.
Performance: We have worked with computer systems long enough to all agree that micro-benchmarks have little to do with performance on real world scenarios. We also know that Duetto still produces suboptimal code for many common cases that we plan to fix. We look forward to try and profile Duetto generated code from a large, real world code base, to find out how it performs and what needs to be optimized.
In summary, we think that Alon’s review of our technology is pretty accurate, and we look forward making more and more of Duetto’s features known to the community, with dedicated posts and documentation. We have been impressed by the feedback we got with our first release — and look forward to hear your opinion.
Duetto release 0.9.1 is now available as source tarballs and binaries for Ubuntu/Debian (PPA) and Windows. MacOSX builds will be available soon. Feel free to report bugs, and request features here.
Today Leaning Technologies Ltd. releases duetto, a compiler designed to develop integrated (backend and frontend) web applications in C++. duetto is now available to the public as FOSS here, and will be offered in a commercial package with closed-source-friendly licensing soon.
Browser-based applications will undoubtedly play a big role in the future of both consumer and business applications. Web apps have some clear advantages compared to native applications, like being natively cloud-based and collaborative, while also being inherently portable, especially thanks to the increased attention that browser vendors have dedicated to web standard compliancy. Modern web standards, that usually go under the umbrella term HTML5, provide incredible close-to-native capabilities and this gap will be shrinking with time.
Duetto combines the advantages of emscripten and node.js by allowing the programmer to:
write web applications in C++, reusing existing code and making porting of whole applications and games to the browser plausible.
code both the frontend and the backend of a web application in the same language and codebase
In addition to this, duetto provides some nice features:
Bring the robustness and proven scalability of C++ programming to the Web
You can access all browser APIs directly. Duetto inherits the C++ philosophy of exposing the platform capabilities (and limitations) to the users. There is no middle man.
Duetto is based on LLVM/clang. An industry standard C++ compiler is a programmer’s best friend: code sanity is verified as compile time, including RPC signatures.
Contrarily to emscripten we do not try to emulate a traditional address space using typed arrays, but directly map C++ objects to JS objects. This reduces memory consumption since the garbage collector can delete unused object.
Are you interested in using duetto? You can now. We have just released the whole technology as a FOSS project, available here. Also, we’ve stuck to our promise of releasing in six months from our first announcement in April
The compiler itself, derived from LLVM/clang is released with the same license as LLVM/clang (UI/NCSA) and we plan to contribute generally useful code upstream. Accompanying headers and libraries will be released as GPLv2+. We will be also offering closed source friendly licensing of such components for a fee. Here are links to the various repositories on github:
duetto enabled LLVM
duetto enabled clang
duetto headers and utilities
Release tarballs will be available shortly on the Launchpad page of the project: https://launchpad.net/duetto. Launchpad will also be our bug reporting system, so please use it to report any problem you might find. We will be also providing builds of duetto for Ubuntu linux using a PPA. We plan to provide binary builds for Mac OS X and Windows as well, but we are still working on automating those build.
Please subscribe to our Launchpad group/mailing list to discuss bugs, desired features and usability issues on https://launchpad.net/~duetto-users. You can also follow us on twitter (@leaningtech) and visit our website to be updated on duetto related news:
Duetto — our C++ compiler for the Web — is getting close to its first public release, due in a few days, and we would like to spend a few lines to present and explain our licensing model, as many people have been asking recently.
When we started working on duetto around a year ago, we knew that we were going to release it as a FOSS project. We believe that duetto, by enabling truly seamless client/server C++ programming, can become a corner-stone of Web applications and games development. But this can only happen if an ecosystem of libraries, extensions and frameworks will be built around duetto itself. We know that the only way for this to happen is to make the core of our technology available to everyone, for free. In addition to that, since duetto is heavily based on the LLVM/clang framework and could not have existed otherwise, we really feel that giving back to the community is the most fair thing to do.
At the same time, we have also always been determined to build a company around this project. For this reason, we will be offering a premium package that includes the same core FOSS technology plus personalized support, proprietary friendly licensing and auxiliary perks including, but not limited to, integration with development environments.
Let’s now talk licenses. The duetto compiler itself is derived from LLVM and clang, both released under the University of Illinois/NCSA Open Source License. We are releasing our duetto compiler under the same license, both to make it easier for the community to contribute with code, to contribute back with some of our code to LLVM and clang, and because it does not make much sense to use a dual licensing scheme on a compiler.
In addition to the compiler, there are a bunch of headers and libraries which are essentials to develop using duetto. All those will be released under a classic dual licensing scheme: GPLv2 for everyone and a proprietary non-copyleft license for paying customers.
To summarize, here is how duetto licensing will work:
duetto core compiler: University of Illinois/NCSA Open Source License (like the LLVM and clang projects)
duetto core libraries/headers: dual license GPLv2; proprietary non-copyleft license available for a fee
duetto premium features: proprietary and closed source, available for a fee
We think this choice represents a good balance between being open and giving back to the open source community, and building a viable business model. We hope the open source community will appreciate our choice!
Well, if you ask me, that is the most relevant feature of C++. I don’t see it as a limitation, but as a framework that helps structuring my ideas into machine compatible form. Over time I have grown to appreciate that the C++ compiler is a programmer’s best friend.
More recently I’ve also become fascinated with the incredible progress that browsers vendors have made in bringing standard-compliant modern capabilities to the general user. While HTML5 canvas and WebGL are definitely the most visible additions, we should not forget about local storage, programmatic audio support, video support, web sockets and everything that is being standardized. This extensive feature set is amazing and, to me, this depicts a future where almost-native applications and games are available on every single platform through browsers.
So we have some pretty awesome Web APIs, tightly coupled in a programming environment that, to me, looks like a joke. Since that sounds like a problem to me, and I like to solve problems , I decided to bring my language of choice, C++, to the Web platform.
My solution was to write duetto, a C++ compiler for the Web. With a couple of friends, we are trying to build a company around it.
What we want to do is to provide a better programming language for the Web, to take the most advantage of all the fantastic underlying technologies such as HTML5 and CSS, which do their job more than well. With duetto, designers can still work using tools they know, while developers can write a sound application logic in robust C++ code. And, by the way, we are not trying to provide an half-backed widget library that would always be lagging behind what browser APIs can provide. With duetto you access the browser directly, with no need of any middle layer.
One of the intended uses of duetto is to port existing C++ applications, especially games, to the Web. But we also want to make it a primary choice to write new web apps. One of the requirements of modern apps is continuous communication with remote servers, for example to receive other users’ updates. This is why we have also introduced an experimental support for transparent RPC to server side code.
Calling this RPC is actually kind of limiting. Our objective is to provide a seamless experience to make it possible to write client and server side code in the same codebase and let the compiler generate the two sides of compiled code. Take a look at some example code.
The release date for duetto is now getting really close and we are really excited about shipping it and see what people can do with this technology. If you want to take a preview please visit http://leaningtech.com, there are still a few slots open in our extended beta program.
 Maybe I like solving problems too much, I wrote Lightspark because Adobe’s Flash Player was too slow in rendering full screen videos of kittens on my workstation
Well folks, we are very happy about the success of our limited beta campaign having already filled up the 100 slots we have offered for our first public beta. We decided to raise the threshold to up to 200 beta testers. Apply here:
and you will receive a link to our most recent build.
Moreover, take a look at the incredibly awesome demo that our friend Daniele Di Proietto made for us using an earlier beta. It is an implementation of the “Not Tetris” concept. And just to be clear, it’s fully implemented in C++ using direct access to WebGL. For more info, see the demo page.
For more info take a look at our original blog post here.
Did we ever tell you that writing a full-feature compiler is not a job for the faint of heart? During the last couple of months we have been marching forward in our quest to bring a great, seamless C++ programming experience for the Web. And we see victory at the end of our journey!
We are aware, of course, that there are already other solutions, such as Emscripten, to solve this problem. What makes Duetto different is that it is a C++ compiler with a C++ mindset.
We have worked long and hard to polish Duetto and we are now almost satisfied by its capabilities and robustness. We are now ready to share this tool with the first 100 developers that are going to share their interest, in order to collect the most feedback and provide the best possible support and experience.
Interested in trying out Duetto for free? Visit http://leaningtech.com to get your copy.
If you’re not one of the 100 lucky beta testers, do not despair. We will be releasing Duetto in roughly a month. Just enough time to hammer out some bugs, add some more awesomeness and figure out last details on our licensing model. Duetto will be released under a dual licensing scheme, so it will be an open source/free software project, available for anyone. We will also be offering paid commercial licenses for people which are not comfortable with free software libraries and/or want enterprise-grade support.
UPDATE: I forgot to link the incredibly awesome demo by Daniele Di Proietto. He used Duetto to port his own implementation of the “Not tetris” concept to the Web.
At LeaningTech we have been working hard and it’s time for some news and updates on the development of duetto, our tool for programming the Web in C++.
The compilation of bullet in JS with duetto requires very minimal adaptation to the vanilla code, and can be easily performed in full integration with CMake infrastructure. We built a minimal demo to show the result of our tool in action: a triangle falling on the ground attracted by gravity. Please note that the demo is completely not optimized, not even by minimizing the JS. Any performance must be considered preliminary and there is vast room for improvement that we plan to fully exploit
This was our first trial at compiling a complex codebase, and it went surprisingly smoothly!
The following features are now fully supported in duetto:
- A fully working libc and libc++
Our current beta features a full implementation of the standard C library (based on the newlib implementation) and of the standard C++ library (based on the LLVM/libc++). Both libraries are standard implementations, support for duetto was included using the same infrastructure used for other, native, targets.
- Integration in the CMake/autotool toolchain as a target
Duetto is now fully integrated with the CMake, using the infrastructure normally used for cross compiling (i.e toolchain definition files). This means you can compile complex projects that use CMake with little effort. We also plan to integrate with autotools in the future.
- Full support of the DOM and HTML5 APIs (such as WebGL)
Our latest beta features an improved support for using the DOM and WebGL API from C++. This is done by defining a set of headers which allows to transparently access all the browser DOM and libraries. It is also very easy to add support for any JS libraries, following the same approach.
Four months have passed since our initial post about duetto. At that time we promised what we would have released our brainchild in a six months time frame. Well, I’m very happy to say that we are fully in schedule and we will be able to release duetto in the fall, under a dual licensing scheme: open-source for open-source and non-commercial use, and a paid license for commercial use.
We have already started sharing a limited private beta with a few developers and are interested in expanding its release to some more, if interested please contact us at email@example.com
Around a month ago we posted a first overview of Duetto: our integrated LLVM-based solution for programming both the client and the server side of Web applications using C++. We have been completely overwhelmed by the interest generated by that post!
We tried to keep track of the many insightful comments and constructive criticism we received on this blog and on aggregators. Here are the most common questions that arose:
1) Why didn’t you compare Duetto with asm.js?
We believe that comparing to asm.js-enabled code is not an apples-to-apples comparison. In the current state the Firefox asm.js Ahead-Of-Time compiler can only be enabled on code which is generated using the emscripten approach to memory allocation, so code generated by Duetto has no way to benefit from it. We are open to discuss how the asm.js approach of validating typed-ness of code AOT can be extended so that Duetto, and potentially other solutions, may benefit from it as well.
Still, it is clearly very interesting to compare Duetto performance to asm.js, so we have made a new round of benchmarking that includes it. The results are pretty interesting. As you can see, although asm.js is always faster than emscripten, there are cases in which Duetto on V8 outperforms asm.js (on Spidermonkey) as well. We think that this is caused by the highly efficient handling of objects that V8 provides and shows that there is no clear winner. It is very hard to predict what approach will perform better, and by how much, on real world scenarios.
2) Emscripten is efficient because it effectively disables the Garbage Collector. How does Duetto handle dynamic memory?
Duetto maps C++ objects to native JS objects. This of course means that, every now and then, the GC will run. Emscripten handles memory by pre-allocating all the needed memory and then assigning slices of it using a malloc implementation. So yes, since the memory is not really dynamic it can avoid all the performance impact of the garbage collector
But this advantage does not come for free when you include memory consumption into account. Memory is a resource shared will all the other applications on the system, both native ones and other web apps. Eagerly reserving memory can have a large effect on the performance of the system as a whole.
While the GC may cause a sizeable overhead in terms of CPU cycles, it also decreases memory usage by freeing unneeded memory. Generally speaking, even when programming on traditional platforms, such as x86, dynamic memory allocations and deallocations should be avoided in performance sensitive code. Pre-allocating the required memory directly in the application code is indeed possible with Duetto as well.
3) Is Duetto code interoperable with existing JS code, libraries and vice-versa?
We currently automatically generate the header for all the HTML5/DOM APIs using headers originally written for TypeScript. We also plan to write headers for the most popular JS libraries, but we would like to stress that the needed headers are just plain C++ class declarations and no special additional support in the compiler is required to support new APIs or new libraries. So when the newest and greatest Web API comes out anyone will be able to write the header to use it with Duetto.
4) Why don’t you release immediately?
Because we want to make the developer experience sufficiently polished before giving it to users. Anyone how had the misfortune of using buggy compiler will probably appreciate this. We plan to release it in 6 months from the previous post (so around 5 months from now, time flies) under a dual licensing scheme: as Open Source for open source and non commercial projects, and with a paid license for commercial development.
5) Will Duetto make reverse engineering of original code easy?
6) Will other languages will be supported?
We choose to start from C++ because it’s a language that has proven to be good enough for very large scale projects and also because it’s the one we know best and love. Currently we have no plans to expand the Duetto architecture to other languages, but it is definitely possible that in the future, based on user demand, we could bring our awesome integrated client/server development model to other languages as well.
We (Leaning Technologies Ltd) would like to introduce Duetto, our own LLVM-based solution for programming the Web using C++. And by the Web, we mean both the client and server side of it, but let’s talk about the client side first.
So, yes, Duetto does need some minimal porting to bring C++ code to the Web while emscripten makes it mostly free. What you get in exchange for that is faster performance with no need of a special VM and deep integration with the browser. Duetto creates a really seamless C++ programming experience for the Web:
The Duetto backend is already in a very advanced state, and we believe it’s already suitable to bring the first applications to the Web. Especially games, which are our primary target. Unfortunately our front end is not yet as polished as we would like, as we want to improve the error reporting to make the porting experience as smooth as possible.
We are not yet ready to release Duetto, but we are eager to start opening collaborations, so if you are interested in bringing your C++ application or game to the web, feel free to contact me (firstname.lastname@example.org). We believe that in six months or less from now we will be able to release a robust product, most probably capable of generating even faster code. And we want to release it as open source.
For more information please visit our site: http://www.leaningtech.com