Archive for category Leaningtech
Duetto 0.9.6: debugging C++ web applications within the browser
Posted by Alessandro Pignotti in Leaningtech on April 18, 2014
Hello everyone! Leaning Technologies is happy to present yet another feature-packed release of duetto — our C++ compiler for the Web.
Duetto 0.9.6 introduces the initial support for a new, highly-requested feature: in-browser debugging of your C++ web application. We hope you’ll like this! We also introduced some new powerful optimizations to reduce memory usage and pressure on the garbage collector, and extended our support for the standard library.
Initial support for integrated debugging using Source Maps
Source maps is a standard technology to debug code compiled to JavaScript. It’s already used by many tools that generate JavaScript, both from other languages and from JavaScript itself. Duetto is now able to seamlessly generate source maps from C++ code.
Modern browsers can automatically load the source map and display the original C++ code when an error occurs. You can also set breakpoints in the C++ file and the JavaScript execution will stop when the corresponding line is reached. You can even single step the execution while looking at C++ code.
To generate a source map, you can use the following command (remember to enable debug information in clang using the –g option):
/opt/duetto/bin/clang++ -target duetto -g test.cpp -o test.js -duetto-sourcemap=test.js.map
Enabled Scalar Replacement of Aggregates (SROA) optimization
SROA is an standard compiler optimization which tries to replace complex objects with their members whenever possible. It’s a very critical optimization, especially for duetto, as every object which removed by SROA reduces the pressure on the garbage collector. LLVM has SROA support out of the box, but the existing implementation would often generate type-unsafe code that would break the duetto backend, so we have been forced to disable SROA up to now.
This release includes an improved SROA which always generates duetto-safe code, so we have been able to enable it again.
First support for collapsing allocations
We have introduced a new optimizations which tries to reduce memory pressure even more by recycling memory allocated for local variables. Whenever two or more allocations of the same type are used in disjoint sections of the same function they are merged into one.
Improved standard library support
We have fixed support for std::map, std::multimap, std::unordered_map, std::std, std::multiset and std::unordered_set which should be now fully working. sets and maps having pointers as the key are not yet supported.
Strict-mode JavaScript
Duetto-generated JavaScript now starts with “use strict”, a couple of small fixes have been made to generate fully strict-mode compliant code.
You can download duetto for Windows and Mac OS X on launchpad. Packages for Ubuntu and Debian are available from our PPA. You can also get the source code on GitHub.
Follow us on @leaningtech, Facebook and at www.leaningtech.com for updates.
Duetto (C++ for the Web) 0.9.3 is out — Usability and standard library improvements, Windows installer
Posted by Alessandro Pignotti in Leaningtech on January 11, 2014
At Leaning Technologies we are glad to announce our newest release of duetto, our C++ compiler for Web applications.
This release, marked as 0.9.3, contains some improvements:
-
Generated JS files are now self contained and do not need the duetto.js helper file
-
C++11 is now the default compilation mode
-
New Windows installer
-
Improvements to standard C++ library support, in particular now iostream, including cout and cerr are supported
-
stdout and stderr support, using browser console
We also have a new major release in the works, which will provide some quite interesting new features to duetto. We will announce some of them at the mloc.js conference in Budapest this February 13 – 14.
You can find binaries and installers of duetto for Windows and Mac OS X on launchpad. Packages for Ubuntu and Debian are available from our PPA. Source tarballs are available on launchpad as well.
Installation and getting started guides are available at wiki.leaningtech.com
Follow us on @leaningtech, Facebook and at www.leaningtech.com for updates.
Duetto (C++ for the Web) 0.9.2 is out — OpenGL ES implementation in WebGL and toolchain improvements
Posted by Alessandro Pignotti in Leaningtech on December 2, 2013
A lot of progress was made here at Leaningtech during the last few weeks, leading to our latest release of duetto — version 0.9.2.
For this version we have been focusing mainly on usability. The most relevant updates include:
-
A WebGL-based OpenGL ES implementation
-
Automatic linking of system libraries;
-
General toolchain usability improvement, including simplified command line syntax for common operations.
Duetto’s OpenGL ES implementation exposes the standard API available on desktop and mobile to Web developers. We have designed it to make porting of existing GLES apps and games easier, but it is of course possible to use it in new code. Moreover, we expose the underlying WebGL context to make it possible to mix and match between GLES and WebGL code, to take advantage of the native capabilities of the browser like using JPEG and PNG compressed images as textures. This GLES implementation is used in this Nontetris game, developed in C++ using duetto. You can find its source code here.
The last two points are aimed at a general effort of increasing the ease of use of duetto — and reducing the likelihood of the most frequent errors related to missing links to libraries.
You can find binary archives of duetto for Windows and Mac OS X on launchpad. Packages for Ubuntu and Debian are available from our PPA. Source tarballs are available on launchpad as well.
We have started an open wiki here. On the wiki you can already find installation instructions for the various platforms, build instructions and the getting started guide. Feel also free to contribute to the wiki if you feel something is missing.
We are also happy to announce that we will be speaking at the mloc.js conference in February in Budapest. We look forward to it as a great chance to meet some awesome people working on JS as a compiler target and discuss about our technology.
Follow us on @leaningtech, Facebook and at www.leaningtech.com for updates.
Comparing Duetto with Emscripten: a follow-up
Posted by Alessandro Pignotti in Leaningtech on November 15, 2013
Yesterday, Alon Zakai (alias kripken, creator of Emscripten) wrote an excellent post comparing three C++ to JS compilers: Mandreel, Emscripten and Duetto. We would like to provide a few more details regarding Duetto as we feel that, although the comparison was definitely fair, we can add some points which have been missed.
Memory model: Contrarily to Mandreel and Emscripten, which emulate a traditional flat address space using a singleton typed array, Duetto maps C++ objects to JS objects. This naturally implies an overhead both in terms of meta-data that the JS engine manages and in terms of computational work that the Garbage Collector (GC) needs to do. While these may look like major disadvantages, other factors should be taken into account. The computational overhead does not go wasted, as it is functional to keep memory usage in check and make sure that the amount of occupied memory is closer to the amount actually required.
In addition, the typed array based memory model incurs in some overhead as well, as memory areas are still managed using some kind of malloc. And they also suffer a lot from fragmentation as the browser can’t reclaim memory in the middle of the array which is not currently being used.
We strongly believe that the duetto memory model being based on JS objects is more fair to other applications, both native and web-based and to the user’s system as a whole. If the system gets tight on memory, it is possible for the browser to reclaim currently unused objects used in code generated by duetto, on the other hand when a large typed array is used, the system can at best page-out unused or seldom used pages to swap since it has no way to know what memory is actually used and what is not. On the other hand it is definitely true that frequently allocating and deallocating objects may cause an unacceptable slowdown, but this is actually true for native programs as well and the solution is to re-use the same objects over and over (for example by creating pools) at the application level, and this is something that is, of course, possible using duetto as well. Generally we believe that using JS objects also make duetto compiled code more similar to handwritten JS code, which is already handled very well by JS engines, with the added advantage of guaranteed type immutability which can be exploited by the engine to generate faster code. Still, duetto is flexible enough that it will be possible, in the future, to support using an emscripten like memory model on a type-by-type basis. This would be useful to adapt to various workloads and for compatibility with code compiled by emscripten.
APIs: Duetto is built around the concept that the browser is a platform having JS as the machine language. Such platform has APIs like WebGL, HTML5 Canvas, DOM events and so on which are the lowest level accessible functionalities. With Duetto you can use all of them directly from C++ and no JS wrapper code is required.
What about the cross platform APIs used on native systems like SDL and OpenAL? Well, we don’t currently support them, but they can be ported to Duetto by writing a new backend for the browser platform, similarly to what has been already done to port them to Windows, Linux and Mac. And using duetto it is possible to write such porting code directly in C++ because you have direct access to the browser APIs. Once the cross platform APIs support duetto, applications using them will run as well without much (or any) work.
And what about GLES? Well, we received so many requests for this functionality that we are working on providing a WebGL based GLES implementation ourselves. We plan to release it very soon. With that it should be possible to build programs using GLES with duetto by only changing the windowing system initialization code, similarly to porting from windows’ WGL to X11 glX.
C++ compatibility: Alon correctly states that duetto has “partial” C++ compatibility. This is true, but given the incredible amount of capabilities that the C++ language has, it is important to clarify what these limitations are.
Let’s start from the basis: our C++ compiler was of course not written from scratch, but was based on clang. This means that we inherit complete support for all C++ features, including recent and advanced ones like C++11, template support and lambda. And we actually use such features in the C++ code which manages transparent client/server RPCs.
The limitations are more about a few specific unsupported capabilities, currently the major ones are: missing support for virtual inheritance/virtual base classes, some issues with pointer comparison in a couple of corner cases and missing support for some parts of the standard library. But we would like to stress that those limitations are (mostly) non structural and we will do our best to increase the compatibility as much as possible while still be coherent with C++ philosophy of exposing all platform capabilities and limitations to the user.
Performance: We have worked with computer systems long enough to all agree that micro-benchmarks have little to do with performance on real world scenarios. We also know that Duetto still produces suboptimal code for many common cases that we plan to fix. We look forward to try and profile Duetto generated code from a large, real world code base, to find out how it performs and what needs to be optimized.
In summary, we think that Alon’s review of our technology is pretty accurate, and we look forward making more and more of Duetto’s features known to the community, with dedicated posts and documentation. We have been impressed by the feedback we got with our first release — and look forward to hear your opinion.
Duetto release 0.9.1 is now available as source tarballs and binaries for Ubuntu/Debian (PPA) and Windows. MacOSX builds will be available soon. Feel free to report bugs, and request features here.
Follow us on @leaningtech, Facebook and at www.leaningtech.com for updates.
Duetto (C++ for the Web) to be released soon: licensing model
Posted by Alessandro Pignotti in Leaningtech on October 24, 2013
Duetto — our C++ compiler for the Web — is getting close to its first public release, due in a few days, and we would like to spend a few lines to present and explain our licensing model, as many people have been asking recently.
When we started working on duetto around a year ago, we knew that we were going to release it as a FOSS project. We believe that duetto, by enabling truly seamless client/server C++ programming, can become a corner-stone of Web applications and games development. But this can only happen if an ecosystem of libraries, extensions and frameworks will be built around duetto itself. We know that the only way for this to happen is to make the core of our technology available to everyone, for free. In addition to that, since duetto is heavily based on the LLVM/clang framework and could not have existed otherwise, we really feel that giving back to the community is the most fair thing to do.
At the same time, we have also always been determined to build a company around this project. For this reason, we will be offering a premium package that includes the same core FOSS technology plus personalized support, proprietary friendly licensing and auxiliary perks including, but not limited to, integration with development environments.
Let’s now talk licenses. The duetto compiler itself is derived from LLVM and clang, both released under the University of Illinois/NCSA Open Source License. We are releasing our duetto compiler under the same license, both to make it easier for the community to contribute with code, to contribute back with some of our code to LLVM and clang, and because it does not make much sense to use a dual licensing scheme on a compiler.
In addition to the compiler, there are a bunch of headers and libraries which are essentials to develop using duetto. All those will be released under a classic dual licensing scheme: GPLv2 for everyone and a proprietary non-copyleft license for paying customers.
To summarize, here is how duetto licensing will work:
-
duetto core compiler: University of Illinois/NCSA Open Source License (like the LLVM and clang projects)
-
duetto core libraries/headers: dual license GPLv2; proprietary non-copyleft license available for a fee
-
duetto premium features: proprietary and closed source, available for a fee
We think this choice represents a good balance between being open and giving back to the open source community, and building a viable business model. We hope the open source community will appreciate our choice!
Follow us on twitter @leaningtech and visit our website http://www.leaningtech.com to get updates on our upcoming release and for further information.
Bringing C++ to the Web: some background on Duetto
Posted by Alessandro Pignotti in Leaningtech on October 10, 2013
While I am a big-time fan of C++, I don’t go so far as to say it’s “the best programming language ever, period.” As an engineer, still, I really believe it’s an extremely effective tool for proper programming. Fans and advocates of more dynamic programming languages such as Python, Ruby and JavaScript typically consider C++ a boring, verbose language with a pedantic compiler that never stops complaining about seemingly unimportant details.
Well, if you ask me, that is the most relevant feature of C++. I don’t see it as a limitation, but as a framework that helps structuring my ideas into machine compatible form. Over time I have grown to appreciate that the C++ compiler is a programmer’s best friend.
More recently I’ve also become fascinated with the incredible progress that browsers vendors have made in bringing standard-compliant modern capabilities to the general user. While HTML5 canvas and WebGL are definitely the most visible additions, we should not forget about local storage, programmatic audio support, video support, web sockets and everything that is being standardized. This extensive feature set is amazing and, to me, this depicts a future where almost-native applications and games are available on every single platform through browsers.
So far, taking advantage of the great platform that the Web is, has required programming in JavaScript. JavaScript is terrible. I am no JS ninja, and I find it incredibly frustrating having no hint about what kind of thing will be passed as an argument of a function, or if anything at all will be passed, for that matter. How some people can build and effectively maintain a large-scale code base in JS is beyond my comprehension. I am sure many developers feel that same way that I do.
So we have some pretty awesome Web APIs, tightly coupled in a programming environment that, to me, looks like a joke. Since that sounds like a problem to me, and I like to solve problems [1], I decided to bring my language of choice, C++, to the Web platform.
My solution was to write duetto, a C++ compiler for the Web. With a couple of friends, we are trying to build a company around it.
duetto is a C++ compiler for the Web, not for JavaScript. Most of the work involved in writing duetto was related to finding a way to generate good JavaScript from C++. To do that, we wrote a special LLVM backend, based on a couple of (hopefully) clever tricks. But we did that only because we had no choice. JavaScript is currently the only practical way to deploy code to browsers. But in our vision JS is only a mean to our goal of enabling C++ as a programming language for web apps and games.
What we want to do is to provide a better programming language for the Web, to take the most advantage of all the fantastic underlying technologies such as HTML5 and CSS, which do their job more than well. With duetto, designers can still work using tools they know, while developers can write a sound application logic in robust C++ code. And, by the way, we are not trying to provide an half-backed widget library that would always be lagging behind what browser APIs can provide. With duetto you access the browser directly, with no need of any middle layer.
One of the intended uses of duetto is to port existing C++ applications, especially games, to the Web. But we also want to make it a primary choice to write new web apps. One of the requirements of modern apps is continuous communication with remote servers, for example to receive other users’ updates. This is why we have also introduced an experimental support for transparent RPC to server side code.
Calling this RPC is actually kind of limiting. Our objective is to provide a seamless experience to make it possible to write client and server side code in the same codebase and let the compiler generate the two sides of compiled code. Take a look at some example code.
We are all standing on the shoulder of giants. In our case the giant is the LLVM/clang project. Generating JavaScript required some improvements to the compiler core that we hope to include in the upstream projects. LLVM provide a large array of optimizations passes on code generated by clang, which means that JavaScript code reaching the browser has been already heavily optimized, lowering the burden on the JS engine. In general, the JS generated by duetto tries to play well with modern JS engines, by providing hints that should make it possible to generate good native code. Currently the code is generic, but we plan to explore the possibility of generating code tailored for each JS engine (e.g. Chrome’s V8 or Mozilla’s SpiderMonkey). In the end this would be not different from the CPU specific optimizations that are available on regular architectures.
The release date for duetto is now getting really close and we are really excited about shipping it and see what people can do with this technology. If you want to take a preview please visit http://leaningtech.com, there are still a few slots open in our extended beta program.
[1] Maybe I like solving problems too much, I wrote Lightspark because Adobe’s Flash Player was too slow in rendering full screen videos of kittens on my workstation
Update: Duetto (C++ for the Web) limited public beta extended!
Posted by Alessandro Pignotti in Leaningtech on October 4, 2013
Well folks, we are very happy about the success of our limited beta campaign having already filled up the 100 slots we have offered for our first public beta. We decided to raise the threshold to up to 200 beta testers. Apply here:
and you will receive a link to our most recent build.
Moreover, take a look at the incredibly awesome demo that our friend Daniele Di Proietto made for us using an earlier beta. It is an implementation of the “Not Tetris” concept. And just to be clear, it’s fully implemented in C++ using direct access to WebGL. For more info, see the demo page.
For more info take a look at our original blog post here.
Duetto (C++ for the Web) available for download: limited public beta
Posted by Alessandro Pignotti in Leaningtech on October 2, 2013
Did we ever tell you that writing a full-feature compiler is not a job for the faint of heart? During the last couple of months we have been marching forward in our quest to bring a great, seamless C++ programming experience for the Web. And we see victory at the end of our journey!
Duetto is our clang-derived C++ compiler which makes it possible to really translate the value of C++ to Web programming. What we want is for you to be able to reuse your oh-so-hard-to-reimplement C++ code (like simulation libraries, or physics engines) in your shiny new HTML5-based app or videogame, with no need of rewriting it in JavaScript. But not only.
We are aware, of course, that there are already other solutions, such as Emscripten, to solve this problem. What makes Duetto different is that it is a C++ compiler with a C++ mindset.
We believe that the strength of C++ comes from enhancing the platform capabilities, without trying to hide the limitations from the developers. In Duetto, JavaScript is our machine architecture and an HTML5 browser is our OS. Our tool compiles standard, compliant C++11 code to JavaScript and exposes all the awesome HTML5 features of modern browsers, including WebGL. There is no middle layer, no inefficiencies, no custom half-crippled widget library. You write you web page in HTML and CSS, and code the logic with C++. We truly believe this is taking best of both worlds.
We have worked long and hard to polish Duetto and we are now almost satisfied by its capabilities and robustness. We are now ready to share this tool with the first 100 developers that are going to share their interest, in order to collect the most feedback and provide the best possible support and experience.
Interested in trying out Duetto for free? Visit http://leaningtech.com to get your copy.
If you’re not one of the 100 lucky beta testers, do not despair. We will be releasing Duetto in roughly a month. Just enough time to hammer out some bugs, add some more awesomeness and figure out last details on our licensing model. Duetto will be released under a dual licensing scheme, so it will be an open source/free software project, available for anyone. We will also be offering paid commercial licenses for people which are not comfortable with free software libraries and/or want enterprise-grade support.
UPDATE: I forgot to link the incredibly awesome demo by Daniele Di Proietto. He used Duetto to port his own implementation of the “Not tetris” concept to the Web.
Duetto (C++ for the Web): CMake integration, Bullet and WebGL demo
Posted by Alessandro Pignotti in Leaningtech on July 17, 2013
At LeaningTech we have been working hard and it’s time for some news and updates on the development of duetto, our tool for programming the Web in C++.
Thanks to some key developments that we are about to describe, we managed to compile our first real-world test case in Javascript, the bullet physics library — and a small demo in webGL that takes advantage of it.
The compilation of bullet in JS with duetto requires very minimal adaptation to the vanilla code, and can be easily performed in full integration with CMake infrastructure. We built a minimal demo to show the result of our tool in action: a triangle falling on the ground attracted by gravity. Please note that the demo is completely not optimized, not even by minimizing the JS. Any performance must be considered preliminary and there is vast room for improvement that we plan to fully exploit
http://leaningtech.com/duettodemo/HelloWorld.html
This was our first trial at compiling a complex codebase, and it went surprisingly smoothly!
The following features are now fully supported in duetto:
- A fully working libc and libc++
Our current beta features a full implementation of the standard C library (based on the newlib implementation) and of the standard C++ library (based on the LLVM/libc++). Both libraries are standard implementations, support for duetto was included using the same infrastructure used for other, native, targets.
- Integration in the CMake/autotool toolchain as a target
Duetto is now fully integrated with the CMake, using the infrastructure normally used for cross compiling (i.e toolchain definition files). This means you can compile complex projects that use CMake with little effort. We also plan to integrate with autotools in the future.
- Full support of the DOM and HTML5 APIs (such as WebGL)
Our latest beta features an improved support for using the DOM and WebGL API from C++. This is done by defining a set of headers which allows to transparently access all the browser DOM and libraries. It is also very easy to add support for any JS libraries, following the same approach.
Four months have passed since our initial post about duetto. At that time we promised what we would have released our brainchild in a six months time frame. Well, I’m very happy to say that we are fully in schedule and we will be able to release duetto in the fall, under a dual licensing scheme: open-source for open-source and non-commercial use, and a paid license for commercial use.
We have already started sharing a limited private beta with a few developers and are interested in expanding its release to some more, if interested please contact us at info@leaningtech.com
Duetto (C++ for the Web): comparison to asm.js and other clarifications
Posted by Alessandro Pignotti in Leaningtech on May 28, 2013
Around a month ago we posted a first overview of Duetto: our integrated LLVM-based solution for programming both the client and the server side of Web applications using C++. We have been completely overwhelmed by the interest generated by that post!
We tried to keep track of the many insightful comments and constructive criticism we received on this blog and on aggregators. Here are the most common questions that arose:
1) Why didn’t you compare Duetto with asm.js?
We believe that comparing to asm.js-enabled code is not an apples-to-apples comparison. In the current state the Firefox asm.js Ahead-Of-Time compiler can only be enabled on code which is generated using the emscripten approach to memory allocation, so code generated by Duetto has no way to benefit from it. We are open to discuss how the asm.js approach of validating typed-ness of code AOT can be extended so that Duetto, and potentially other solutions, may benefit from it as well.
Still, it is clearly very interesting to compare Duetto performance to asm.js, so we have made a new round of benchmarking that includes it. The results are pretty interesting. As you can see, although asm.js is always faster than emscripten, there are cases in which Duetto on V8 outperforms asm.js (on Spidermonkey) as well. We think that this is caused by the highly efficient handling of objects that V8 provides and shows that there is no clear winner. It is very hard to predict what approach will perform better, and by how much, on real world scenarios.
2) Emscripten is efficient because it effectively disables the Garbage Collector. How does Duetto handle dynamic memory?
Duetto maps C++ objects to native JS objects. This of course means that, every now and then, the GC will run. Emscripten handles memory by pre-allocating all the needed memory and then assigning slices of it using a malloc implementation. So yes, since the memory is not really dynamic it can avoid all the performance impact of the garbage collector
But this advantage does not come for free when you include memory consumption into account. Memory is a resource shared will all the other applications on the system, both native ones and other web apps. Eagerly reserving memory can have a large effect on the performance of the system as a whole.
While the GC may cause a sizeable overhead in terms of CPU cycles, it also decreases memory usage by freeing unneeded memory. Generally speaking, even when programming on traditional platforms, such as x86, dynamic memory allocations and deallocations should be avoided in performance sensitive code. Pre-allocating the required memory directly in the application code is indeed possible with Duetto as well.
Since I do not expect native platforms (i.e. GLibc) to preallocate gigs of memory at application startup to make it faster when dynamic memory is actually used, I do not expect this from a compiler for the JavaScript target either.
3) Is Duetto code interoperable with existing JS code, libraries and vice-versa?
Yes, absolutely. Methods compiled by Duetto can be exported using the regular C++ mangling or even without the mangling if they are declared as “extern C”. In both cases the can be freely invoked from pure JavaScript code. At the same time Duetto compiled code will be able to seamlessly access all the HTML5/DOM APIs and any JS function. The only requirement is that the JS interface needs to be declared in a C++ header.
We currently automatically generate the header for all the HTML5/DOM APIs using headers originally written for TypeScript. We also plan to write headers for the most popular JS libraries, but we would like to stress that the needed headers are just plain C++ class declarations and no special additional support in the compiler is required to support new APIs or new libraries. So when the newest and greatest Web API comes out anyone will be able to write the header to use it with Duetto.
4) Why don’t you release immediately?
Because we want to make the developer experience sufficiently polished before giving it to users. Anyone how had the misfortune of using buggy compiler will probably appreciate this. We plan to release it in 6 months from the previous post (so around 5 months from now, time flies) under a dual licensing scheme: as Open Source for open source and non commercial projects, and with a paid license for commercial development.
5) Will Duetto make reverse engineering of original code easy?
In Duetto, C++ code passes through the regular Clang/LLVM pipeline before being converted to JavaScript. This is the same path used to compile assembly for any other architecture, like x86 or ARM. The generated JavaScript is no easier to understand than the corresponding machine code or assembly. Moreover, one of the typical optimization steps for JavaScript, which we can employ as well, is ‘minimization’ which destroys any residual information about variables and method names.
6) Will other languages will be supported?
We choose to start from C++ because it’s a language that has proven to be good enough for very large scale projects and also because it’s the one we know best and love. Currently we have no plans to expand the Duetto architecture to other languages, but it is definitely possible that in the future, based on user demand, we could bring our awesome integrated client/server development model to other languages as well.