Archive for category Leaningtech

Duetto 0.9.6: debugging C++ web applications within the browser

logo_duetto_quadrato_192

Hello every­one! Lean­ing Tech­nolo­gies is happy to present yet another feature-packed release of duetto — our C++ com­piler for the Web.

Duetto 0.9.6 intro­duces the ini­tial sup­port for a new, highly-requested fea­ture: in-browser debug­ging of your C++ web appli­ca­tion. We hope you’ll like this! We also intro­duced some new pow­er­ful opti­miza­tions to reduce mem­ory usage and pres­sure on the garbage col­lec­tor, and extended our sup­port for the stan­dard library.

Ini­tial sup­port for inte­grated debug­ging using Source Maps

Source maps is a stan­dard tech­nol­ogy to debug code com­piled to JavaScript. It’s already used by many tools that gen­er­ate JavaScript, both from other lan­guages and from JavaScript itself. Duetto is now able to seam­lessly gen­er­ate source maps from C++ code.

Mod­ern browsers can auto­mat­i­cally load the source map and dis­play the orig­i­nal C++ code when an error occurs. You can also set break­points in the C++ file and the JavaScript exe­cu­tion will stop when the cor­re­spond­ing line is reached. You can even sin­gle step the exe­cu­tion while look­ing at C++ code.


To gen­er­ate a source map, you can use the fol­low­ing com­mand (remem­ber to enable debug infor­ma­tion in clang using the –g option):

/opt/duetto/bin/clang++ -target duetto -g test.cpp -o test.js -duetto-sourcemap=test.js.map

Enabled Scalar Replace­ment of Aggre­gates (SROA) optimization

SROA is an stan­dard com­piler opti­miza­tion which tries to replace com­plex objects with their mem­bers when­ever pos­si­ble. It’s a very crit­i­cal opti­miza­tion, espe­cially for duetto, as every object which removed by SROA reduces the pres­sure on the garbage col­lec­tor. LLVM has SROA sup­port out of the box, but the exist­ing imple­men­ta­tion would often gen­er­ate type-unsafe code that would break the duetto back­end, so we have been forced to dis­able SROA up to now.

This release includes an improved SROA which always gen­er­ates duetto-safe code, so we have been able to enable it again.

First sup­port for col­laps­ing allocations

We have intro­duced a new opti­miza­tions which tries to reduce mem­ory pres­sure even more by recy­cling mem­ory allo­cated for local vari­ables. When­ever two or more allo­ca­tions of the same type are used in dis­joint sec­tions of the same func­tion they are merged into one.

Improved stan­dard library support

We have fixed sup­port for std::map, std::multimap, std::unordered_map, std::std, std::multiset and std::unordered_set which should be now fully work­ing. sets and maps hav­ing point­ers as the key are not yet supported.

Strict-mode JavaScript

Duetto-generated JavaScript now starts with “use strict”, a cou­ple of small fixes have been made to gen­er­ate fully strict-mode com­pli­ant code.

You can down­load duetto for Win­dows and Mac OS X on launch­pad. Pack­ages for Ubuntu and Debian are avail­able from our PPA. You can also get the source code on GitHub.

Fol­low us on @leaningtech, Face­book and at www.leaningtech.com for updates.

No Comments

Duetto (C++ for the Web) 0.9.3 is out — Usability and standard library improvements, Windows installer

logo_duetto_quadrato_192

At Lean­ing Tech­nolo­gies we are glad to announce our newest release of duetto, our C++ com­piler for Web applications.

This release, marked as 0.9.3, con­tains some improvements:

  • Gen­er­ated JS files are now self con­tained and do not need the duetto.js helper file

  • C++11 is now the default com­pi­la­tion mode

  • New Win­dows installer

  • Improve­ments to stan­dard C++ library sup­port, in par­tic­u­lar now iostream, includ­ing cout and cerr are supported

  • std­out and stderr sup­port, using browser console

We also have a new major release in the works, which will pro­vide some quite inter­est­ing new fea­tures to duetto. We will announce some of them at the mloc.js con­fer­ence in Budapest this Feb­ru­ary 13 – 14.

You can find bina­ries and installers of duetto for Win­dows and Mac OS X on launch­pad. Pack­ages for Ubuntu and Debian are avail­able from our PPA. Source tar­balls are avail­able on launch­pad as well.

Instal­la­tion and get­ting started guides are avail­able at wiki.leaningtech.com

Fol­low us on @leaningtech, Face­book and at www.leaningtech.com for updates.

No Comments

Duetto (C++ for the Web) 0.9.2 is out — OpenGL ES implementation in WebGL and toolchain improvements


A lot of progress was made here at Lean­ingtech dur­ing the last few weeks, lead­ing to our lat­est release of duetto — ver­sion 0.9.2.

For this ver­sion we have been focus­ing mainly on usabil­ity. The most rel­e­vant updates include:

  • A WebGL-based OpenGL ES implementation

  • Auto­matic link­ing of sys­tem libraries;

  • Gen­eral tool­chain usabil­ity improve­ment, includ­ing sim­pli­fied com­mand line syn­tax for com­mon operations.

Duetto’s OpenGL ES imple­men­ta­tion exposes the stan­dard API avail­able on desk­top and mobile to Web devel­op­ers. We have designed it to make port­ing of exist­ing GLES apps and games eas­ier, but it is of course pos­si­ble to use it in new code. More­over, we expose the under­ly­ing WebGL con­text to make it pos­si­ble to mix and match between GLES and WebGL code, to take advan­tage of the native capa­bil­i­ties of the browser like using JPEG and PNG com­pressed images as tex­tures. This GLES imple­men­ta­tion is used in this Non­tetris game, devel­oped in C++ using duetto. You can find its source code here.

The last two points are aimed at a gen­eral effort of increas­ing the ease of use of duetto — and reduc­ing the like­li­hood of the most fre­quent errors related to miss­ing links to libraries.

You can find binary archives of duetto for Win­dows and Mac OS X on launch­pad. Pack­ages for Ubuntu and Debian are avail­able from our PPA. Source tar­balls are avail­able on launch­pad as well.

We have started an open wiki here. On the wiki you can already find instal­la­tion instruc­tions for the var­i­ous plat­forms, build instruc­tions and the get­ting started guide. Feel also free to con­tribute to the wiki if you feel some­thing is missing.

We are also happy to announce that we will be speak­ing at the mloc.js con­fer­ence in Feb­ru­ary in Budapest. We look for­ward to it as a great chance to meet some awe­some peo­ple work­ing on JS as a com­piler tar­get and dis­cuss about our technology.

Fol­low us on @leaningtech, Face­book and at www.leaningtech.com for updates.

No Comments

Comparing Duetto with Emscripten: a follow-up

logo_duetto_quadrato_192

Yes­ter­day, Alon Zakai (alias krip­ken, cre­ator of Emscripten) wrote an excel­lent post com­par­ing three C++ to JS com­pil­ers: Man­dreel, Emscripten and Duetto. We would like to pro­vide a few more details regard­ing Duetto as we feel that, although the com­par­i­son was def­i­nitely fair, we can add some points which have been missed.

Mem­ory model: Con­trar­ily to Man­dreel and Emscripten, which emu­late a tra­di­tional flat address space using a sin­gle­ton typed array, Duetto maps C++ objects to JS objects. This nat­u­rally implies an over­head both in terms of meta-data that the JS engine man­ages and in terms of com­pu­ta­tional work that the Garbage Col­lec­tor (GC) needs to do. While these may look like major dis­ad­van­tages, other fac­tors should be taken into account. The com­pu­ta­tional over­head does not go wasted, as it is func­tional to keep mem­ory usage in check and make sure that the amount of occu­pied mem­ory is closer to the amount actu­ally required.

In addi­tion, the typed array based mem­ory model incurs in some over­head as well,  as mem­ory areas are still man­aged using some kind of mal­loc. And they also suf­fer a lot from frag­men­ta­tion as the browser can’t reclaim mem­ory in the mid­dle of the array which is not cur­rently being used.

We strongly believe that the duetto mem­ory model being based on JS objects is more fair to other appli­ca­tions, both native and web-based and to the user’s sys­tem as a whole. If the sys­tem gets tight on mem­ory, it is pos­si­ble for the browser to reclaim cur­rently unused objects used in code gen­er­ated by duetto, on the other hand when a large typed array is used, the sys­tem can at best page-out unused or sel­dom used pages to swap since it has no way to know what mem­ory is actu­ally used and what is not. On the other hand it is def­i­nitely true that fre­quently allo­cat­ing and deal­lo­cat­ing objects may cause an unac­cept­able slow­down, but this is actu­ally true for native pro­grams as well and the solu­tion is to re-use the same objects over and over (for exam­ple by cre­at­ing pools) at the appli­ca­tion level, and this is some­thing that is, of course, pos­si­ble using duetto as well. Gen­er­ally we believe that using JS objects also make duetto com­piled code more sim­i­lar to hand­writ­ten JS code, which is already han­dled very well by JS engines, with the added advan­tage of guar­an­teed type immutabil­ity which can be exploited by the engine to gen­er­ate faster code. Still, duetto is flex­i­ble enough that it will be pos­si­ble, in the future, to sup­port using an emscripten like mem­ory model on a type-by-type basis. This would be use­ful to adapt to var­i­ous work­loads and for com­pat­i­bil­ity with code com­piled by emscripten.

APIs: Duetto is built around the con­cept that the browser is a plat­form hav­ing JS as the machine lan­guage. Such plat­form has APIs like WebGL, HTML5 Can­vas, DOM events and so on which are the low­est level acces­si­ble func­tion­al­i­ties. With Duetto you can use all of them directly from C++ and no JS wrap­per code is required.

What about the cross plat­form APIs used on native sys­tems like SDL and Ope­nAL? Well, we don’t cur­rently sup­port them, but they can be ported to Duetto by writ­ing a new back­end for the browser plat­form, sim­i­larly to what has been already done to port them to Win­dows, Linux and Mac. And using duetto it is pos­si­ble to write such port­ing code directly in C++ because you have direct access to the browser APIs. Once the cross plat­form APIs sup­port duetto, appli­ca­tions using them will run as well with­out much (or any) work.

And what about GLES? Well, we received so many requests for this func­tion­al­ity that we are work­ing on pro­vid­ing a WebGL based GLES imple­men­ta­tion our­selves. We plan to release it very soon. With that it should be pos­si­ble to build pro­grams using GLES with duetto by only chang­ing the win­dow­ing sys­tem ini­tial­iza­tion code, sim­i­larly to port­ing from win­dows’ WGL to X11 glX.

C++ com­pat­i­bil­ity: Alon cor­rectly states that duetto has “par­tial” C++ com­pat­i­bil­ity. This is true, but given the incred­i­ble amount of capa­bil­i­ties that the C++ lan­guage has, it is impor­tant to clar­ify what these lim­i­ta­tions are.

Let’s start from the basis: our C++ com­piler was of course not writ­ten from scratch, but was based on clang. This means that we inherit com­plete sup­port for all C++ fea­tures, includ­ing recent and advanced ones like C++11, tem­plate sup­port and lambda. And we actu­ally use such fea­tures in the C++ code which man­ages trans­par­ent client/server RPCs.

The lim­i­ta­tions are more about a few spe­cific unsup­ported capa­bil­i­ties, cur­rently the major ones are: miss­ing sup­port for vir­tual inheritance/virtual base classes, some issues with pointer com­par­i­son in a cou­ple of cor­ner cases and miss­ing sup­port for some parts of the stan­dard library. But we would like to stress that those lim­i­ta­tions are (mostly) non struc­tural and we will do our best to increase the com­pat­i­bil­ity as much as pos­si­ble while still be coher­ent with C++ phi­los­o­phy of expos­ing all plat­form capa­bil­i­ties and lim­i­ta­tions to the user.

Per­for­mance: We have worked with com­puter sys­tems long enough to all agree that micro-benchmarks have lit­tle to do with per­for­mance on real world sce­nar­ios. We also know that Duetto still pro­duces sub­op­ti­mal code for many com­mon cases that we plan to fix. We look for­ward to try and pro­file Duetto gen­er­ated code from a large, real world code base, to find out how it per­forms and what needs to be optimized.

In sum­mary, we think that Alon’s review of our tech­nol­ogy is pretty accu­rate, and we look for­ward mak­ing more and more of Duetto’s fea­tures known to the com­mu­nity, with ded­i­cated posts and doc­u­men­ta­tion. We have been impressed by the feed­back we got with our first release — and look for­ward to hear your opinion.

Duetto release 0.9.1 is now avail­able as source tar­balls and bina­ries for Ubuntu/Debian (PPA) and Win­dows. MacOSX builds will be avail­able soon. Feel free to report bugs, and request fea­tures here.

Fol­low us on @leaningtech, Face­book and at www.leaningtech.com for updates.

No Comments

Duetto (C++ for the Web) to be released soon: licensing model

logo_duetto_quadrato_192

Duetto — our C++ com­piler for the Web — is get­ting close to its first pub­lic release, due in a few days, and we would like to spend a few lines to present and explain our licens­ing model, as many peo­ple have been ask­ing recently.

When we started  work­ing on duetto around a year ago, we knew that we were going to release it as a FOSS project. We believe that duetto, by enabling truly seam­less client/server C++ pro­gram­ming, can become a corner-stone of Web appli­ca­tions and games devel­op­ment. But this can only hap­pen if an ecosys­tem of libraries, exten­sions and frame­works will be built around duetto itself. We know that the only way for this to hap­pen is to make the core of our tech­nol­ogy avail­able to every­one, for free. In addi­tion to that, since duetto is heav­ily based on the LLVM/clang frame­work and could not have existed oth­er­wise, we really feel that giv­ing back to the com­mu­nity is the most fair thing to do.

At the same time, we have also always been deter­mined to build a com­pany around this project. For this rea­son, we will be offer­ing a pre­mium pack­age that includes the same core FOSS tech­nol­ogy plus per­son­al­ized sup­port, pro­pri­etary friendly licens­ing and aux­il­iary perks includ­ing, but not lim­ited to, inte­gra­tion with devel­op­ment environments.

Let’s now talk licenses. The duetto com­piler itself is derived from LLVM and clang, both released under the Uni­ver­sity of Illinois/NCSA Open Source License. We are releas­ing our duetto com­piler under the same license, both to make it eas­ier for the com­mu­nity to con­tribute with code, to con­tribute back with some of our code to LLVM and clang, and because it does not make much sense to use a dual licens­ing scheme on a compiler.

In addi­tion to the com­piler, there are a bunch of head­ers and libraries which are essen­tials to develop using duetto. All those will be released under a clas­sic dual licens­ing scheme: GPLv2 for every­one and a pro­pri­etary non-copyleft license for pay­ing customers.

To sum­ma­rize, here is how duetto licens­ing will work:

  • duetto core com­piler: Uni­ver­sity of Illinois/NCSA Open Source License (like the LLVM and clang projects)

  • duetto core libraries/headers: dual license GPLv2; pro­pri­etary non-copyleft license avail­able for a fee

  • duetto pre­mium fea­tures: pro­pri­etary and closed source, avail­able for a fee

We think this choice rep­re­sents a good bal­ance between being open and giv­ing back to the open source com­mu­nity, and build­ing a viable busi­ness model. We hope the open source com­mu­nity will appre­ci­ate our choice!

 Fol­low us on twit­ter @leaningtech and visit our web­site http://www.leaningtech.com to get updates on our upcom­ing release and for fur­ther information.

6 Comments

Bringing C++ to the Web: some background on Duetto

logo_duetto_quadrato_192

While I am a big-time fan of C++, I don’t go so far as to say it’s “the best pro­gram­ming lan­guage ever, period.” As an engi­neer, still, I really believe it’s an extremely effec­tive tool for proper pro­gram­ming. Fans and advo­cates of more dynamic pro­gram­ming lan­guages such as Python, Ruby and JavaScript typ­i­cally con­sider C++ a bor­ing, ver­bose lan­guage with a pedan­tic com­piler that never stops com­plain­ing about seem­ingly unim­por­tant details.

Well, if you ask me, that is the most rel­e­vant fea­ture of C++. I don’t see it as a lim­i­ta­tion, but as a frame­work that helps struc­tur­ing my ideas into machine com­pat­i­ble form. Over time I have grown to appre­ci­ate that the C++ com­piler is a programmer’s best friend.

More recently I’ve also become fas­ci­nated with the incred­i­ble progress that browsers ven­dors have made in bring­ing standard-compliant mod­ern capa­bil­i­ties to the gen­eral user. While HTML5 can­vas and WebGL are def­i­nitely the most vis­i­ble addi­tions, we should not for­get about local stor­age, pro­gram­matic audio sup­port, video sup­port, web sock­ets and every­thing that is being stan­dard­ized. This exten­sive fea­ture set is amaz­ing and, to me, this depicts a future where almost-native appli­ca­tions and games are avail­able on every sin­gle plat­form through browsers.

So far, tak­ing advan­tage of the great plat­form that the Web is, has required pro­gram­ming in JavaScript. JavaScript is ter­ri­ble. I am no JS ninja, and I find it incred­i­bly frus­trat­ing hav­ing no hint about what kind of thing will be passed as an argu­ment of a func­tion, or if any­thing at all will be passed, for that mat­ter. How some peo­ple can build and effec­tively main­tain a large-scale code base in JS is beyond my com­pre­hen­sion. I am sure many devel­op­ers feel that same way that I do.

So we have some pretty awe­some Web APIs, tightly cou­pled in a pro­gram­ming envi­ron­ment that, to me, looks like a joke. Since that sounds like a prob­lem to me, and I like to solve prob­lems [1], I decided to bring my lan­guage of choice, C++, to the Web platform.

My solu­tion was to write duetto, a C++ com­piler for the Web. With a cou­ple of friends, we are try­ing to build a com­pany around it.

duetto is a C++ com­piler for the Web, not for JavaScript. Most of the work involved in writ­ing duetto was related to find­ing a way to gen­er­ate good JavaScript from C++. To do that, we wrote a spe­cial LLVM back­end, based on a cou­ple of (hope­fully) clever tricks. But we did that only because we had no choice. JavaScript is cur­rently the only prac­ti­cal way to deploy code to browsers. But in our vision JS is only a mean to our goal of enabling C++ as a pro­gram­ming lan­guage for web apps and games.

What we want to do is to pro­vide a bet­ter pro­gram­ming lan­guage for the Web, to take the most advan­tage of all the fan­tas­tic under­ly­ing tech­nolo­gies such as HTML5 and CSS, which do their job more than well. With duetto, design­ers can still work using tools they know, while devel­op­ers can write a sound appli­ca­tion logic in robust C++ code. And, by the way, we are not try­ing to pro­vide an half-backed wid­get library that would always be lag­ging behind what browser APIs can pro­vide. With duetto you access the browser directly, with no need of any mid­dle layer.

One of the intended uses of duetto is to port exist­ing C++ appli­ca­tions, espe­cially games, to the Web. But we also want to make it a pri­mary choice to write new web apps. One of the require­ments of mod­ern apps is con­tin­u­ous com­mu­ni­ca­tion with remote servers, for exam­ple to receive other users’ updates. This is why we have also intro­duced an exper­i­men­tal sup­port for trans­par­ent RPC to server side code.

Call­ing this RPC is actu­ally kind of lim­it­ing. Our objec­tive is to pro­vide a seam­less expe­ri­ence to make it pos­si­ble to write client and server side code in the same code­base and let the com­piler gen­er­ate the two sides of com­piled code. Take a look at some exam­ple code.

We are all stand­ing on the shoul­der of giants. In our case the giant is the LLVM/clang project. Gen­er­at­ing JavaScript required some improve­ments to the com­piler core that we hope to include in the upstream projects. LLVM pro­vide a large array of opti­miza­tions passes on code gen­er­ated  by clang, which means that JavaScript code reach­ing the browser has been already heav­ily opti­mized, low­er­ing the bur­den on the JS engine. In gen­eral, the JS gen­er­ated by duetto tries to play well with mod­ern JS engines, by pro­vid­ing hints that should make it pos­si­ble to gen­er­ate good native code. Cur­rently the code is generic, but we plan to explore the pos­si­bil­ity of gen­er­at­ing code tai­lored for each JS engine (e.g. Chrome’s V8 or Mozilla’s Spi­der­Mon­key). In the end this would be not dif­fer­ent from the CPU spe­cific opti­miza­tions that are avail­able on reg­u­lar architectures.

The release date for duetto is now get­ting really close and we are really excited about ship­ping it and see what peo­ple can do with this tech­nol­ogy. If you want to take a pre­view please visit http://leaningtech.com, there are still a few slots open in our extended beta program.

[1] Maybe I like solv­ing prob­lems too much, I wrote Lightspark because Adobe’s Flash Player was too slow in ren­der­ing full screen videos of kit­tens on my workstation

1 Comment

Update: Duetto (C++ for the Web) limited public beta extended!

nontetris

Well folks, we are very happy about the suc­cess of our lim­ited beta cam­paign hav­ing already filled up the 100 slots we have offered for our first pub­lic beta. We decided to raise the thresh­old to up to 200 beta testers. Apply here:

http://www.leaningtech.com

and you will receive a link to our most recent build.

More­over,  take a look at the incred­i­bly awe­some demo that our friend Daniele Di Proi­etto made for us using an ear­lier beta. It is an imple­men­ta­tion of the “Not Tetris” con­cept. And just to be clear, it’s fully imple­mented in C++ using direct access to WebGL. For more info, see the demo page.

For more info take a look at our orig­i­nal blog post here.

No Comments

Duetto (C++ for the Web) available for download: limited public beta

logo_duetto_quadrato_192Did we ever tell you that writ­ing a full-feature com­piler is not a job for the faint of heart? Dur­ing the last cou­ple of months we have been march­ing for­ward in our quest to bring a great, seam­less C++ pro­gram­ming expe­ri­ence for the Web. And we see vic­tory at the end of our journey!

Duetto is our clang-derived C++ com­piler which makes it pos­si­ble to really trans­late the value of C++ to Web pro­gram­ming. What we want is for you to be able to reuse your oh-so-hard-to-reimplement C++ code (like sim­u­la­tion libraries, or physics engines) in your shiny new HTML5-based app or videogame, with no need of rewrit­ing it in JavaScript. But not only.

We are aware, of course, that there are already other solu­tions, such as Emscripten, to solve this prob­lem. What makes Duetto dif­fer­ent is that it is a C++ com­piler with a C++ mindset.

We believe that the strength of C++ comes from enhanc­ing the plat­form capa­bil­i­ties, with­out try­ing to hide the lim­i­ta­tions from the devel­op­ers. In Duetto, JavaScript is our machine archi­tec­ture and an HTML5 browser is our OS. Our tool com­piles stan­dard, com­pli­ant C++11 code to JavaScript and exposes all the awe­some HTML5 fea­tures of mod­ern browsers, includ­ing WebGL. There is no mid­dle layer, no inef­fi­cien­cies, no cus­tom half-crippled wid­get library. You write you web page in HTML and CSS, and code the logic with C++. We truly believe this is tak­ing best of both worlds.

We have worked long and hard to pol­ish Duetto and we are now almost sat­is­fied by its capa­bil­i­ties and robust­ness. We are now ready to share this tool with the first 100 devel­op­ers that are going to share their inter­est, in order to col­lect the most feed­back and pro­vide the best pos­si­ble sup­port and experience.

Inter­ested in try­ing out Duetto for free? Visit http://leaningtech.com to get your copy.

If you’re not one of the 100 lucky beta testers, do not despair. We will be releas­ing Duetto in roughly a month. Just enough time to ham­mer out some bugs, add some more awe­some­ness and fig­ure out last details on our licens­ing model. Duetto will be released under a dual licens­ing scheme, so it will be an open source/free soft­ware project, avail­able for any­one. We will also be offer­ing paid com­mer­cial licenses for peo­ple which are not com­fort­able with free soft­ware libraries and/or want enterprise-grade support.

UPDATE: I for­got to link the incred­i­bly awe­some demo by Daniele Di Proi­etto. He used Duetto to port his own imple­men­ta­tion of the “Not tetris” con­cept to the Web.

13 Comments

Duetto (C++ for the Web): CMake integration, Bullet and WebGL demo

At Lean­ingTech we have been work­ing hard and it’s time for some news and updates on the devel­op­ment of duetto, our tool for pro­gram­ming the Web in C++.

Thanks to some key devel­op­ments that we are about to describe, we man­aged to com­pile our first real-world test case in Javascript, the bul­let physics library — and a small demo in webGL that takes advan­tage of it.

The com­pi­la­tion of bul­let in JS with duetto requires very min­i­mal adap­ta­tion to the vanilla code, and can be eas­ily per­formed in full inte­gra­tion with CMake infra­struc­ture. We built a min­i­mal demo to show the result of our tool in action: a tri­an­gle falling on the ground attracted by grav­ity. Please note that the demo is com­pletely not opti­mized, not even by min­i­miz­ing the JS. Any per­for­mance must be con­sid­ered pre­lim­i­nary and there is vast room for improve­ment that we plan to fully exploit

http://leaningtech.com/duettodemo/HelloWorld.html

This was our first trial at com­pil­ing a com­plex code­base, and it went sur­pris­ingly smoothly!

The fol­low­ing fea­tures are now fully sup­ported in duetto:

  • A fully work­ing libc and libc++

Our cur­rent beta fea­tures a full imple­men­ta­tion of the stan­dard C library (based on the newlib imple­men­ta­tion) and of the stan­dard C++ library (based on the LLVM/libc++). Both libraries are stan­dard imple­men­ta­tions, sup­port for duetto was included using the same infra­struc­ture used for other, native, targets.

  • Inte­gra­tion in the CMake/autotool tool­chain as a target

Duetto is now fully inte­grated with the CMake, using the infra­struc­ture nor­mally used for cross com­pil­ing (i.e tool­chain def­i­n­i­tion files). This means you can com­pile com­plex projects that use CMake with lit­tle effort. We also plan to inte­grate with auto­tools in the future.

  • Full sup­port of the DOM and HTML5 APIs (such as WebGL)

Our lat­est beta fea­tures an improved sup­port for using the DOM and WebGL API from C++. This is done by defin­ing a set of head­ers which allows to trans­par­ently access all the browser DOM and libraries. It is also very easy to add sup­port for any JS libraries, fol­low­ing the same approach.

Four months have passed since our ini­tial post about duetto. At that time we promised what we would have released our brain­child in a six months time frame. Well, I’m very happy to say that we are fully in sched­ule and we will be able to release duetto in the fall, under a dual licens­ing scheme: open-source for open-source and non-commercial use, and a paid license for com­mer­cial use.

We have already started shar­ing a lim­ited pri­vate beta with a few devel­op­ers and are inter­ested in expand­ing its release to some more, if inter­ested please con­tact us at info@leaningtech.com

1 Comment

Duetto (C++ for the Web): comparison to asm.js and other clarifications

Around a month ago we posted a first overview of Duetto: our inte­grated LLVM-based solu­tion for pro­gram­ming both the client and the server side of Web appli­ca­tions using C++. We have been com­pletely over­whelmed by the inter­est gen­er­ated by that post!

We tried to keep track of the many insight­ful com­ments and con­struc­tive crit­i­cism we received on this blog and on aggre­ga­tors. Here are the most com­mon ques­tions that arose:

1) Why didn’t you com­pare Duetto with asm.js?

We believe that com­par­ing to asm.js-enabled code is not an apples-to-apples com­par­i­son. In the cur­rent state the Fire­fox asm.js Ahead-Of-Time com­piler can only be enabled on code which is gen­er­ated using the emscripten approach to mem­ory allo­ca­tion, so code gen­er­ated by Duetto has no way to ben­e­fit from it. We are open to dis­cuss how the asm.js approach of val­i­dat­ing typed-ness of code AOT can be extended so that Duetto, and poten­tially other solu­tions, may ben­e­fit from it as well.

Still, it is clearly very inter­est­ing to com­pare Duetto per­for­mance to asm.js, so we have made a new round of bench­mark­ing that includes it. The results are pretty inter­est­ing. As you can see, although asm.js is always faster than emscripten, there are cases in which Duetto on V8 out­per­forms asm.js (on Spi­der­mon­key) as well. We think that this is caused by the highly effi­cient han­dling of objects that V8 pro­vides and shows that there is no clear win­ner. It is very hard to pre­dict what approach will per­form bet­ter, and by how much, on real world scenarios.

 

benchmarks2

For each bench­mark the best time in 10 runs has been selected. The V8 and Spi­der­monky JavaScript shells have been used. The respec­tive com­mits are b13921fa78ce7d7a94ce74f6198db79e075a2e03 and afb7995ef276. Flags: Emscripten (emcc –O2), Asm.js (emcc –O2 –s ASM_JS=1), Duetto and native (clang –fno-math-errno –O2). –fno-math-errno has been added for coherency with inter­nal emcc flags. *The fasta bench­mark has been mod­i­fied by remov­ing a mem­ory allo­ca­tion in a tight loop.

2) Emscripten is effi­cient because it effec­tively dis­ables the Garbage Col­lec­tor. How does Duetto han­dle dynamic memory?

Duetto maps C++ objects to native JS objects. This of course means that, every now and then, the GC will run. Emscripten han­dles mem­ory by pre-allocating all the needed mem­ory and then assign­ing slices of it using a mal­loc imple­men­ta­tion. So yes, since the mem­ory is not really dynamic it can avoid all the per­for­mance impact of the garbage collector

But this advan­tage does not come for free when you include mem­ory con­sump­tion into account. Mem­ory is a resource shared will all the other appli­ca­tions on the sys­tem, both native ones and other web apps. Eagerly reserv­ing mem­ory can have a large effect on the per­for­mance of the sys­tem as a whole.

While the GC may cause a size­able over­head in terms of CPU cycles, it also decreases mem­ory usage by free­ing unneeded mem­ory. Gen­er­ally speak­ing, even when pro­gram­ming on tra­di­tional plat­forms, such as x86, dynamic mem­ory allo­ca­tions and deal­lo­ca­tions should be avoided in per­for­mance sen­si­tive code. Pre-allocating the required mem­ory directly in the appli­ca­tion code is indeed pos­si­ble with Duetto as well.

Since I do not expect native plat­forms (i.e. GLibc) to pre­al­lo­cate gigs of mem­ory at appli­ca­tion startup to make it faster when dynamic mem­ory is actu­ally used, I do not expect this from a com­piler for the JavaScript tar­get either.

3) Is Duetto code inter­op­er­a­ble with exist­ing JS code, libraries and vice-versa?

Yes, absolutely. Meth­ods com­piled by Duetto can be exported using the reg­u­lar C++ man­gling or even with­out the man­gling if they are declared as “extern C”. In both cases the can be freely invoked from pure JavaScript code. At the same time Duetto com­piled code will be able to seam­lessly access all the HTML5/DOM APIs and any JS func­tion. The only require­ment is that the JS inter­face needs to be declared in a C++ header.

We cur­rently auto­mat­i­cally gen­er­ate the header for all the HTML5/DOM APIs using head­ers orig­i­nally writ­ten for Type­Script. We also plan to write head­ers for the most pop­u­lar JS libraries, but we would like to stress that the needed head­ers are just plain C++ class dec­la­ra­tions and no spe­cial addi­tional sup­port in the com­piler is required to sup­port new APIs or new libraries. So when the newest and great­est Web API comes out any­one will be able to write the header to use it with Duetto.

4) Why don’t you release immediately?

Because we want to make the devel­oper expe­ri­ence suf­fi­ciently pol­ished before giv­ing it to users. Any­one how had the mis­for­tune of using buggy com­piler will prob­a­bly appre­ci­ate this. We plan to release it in 6 months from the pre­vi­ous post (so around 5 months from now, time flies) under a dual licens­ing scheme: as Open Source for open source and non com­mer­cial projects, and with a paid license for com­mer­cial development.

5) Will Duetto make reverse engi­neer­ing of orig­i­nal code easy?

In Duetto, C++ code passes through the reg­u­lar Clang/LLVM pipeline before being con­verted to JavaScript. This is the same path used to com­pile assem­bly for any other archi­tec­ture, like x86 or ARM. The gen­er­ated JavaScript is no eas­ier to under­stand than the cor­re­spond­ing machine code or assem­bly. More­over, one of the typ­i­cal opti­miza­tion steps for JavaScript, which we can employ as well, is ‘min­i­miza­tion’ which destroys any resid­ual infor­ma­tion about vari­ables and method names.

6) Will other lan­guages will be supported?

We choose to start from C++ because it’s a lan­guage that has proven to be good enough for very large scale projects and also because it’s the one we know best and love. Cur­rently we have no plans to expand the Duetto archi­tec­ture to other lan­guages, but it is def­i­nitely pos­si­ble that in the future, based on user demand, we could bring our awe­some inte­grated client/server devel­op­ment model to other lan­guages as well.

2 Comments