Archive for May, 2013

Duetto (C++ for the Web): comparison to asm.js and other clarifications

Around a month ago we posted a first overview of Duetto: our inte­grated LLVM-based solu­tion for pro­gram­ming both the client and the server side of Web appli­ca­tions using C++. We have been com­pletely over­whelmed by the inter­est gen­er­ated by that post!

We tried to keep track of the many insight­ful com­ments and con­struc­tive crit­i­cism we received on this blog and on aggre­ga­tors. Here are the most com­mon ques­tions that arose:

1) Why didn’t you com­pare Duetto with asm.js?

We believe that com­par­ing to asm.js-enabled code is not an apples-to-apples com­par­i­son. In the cur­rent state the Fire­fox asm.js Ahead-Of-Time com­piler can only be enabled on code which is gen­er­ated using the emscripten approach to mem­ory allo­ca­tion, so code gen­er­ated by Duetto has no way to ben­e­fit from it. We are open to dis­cuss how the asm.js approach of val­i­dat­ing typed-ness of code AOT can be extended so that Duetto, and poten­tially other solu­tions, may ben­e­fit from it as well.

Still, it is clearly very inter­est­ing to com­pare Duetto per­for­mance to asm.js, so we have made a new round of bench­mark­ing that includes it. The results are pretty inter­est­ing. As you can see, although asm.js is always faster than emscripten, there are cases in which Duetto on V8 out­per­forms asm.js (on Spi­der­mon­key) as well. We think that this is caused by the highly effi­cient han­dling of objects that V8 pro­vides and shows that there is no clear win­ner. It is very hard to pre­dict what approach will per­form bet­ter, and by how much, on real world scenarios.

 

benchmarks2

For each bench­mark the best time in 10 runs has been selected. The V8 and Spi­der­monky JavaScript shells have been used. The respec­tive com­mits are b13921fa78ce7d7a94ce74f6198db79e075a2e03 and afb7995ef276. Flags: Emscripten (emcc –O2), Asm.js (emcc –O2 –s ASM_JS=1), Duetto and native (clang –fno-math-errno –O2). –fno-math-errno has been added for coherency with inter­nal emcc flags. *The fasta bench­mark has been mod­i­fied by remov­ing a mem­ory allo­ca­tion in a tight loop.

2) Emscripten is effi­cient because it effec­tively dis­ables the Garbage Col­lec­tor. How does Duetto han­dle dynamic memory?

Duetto maps C++ objects to native JS objects. This of course means that, every now and then, the GC will run. Emscripten han­dles mem­ory by pre-allocating all the needed mem­ory and then assign­ing slices of it using a mal­loc imple­men­ta­tion. So yes, since the mem­ory is not really dynamic it can avoid all the per­for­mance impact of the garbage collector

But this advan­tage does not come for free when you include mem­ory con­sump­tion into account. Mem­ory is a resource shared will all the other appli­ca­tions on the sys­tem, both native ones and other web apps. Eagerly reserv­ing mem­ory can have a large effect on the per­for­mance of the sys­tem as a whole.

While the GC may cause a size­able over­head in terms of CPU cycles, it also decreases mem­ory usage by free­ing unneeded mem­ory. Gen­er­ally speak­ing, even when pro­gram­ming on tra­di­tional plat­forms, such as x86, dynamic mem­ory allo­ca­tions and deal­lo­ca­tions should be avoided in per­for­mance sen­si­tive code. Pre-allocating the required mem­ory directly in the appli­ca­tion code is indeed pos­si­ble with Duetto as well.

Since I do not expect native plat­forms (i.e. GLibc) to pre­al­lo­cate gigs of mem­ory at appli­ca­tion startup to make it faster when dynamic mem­ory is actu­ally used, I do not expect this from a com­piler for the JavaScript tar­get either.

3) Is Duetto code inter­op­er­a­ble with exist­ing JS code, libraries and vice-versa?

Yes, absolutely. Meth­ods com­piled by Duetto can be exported using the reg­u­lar C++ man­gling or even with­out the man­gling if they are declared as “extern C”. In both cases the can be freely invoked from pure JavaScript code. At the same time Duetto com­piled code will be able to seam­lessly access all the HTML5/DOM APIs and any JS func­tion. The only require­ment is that the JS inter­face needs to be declared in a C++ header.

We cur­rently auto­mat­i­cally gen­er­ate the header for all the HTML5/DOM APIs using head­ers orig­i­nally writ­ten for Type­Script. We also plan to write head­ers for the most pop­u­lar JS libraries, but we would like to stress that the needed head­ers are just plain C++ class dec­la­ra­tions and no spe­cial addi­tional sup­port in the com­piler is required to sup­port new APIs or new libraries. So when the newest and great­est Web API comes out any­one will be able to write the header to use it with Duetto.

4) Why don’t you release immediately?

Because we want to make the devel­oper expe­ri­ence suf­fi­ciently pol­ished before giv­ing it to users. Any­one how had the mis­for­tune of using buggy com­piler will prob­a­bly appre­ci­ate this. We plan to release it in 6 months from the pre­vi­ous post (so around 5 months from now, time flies) under a dual licens­ing scheme: as Open Source for open source and non com­mer­cial projects, and with a paid license for com­mer­cial development.

5) Will Duetto make reverse engi­neer­ing of orig­i­nal code easy?

In Duetto, C++ code passes through the reg­u­lar Clang/LLVM pipeline before being con­verted to JavaScript. This is the same path used to com­pile assem­bly for any other archi­tec­ture, like x86 or ARM. The gen­er­ated JavaScript is no eas­ier to under­stand than the cor­re­spond­ing machine code or assem­bly. More­over, one of the typ­i­cal opti­miza­tion steps for JavaScript, which we can employ as well, is ‘min­i­miza­tion’ which destroys any resid­ual infor­ma­tion about vari­ables and method names.

6) Will other lan­guages will be supported?

We choose to start from C++ because it’s a lan­guage that has proven to be good enough for very large scale projects and also because it’s the one we know best and love. Cur­rently we have no plans to expand the Duetto archi­tec­ture to other lan­guages, but it is def­i­nitely pos­si­ble that in the future, based on user demand, we could bring our awe­some inte­grated client/server devel­op­ment model to other lan­guages as well.

2 Comments

Duetto (C++ for the Web): comparison to asm.js and other clarifications

Around a month ago we posted a first overview of Duetto: our inte­grated LLVM-based solu­tion for pro­gram­ming both the client and the server side of Web appli­ca­tions using C++. We have been com­pletely over­whelmed by the inter­est gen­er­ated by that post!

We tried to keep track of the many insight­ful com­ments and con­struc­tive crit­i­cism we received on this blog and on aggre­ga­tors. Here are the most com­mon ques­tions that arose:

1) Why didn’t you com­pare Duetto with asm.js?

We believe that com­par­ing to asm.js-enabled code is not an apples-to-apples com­par­i­son. In the cur­rent state the Fire­fox asm.js Ahead-Of-Time com­piler can only be enabled on code which is gen­er­ated using the emscripten approach to mem­ory allo­ca­tion, so code gen­er­ated by Duetto has no way to ben­e­fit from it. We are open to dis­cuss how the asm.js approach of val­i­dat­ing typed-ness of code AOT can be extended so that Duetto, and poten­tially other solu­tions, may ben­e­fit from it as well.

Still, it is clearly very inter­est­ing to com­pare Duetto per­for­mance to asm.js, so we have made a new round of bench­mark­ing that includes it. The results are pretty inter­est­ing. As you can see, although asm.js is always faster than emscripten, there are cases in which Duetto on V8 out­per­forms asm.js (on Spi­der­mon­key) as well. We think that this is caused by the highly effi­cient han­dling of objects that V8 pro­vides and shows that there is no clear win­ner. It is very hard to pre­dict what approach will per­form bet­ter, and by how much, on real world scenarios.

 

benchmarks2

For each bench­mark the best time in 10 runs has been selected. The V8 and Spi­der­monky JavaScript shells have been used. The respec­tive com­mits are b13921fa78ce7d7a94ce74f6198db79e075a2e03 and afb7995ef276. Flags: Emscripten (emcc –O2), Asm.js (emcc –O2 –s ASM_JS=1), Duetto and native (clang –fno-math-errno –O2). –fno-math-errno has been added for coherency with inter­nal emcc flags. *The fasta bench­mark has been mod­i­fied by remov­ing a mem­ory allo­ca­tion in a tight loop.

2) Emscripten is effi­cient because it effec­tively dis­ables the Garbage Col­lec­tor. How does Duetto han­dle dynamic memory?

Duetto maps C++ objects to native JS objects. This of course means that, every now and then, the GC will run. Emscripten han­dles mem­ory by pre-allocating all the needed mem­ory and then assign­ing slices of it using a mal­loc imple­men­ta­tion. So yes, since the mem­ory is not really dynamic it can avoid all the per­for­mance impact of the garbage collector

But this advan­tage does not come for free when you include mem­ory con­sump­tion into account. Mem­ory is a resource shared will all the other appli­ca­tions on the sys­tem, both native ones and other web apps. Eagerly reserv­ing mem­ory can have a large effect on the per­for­mance of the sys­tem as a whole.

While the GC may cause a size­able over­head in terms of CPU cycles, it also decreases mem­ory usage by free­ing unneeded mem­ory. Gen­er­ally speak­ing, even when pro­gram­ming on tra­di­tional plat­forms, such as x86, dynamic mem­ory allo­ca­tions and deal­lo­ca­tions should be avoided in per­for­mance sen­si­tive code. Pre-allocating the required mem­ory directly in the appli­ca­tion code is indeed pos­si­ble with Duetto as well.

Since I do not expect native plat­forms (i.e. GLibc) to pre­al­lo­cate gigs of mem­ory at appli­ca­tion startup to make it faster when dynamic mem­ory is actu­ally used, I do not expect this from a com­piler for the JavaScript tar­get either.

3) Is Duetto code inter­op­er­a­ble with exist­ing JS code, libraries and vice-versa?

Yes, absolutely. Meth­ods com­piled by Duetto can be exported using the reg­u­lar C++ man­gling or even with­out the man­gling if they are declared as “extern C”. In both cases the can be freely invoked from pure JavaScript code. At the same time Duetto com­piled code will be able to seam­lessly access all the HTML5/DOM APIs and any JS func­tion. The only require­ment is that the JS inter­face needs to be declared in a C++ header.

We cur­rently auto­mat­i­cally gen­er­ate the header for all the HTML5/DOM APIs using head­ers orig­i­nally writ­ten for Type­Script. We also plan to write head­ers for the most pop­u­lar JS libraries, but we would like to stress that the needed head­ers are just plain C++ class dec­la­ra­tions and no spe­cial addi­tional sup­port in the com­piler is required to sup­port new APIs or new libraries. So when the newest and great­est Web API comes out any­one will be able to write the header to use it with Duetto.

4) Why don’t you release immediately?

Because we want to make the devel­oper expe­ri­ence suf­fi­ciently pol­ished before giv­ing it to users. Any­one how had the mis­for­tune of using buggy com­piler will prob­a­bly appre­ci­ate this. We plan to release it in 6 months from the pre­vi­ous post (so around 5 months from now, time flies) under a dual licens­ing scheme: as Open Source for open source and non com­mer­cial projects, and with a paid license for com­mer­cial development.

5) Will Duetto make reverse engi­neer­ing of orig­i­nal code easy?

In Duetto, C++ code passes through the reg­u­lar Clang/LLVM pipeline before being con­verted to JavaScript. This is the same path used to com­pile assem­bly for any other archi­tec­ture, like x86 or ARM. The gen­er­ated JavaScript is no eas­ier to under­stand than the cor­re­spond­ing machine code or assem­bly. More­over, one of the typ­i­cal opti­miza­tion steps for JavaScript, which we can employ as well, is ‘min­i­miza­tion’ which destroys any resid­ual infor­ma­tion about vari­ables and method names.

6) Will other lan­guages will be supported?

We choose to start from C++ because it’s a lan­guage that has proven to be good enough for very large scale projects and also because it’s the one we know best and love. Cur­rently we have no plans to expand the Duetto archi­tec­ture to other lan­guages, but it is def­i­nitely pos­si­ble that in the future, based on user demand, we could bring our awe­some inte­grated client/server devel­op­ment model to other lan­guages as well.

2 Comments