Duetto 0.9.6: debugging C++ web applications within the browser

logo_duetto_quadrato_192

Hello every­one! Lean­ing Tech­nolo­gies is happy to present yet another feature-packed release of duetto — our C++ com­piler for the Web.

Duetto 0.9.6 intro­duces the ini­tial sup­port for a new, highly-requested fea­ture: in-browser debug­ging of your C++ web appli­ca­tion. We hope you’ll like this! We also intro­duced some new pow­er­ful opti­miza­tions to reduce mem­ory usage and pres­sure on the garbage col­lec­tor, and extended our sup­port for the stan­dard library.

Ini­tial sup­port for inte­grated debug­ging using Source Maps

Source maps is a stan­dard tech­nol­ogy to debug code com­piled to JavaScript. It’s already used by many tools that gen­er­ate JavaScript, both from other lan­guages and from JavaScript itself. Duetto is now able to seam­lessly gen­er­ate source maps from C++ code.

Mod­ern browsers can auto­mat­i­cally load the source map and dis­play the orig­i­nal C++ code when an error occurs. You can also set break­points in the C++ file and the JavaScript exe­cu­tion will stop when the cor­re­spond­ing line is reached. You can even sin­gle step the exe­cu­tion while look­ing at C++ code.


To gen­er­ate a source map, you can use the fol­low­ing com­mand (remem­ber to enable debug infor­ma­tion in clang using the –g option):

/opt/duetto/bin/clang++ -target duetto -g test.cpp -o test.js -duetto-sourcemap=test.js.map

Enabled Scalar Replace­ment of Aggre­gates (SROA) optimization

SROA is an stan­dard com­piler opti­miza­tion which tries to replace com­plex objects with their mem­bers when­ever pos­si­ble. It’s a very crit­i­cal opti­miza­tion, espe­cially for duetto, as every object which removed by SROA reduces the pres­sure on the garbage col­lec­tor. LLVM has SROA sup­port out of the box, but the exist­ing imple­men­ta­tion would often gen­er­ate type-unsafe code that would break the duetto back­end, so we have been forced to dis­able SROA up to now.

This release includes an improved SROA which always gen­er­ates duetto-safe code, so we have been able to enable it again.

First sup­port for col­laps­ing allocations

We have intro­duced a new opti­miza­tions which tries to reduce mem­ory pres­sure even more by recy­cling mem­ory allo­cated for local vari­ables. When­ever two or more allo­ca­tions of the same type are used in dis­joint sec­tions of the same func­tion they are merged into one.

Improved stan­dard library support

We have fixed sup­port for std::map, std::multimap, std::unordered_map, std::std, std::multiset and std::unordered_set which should be now fully work­ing. sets and maps hav­ing point­ers as the key are not yet supported.

Strict-mode JavaScript

Duetto-generated JavaScript now starts with “use strict”, a cou­ple of small fixes have been made to gen­er­ate fully strict-mode com­pli­ant code.

You can down­load duetto for Win­dows and Mac OS X on launch­pad. Pack­ages for Ubuntu and Debian are avail­able from our PPA. You can also get the source code on GitHub.

Fol­low us on @leaningtech, Face­book and at www.leaningtech.com for updates.

No Comments

Duetto 0.9.6: debugging C++ web applications within the browser

logo_duetto_quadrato_192

Hello every­one! Lean­ing Tech­nolo­gies is happy to present yet another feature-packed release of duetto — our C++ com­piler for the Web.

Duetto 0.9.6 intro­duces the ini­tial sup­port for a new, highly-requested fea­ture: in-browser debug­ging of your C++ web appli­ca­tion. We hope you’ll like this! We also intro­duced some new pow­er­ful opti­miza­tions to reduce mem­ory usage and pres­sure on the garbage col­lec­tor, and extended our sup­port for the stan­dard library.

Ini­tial sup­port for inte­grated debug­ging using Source Maps

Source maps is a stan­dard tech­nol­ogy to debug code com­piled to JavaScript. It’s already used by many tools that gen­er­ate JavaScript, both from other lan­guages and from JavaScript itself. Duetto is now able to seam­lessly gen­er­ate source maps from C++ code.

Mod­ern browsers can auto­mat­i­cally load the source map and dis­play the orig­i­nal C++ code when an error occurs. You can also set break­points in the C++ file and the JavaScript exe­cu­tion will stop when the cor­re­spond­ing line is reached. You can even sin­gle step the exe­cu­tion while look­ing at C++ code.


To gen­er­ate a source map, you can use the fol­low­ing com­mand (remem­ber to enable debug infor­ma­tion in clang using the –g option):

/opt/duetto/bin/clang++ -target duetto -g test.cpp -o test.js -duetto-sourcemap=test.js.map

Enabled Scalar Replace­ment of Aggre­gates (SROA) optimization

SROA is an stan­dard com­piler opti­miza­tion which tries to replace com­plex objects with their mem­bers when­ever pos­si­ble. It’s a very crit­i­cal opti­miza­tion, espe­cially for duetto, as every object which removed by SROA reduces the pres­sure on the garbage col­lec­tor. LLVM has SROA sup­port out of the box, but the exist­ing imple­men­ta­tion would often gen­er­ate type-unsafe code that would break the duetto back­end, so we have been forced to dis­able SROA up to now.

This release includes an improved SROA which always gen­er­ates duetto-safe code, so we have been able to enable it again.

First sup­port for col­laps­ing allocations

We have intro­duced a new opti­miza­tions which tries to reduce mem­ory pres­sure even more by recy­cling mem­ory allo­cated for local vari­ables. When­ever two or more allo­ca­tions of the same type are used in dis­joint sec­tions of the same func­tion they are merged into one.

Improved stan­dard library support

We have fixed sup­port for std::map, std::multimap, std::unordered_map, std::std, std::multiset and std::unordered_set which should be now fully work­ing. sets and maps hav­ing point­ers as the key are not yet supported.

Strict-mode JavaScript

Duetto-generated JavaScript now starts with “use strict”, a cou­ple of small fixes have been made to gen­er­ate fully strict-mode com­pli­ant code.

You can down­load duetto for Win­dows and Mac OS X on launch­pad. Pack­ages for Ubuntu and Debian are avail­able from our PPA. You can also get the source code on GitHub.

Fol­low us on @leaningtech, Face­book and at www.leaningtech.com for updates.

No Comments

Duetto (C++ for the Web) 0.9.4: Async RPCs, Promises, JavaScript interoperability

logo_duetto_quadrato_192

Around a month has passed since our talk at mloc.js. There we had a great oppor­tu­nity to talk about the tech­nol­ogy behind the duetto C++ com­piler for the Web. We also made our first pub­lic announce­ment of our promise based sup­port for asyn­chro­nous type safe RPC, which fits in our vision of C++ as the plat­form lan­guage for the Web, both on the client and on the server side.

Today, we announce the new, feature-packed release of Duetto — 0.9.4! This time, we really splurged, and added some qual­ity fea­tures that we’re sure you’ll find interesting:

  • Ini­tial sup­port for promise-based async RPC

  • Strong inter­op­er­abil­ity with JavaScript

    • __asm__ to exe­cute JavaScript from C++ code

    • [[jsex­port]] attribute to use C++ objects from JavaScript

  • Sup­port for C++ mutex/atomic/thread headers

As usual, you can find the full source code of the release here, plus binary pack­ages for Debian/Ubuntu on our PPA, a binary setup for win­dows and a binary archive for Mac OS X here.

Asyn­chro­nous RPC using promises

With duetto you can “tag” a method to let the com­piler know that the code should be com­piled for the server side. The com­piler also takes care of gen­er­at­ing RPC boil­er­plate includ­ing type safe seri­al­iza­tion of para­me­ters and dese­ri­al­iza­tion of the return value. While the basic model is syn­chro­nous, you can take advan­tage of promises to use fully type safe asyn­chro­nous RPC in duetto. See e.g. the fol­low­ing code:

// Pending requests for messages
vector<Promise<string>*> pendingRequests;
Promise<string>* getChatMessageRemote() [[server]]
{
     auto ret=new Promise<string>();
     // Store the new promise, you can use it later to complete
     // the request in asynchronous manner
     pendingRequests.push_back(ret);
     return ret;
}
void sendMessageRemote(const string& str) [[server]]
{
     for (auto p: pendingRequests)
          p->done(str);
     pendingRequests.clear();
}
void messageHandler(const string& newMessage) [[client]]
{
     client::console.log("Message received",newMessage.c_str());
}
void webMain() [[client]]
{
     // First wait for messages
     auto promise = getChatMessageRemote();
     // Add a callback to the promise
     promise->then(messageHandler);
     // Tell the promise that no more callbacks will be added
     promise->complete();
     // Now send a message, it will be echoed back
     sendMessageRemote("Test message");
}

The code is a sim­ple but work­ing exam­ple of a multi-user chat appli­ca­tion. The getChatMes­sageR­e­mote method returns a promise imme­di­ately after being called, with­out wait­ing for the server answer. On the server side we cre­ate a promise and return it. We also need to store it some­where to even­tu­ally ful­fill the promise by send­ing a value.

The client code needs to set a call­back to receive the value when the promise will be even­tu­ally ful­filled. Later on the client send a chat mes­sage using the sendMes­sageR­e­mote server method. This method is syn­chro­nous and will return only after the server side exe­cu­tion is com­pleted. The server echoes the mes­sage back to all con­nected clients by ful­fill­ing all pend­ing promises.

Strong JS interoperability

This release of duetto includes two fea­tures that pro­vide a supe­rior inte­gra­tion with man­u­ally writ­ten JavaScript code: the [[jsex­port]] class attribute and __asm__ sup­port for inline JavaScript.

[[jsex­port]] class attribute

We have intro­duced pre­lim­i­nary sup­port for a new attribute: [[jsex­port]]. This attribute is designed to be applied on C++ classes, doing so will cause make the class avail­able to man­u­ally writ­ten JavaScript code. The fol­low­ing exam­ple shows its typ­i­cal use:

class [[jsexport]] ExportExample
{
private:
    float a;
    int b;
public:
     ExportExample(float _a, int _b):a(_a),b(_b)
     {
     }
     void testMethod()
     {
          client::console.log(“Float value is”, a);
     }
};

Dur­ing com­pi­la­tion duetto will gen­er­ate bridge code to expose the Expor­tEx­am­ple C++ class to JavaScript code. You can then use the class when writ­ing JS code

// Using jsexport-ed classes from JavaScript code
var cppobj = new ExportExample(0.1, 42);
cppobj.testMethod();

The [[jex­port]] attribute places some lim­i­ta­tions on the class. In par­tic­u­lar, the object must have a triv­ial destruc­tor and no over­loaded meth­ods. More­over oper­a­tors can’t be exported to JavaScript. Hav­ing a triv­ial destruc­tor is nec­es­sary to make sure that the reg­u­lar Garbage Col­lec­tion mech­a­nism is suf­fi­cient to reclaim the object. We plan to expand this fea­ture in the future to reduce any lim­i­ta­tions as much as possible.

Inline JavaScript using __asm__

This release also adds sup­port for insert­ing arbi­trary JavaScript code inside C++ code using the __asm__ func­tion­al­ity. Our design choice has been to put no limit on what you can do inside __asm__, includ­ing break­ing the pro­gram, so be care­ful when tak­ing advan­tage of this capability.

void webMain()
{
      __asm__(“alert(‘JavaScript code inside __asm__’);”);
}

At the moment only this sim­ple form of JS code inser­tion is sup­ported. It’s not cur­rently pos­si­ble to pass vari­ables from sur­round­ing C++ code or return a value from __asm__, but we will extend this sup­port in the future.

Sup­port for C++ mutex/atomic/thread headers

Some C++ func­tion­al­i­ties, espe­cially the ones related to multi-threading, do not have a direct map­ping on the browser plat­form. That said, we want to pro­vide as much com­pat­i­bil­ity as pos­si­ble with exist­ing C++ code to reduce the effort when using duetto for porting.

As of duetto 0.9.4, we sup­port the fol­low­ing C++ thread­ing primitives:

  • mutex: Imple­mented as plain coun­ters, mutexes can­not block exe­cu­tion but report an error if a non-recursive mutex is acquired twice.

  • atomic: Imple­mented as plain inte­gers, since the browser is not actu­ally con­cur­rent there is no real sup­port for atomic operations.

  • thread: Mod­ern browsers sup­port Web­Work­ers, but the thread­ing model does not map to the one expected by C++ threads. We chose to make the user aware that C++ threads are not sup­ported by adding an explicit error mes­sage when the thread header is included in code com­piled with duetto.

 Fol­low us on @leaningtech, Face­book and at www.leaningtech.com for updates.

2 Comments

Duetto (C++ for the Web) 0.9.4: Async RPCs, Promises, JavaScript interoperability

logo_duetto_quadrato_192

Around a month has passed since our talk at mloc.js. There we had a great oppor­tu­nity to talk about the tech­nol­ogy behind the duetto C++ com­piler for the Web. We also made our first pub­lic announce­ment of our promise based sup­port for asyn­chro­nous type safe RPC, which fits in our vision of C++ as the plat­form lan­guage for the Web, both on the client and on the server side.

Today, we announce the new, feature-packed release of Duetto — 0.9.4! This time, we really splurged, and added some qual­ity fea­tures that we’re sure you’ll find interesting:

  • Ini­tial sup­port for promise-based async RPC

  • Strong inter­op­er­abil­ity with JavaScript

    • __asm__ to exe­cute JavaScript from C++ code

    • [[jsex­port]] attribute to use C++ objects from JavaScript

  • Sup­port for C++ mutex/atomic/thread headers

As usual, you can find the full source code of the release here, plus binary pack­ages for Debian/Ubuntu on our PPA, a binary setup for win­dows and a binary archive for Mac OS X here.

Asyn­chro­nous RPC using promises

With duetto you can “tag” a method to let the com­piler know that the code should be com­piled for the server side. The com­piler also takes care of gen­er­at­ing RPC boil­er­plate includ­ing type safe seri­al­iza­tion of para­me­ters and dese­ri­al­iza­tion of the return value. While the basic model is syn­chro­nous, you can take advan­tage of promises to use fully type safe asyn­chro­nous RPC in duetto. See e.g. the fol­low­ing code:

// Pending requests for messages
vector<Promise<string>*> pendingRequests;
Promise<string>* getChatMessageRemote() [[server]]
{
     auto ret=new Promise<string>();
     // Store the new promise, you can use it later to complete
     // the request in asynchronous manner
     pendingRequests.push_back(ret);
     return ret;
}
void sendMessageRemote(const string& str) [[server]]
{
     for (auto p: pendingRequests)
          p->done(str);
     pendingRequests.clear();
}
void messageHandler(const string& newMessage) [[client]]
{
     client::console.log("Message received",newMessage.c_str());
}
void webMain() [[client]]
{
     // First wait for messages
     auto promise = getChatMessageRemote();
     // Add a callback to the promise
     promise->then(messageHandler);
     // Tell the promise that no more callbacks will be added
     promise->complete();
     // Now send a message, it will be echoed back
     sendMessageRemote("Test message");
}

The code is a sim­ple but work­ing exam­ple of a multi-user chat appli­ca­tion. The getChatMes­sageR­e­mote method returns a promise imme­di­ately after being called, with­out wait­ing for the server answer. On the server side we cre­ate a promise and return it. We also need to store it some­where to even­tu­ally ful­fill the promise by send­ing a value.

The client code needs to set a call­back to receive the value when the promise will be even­tu­ally ful­filled. Later on the client send a chat mes­sage using the sendMes­sageR­e­mote server method. This method is syn­chro­nous and will return only after the server side exe­cu­tion is com­pleted. The server echoes the mes­sage back to all con­nected clients by ful­fill­ing all pend­ing promises.

Strong JS interoperability

This release of duetto includes two fea­tures that pro­vide a supe­rior inte­gra­tion with man­u­ally writ­ten JavaScript code: the [[jsex­port]] class attribute and __asm__ sup­port for inline JavaScript.

[[jsex­port]] class attribute

We have intro­duced pre­lim­i­nary sup­port for a new attribute: [[jsex­port]]. This attribute is designed to be applied on C++ classes, doing so will cause make the class avail­able to man­u­ally writ­ten JavaScript code. The fol­low­ing exam­ple shows its typ­i­cal use:

class [[jsexport]] ExportExample
{
private:
    float a;
    int b;
public:
     ExportExample(float _a, int _b):a(_a),b(_b)
     {
     }
     void testMethod()
     {
          client::console.log(“Float value is”, a);
     }
};

Dur­ing com­pi­la­tion duetto will gen­er­ate bridge code to expose the Expor­tEx­am­ple C++ class to JavaScript code. You can then use the class when writ­ing JS code

// Using jsexport-ed classes from JavaScript code
var cppobj = new ExportExample(0.1, 42);
cppobj.testMethod();

The [[jex­port]] attribute places some lim­i­ta­tions on the class. In par­tic­u­lar, the object must have a triv­ial destruc­tor and no over­loaded meth­ods. More­over oper­a­tors can’t be exported to JavaScript. Hav­ing a triv­ial destruc­tor is nec­es­sary to make sure that the reg­u­lar Garbage Col­lec­tion mech­a­nism is suf­fi­cient to reclaim the object. We plan to expand this fea­ture in the future to reduce any lim­i­ta­tions as much as possible.

Inline JavaScript using __asm__

This release also adds sup­port for insert­ing arbi­trary JavaScript code inside C++ code using the __asm__ func­tion­al­ity. Our design choice has been to put no limit on what you can do inside __asm__, includ­ing break­ing the pro­gram, so be care­ful when tak­ing advan­tage of this capability.

void webMain()
{
      __asm__(“alert(‘JavaScript code inside __asm__’);”);
}

At the moment only this sim­ple form of JS code inser­tion is sup­ported. It’s not cur­rently pos­si­ble to pass vari­ables from sur­round­ing C++ code or return a value from __asm__, but we will extend this sup­port in the future.

Sup­port for C++ mutex/atomic/thread headers

Some C++ func­tion­al­i­ties, espe­cially the ones related to multi-threading, do not have a direct map­ping on the browser plat­form. That said, we want to pro­vide as much com­pat­i­bil­ity as pos­si­ble with exist­ing C++ code to reduce the effort when using duetto for porting.

As of duetto 0.9.4, we sup­port the fol­low­ing C++ thread­ing primitives:

  • mutex: Imple­mented as plain coun­ters, mutexes can­not block exe­cu­tion but report an error if a non-recursive mutex is acquired twice.

  • atomic: Imple­mented as plain inte­gers, since the browser is not actu­ally con­cur­rent there is no real sup­port for atomic operations.

  • thread: Mod­ern browsers sup­port Web­Work­ers, but the thread­ing model does not map to the one expected by C++ threads. We chose to make the user aware that C++ threads are not sup­ported by adding an explicit error mes­sage when the thread header is included in code com­piled with duetto.

 Fol­low us on @leaningtech, Face­book and at www.leaningtech.com for updates.

2 Comments

Duetto (C++ for the Web) 0.9.3 is out — Usability and standard library improvements, Windows installer

logo_duetto_quadrato_192

At Lean­ing Tech­nolo­gies we are glad to announce our newest release of duetto, our C++ com­piler for Web applications.

This release, marked as 0.9.3, con­tains some improvements:

  • Gen­er­ated JS files are now self con­tained and do not need the duetto.js helper file

  • C++11 is now the default com­pi­la­tion mode

  • New Win­dows installer

  • Improve­ments to stan­dard C++ library sup­port, in par­tic­u­lar now iostream, includ­ing cout and cerr are supported

  • std­out and stderr sup­port, using browser console

We also have a new major release in the works, which will pro­vide some quite inter­est­ing new fea­tures to duetto. We will announce some of them at the mloc.js con­fer­ence in Budapest this Feb­ru­ary 13 – 14.

You can find bina­ries and installers of duetto for Win­dows and Mac OS X on launch­pad. Pack­ages for Ubuntu and Debian are avail­able from our PPA. Source tar­balls are avail­able on launch­pad as well.

Instal­la­tion and get­ting started guides are avail­able at wiki.leaningtech.com

Fol­low us on @leaningtech, Face­book and at www.leaningtech.com for updates.

No Comments

Duetto (C++ for the Web) 0.9.3 is out — Usability and standard library improvements, Windows installer

logo_duetto_quadrato_192

At Lean­ing Tech­nolo­gies we are glad to announce our newest release of duetto, our C++ com­piler for Web applications.

This release, marked as 0.9.3, con­tains some improvements:

  • Gen­er­ated JS files are now self con­tained and do not need the duetto.js helper file

  • C++11 is now the default com­pi­la­tion mode

  • New Win­dows installer

  • Improve­ments to stan­dard C++ library sup­port, in par­tic­u­lar now iostream, includ­ing cout and cerr are supported

  • std­out and stderr sup­port, using browser console

We also have a new major release in the works, which will pro­vide some quite inter­est­ing new fea­tures to duetto. We will announce some of them at the mloc.js con­fer­ence in Budapest this Feb­ru­ary 13 – 14.

You can find bina­ries and installers of duetto for Win­dows and Mac OS X on launch­pad. Pack­ages for Ubuntu and Debian are avail­able from our PPA. Source tar­balls are avail­able on launch­pad as well.

Instal­la­tion and get­ting started guides are avail­able at wiki.leaningtech.com

Fol­low us on @leaningtech, Face­book and at www.leaningtech.com for updates.

No Comments

Duetto (C++ for the Web) 0.9.2 is out — OpenGL ES implementation in WebGL and toolchain improvements


A lot of progress was made here at Lean­ingtech dur­ing the last few weeks, lead­ing to our lat­est release of duetto — ver­sion 0.9.2.

For this ver­sion we have been focus­ing mainly on usabil­ity. The most rel­e­vant updates include:

  • A WebGL-based OpenGL ES implementation

  • Auto­matic link­ing of sys­tem libraries;

  • Gen­eral tool­chain usabil­ity improve­ment, includ­ing sim­pli­fied com­mand line syn­tax for com­mon operations.

Duetto’s OpenGL ES imple­men­ta­tion exposes the stan­dard API avail­able on desk­top and mobile to Web devel­op­ers. We have designed it to make port­ing of exist­ing GLES apps and games eas­ier, but it is of course pos­si­ble to use it in new code. More­over, we expose the under­ly­ing WebGL con­text to make it pos­si­ble to mix and match between GLES and WebGL code, to take advan­tage of the native capa­bil­i­ties of the browser like using JPEG and PNG com­pressed images as tex­tures. This GLES imple­men­ta­tion is used in this Non­tetris game, devel­oped in C++ using duetto. You can find its source code here.

The last two points are aimed at a gen­eral effort of increas­ing the ease of use of duetto — and reduc­ing the like­li­hood of the most fre­quent errors related to miss­ing links to libraries.

You can find binary archives of duetto for Win­dows and Mac OS X on launch­pad. Pack­ages for Ubuntu and Debian are avail­able from our PPA. Source tar­balls are avail­able on launch­pad as well.

We have started an open wiki here. On the wiki you can already find instal­la­tion instruc­tions for the var­i­ous plat­forms, build instruc­tions and the get­ting started guide. Feel also free to con­tribute to the wiki if you feel some­thing is missing.

We are also happy to announce that we will be speak­ing at the mloc.js con­fer­ence in Feb­ru­ary in Budapest. We look for­ward to it as a great chance to meet some awe­some peo­ple work­ing on JS as a com­piler tar­get and dis­cuss about our technology.

Fol­low us on @leaningtech, Face­book and at www.leaningtech.com for updates.

No Comments

Duetto (C++ for the Web) 0.9.2 is out — OpenGL ES implementation in WebGL and toolchain improvements


A lot of progress was made here at Lean­ingtech dur­ing the last few weeks, lead­ing to our lat­est release of duetto — ver­sion 0.9.2.

For this ver­sion we have been focus­ing mainly on usabil­ity. The most rel­e­vant updates include:

  • A WebGL-based OpenGL ES implementation

  • Auto­matic link­ing of sys­tem libraries;

  • Gen­eral tool­chain usabil­ity improve­ment, includ­ing sim­pli­fied com­mand line syn­tax for com­mon operations.

Duetto’s OpenGL ES imple­men­ta­tion exposes the stan­dard API avail­able on desk­top and mobile to Web devel­op­ers. We have designed it to make port­ing of exist­ing GLES apps and games eas­ier, but it is of course pos­si­ble to use it in new code. More­over, we expose the under­ly­ing WebGL con­text to make it pos­si­ble to mix and match between GLES and WebGL code, to take advan­tage of the native capa­bil­i­ties of the browser like using JPEG and PNG com­pressed images as tex­tures. This GLES imple­men­ta­tion is used in this Non­tetris game, devel­oped in C++ using duetto. You can find its source code here.

The last two points are aimed at a gen­eral effort of increas­ing the ease of use of duetto — and reduc­ing the like­li­hood of the most fre­quent errors related to miss­ing links to libraries.

You can find binary archives of duetto for Win­dows and Mac OS X on launch­pad. Pack­ages for Ubuntu and Debian are avail­able from our PPA. Source tar­balls are avail­able on launch­pad as well.

We have started an open wiki here. On the wiki you can already find instal­la­tion instruc­tions for the var­i­ous plat­forms, build instruc­tions and the get­ting started guide. Feel also free to con­tribute to the wiki if you feel some­thing is missing.

We are also happy to announce that we will be speak­ing at the mloc.js con­fer­ence in Feb­ru­ary in Budapest. We look for­ward to it as a great chance to meet some awe­some peo­ple work­ing on JS as a com­piler tar­get and dis­cuss about our technology.

Fol­low us on @leaningtech, Face­book and at www.leaningtech.com for updates.

No Comments

Comparing Duetto with Emscripten: a follow-up

logo_duetto_quadrato_192

Yes­ter­day, Alon Zakai (alias krip­ken, cre­ator of Emscripten) wrote an excel­lent post com­par­ing three C++ to JS com­pil­ers: Man­dreel, Emscripten and Duetto. We would like to pro­vide a few more details regard­ing Duetto as we feel that, although the com­par­i­son was def­i­nitely fair, we can add some points which have been missed.

Mem­ory model: Con­trar­ily to Man­dreel and Emscripten, which emu­late a tra­di­tional flat address space using a sin­gle­ton typed array, Duetto maps C++ objects to JS objects. This nat­u­rally implies an over­head both in terms of meta-data that the JS engine man­ages and in terms of com­pu­ta­tional work that the Garbage Col­lec­tor (GC) needs to do. While these may look like major dis­ad­van­tages, other fac­tors should be taken into account. The com­pu­ta­tional over­head does not go wasted, as it is func­tional to keep mem­ory usage in check and make sure that the amount of occu­pied mem­ory is closer to the amount actu­ally required.

In addi­tion, the typed array based mem­ory model incurs in some over­head as well,  as mem­ory areas are still man­aged using some kind of mal­loc. And they also suf­fer a lot from frag­men­ta­tion as the browser can’t reclaim mem­ory in the mid­dle of the array which is not cur­rently being used.

We strongly believe that the duetto mem­ory model being based on JS objects is more fair to other appli­ca­tions, both native and web-based and to the user’s sys­tem as a whole. If the sys­tem gets tight on mem­ory, it is pos­si­ble for the browser to reclaim cur­rently unused objects used in code gen­er­ated by duetto, on the other hand when a large typed array is used, the sys­tem can at best page-out unused or sel­dom used pages to swap since it has no way to know what mem­ory is actu­ally used and what is not. On the other hand it is def­i­nitely true that fre­quently allo­cat­ing and deal­lo­cat­ing objects may cause an unac­cept­able slow­down, but this is actu­ally true for native pro­grams as well and the solu­tion is to re-use the same objects over and over (for exam­ple by cre­at­ing pools) at the appli­ca­tion level, and this is some­thing that is, of course, pos­si­ble using duetto as well. Gen­er­ally we believe that using JS objects also make duetto com­piled code more sim­i­lar to hand­writ­ten JS code, which is already han­dled very well by JS engines, with the added advan­tage of guar­an­teed type immutabil­ity which can be exploited by the engine to gen­er­ate faster code. Still, duetto is flex­i­ble enough that it will be pos­si­ble, in the future, to sup­port using an emscripten like mem­ory model on a type-by-type basis. This would be use­ful to adapt to var­i­ous work­loads and for com­pat­i­bil­ity with code com­piled by emscripten.

APIs: Duetto is built around the con­cept that the browser is a plat­form hav­ing JS as the machine lan­guage. Such plat­form has APIs like WebGL, HTML5 Can­vas, DOM events and so on which are the low­est level acces­si­ble func­tion­al­i­ties. With Duetto you can use all of them directly from C++ and no JS wrap­per code is required.

What about the cross plat­form APIs used on native sys­tems like SDL and Ope­nAL? Well, we don’t cur­rently sup­port them, but they can be ported to Duetto by writ­ing a new back­end for the browser plat­form, sim­i­larly to what has been already done to port them to Win­dows, Linux and Mac. And using duetto it is pos­si­ble to write such port­ing code directly in C++ because you have direct access to the browser APIs. Once the cross plat­form APIs sup­port duetto, appli­ca­tions using them will run as well with­out much (or any) work.

And what about GLES? Well, we received so many requests for this func­tion­al­ity that we are work­ing on pro­vid­ing a WebGL based GLES imple­men­ta­tion our­selves. We plan to release it very soon. With that it should be pos­si­ble to build pro­grams using GLES with duetto by only chang­ing the win­dow­ing sys­tem ini­tial­iza­tion code, sim­i­larly to port­ing from win­dows’ WGL to X11 glX.

C++ com­pat­i­bil­ity: Alon cor­rectly states that duetto has “par­tial” C++ com­pat­i­bil­ity. This is true, but given the incred­i­ble amount of capa­bil­i­ties that the C++ lan­guage has, it is impor­tant to clar­ify what these lim­i­ta­tions are.

Let’s start from the basis: our C++ com­piler was of course not writ­ten from scratch, but was based on clang. This means that we inherit com­plete sup­port for all C++ fea­tures, includ­ing recent and advanced ones like C++11, tem­plate sup­port and lambda. And we actu­ally use such fea­tures in the C++ code which man­ages trans­par­ent client/server RPCs.

The lim­i­ta­tions are more about a few spe­cific unsup­ported capa­bil­i­ties, cur­rently the major ones are: miss­ing sup­port for vir­tual inheritance/virtual base classes, some issues with pointer com­par­i­son in a cou­ple of cor­ner cases and miss­ing sup­port for some parts of the stan­dard library. But we would like to stress that those lim­i­ta­tions are (mostly) non struc­tural and we will do our best to increase the com­pat­i­bil­ity as much as pos­si­ble while still be coher­ent with C++ phi­los­o­phy of expos­ing all plat­form capa­bil­i­ties and lim­i­ta­tions to the user.

Per­for­mance: We have worked with com­puter sys­tems long enough to all agree that micro-benchmarks have lit­tle to do with per­for­mance on real world sce­nar­ios. We also know that Duetto still pro­duces sub­op­ti­mal code for many com­mon cases that we plan to fix. We look for­ward to try and pro­file Duetto gen­er­ated code from a large, real world code base, to find out how it per­forms and what needs to be optimized.

In sum­mary, we think that Alon’s review of our tech­nol­ogy is pretty accu­rate, and we look for­ward mak­ing more and more of Duetto’s fea­tures known to the com­mu­nity, with ded­i­cated posts and doc­u­men­ta­tion. We have been impressed by the feed­back we got with our first release — and look for­ward to hear your opinion.

Duetto release 0.9.1 is now avail­able as source tar­balls and bina­ries for Ubuntu/Debian (PPA) and Win­dows. MacOSX builds will be avail­able soon. Feel free to report bugs, and request fea­tures here.

Fol­low us on @leaningtech, Face­book and at www.leaningtech.com for updates.

No Comments

Comparing Duetto with Emscripten: a follow-up

logo_duetto_quadrato_192

Yes­ter­day, Alon Zakai (alias krip­ken, cre­ator of Emscripten) wrote an excel­lent post com­par­ing three C++ to JS com­pil­ers: Man­dreel, Emscripten and Duetto. We would like to pro­vide a few more details regard­ing Duetto as we feel that, although the com­par­i­son was def­i­nitely fair, we can add some points which have been missed.

Mem­ory model: Con­trar­ily to Man­dreel and Emscripten, which emu­late a tra­di­tional flat address space using a sin­gle­ton typed array, Duetto maps C++ objects to JS objects. This nat­u­rally implies an over­head both in terms of meta-data that the JS engine man­ages and in terms of com­pu­ta­tional work that the Garbage Col­lec­tor (GC) needs to do. While these may look like major dis­ad­van­tages, other fac­tors should be taken into account. The com­pu­ta­tional over­head does not go wasted, as it is func­tional to keep mem­ory usage in check and make sure that the amount of occu­pied mem­ory is closer to the amount actu­ally required.

In addi­tion, the typed array based mem­ory model incurs in some over­head as well,  as mem­ory areas are still man­aged using some kind of mal­loc. And they also suf­fer a lot from frag­men­ta­tion as the browser can’t reclaim mem­ory in the mid­dle of the array which is not cur­rently being used.

We strongly believe that the duetto mem­ory model being based on JS objects is more fair to other appli­ca­tions, both native and web-based and to the user’s sys­tem as a whole. If the sys­tem gets tight on mem­ory, it is pos­si­ble for the browser to reclaim cur­rently unused objects used in code gen­er­ated by duetto, on the other hand when a large typed array is used, the sys­tem can at best page-out unused or sel­dom used pages to swap since it has no way to know what mem­ory is actu­ally used and what is not. On the other hand it is def­i­nitely true that fre­quently allo­cat­ing and deal­lo­cat­ing objects may cause an unac­cept­able slow­down, but this is actu­ally true for native pro­grams as well and the solu­tion is to re-use the same objects over and over (for exam­ple by cre­at­ing pools) at the appli­ca­tion level, and this is some­thing that is, of course, pos­si­ble using duetto as well. Gen­er­ally we believe that using JS objects also make duetto com­piled code more sim­i­lar to hand­writ­ten JS code, which is already han­dled very well by JS engines, with the added advan­tage of guar­an­teed type immutabil­ity which can be exploited by the engine to gen­er­ate faster code. Still, duetto is flex­i­ble enough that it will be pos­si­ble, in the future, to sup­port using an emscripten like mem­ory model on a type-by-type basis. This would be use­ful to adapt to var­i­ous work­loads and for com­pat­i­bil­ity with code com­piled by emscripten.

APIs: Duetto is built around the con­cept that the browser is a plat­form hav­ing JS as the machine lan­guage. Such plat­form has APIs like WebGL, HTML5 Can­vas, DOM events and so on which are the low­est level acces­si­ble func­tion­al­i­ties. With Duetto you can use all of them directly from C++ and no JS wrap­per code is required.

What about the cross plat­form APIs used on native sys­tems like SDL and Ope­nAL? Well, we don’t cur­rently sup­port them, but they can be ported to Duetto by writ­ing a new back­end for the browser plat­form, sim­i­larly to what has been already done to port them to Win­dows, Linux and Mac. And using duetto it is pos­si­ble to write such port­ing code directly in C++ because you have direct access to the browser APIs. Once the cross plat­form APIs sup­port duetto, appli­ca­tions using them will run as well with­out much (or any) work.

And what about GLES? Well, we received so many requests for this func­tion­al­ity that we are work­ing on pro­vid­ing a WebGL based GLES imple­men­ta­tion our­selves. We plan to release it very soon. With that it should be pos­si­ble to build pro­grams using GLES with duetto by only chang­ing the win­dow­ing sys­tem ini­tial­iza­tion code, sim­i­larly to port­ing from win­dows’ WGL to X11 glX.

C++ com­pat­i­bil­ity: Alon cor­rectly states that duetto has “par­tial” C++ com­pat­i­bil­ity. This is true, but given the incred­i­ble amount of capa­bil­i­ties that the C++ lan­guage has, it is impor­tant to clar­ify what these lim­i­ta­tions are.

Let’s start from the basis: our C++ com­piler was of course not writ­ten from scratch, but was based on clang. This means that we inherit com­plete sup­port for all C++ fea­tures, includ­ing recent and advanced ones like C++11, tem­plate sup­port and lambda. And we actu­ally use such fea­tures in the C++ code which man­ages trans­par­ent client/server RPCs.

The lim­i­ta­tions are more about a few spe­cific unsup­ported capa­bil­i­ties, cur­rently the major ones are: miss­ing sup­port for vir­tual inheritance/virtual base classes, some issues with pointer com­par­i­son in a cou­ple of cor­ner cases and miss­ing sup­port for some parts of the stan­dard library. But we would like to stress that those lim­i­ta­tions are (mostly) non struc­tural and we will do our best to increase the com­pat­i­bil­ity as much as pos­si­ble while still be coher­ent with C++ phi­los­o­phy of expos­ing all plat­form capa­bil­i­ties and lim­i­ta­tions to the user.

Per­for­mance: We have worked with com­puter sys­tems long enough to all agree that micro-benchmarks have lit­tle to do with per­for­mance on real world sce­nar­ios. We also know that Duetto still pro­duces sub­op­ti­mal code for many com­mon cases that we plan to fix. We look for­ward to try and pro­file Duetto gen­er­ated code from a large, real world code base, to find out how it per­forms and what needs to be optimized.

In sum­mary, we think that Alon’s review of our tech­nol­ogy is pretty accu­rate, and we look for­ward mak­ing more and more of Duetto’s fea­tures known to the com­mu­nity, with ded­i­cated posts and doc­u­men­ta­tion. We have been impressed by the feed­back we got with our first release — and look for­ward to hear your opinion.

Duetto release 0.9.1 is now avail­able as source tar­balls and bina­ries for Ubuntu/Debian (PPA) and Win­dows. MacOSX builds will be avail­able soon. Feel free to report bugs, and request fea­tures here.

Fol­low us on @leaningtech, Face­book and at www.leaningtech.com for updates.

No Comments