Archive for category Uncategorized

Duetto (C++ for the Web) 0.9.4: Async RPCs, Promises, JavaScript interoperability

logo_duetto_quadrato_192

Around a month has passed since our talk at mloc.js. There we had a great oppor­tu­nity to talk about the tech­nol­ogy behind the duetto C++ com­piler for the Web. We also made our first pub­lic announce­ment of our promise based sup­port for asyn­chro­nous type safe RPC, which fits in our vision of C++ as the plat­form lan­guage for the Web, both on the client and on the server side.

Today, we announce the new, feature-packed release of Duetto — 0.9.4! This time, we really splurged, and added some qual­ity fea­tures that we’re sure you’ll find interesting:

  • Ini­tial sup­port for promise-based async RPC

  • Strong inter­op­er­abil­ity with JavaScript

    • __asm__ to exe­cute JavaScript from C++ code

    • [[jsex­port]] attribute to use C++ objects from JavaScript

  • Sup­port for C++ mutex/atomic/thread headers

As usual, you can find the full source code of the release here, plus binary pack­ages for Debian/Ubuntu on our PPA, a binary setup for win­dows and a binary archive for Mac OS X here.

Asyn­chro­nous RPC using promises

With duetto you can “tag” a method to let the com­piler know that the code should be com­piled for the server side. The com­piler also takes care of gen­er­at­ing RPC boil­er­plate includ­ing type safe seri­al­iza­tion of para­me­ters and dese­ri­al­iza­tion of the return value. While the basic model is syn­chro­nous, you can take advan­tage of promises to use fully type safe asyn­chro­nous RPC in duetto. See e.g. the fol­low­ing code:

// Pending requests for messages
vector<Promise<string>*> pendingRequests;
Promise<string>* getChatMessageRemote() [[server]]
{
     auto ret=new Promise<string>();
     // Store the new promise, you can use it later to complete
     // the request in asynchronous manner
     pendingRequests.push_back(ret);
     return ret;
}
void sendMessageRemote(const string& str) [[server]]
{
     for (auto p: pendingRequests)
          p->done(str);
     pendingRequests.clear();
}
void messageHandler(const string& newMessage) [[client]]
{
     client::console.log("Message received",newMessage.c_str());
}
void webMain() [[client]]
{
     // First wait for messages
     auto promise = getChatMessageRemote();
     // Add a callback to the promise
     promise->then(messageHandler);
     // Tell the promise that no more callbacks will be added
     promise->complete();
     // Now send a message, it will be echoed back
     sendMessageRemote("Test message");
}

The code is a sim­ple but work­ing exam­ple of a multi-user chat appli­ca­tion. The getChatMes­sageR­e­mote method returns a promise imme­di­ately after being called, with­out wait­ing for the server answer. On the server side we cre­ate a promise and return it. We also need to store it some­where to even­tu­ally ful­fill the promise by send­ing a value.

The client code needs to set a call­back to receive the value when the promise will be even­tu­ally ful­filled. Later on the client send a chat mes­sage using the sendMes­sageR­e­mote server method. This method is syn­chro­nous and will return only after the server side exe­cu­tion is com­pleted. The server echoes the mes­sage back to all con­nected clients by ful­fill­ing all pend­ing promises.

Strong JS interoperability

This release of duetto includes two fea­tures that pro­vide a supe­rior inte­gra­tion with man­u­ally writ­ten JavaScript code: the [[jsex­port]] class attribute and __asm__ sup­port for inline JavaScript.

[[jsex­port]] class attribute

We have intro­duced pre­lim­i­nary sup­port for a new attribute: [[jsex­port]]. This attribute is designed to be applied on C++ classes, doing so will cause make the class avail­able to man­u­ally writ­ten JavaScript code. The fol­low­ing exam­ple shows its typ­i­cal use:

class [[jsexport]] ExportExample
{
private:
    float a;
    int b;
public:
     ExportExample(float _a, int _b):a(_a),b(_b)
     {
     }
     void testMethod()
     {
          client::console.log(“Float value is”, a);
     }
};

Dur­ing com­pi­la­tion duetto will gen­er­ate bridge code to expose the Expor­tEx­am­ple C++ class to JavaScript code. You can then use the class when writ­ing JS code

// Using jsexport-ed classes from JavaScript code
var cppobj = new ExportExample(0.1, 42);
cppobj.testMethod();

The [[jex­port]] attribute places some lim­i­ta­tions on the class. In par­tic­u­lar, the object must have a triv­ial destruc­tor and no over­loaded meth­ods. More­over oper­a­tors can’t be exported to JavaScript. Hav­ing a triv­ial destruc­tor is nec­es­sary to make sure that the reg­u­lar Garbage Col­lec­tion mech­a­nism is suf­fi­cient to reclaim the object. We plan to expand this fea­ture in the future to reduce any lim­i­ta­tions as much as possible.

Inline JavaScript using __asm__

This release also adds sup­port for insert­ing arbi­trary JavaScript code inside C++ code using the __asm__ func­tion­al­ity. Our design choice has been to put no limit on what you can do inside __asm__, includ­ing break­ing the pro­gram, so be care­ful when tak­ing advan­tage of this capability.

void webMain()
{
      __asm__(“alert(‘JavaScript code inside __asm__’);”);
}

At the moment only this sim­ple form of JS code inser­tion is sup­ported. It’s not cur­rently pos­si­ble to pass vari­ables from sur­round­ing C++ code or return a value from __asm__, but we will extend this sup­port in the future.

Sup­port for C++ mutex/atomic/thread headers

Some C++ func­tion­al­i­ties, espe­cially the ones related to multi-threading, do not have a direct map­ping on the browser plat­form. That said, we want to pro­vide as much com­pat­i­bil­ity as pos­si­ble with exist­ing C++ code to reduce the effort when using duetto for porting.

As of duetto 0.9.4, we sup­port the fol­low­ing C++ thread­ing primitives:

  • mutex: Imple­mented as plain coun­ters, mutexes can­not block exe­cu­tion but report an error if a non-recursive mutex is acquired twice.

  • atomic: Imple­mented as plain inte­gers, since the browser is not actu­ally con­cur­rent there is no real sup­port for atomic operations.

  • thread: Mod­ern browsers sup­port Web­Work­ers, but the thread­ing model does not map to the one expected by C++ threads. We chose to make the user aware that C++ threads are not sup­ported by adding an explicit error mes­sage when the thread header is included in code com­piled with duetto.

 Fol­low us on @leaningtech, Face­book and at www.leaningtech.com for updates.

2 Comments

Duetto: a C++ compiler for the Web going beyond emscripten and node.js

logo_duetto_quadrato_192

Today Lean­ing Tech­nolo­gies Ltd. releases duetto, a com­piler designed to develop inte­grated (back­end and fron­tend) web appli­ca­tions in C++. duetto is now avail­able to the pub­lic as FOSS here, and will be offered in a com­mer­cial pack­age with closed-source-friendly licens­ing soon.

Browser-based appli­ca­tions will undoubt­edly play a big role in the future of both con­sumer and busi­ness appli­ca­tions. Web apps have some clear advan­tages com­pared to native appli­ca­tions, like being natively cloud-based and col­lab­o­ra­tive, while also being inher­ently portable, espe­cially thanks to the increased atten­tion that browser ven­dors have ded­i­cated to web stan­dard com­pli­ancy. Mod­ern web stan­dards, that usu­ally go under the umbrella term HTML5, pro­vide incred­i­ble close-to-native capa­bil­i­ties and this gap will be shrink­ing with time.

Sev­eral tools have been devel­oped to help pro­gram­mers in writ­ing large-scale Web appli­ca­tion fron­tends, reduc­ing the lim­i­ta­tions and obsta­cles given by Javascript: Coffe­Script, Microsoft Type­Script, Google Dart, Google GWT and, more recently, emscripten, which com­piles LLVM byte­code (and there­fore C++) to Javascript. On the server side, it is com­mon to use lan­guages such as PHP, Python and Ruby, lead­ing to a sep­a­rate back­end code­base in a lan­guage other than Javascript. This causes dupli­ca­tion of code (and new bugs), hav­ing to deal with RPC man­u­ally, and over­all takes time away from proper devel­op­ment. A very pop­u­lar solu­tion for this has been to use Javascript on the server as well with node.js.

We present duetto, our C++ com­piler for the Web, which is now avail­able to the pub­lic. Our solu­tion inte­grates and sur­passes fea­tures of exist­ing tools, and allows to pro­gram both the fron­tend and the back­end of a Web appli­ca­tion in an inte­grated C++ code­base, com­pil­ing them respec­tively to JavaScript and native code.

Duetto com­bines the advan­tages of emscripten and node.js by allow­ing the pro­gram­mer to:

  • write web appli­ca­tions in C++, reusing exist­ing code and mak­ing port­ing of whole appli­ca­tions and games to the browser plausible.

  • code both the fron­tend and the back­end of a web appli­ca­tion in the same lan­guage and codebase

In addi­tion to this, duetto pro­vides some nice features:

  • Bring the robust­ness and proven scal­a­bil­ity of C++ pro­gram­ming to the Web

  • You can access all browser APIs directly. Duetto inher­its the C++ phi­los­o­phy of expos­ing the plat­form capa­bil­i­ties (and lim­i­ta­tions) to the users. There is no mid­dle man.

  • Duetto is based on LLVM/clang. An indus­try stan­dard C++ com­piler is a programmer’s best friend: code san­ity is ver­i­fied as com­pile time, includ­ing RPC signatures.

  • The LLVM tool­chain also guar­an­tees that a mind-blowing set of opti­miza­tions is run at com­pile time, gen­er­at­ing highly effi­cient code. This decreases the bur­den on JavaScript JIT com­piler at runtime.

  • Con­trar­ily to emscripten we do not try to emu­late a tra­di­tional address space using typed arrays, but directly map C++ objects to JS objects. This reduces mem­ory con­sump­tion since the garbage col­lec­tor can delete unused object.

Are you inter­ested in using duetto? You can now. We have just released the whole tech­nol­ogy as a FOSS project, avail­able here.  Also, we’ve stuck to our promise of releas­ing in six months from our first announce­ment in April

The com­piler itself, derived from LLVM/clang is released with the same license as LLVM/clang (UI/NCSA) and we plan to con­tribute gen­er­ally use­ful code upstream. Accom­pa­ny­ing head­ers and libraries will be released as GPLv2+. We will be also offer­ing closed source friendly licens­ing of such com­po­nents for a fee. Here are links to the var­i­ous repos­i­to­ries on github:

Release tar­balls will be avail­able shortly on the Launch­pad page of the project: https://launchpad.net/duetto. Launch­pad will also be our bug report­ing sys­tem, so please use it to report any prob­lem you might find. We will be also pro­vid­ing builds of duetto for Ubuntu linux using a PPA. We plan to pro­vide binary builds for Mac OS X and Win­dows as well, but we are still work­ing on automat­ing those build.

Please sub­scribe to our Launch­pad group/mailing list to dis­cuss bugs, desired fea­tures and usabil­ity issues on https://launchpad.net/~duetto-users. You can also fol­low us on twit­ter (@leaningtech) and visit our web­site to be updated on duetto related news:

6 Comments

Linux support for Asus Xonar U1 USB audio device

I got as a Christ­mas gift an exter­nal USB audio device: an Asus Xonar U1. It’s a nice device with a decent audio qual­ity. The audio itself, both in and out works per­fectly with the stan­dard snd-usb-audio ker­nel mod­ule. Unfor­tu­nately the audio con­trols on the device are not stan­dard, so I devel­oped a sim­ple dae­mon and accom­pa­ny­ing udev and pm-utils script to get the device to work. I’ve pub­lished every­thing under the GPL hop­ing that they might be use­ful for some­one else. Feel free to clone and fork my repos­i­tory on github

https://github.com/alexp-sssup/asus-xonar-u1-utils

Flattr this

3 Comments

Lightspark 0.6.0.1 released

I’m very happy to annouce a new major release for Lightspark, the open source flash player imple­men­ta­tion. This release includes quite a lot of fixes, both inter­nal and vis­i­ble to users. The most impor­tant ones from a user point of view are prob­a­bly the improved sup­port for PulseAu­dio flat vol­umes, which makes it impos­si­ble for lightspark to manip­u­late the sys­tem vol­ume and the newly added sup­port for the BBC video player. To use BBC site you might need to use AdBlock, which is gen­er­ally rec­om­mended since more often than not flash adver­tise­ments are not sup­ported by Lightspark and causes fail­ures in oth­er­wise sup­ported sites.

From the ChangeLog:
* Enable RTMP sup­port by default, requires librtmp
* Fixed sup­port for IEvent­Dis­patcher imple­men­ta­tion pat­tern
* Improved seri­al­iza­tion robust­ness
* Improved matrix han­dling
* Imple­ment string and name­space pool­ing to reduce mem­ory con­sump­tion
* Proper sup­port for pri­vate name­spaces
* Improved sup­port for fonts
* Sup­port LLVM 3.1
* Fix full vol­ume issue when PulseAu­dio flat vol­umes are enabled
* Ini­tial sup­port for AIR desk­top appli­ca­tions
* Sup­port for www.bbc.co.uk video player

Source tar­ball is, as usual avail­able from Launch­pad.


Flattr this

5 Comments

iCTF hacking competition 2010: solution of some challenges

This year, hav­ing joined the com­puter secu­rity group at UCSB for my phd, I’ve helped in the orga­ni­za­tion of the 2010rh edi­tion of the iCTF, the biggest inter­na­tional online hack­ing com­pe­ti­tion. It has been plenty of fun, with more than 70 teams par­tic­i­pat­ing from all over the world. The CMU team “Plaid par­lia­ment of pwn­ing” won it,  get­ting the  1000$ price (thanks Adobe and IEEE Secu­rity & Pri­vacy mag­a­zine for the sponsorships!).

Since peo­ple have been ask­ing for the solu­tions of the two chal­lenges I wrote, so here they are.

chal­lenge 5:

This chal­lenge is easy, I encour­age you to give it a try, it’s fun!

Ques­tion: “Who’re you’re gonna call?”

File: call
Points: 300
Teams that have com­pleted it: 44 (congrats!)

Solu­tion: in an html com­ment fol­low­ing this line

chal­lenge 4:

This chal­lenge is a lit­tle more dif­fi­cult, but if you know python you have all the skills nec­es­sary to beat it.
Question:

Hello Anony­mous,
it has come to our knowl­edge that the Lity­van Secret Ser­vice (note: this was part of a more com­plex story that ran across all the CTF)
is installing a remote con­trol sys­tem on their sub­marines. This would
allow a hand­ful of man to launch a mas­sive attack from a secure
loca­tion.
This threat is not accept­able: your mis­sion is to break into one
sub­ma­rine remote con­trol sys­tem and launch a mis­sile against Navy Con­trol Center,
so to shut down the sys­tem for good.
The details of this sys­tems are unknown, but we have recov­ered from a
stolen usb device two files that might give some insights to you: they
are attached. We dis­cov­ered what we sus­pect to be one of the entry point
of this sys­tem: it is located at 10.15.42.42, on TCP port 5000.
Should you com­plete this mis­sion, you will be lav­ishly remunerated.
Good luck!
X

To make it run, unpack this file and run server.py (it’s all python, you can check it for back­doors). To start the chal­lenge, you should read only the con­tent of the “pub” direc­tory: that was the mate­r­ial that was given in the ctf.

Points: 500
Teams that have com­pleted it: 0, as most of the dif­fi­cult chal­lenges — because of the struc­ture of the iCTF 2010, it turned out that it was more con­ve­nient to focus only on the easy chal­lenges. We’ll have to fix it next year!

Solu­tion: in an html com­ment fol­low­ing this line. The given file also con­tains a script that can solve the chal­lenge (the test_* files)

For the solu­tion of another dif­fi­cult chal­lenge, head over to Bryce’s blog.

No Comments

Lightspark 0.4.5 RC is out!

Finally, after a long delay caused by sev­eral issues Lightspark 0.4.5 is out. Here it is a brief Changelog.

  • Include the new Advanced Graph­ics Engine, that should pro­vide smoother and faster graph­ics and sup­port for clipping
  • More robust input sup­port (makes it pos­si­ble to use Play/Pause on YouTube)

The work on this released has been espe­cially slowed down by an issue found in libxml++ and another one found in mesa. Pack­agers should pay atten­tion to ful­fill the fol­low­ing depen­dency requirements:

  • Libxml++ ver­sion 2.33.1 or bet­ter. If an older ver­sion on libxml++ must be sup­ported some com­mits must be back­ported. More info about this in the README file
  • Mesa should include the fix for this bug. The issue affects some radeon and maybe intel cards.

Pack­ages will be avail­able for Fedora Rawhide and are already uploaded on the PPA for Ubuntu Mav­er­ick

Flattr this

5 Comments

See you in a week!

Me and most of my engi­neer­ing col­leagues here at Sant’Anna School for Advanced Stud­ies are leav­ing tomor­row morn­ing for a week long tour of uni­ver­si­ties and labs in Bel­gium and Nether­lands. This means I’ll have no time to work on Lightspark, sorry :-)

Any­way the 0.4.5 release will be prob­a­bly out in the last days of the month. The crit­i­cal bug in libxml++ is now fixed as my patch was accepted upstream. If (as I believe) a new ver­sion of the library will be released and pack­aged before i get back there will be no rea­son to hold the release any­more.

Flattr this

3 Comments

Lightspark 0.4.4.2 released

Another week, another bug­fix release for Lightspark! Apart from restor­ing the sup­port for YouTube this release fea­tures the new plu­gin based audio frame­work that makes it pos­si­ble to sup­port other back­ends beside PulseAu­dio. At the moment both an ALSA and Ope­nAL plu­g­ins are being worked on.

As always you can grab the release here on Launch­pad


Flattr this

21 Comments

Getting Things GNOME!          —          GSoC review (#11)

Hello planet!
I’m back from GUADEC. It was my first con­fer­ence about open source and it was great.
I’ve found par­tic­u­larly inspir­ing the talk by Guil­laume Desmottes about Telepa­thy and Epiphany, which can be great to extend GTG pos­si­bil­i­ties in col­lab­o­ra­tion.
The talk by Jake Edge about pro­mot­ing free soft­ware projects was also very inter­est­ing, in par­tic­u­lar for the young Lightspark project (that went com­pletely unno­ticed for a few months before show­ing up on planet GNOME).

Thanks to the excit­ing talks and peo­ple at GUADEC,   the GTG team (even the peo­ple who were not there!) has been work­ing fer­vently on a nice rewrite of some parts of GTG core, along with a lot of unit-tests. Hope­fully, a lot of bugs will be closed thanks to this, and GTG will be nicer to code.

Some of the GTG peo­ple at GUADEC. From left to right: Bertrand Rousseau, Karlo Jez, Lionel Dri­cot and me.

As for my Google Sum­mer of Code on Get­ting Things Gnome sup­port for mul­ti­ple back­ends, this week has seen:

  • a port of my Evo­lu­tion plu­gin as a back­end (that was the last one planned)
  • refac­tor­ing of the Twit­ter plu­gin to get autho­riza­tion through Oauth (using  the tweepy library, thanks Tante for the hint)
  • docs, docs, docs

Next week, I’ll keep doc­u­ment­ing and test­ing. I should also write a guide on how to write new back­ends. See you next week!

No Comments

Lightspark 0.4.2.2 released

Try­ing to keep up with the old rule “Release early, release often” I’ happy to announce release 0.4.2.2 open source flash player.

This appar­ently small point release actu­ally includes the biggest fea­ture plan­nend for the upcom­ing 0.4.3 release, namely Gnash fall­back on older SWF clips.  Lightspark cur­rently relies no Gnash for any Flash con­tent that does not require AVM2 (Action­Script 3) support.

I would also like to explain an issue that many users and testers reported. Fire­fox is not able to han­dle mul­ti­ple plu­gin for the same file type! Not even if only one of those plu­g­ins is actu­ally enabled. So, if lightspark is installed along­side adobe’s player or Gnash no flash con­tent will be dis­played. This is a fire­fox bug, I’ve reported the bug and pro­posed a patch that is cur­rently wait­ing the review.

The source of the release is as always avail­able on launch­pad. Binary pack­ages for Ubuntu Lucid and Mav­er­ick will be avail­able on the usual PPA (in a cou­ple hours from now, Launch­pad seems pretty busy at the moment). More­over, since the last announce­ment lightspark has been also included in the debian exper­i­men­tal suite (thanks to Didier Raboud and Luca Falavigna).

Stay tuned, and fol­low the roadmap

9 Comments