RFC: is OpenGL not the Right Thing (TM)?


It’s late at night and I’m back in Pisa for my last year of mas­ter. And, as often hap­pens, a weird idea struck my mind. What if OpenGL is not the right thing for Lightspark? No, I’m not talk­ing about drop­ping hard­ware accel­er­ated ren­der­ing as that’s surely the right way to go, but using OpenGL really looks unnat­ural. In the design of the advanced graph­ics engine OpenGL is being basi­cally used only to upload images ren­dered with cairo to the VRAM, and to blit and com­pos­ite all the ren­dered chunks together... do we really need all the OpenGL com­plex­ity to do this??

Well... OpenGL is basi­cally the only thing we have, the only way to talk with the graph­ics hard­ware. But, here it comes the gal­lium project! As gal­lium splits the API from the dri­ver it could be pos­si­ble, in the­ory, to write a spe­cial­ized gal­lium state tracker to do only the work we need... and maybe do it better.

I’m writ­ing here first because I’m not (yet) expe­ri­enced enough with the gal­lium plat­form to know if the idea is sane, and sec­ond because I some­how feel the same approach could be use­ful for other apps... for exam­ple Lightspark and com­posit­ing win­dow man­agers have sim­i­lar needs. So I’d like to have some feed­back about writ­ing a small API and a gal­lium state tracker to do:

  • DMA accel­er­ated trans­fers of ren­dered image data
  • Blit­ting and com­posit­ing of such data on screen
  • Notify the appli­ca­tion when asyn­chro­nous work (such as DMA trans­fers) has ended (BTW: what’s the right way of doing this in OpenGL?)
  • Enqueue to-be-uploaded-to-vram images and have them sequen­tially transfered
  • Apply sim­ple but pro­gram­ma­ble (shader-like) trans­for­ma­tion to pixel data

Big dis­claimer: I’ve not yet started work­ing on this idea. I’ve not even seri­ously though about it’s fea­si­bil­ity. I’d just like to have some feed­back on this.

Flattr this

, , ,

  • Soroush

    Fire­fox 4 is also using OpenGL to com­pos­ite lay­ers in Linux and Mac.

  • Soroush

    Fire­fox 4 is also using OpenGL to com­pos­ite lay­ers in Linux and Mac.

  • Go Go OpenGL

    “and maybe do it better.”

    Is there any­thing miss­ing from OpenGL? There’s noth­ing wrong with not using all the fea­tures of OpenGL.

    Using some­thing else lim­its the poten­tial out­sider contributions.

  • Go Go OpenGL

    “and maybe do it better.”

    Is there any­thing miss­ing from OpenGL? There’s noth­ing wrong with not using all the fea­tures of OpenGL.

    Using some­thing else lim­its the poten­tial out­sider contributions.

  • Sean

    I have a feel­ing that the real prob­lem is that you’re not using OpenGL exten­sively enough. A very large quan­tity of what you want to do for a Flash-like ren­der­ing API can all be done on the GPU, with GLSL. This in effect is what OpenVG or Direct2D set out to accom­plish: pow­er­ful 2D hardware-accelerated ren­der­ing on top of a system’s 3D dri­ver stack. Toss in some OpenCL for the few things GLSL isn’t quite cut out to do (poly­gon tes­sel­la­tion for pathing, etc.) and you’re in pretty good shape. If you’re already using cairo, then get­ting this level of sup­port is a lot eas­ier than you might think, given the renewed vigor in the cairo OpenGL sur­face support.

    Of course, the Open Source OpenGL/OpenCL stack is (at this moment) pure crap and I wouldn’t be sur­prised at all if pure-software ren­der­ing is faster. Time to take care of that, hopefully.

  • Sean

    I have a feel­ing that the real prob­lem is that you’re not using OpenGL exten­sively enough. A very large quan­tity of what you want to do for a Flash-like ren­der­ing API can all be done on the GPU, with GLSL. This in effect is what OpenVG or Direct2D set out to accom­plish: pow­er­ful 2D hardware-accelerated ren­der­ing on top of a system’s 3D dri­ver stack. Toss in some OpenCL for the few things GLSL isn’t quite cut out to do (poly­gon tes­sel­la­tion for pathing, etc.) and you’re in pretty good shape. If you’re already using cairo, then get­ting this level of sup­port is a lot eas­ier than you might think, given the renewed vigor in the cairo OpenGL sur­face support.

    Of course, the Open Source OpenGL/OpenCL stack is (at this moment) pure crap and I wouldn’t be sur­prised at all if pure-software ren­der­ing is faster. Time to take care of that, hopefully.

  • http://rpg-314.blogspot.com Rohit

    I’d sug­gest that you get a work­ing ogl based imple­men­ta­tion first before you go the gal­lium route.

    Gal­lium is pretty imma­ture com­pared to ogl, espe­cially on linux.

  • http://rpg-314.blogspot.com Rohit

    I’d sug­gest that you get a work­ing ogl based imple­men­ta­tion first before you go the gal­lium route.

    Gal­lium is pretty imma­ture com­pared to ogl, espe­cially on linux.

  • Anony­mous

    As another com­ment sug­gested, you could poten­tially make bet­ter use of GLSL to offload some of the ren­der­ing to the graph­ics card.

    How­ever, a few dif­fer­ent alter­na­tives exist that would prove sig­nif­i­cantly less rad­i­cal than gallium.

    For instance, you might con­sider try­ing out cairo’s new GL back­end, which lets cairo help you do hardware-accelerated 2D rendering.

    If you don’t mind try­ing some­thing non-portable, you could also try the DRI2 X exten­sion, which gives you a fairly direct inter­face to manip­u­late pixmaps in graph­ics memory.

  • Anony­mous

    As another com­ment sug­gested, you could poten­tially make bet­ter use of GLSL to offload some of the ren­der­ing to the graph­ics card.

    How­ever, a few dif­fer­ent alter­na­tives exist that would prove sig­nif­i­cantly less rad­i­cal than gallium.

    For instance, you might con­sider try­ing out cairo’s new GL back­end, which lets cairo help you do hardware-accelerated 2D rendering.

    If you don’t mind try­ing some­thing non-portable, you could also try the DRI2 X exten­sion, which gives you a fairly direct inter­face to manip­u­late pixmaps in graph­ics memory.

  • AdamK

    Please, stick to OpenGL. Gal­lium, while great in the­ory atm. is słow and not avi­alivle every­where, while OpenGL is fast and avi­al­i­ble even on my pocket watch ;)

  • AdamK

    Please, stick to OpenGL. Gal­lium, while great in the­ory atm. is słow and not avi­alivle every­where, while OpenGL is fast and avi­al­i­ble even on my pocket watch ;)

  • E Gorodin­sky

    I was going to say it’s a good idea, but then read the com­ments and one of them men­tioned OpenVG. If I’m not mis­taken the openvg state tracker is already there. Have a look — per­haps that is exactly what you need.

  • E Gorodin­sky

    I was going to say it’s a good idea, but then read the com­ments and one of them men­tioned OpenVG. If I’m not mis­taken the openvg state tracker is already there. Have a look — per­haps that is exactly what you need.

  • MostAwe­some­Dude

    I am con­fused by the com­ments that imply Gal­lium is slow. Not only are the production-quality Gallium-based dri­vers fast, but they’re faster than clas­sic Mesa dri­vers in some situations.

    Gal­lium oper­ates on BOs directly, so you don’t have to worry about DMA sched­ul­ing. You just have a BO, and map/unmap as needed. Blit­ting and com­posit­ing are pretty sim­ple to set up and exe­cute; we have blit­ter util­ity func­tions already. Get­ting stuff into VRAM isn’t really a worry; just put stuff into BOs and let the oper­at­ing sys­tem move them to VRAM when it likes. And, of course, Gal­lium is fully shader­ful and guar­an­teed to sup­port at least D3D9 shaders.

  • MostAwe­some­Dude

    I am con­fused by the com­ments that imply Gal­lium is slow. Not only are the production-quality Gallium-based dri­vers fast, but they’re faster than clas­sic Mesa dri­vers in some situations.

    Gal­lium oper­ates on BOs directly, so you don’t have to worry about DMA sched­ul­ing. You just have a BO, and map/unmap as needed. Blit­ting and com­posit­ing are pretty sim­ple to set up and exe­cute; we have blit­ter util­ity func­tions already. Get­ting stuff into VRAM isn’t really a worry; just put stuff into BOs and let the oper­at­ing sys­tem move them to VRAM when it likes. And, of course, Gal­lium is fully shader­ful and guar­an­teed to sup­port at least D3D9 shaders.

  • Rémi

    “As gal­lium splits the API from the dri­ver it could be pos­si­ble, in the­ory, to write a spe­cial­ized gal­lium state tracker to do only the work we need... and maybe do it better.”

    For your own sake, pro­file your own work. Don’t jump to any con­clu­sion which could lead you the wrong way with­out any hard data.

    Num­bers first, deci­sion second.

  • Rémi

    “As gal­lium splits the API from the dri­ver it could be pos­si­ble, in the­ory, to write a spe­cial­ized gal­lium state tracker to do only the work we need... and maybe do it better.”

    For your own sake, pro­file your own work. Don’t jump to any con­clu­sion which could lead you the wrong way with­out any hard data.

    Num­bers first, deci­sion second.

  • Qwerty

    AFAIK, Moon­light is also using Gallium3d for shaders. You might be on to some­thing here — I say go for it!

  • Qwerty

    AFAIK, Moon­light is also using Gallium3d for shaders. You might be on to some­thing here — I say go for it!

  • Anony­mous

    Maybe I’m not under­stand­ing the prob­lem fully, but what’s wrong with using Cairo directly for every­thing? Per­haps with the OpenGL back­end, per­haps with the OpenVG back­end. Maybe even the X back­end will be just fine — the xren­der imple­men­ta­tions are hard­ware accel­er­ated by the 3d engine these days too, as I under­stand things.

    It seems coun­ter­pro­duc­tive to soft­ware ren­der with Cairo, upload to vram, and then do only the blit­ting with acceleration.

  • Anony­mous

    Maybe I’m not under­stand­ing the prob­lem fully, but what’s wrong with using Cairo directly for every­thing? Per­haps with the OpenGL back­end, per­haps with the OpenVG back­end. Maybe even the X back­end will be just fine — the xren­der imple­men­ta­tions are hard­ware accel­er­ated by the 3d engine these days too, as I under­stand things.

    It seems coun­ter­pro­duc­tive to soft­ware ren­der with Cairo, upload to vram, and then do only the blit­ting with acceleration.

  • Xavier Bestel

    Well, the big advan­tage of OpenGL is that it’s cross-platform, so Lightspark would work on OSX, Win­dows & all sorts of mobile phones as well, which should the­o­ret­i­cally give you more developers.

  • Xavier Bestel

    Well, the big advan­tage of OpenGL is that it’s cross-platform, so Lightspark would work on OSX, Win­dows & all sorts of mobile phones as well, which should the­o­ret­i­cally give you more developers.

  • http://zifre.myopenid.com/ Zifre

    OpenVG is prob­a­bly exactly what you need (it is 2D, so it removes a lot of the com­plex­ity of OpenGL). There is an OpenVG state tracker for Gal­lium, but I’m not sure how com­plete it is. The other good option is OpenGL ES. This will run on Gal­lium dri­vers and most embed­ded dri­vers (whereas full OpenGL won’t), and with some tricks and small mod­i­fi­ca­tions, your OpenGL ES ren­derer will work on desk­top OpenGL dri­vers too. Please don’t use Cairo; OpenVG is a Khronos stan­dard and much more likely to be found out­side of desk­top Linux.

    Also, Gal­lium is not meant to be used by appli­ca­tions directly. You should really use some API that has a Gal­lium state tracker.

  • http://zifre.myopenid.com/ Zifre

    OpenVG is prob­a­bly exactly what you need (it is 2D, so it removes a lot of the com­plex­ity of OpenGL). There is an OpenVG state tracker for Gal­lium, but I’m not sure how com­plete it is. The other good option is OpenGL ES. This will run on Gal­lium dri­vers and most embed­ded dri­vers (whereas full OpenGL won’t), and with some tricks and small mod­i­fi­ca­tions, your OpenGL ES ren­derer will work on desk­top OpenGL dri­vers too. Please don’t use Cairo; OpenVG is a Khronos stan­dard and much more likely to be found out­side of desk­top Linux.

    Also, Gal­lium is not meant to be used by appli­ca­tions directly. You should really use some API that has a Gal­lium state tracker.

  • Sgellert

    I for one urge you not to go down the Gallium3d state tracker path, as this make Lightspark unus­able in the mobile/embedded world.

  • Sgellert

    I for one urge you not to go down the Gallium3d state tracker path, as this make Lightspark unus­able in the mobile/embedded world.

RFC: is OpenGL not the Right Thing (TM)?


It’s late at night and I’m back in Pisa for my last year of mas­ter. And, as often hap­pens, a weird idea struck my mind. What if OpenGL is not the right thing for Lightspark? No, I’m not talk­ing about drop­ping hard­ware accel­er­ated ren­der­ing as that’s surely the right way to go, but using OpenGL really looks unnat­ural. In the design of the advanced graph­ics engine OpenGL is being basi­cally used only to upload images ren­dered with cairo to the VRAM, and to blit and com­pos­ite all the ren­dered chunks together... do we really need all the OpenGL com­plex­ity to do this??

Well... OpenGL is basi­cally the only thing we have, the only way to talk with the graph­ics hard­ware. But, here it comes the gal­lium project! As gal­lium splits the API from the dri­ver it could be pos­si­ble, in the­ory, to write a spe­cial­ized gal­lium state tracker to do only the work we need... and maybe do it better.

I’m writ­ing here first because I’m not (yet) expe­ri­enced enough with the gal­lium plat­form to know if the idea is sane, and sec­ond because I some­how feel the same approach could be use­ful for other apps... for exam­ple Lightspark and com­posit­ing win­dow man­agers have sim­i­lar needs. So I’d like to have some feed­back about writ­ing a small API and a gal­lium state tracker to do:

  • DMA accel­er­ated trans­fers of ren­dered image data
  • Blit­ting and com­posit­ing of such data on screen
  • Notify the appli­ca­tion when asyn­chro­nous work (such as DMA trans­fers) has ended (BTW: what’s the right way of doing this in OpenGL?)
  • Enqueue to-be-uploaded-to-vram images and have them sequen­tially transfered
  • Apply sim­ple but pro­gram­ma­ble (shader-like) trans­for­ma­tion to pixel data

Big dis­claimer: I’ve not yet started work­ing on this idea. I’ve not even seri­ously though about it’s fea­si­bil­ity. I’d just like to have some feed­back on this.

Flattr this

, , ,