Technology Temple

Dur­ing the pESA­pod project we worked on the telecom­mu­ni­ca­tion and teleme­try sys­tem for the robot. The com­put­ing infra­struc­ture was very com­plex (maybe too much). We had three Altera FPGA on board and a very low power con­sump­tion PC, a VIA Epia board. Using devices that are light on power needs is a must for mobile robots. But we ended up using more power for the elec­tron­ics than for the motors. I guess the Altera’s boards are very heavy on power, being pro­to­typ­ing devices.

Any­way the Epia with the onboard Eden proces­sor is a very nice machine. It is fully x86 com­pat­i­ble, and we man­aged to run linux on it with­out prob­lems. It has indeed a very low power foot­print, but the per­for­mance trade­off for this was quite heavy. The orig­i­nal plan was to have four video streams from the robot. A pair of prox­im­ity cam­eras for sam­ple gath­er­ing and a stere­o­cam for nav­i­ga­tion and envi­ron­ment map­ping. We used at the end only the stere­o­cam, but even encod­ing only those two video streams on the Epia was really difficult.

We used lib­FAME for the encod­ing. The name means Fast Assem­bly MPEG Encoder. It is fast indeed, but it is also very poorly man­tained. So we had some prob­lems at firts to make it work. The library accept frames encoded in YUV for­mat, but our cam­era sen­sor data was in bayer encod­ing. So we had to write the for­mat con­ver­sion routine.

RGB to YUV using matrix notation

The con­ver­sion from RGB color space to YUV is quite sim­ple and can be done using lin­ear alge­bra. Our first approach was really naive and based on float­ing point.

// RGB* rgb;
// YUV* yuv;
yuv[i].y=0.299*rgb[i].r + 0.114*rgb[i].g + 0.587*rgb[i].b;
yuv[i].u=128 - 0.168736*rgb[i].r - 0.331264*rgb[i].g + 0.5*rgb[i].b;
yuv[i].v=128 + 0.5*rgb[i].r - 0.418688*rgb[i].g + 0.081312*rgb[i].b;

This was really slow. We later dis­cov­ered to our dis­ap­point­ment that the FPU was clocked at half the speed of the proces­sor. So we changed the imple­men­ta­tion to inte­ger math. The result was some­thing like this:

yuv[i].y=(299*rgb[i].r + 0.114*rgb[i].g + 0.587*rgb[i].b)/1000;
yuv[i].u=128 - (169*rgb[i].r - 331*rgb[i].g + 500*rgb[i].b)/1000;
yuv[i].v=128 + (500*rgb[i].r - 419*rgb[i].g + 81*rgb[i].b)/1000;

This solu­tion almost dou­bled the fram­er­ate. But it was still not enough and we had to dive deep in the magic world of MMX/SSE instruc­tions. The details for the next issue.

Dur­ing the pESA­pod project we worked on the telecom­mu­ni­ca­tion and teleme­try sys­tem for the robot. The com­put­ing infra­struc­ture was very com­plex (maybe too much). We had three Altera FPGA on board and a very low power con­sump­tion PC, a VIA Epia board. Using devices that are light on power needs is a must for mobile robots. But we ended up using more power for the elec­tron­ics than for the motors. I guess the Altera’s boards are very heavy on power, being pro­to­typ­ing devices.

Any­way the Epia with the onboard Eden proces­sor is a very nice machine. It is fully x86 com­pat­i­ble, and we man­aged to run linux on it with­out prob­lems. It has indeed a very low power foot­print, but the per­for­mance trade­off for this was quite heavy. The orig­i­nal plan was to have four video streams from the robot. A pair of prox­im­ity cam­eras for sam­ple gath­er­ing and a stere­o­cam for nav­i­ga­tion and envi­ron­ment map­ping. We used at the end only the stere­o­cam, but even encod­ing only those two video streams on the Epia was really difficult.

We used lib­FAME for the encod­ing. The name means Fast Assem­bly MPEG Encoder. It is fast indeed, but it is also very poorly man­tained. So we had some prob­lems at firts to make it work. The library accept frames encoded in YUV for­mat, but our cam­era sen­sor data was in bayer encod­ing. So we had to write the for­mat con­ver­sion routine.

RGB to YUV using matrix notation

The con­ver­sion from RGB color space to YUV is quite sim­ple and can be done using lin­ear alge­bra. Our first approach was really naive and based on float­ing point.

// RGB* rgb;
// YUV* yuv;
yuv[i].y=0.299*rgb[i].r + 0.114*rgb[i].g + 0.587*rgb[i].b;
yuv[i].u=128 - 0.168736*rgb[i].r - 0.331264*rgb[i].g + 0.5*rgb[i].b;
yuv[i].v=128 + 0.5*rgb[i].r - 0.418688*rgb[i].g + 0.081312*rgb[i].b;

This was really slow. We later dis­cov­ered to our dis­ap­point­ment that the FPU was clocked at half the speed of the proces­sor. So we changed the imple­men­ta­tion to inte­ger math. The result was some­thing like this:

yuv[i].y=(299*rgb[i].r + 0.114*rgb[i].g + 0.587*rgb[i].b)/1000;
yuv[i].u=128 - (169*rgb[i].r - 331*rgb[i].g + 500*rgb[i].b)/1000;
yuv[i].v=128 + (500*rgb[i].r - 419*rgb[i].g + 81*rgb[i].b)/1000;

This solu­tion almost dou­bled the fram­er­ate. But it was still not enough and we had to dive deep in the magic world of MMX/SSE instruc­tions. The details for the next issue.