JPEG Compression for OV7670 images

M

mborgerson

Well-known member
I've downloaded a JPEG compression program and adapted it to run on the T4.1 to which I've connected my OV7670 camera. The original code used malloc() to allocate 256KB of buffers that hold pre-computed constants used during the compression. I moved these tables to arrays DMAMEM. The fact that DMAMEM doesn't get initialized is not an issue as an initialization function called from setup() fills the tables.

The JPEG compressor needs an RGB888 bitmap as input. In the original code, the bitmap was read from a .BMP file. I changed the code to use a bitmap in EXTMEM as the input data.

The process goes like this:

* Capture a VGA-sized image to a buffer in EXTMEM. The image can be either RGB565 or YUV422 encoded. YUV images compress to smaller files due to the reduced spatial color resolution.

* Use the Pixel Pipeline to convert the captured image from the original format to the RGB888 format the encoder wants. The converted image is now 921KB.

* Call the JPG encoder with a pointer to the RGB888 buffer and a file name.

* The encoder compresses the file and writes it to SD card with .jpg extension.

An uncompressed VGA image in RGB565 format is 614KBytes. My compressed test images are about 40KBytes. Here is a test image:
JP_12231320.jpg

The original 600KB RGB565 image is too large to upload to the forum which only allows .bmp files under 390KB in size.

Compressing and writing a VGA image takes about 180MSec. The compression code first converts the RGB888 to YUV, then compresses the YUV data. Since the OV7670 can generate YUV images, I hope to eliminate the code and execution time needed to do the RGB888 to YUV conversion by having the compressor work directly on the YUV data from the camera. The compression code also writes the output to the SD card with one or two-byte writes. I hope to speed this up by writing the output to a buffer, then writing the full buffer to SD Card.

I've got last-minute shopping and wrapping to do, so I probably won't post example code until the weekend.
 
After a bit more research on JPEG encoders, I found tooJpeg by Stephan Brumme. This encoder offered some significant advantages over the earlier version:

1. No requirement for 256KB of precomputed lookup tables in DMAMEM.
2. The ability to change the compression quality.
3. Everything in one C++ file-----no includes or other libraries required.

I modified this file to allow it to accept YCbCr images generated by the OV7670 camera, thus avoiding the necessity to convert the camera RGB565 or YUV422 image to RGB888 in an EXTMEM buffer. This eliminated about 100mSec. of conversion time and the need for a 921KB EXTMEM buffer to hold the VGA image. You do still need a 614KB buffer for a VGA YUV image.

This JPEG converter can be found at https://github.com/mjborgerson/TooJpeg-Teensy4.1

There are other repositories in my github for the OV7670 library, the PXP library, the command processor and the generic data logger. All of these are works in progress, so let me know if you have problems with the libraries or exampled.
 
Couldnt find the source code

Hi, i am currently working on imxrt1170 board and having a similiar issue needing toojpeg accept YCbCr images as input. But couldnt find source code in the github link.
Cqn you push the source code to github....Great thanks indeed...
 
mxgok said:
Hi Joepasquariello,
I have studied the code, the code supports OUTPUT YCbCr color space...As input it takes RGB array 3 bytes per_pixel. The point here is supplying the INPUT as YCbCr not as RBG array.
Click to expand...

Okay. Hopefully @mborgerson will see your question and post the source to his github.
 
JPEG uses the YCbCr color space internally everywhere.

If you look at Stephen Brumme's toojpeg.cpp:writeJpeg() function (starting at line 556 in version 1.5), you can see three 8×8 arrays of floats: Y, Cb, and Cr. In the immediately following nested for loops, the innermost two loops just copy the 8×8 block from the original RGB image to the temporary arrays in YCbCr format: Y is between -128.0f and 127.0, and Cb and Cr between 0.0f and 255.0f. The second set of innermost nested for loops, applied only for 4:2:0, halves the chrominance by averaging the Cr and Cb values from four source pixels: (x,y), (x+1,y), (x,y+1), and (x+1,y+1); and the 8×8 arrays correspond to an area of 16 pixels.
The EncodeBlock() function takes the 8×8 block of data, and compresses and emits it to the JPEG bitstream.

So, the changes needed to work on YCbCr inputs should be quite straightforward, simplifying the existing code. Look for where pixels[] array is used, and where the RGB data is converted to YCbCr using rgb2y(), rgb2cb(), and rgb2cr() helper functions.
 
joepasquariello said:
Okay. Hopefully @mborgerson will see your question and post the source to his github.
Click to expand...

I have not yet posted the code to GitHub for two reasons:

1. I want to make sure that what I post is compatible with the licensing and intentions of the original author, Stephen Brumme.
2. I have encountered system crashes when the first pixel of the image to be compressed is at the start of EXTMEM. During the compression process, the code reads from the byte BEFORE the start of the buffer to get one of Y, Cb, or Cr. There are at least two ways to fix this:
A) Make sure the buffer starts well inside the EXTMEM---I did this with a buffer structure that has 32 bytes of padding before the image itself.
B) change the order of the Y, Cb, and Cr bytes in image. The OV7670 and CSI can handle this, but it has repercussions in a lot of other operations with the images or the pixel pipeline.

I have not yet gone through the steps to implement option B, as I got sidetracked for several months with a driver for a thermal imaging camera. I used option A in the support routines for that camera. The driver I developed had several optimizations over the existing OV7670 library, and I had just started rewriting that library when leaf-raking and the holiday season cut into my programming time.

Rather than post the whole toojpeg.cpp file, I will post the changes I made.

Here is the externally available function that allows selection between RGB and YCbCr. You will have to add the definition to toojpeg.h.
Code: 
// -------------------- externally visible code --------------------

namespace TooJpeg
{
bool YUVMode = false;   /// default to original RGB888 input
void UseYUV(bool bmode){
  if(bmode) YUVMode = true; else YUVMode = false;
}


here is the code that skips RGB conversion, which starts at about line 596 in my code:

Code: 
              if (!isRGB)  {
                Y[deltaY][deltaX] = pixels[pixelPos] - 128.f;
                continue;
              }
              if(YUVMode) { // directly stuff in the YUV values from buffer
                //  Assume order is YCbCr
                Y [deltaY][deltaX] = pixels[2 * pixelPos  ] -128.0f;
                if(pixelPos & 0x01){ // for odd pixels, Cb is at pixelpos -1
                  Cb[deltaY][deltaX] =  pixels[2 * pixelPos - 1]-128.0;
                  Cr[deltaY][deltaX] =  pixels[2 * pixelPos + 1]-128.0;
                }else {  // for even pixels, Cb is at pixelPos +1 
                  Cb[deltaY][deltaX] =  pixels[2 * pixelPos + 1 ]-128.0;
                  Cr[deltaY][deltaX] =  pixels[2 * pixelPos - 1 ]-128.0;
                }                            
              } else {// Do RGB888 to YCbCr conversion           
              // RGB: 3 bytes per pixel (whereas grayscale images have only 1 byte per pixel)
              // color swap??  switch r and b
                auto b = pixels[3 * pixelPos    ];
                auto g = pixels[3 * pixelPos + 1];
                auto r = pixels[3 * pixelPos + 2];

                Y [deltaY][deltaX] = rgb2y (r, g, b) - 128; // again, the JPEG standard requires Y to be shifted by 128
               // YCbCr444 is easy - the more complex YCbCr420 has to be computed about 20 lines below in a second pass
               if (!downsample){
                  Cb[deltaY][deltaX] = rgb2cb(r, g, b); // standard RGB-to-YCbCr conversion
                  Cr[deltaY][deltaX] = rgb2cr(r, g, b);
                }
              } // end of RGB to YCbCr conversion
            }
          }

I hope to update the OV7670 library and toojpeg sometime soon after the start of the new year.
 
I tried and it worked after trials. Very important point is subtracting 128 in floating format as 128.0 or 128.0f otherwise overflow integer subtractions cause index values too large for huffmancode array.
 
You must log in or register to reply here.
Back
Top