Forum Rule: Always post complete source code & details to reproduce any issue!
Results 1 to 11 of 11

Thread: JPEG Compression for OV7670 images

  1. #1
    Senior Member
    Join Date
    Feb 2018
    Location
    Corvallis, OR
    Posts
    431

    JPEG Compression for OV7670 images

    I've downloaded a JPEG compression program and adapted it to run on the T4.1 to which I've connected my OV7670 camera. The original code used malloc() to allocate 256KB of buffers that hold pre-computed constants used during the compression. I moved these tables to arrays DMAMEM. The fact that DMAMEM doesn't get initialized is not an issue as an initialization function called from setup() fills the tables.

    The JPEG compressor needs an RGB888 bitmap as input. In the original code, the bitmap was read from a .BMP file. I changed the code to use a bitmap in EXTMEM as the input data.

    The process goes like this:

    * Capture a VGA-sized image to a buffer in EXTMEM. The image can be either RGB565 or YUV422 encoded. YUV images compress to smaller files due to the reduced spatial color resolution.

    * Use the Pixel Pipeline to convert the captured image from the original format to the RGB888 format the encoder wants. The converted image is now 921KB.

    * Call the JPG encoder with a pointer to the RGB888 buffer and a file name.

    * The encoder compresses the file and writes it to SD card with .jpg extension.

    An uncompressed VGA image in RGB565 format is 614KBytes. My compressed test images are about 40KBytes. Here is a test image:
    Click image for larger version. 

Name:	JP_12231320.jpg 
Views:	68 
Size:	42.4 KB 
ID:	22931

    The original 600KB RGB565 image is too large to upload to the forum which only allows .bmp files under 390KB in size.

    Compressing and writing a VGA image takes about 180MSec. The compression code first converts the RGB888 to YUV, then compresses the YUV data. Since the OV7670 can generate YUV images, I hope to eliminate the code and execution time needed to do the RGB888 to YUV conversion by having the compressor work directly on the YUV data from the camera. The compression code also writes the output to the SD card with one or two-byte writes. I hope to speed this up by writing the output to a buffer, then writing the full buffer to SD Card.

    I've got last-minute shopping and wrapping to do, so I probably won't post example code until the weekend.

  2. #2
    Senior Member
    Join Date
    Feb 2018
    Location
    Corvallis, OR
    Posts
    431
    After a bit more research on JPEG encoders, I found tooJpeg by Stephan Brumme. This encoder offered some significant advantages over the earlier version:

    1. No requirement for 256KB of precomputed lookup tables in DMAMEM.
    2. The ability to change the compression quality.
    3. Everything in one C++ file-----no includes or other libraries required.

    I modified this file to allow it to accept YCbCr images generated by the OV7670 camera, thus avoiding the necessity to convert the camera RGB565 or YUV422 image to RGB888 in an EXTMEM buffer. This eliminated about 100mSec. of conversion time and the need for a 921KB EXTMEM buffer to hold the VGA image. You do still need a 614KB buffer for a VGA YUV image.

    This JPEG converter can be found at https://github.com/mjborgerson/TooJpeg-Teensy4.1

    There are other repositories in my github for the OV7670 library, the PXP library, the command processor and the generic data logger. All of these are works in progress, so let me know if you have problems with the libraries or exampled.

  3. #3
    Senior Member+ KurtE's Avatar
    Join Date
    Jan 2014
    Posts
    11,414
    Sounds like some nice progress on this. May have to play again with this!

  4. #4
    Junior Member
    Join Date
    Dec 2022
    Posts
    4

    Couldnt find the source code

    Hi, i am currently working on imxrt1170 board and having a similiar issue needing toojpeg accept YCbCr images as input. But couldnt find source code in the github link.
    Cqn you push the source code to github....Great thanks indeed...

  5. #5
    Quote Originally Posted by mxgok View Post
    Hi, i am currently working on imxrt1170 board and having a similiar issue needing toojpeg accept YCbCr images as input. But couldnt find source code in the github link.
    Cqn you push the source code to github....Great thanks indeed...
    Suggest you try the original by Stephen Brumme (link below). Docs say it does support YCbCr images.

    https://github.com/stbrumme/toojpeg

  6. #6
    Junior Member
    Join Date
    Dec 2022
    Posts
    4
    Quote Originally Posted by joepasquariello View Post
    Suggest you try the original by Stephen Brumme (link below). Docs say it does support YCbCr images.

    https://github.com/stbrumme/toojpeg
    Hi Joepasquariello,
    I have studied the code, the code supports OUTPUT YCbCr color space...As input it takes RGB array 3 bytes per_pixel. The point here is supplying the INPUT as YCbCr not as RBG array.

  7. #7
    Quote Originally Posted by mxgok View Post
    Hi Joepasquariello,
    I have studied the code, the code supports OUTPUT YCbCr color space...As input it takes RGB array 3 bytes per_pixel. The point here is supplying the INPUT as YCbCr not as RBG array.
    Okay. Hopefully @mborgerson will see your question and post the source to his github.

  8. #8
    Senior Member
    Join Date
    Feb 2015
    Location
    Finland
    Posts
    294
    JPEG uses the YCbCr color space internally everywhere.

    If you look at Stephen Brumme's toojpeg.cpp:writeJpeg() function (starting at line 556 in version 1.5), you can see three 88 arrays of floats: Y, Cb, and Cr. In the immediately following nested for loops, the innermost two loops just copy the 88 block from the original RGB image to the temporary arrays in YCbCr format: Y is between -128.0f and 127.0, and Cb and Cr between 0.0f and 255.0f. The second set of innermost nested for loops, applied only for 4:2:0, halves the chrominance by averaging the Cr and Cb values from four source pixels: (x,y), (x+1,y), (x,y+1), and (x+1,y+1); and the 88 arrays correspond to an area of 16 pixels.
    The EncodeBlock() function takes the 88 block of data, and compresses and emits it to the JPEG bitstream.

    So, the changes needed to work on YCbCr inputs should be quite straightforward, simplifying the existing code. Look for where pixels[] array is used, and where the RGB data is converted to YCbCr using rgb2y(), rgb2cb(), and rgb2cr() helper functions.

  9. #9
    Senior Member
    Join Date
    Feb 2018
    Location
    Corvallis, OR
    Posts
    431
    Quote Originally Posted by joepasquariello View Post
    Okay. Hopefully @mborgerson will see your question and post the source to his github.
    I have not yet posted the code to GitHub for two reasons:

    1. I want to make sure that what I post is compatible with the licensing and intentions of the original author, Stephen Brumme.
    2. I have encountered system crashes when the first pixel of the image to be compressed is at the start of EXTMEM. During the compression process, the code reads from the byte BEFORE the start of the buffer to get one of Y, Cb, or Cr. There are at least two ways to fix this:
    A) Make sure the buffer starts well inside the EXTMEM---I did this with a buffer structure that has 32 bytes of padding before the image itself.
    B) change the order of the Y, Cb, and Cr bytes in image. The OV7670 and CSI can handle this, but it has repercussions in a lot of other operations with the images or the pixel pipeline.

    I have not yet gone through the steps to implement option B, as I got sidetracked for several months with a driver for a thermal imaging camera. I used option A in the support routines for that camera. The driver I developed had several optimizations over the existing OV7670 library, and I had just started rewriting that library when leaf-raking and the holiday season cut into my programming time.

    Rather than post the whole toojpeg.cpp file, I will post the changes I made.

    Here is the externally available function that allows selection between RGB and YCbCr. You will have to add the definition to toojpeg.h.
    Code:
    // -------------------- externally visible code --------------------
    
    namespace TooJpeg
    {
    bool YUVMode = false;   /// default to original RGB888 input
    void UseYUV(bool bmode){
      if(bmode) YUVMode = true; else YUVMode = false;
    }

    here is the code that skips RGB conversion, which starts at about line 596 in my code:

    Code:
                  if (!isRGB)  {
                    Y[deltaY][deltaX] = pixels[pixelPos] - 128.f;
                    continue;
                  }
                  if(YUVMode) { // directly stuff in the YUV values from buffer
                    //  Assume order is YCbCr
                    Y [deltaY][deltaX] = pixels[2 * pixelPos  ] -128.0f;
                    if(pixelPos & 0x01){ // for odd pixels, Cb is at pixelpos -1
                      Cb[deltaY][deltaX] =  pixels[2 * pixelPos - 1]-128.0;
                      Cr[deltaY][deltaX] =  pixels[2 * pixelPos + 1]-128.0;
                    }else {  // for even pixels, Cb is at pixelPos +1 
                      Cb[deltaY][deltaX] =  pixels[2 * pixelPos + 1 ]-128.0;
                      Cr[deltaY][deltaX] =  pixels[2 * pixelPos - 1 ]-128.0;
                    }                            
                  } else {// Do RGB888 to YCbCr conversion           
                  // RGB: 3 bytes per pixel (whereas grayscale images have only 1 byte per pixel)
                  // color swap??  switch r and b
                    auto b = pixels[3 * pixelPos    ];
                    auto g = pixels[3 * pixelPos + 1];
                    auto r = pixels[3 * pixelPos + 2];
    
                    Y [deltaY][deltaX] = rgb2y (r, g, b) - 128; // again, the JPEG standard requires Y to be shifted by 128
                   // YCbCr444 is easy - the more complex YCbCr420 has to be computed about 20 lines below in a second pass
                   if (!downsample){
                      Cb[deltaY][deltaX] = rgb2cb(r, g, b); // standard RGB-to-YCbCr conversion
                      Cr[deltaY][deltaX] = rgb2cr(r, g, b);
                    }
                  } // end of RGB to YCbCr conversion
                }
              }
    I hope to update the OV7670 library and toojpeg sometime soon after the start of the new year.

  10. #10
    Junior Member
    Join Date
    Dec 2022
    Posts
    4
    Hi @mborgerson,
    Thanks for your explanations....clarified the issue.....merry christmas...

  11. #11
    Junior Member
    Join Date
    Dec 2022
    Posts
    4
    I tried and it worked after trials. Very important point is subtracting 128 in floating format as 128.0 or 128.0f otherwise overflow integer subtractions cause index values too large for huffmancode array.

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •