Problem trying to read OV7670 camera under IRQ Teensy 4.0

Status
Not open for further replies.
Looks like we are completing our screen updates faster than we are receiving frames...
It does give you flashing, probably no way for us to get in sync with when the display is set to best update.

I saw lots of flashing with my PC app also. In that case, it required a lot of investigation of how C++ Builder handles the display of offscreen bitmaps. I was able to fixt the PC part to eliminate the flashing, but still saw tearing with camera motion as the Teensy was often reading a changing buffer generated by the CSI at the same time the foreground program was sending out the data via Serial.write(...).

I have solved the tearing problem in my library code by setting a class variable, lastbufferread, to zero, then having a member function return that value (which can be 0, 1, or 2). The CSI DMA End interrupt sets the lastbufferread value to the number of the buffer just completed (1 or 2). The foreground program waits in a tight loop for lastbufferread to become non-zero. At that time, the indicated buffer has just been stored and won't be modified until the other buffer completes. That gives the foreground program at least 33mSec (at 30 FPS) to copy the data to a capture buffer or to send it to the PC. However, a full VGA buffer takes about 40mSec to transmit, so this method doesn't work for VGA at 30FPS. With a 12MHz CSI Clock it seems to work OK, but I'm still working out the details.

I should have the OV7670 library and some example code ready to post in the next few days.
 
Just an FYI - I received my quick and dirty boards that have setup for T41.1, OV7... and ILI9341...

Some of the code Is failing on my new board, but did not solder on PU resistors for Wire yet... Will do next.

Note, the board is setup for the CSI pin numbers, but may still experiment with the reading GPIO code to see how well it works.

What I am curious about is for the CSI_DX pins we use GPIO1(6). Which if you look at which pins are in use:
Code:
1	AD_B0_02	1.02
 0	AD_B0_03	1.03
24/A10	AD_B0_12	1.12
25/A11	AD_B0_13	1.13
19/A5	AD_B1_00	1.16
18/A4	AD_B1_01	1.17
14/A0	AD_B1_02	1.18
15/A1	AD_B1_03	1.19
40/A16	AD_B1_04 	1.20
41/A17	AD_B1_05 	1.21
17/A3	AD_B1_06	1.22
16/A2	AD_B1_07	1.23
22/A8	AD_B1_08	1.24 - CSI_D9
23/A9	AD_B1_09	1.25 - CSI_D8
20/A6	AD_B1_10	1.26 - CSI_D7
21/A7	AD_B1_11	1.27 - CSI_D6
38/A14	AD_B1_12 	1.28 - CSI_D5
39/A5	AD_B1_13 	1.29 - CSI_D4
26/A12	AD_B1_14	1.30 - CSI_D3
27/A13	AD_B1_15	1.31 - CSI_D2
And again not D2 ( bit 0 in 8 bit mode)... But probably bits reversed

So wonder if it will work like:
Code:
uint8_t in =  __RBIT(GPIO6_PSR ); // read all bits in parallel and use instruction to reverse the bits.

WIll try it once this board looks like it is working. Obviously only valid for T4.1....
 
Update: Soldered on PU resistors, now camera is happier?

And I was now able to get both sets of code to work with it. That is the CSI way and the GPIO way.

CSI way was how the board was setup with the IO pins, so it worked. Needed to change code to match the IO pins I used... But other than that worked.

GPIO. I changed pin defines to:
Code:
#if defined USE_CSI_PINS
#define OV7670_PLK   40 //40 // AD_B1_04 CSI_PIXCLK
#define OV7670_XCLK_JUMPER 41 // BUGBUG CSI 41 is NOT a PWM pin so we jumper to it...
#define OV7670_XCLK  37  //41 // AD_B1_05 CSI_MCLK
#define OV7670_HREF  16 // AD_B1_07 CSI_HSYNC
#define OV7670_VSYNC 17 // AD_B1_06 CSI_VSYNC

#define OV7670_D0    27 // AD_B1_15 CSI_D2
#define OV7670_D1    26 // AD_B1_14 CSI_D3
#define OV7670_D2    39 // AD_B1_13 CSI_D4
#define OV7670_D3    38 // AD_B1_12 CSI_D5
#define OV7670_D4    21 // AD_B1_11 CSI_D6
#define OV7670_D5    20 // AD_B1_10 CSI_D7
#define OV7670_D6    23 // AD_B1_09 CSI_D8
#define OV7670_D7    22 // AD_B1_08 CSI_D9
Note there is a hack here. It did not work... Why because the MCLK on pin 41... The problem is 41 is not a PWM pin, so the analogWrite stuff would not work... However a simple hack did. I instead told the camera code to use pin 37 which is a PWM pin AND I was not using. I then had the code pinMode(OV7670_XCLK_JUMPER, INPUT) and ran a jumper from 37 to 41 and it worked!

IMG_1259.jpg

IMG_1258.jpg

Edit: should mention as you can see I only partially assembled the board. I did not put on the two transistors to control backlight. I instead used a 100ohn resistor to jumper to that pin from VIN. Also did not add the quick and dirty sound stuff nor yet put in the USB connections. May at some point depending on if I do anything more with this.

Update: Note: The DMA code does not work yet with the updated pin numbers. Another similar issue where the PIXCLK pin is not an XBAR pin and I was using the XBAR code to use that signal for when to do DMA operation. Can obviously jumper again, may see about alternatives.
 
Last edited:
KurtE: Nice to see you got it working. I ran up against the same Pin 41 issue when using the CSI pins in polled mode testing with my soldered breadboard. I added a jumper to pin 37 also. What kind of clock speed were you able to use in the polled mode? I think I topped out at a CSI clock of about 4MHz.

I also received a couple of ILI9341 boards and was able to get one working with graphics test and the ILI9341_T3n library. I'll eventually get around to integrating it with the CSI OV7670 library, but I first want to get the OV7670 library polished up for presentation.
 
Quick update - I tried it out yesterday with a new ILI9488 board I purchased on Ebay, which looks like the same one I purchased awhile ago. And the code works :D

Right now issue of only working with
#define DEFAULTCLOCK 12

If I change this (about line 532 in OV7670_MB.cpp to 24 I get garbage colors and the like.
And yes with 12 the clock appears to be at 4mhz.

Note: I am going to move that DEFAULTCLOCK define to make it easier to use... Either top of file or...

I know part of the issue. the main init code has a function(s) that set the clock speed for both the CSI and the registerss for OV7670. Our updated Init called after all this, then more or less tells the camera to reboot... So it goes back to original settings and the CSI has the updated speed...

Trying to figure out how to fix. Probably best if can get the updated Init, which we borrowed from the other sketch for running with interrupts or the like, more properly integrated into the init code here.
May first try calling back off with calling back off again to setup the registers for the clock and see if that gets it there.

But here is the current stuff.
 

Attachments

  • CSI_41_main-201112a.zip
    25.4 KB · Views: 51
Right now issue of only working with
#define DEFAULTCLOCK 12

If I change this (about line 532 in OV7670_MB.cpp to 24 I get garbage colors and the like.
And yes with 12 the clock appears to be at 4mhz.
My new OV7670 library includes a SetCSIClock() function with an enum csispeed{CIS12, CSI24} input parameter.

I ran into the same weird color issue with CSI clock at 24MHz. At this speed, the CSI apparently inserts or deletes a byte at the beginning of the RGB565 bitmap, causing the byte order to get swapped. I hacked a simple fix by having my display routine start at the second byte of the bitmap (fb2 +1) and continuing for the proper number of bytes. It's a hack because that causes the display routine to read one byte past the end of the buffer, but you can't see that in the image.

Now that the OV7670 library is fairly stable (and posted to the Project Guidance forum), I plan to see if I can figure out why the switch to 24MHz CSI clock causes this issue. I suspect that it is some combination of VSYNC and HSYNC polarity settings or a required delay in the camera output.

Another work-in-progress is the addition of an ILI9341 display to my wired prototype board.
 
Thanks, I will try to take a look and probably convert over to your library over time.

After all I am just doing this for the heck of it.

What I am wondering is about when the clocks are generated versus when the data is valid. Maybe there is some offset value that is set currently for one clock speed and it needs to be updated...

What I am seeing with Logic Analyzer which with so many IO pins (is only able to run at 16mhz...) is
screenshot.jpg
Top two lines are for the Wire data.... Next 8 are the data bins, next 4 are the other signals.... MCLOCK, PIXCLK, HSYNC, VSYNC

And as you can see it looks like the data and the clock are sort of aligned, where neither edge looks safe, For either leading or trailing. Which may make sense for data not looking correct.
 
I have the 9341's and the same as KurtE new ili9488 here and soon ( not today's mail it seems ) a PCB to build the CSI camera board since have not made time to wire the camera to a T_4.1 yet with added display wires - PCB will be simple soldering.

So able to play along soon with ideally the same second set of cameras @mbogreson and @KurtE now have.
 
@mborgerson - I downloaded your library and built it and looks like it has the same timing issues. I maybe should hook up my faster Pro 8 LA. Won't see all of the bits, but at least it can verify the timing maybe has issues. Wondering if there is some setting to tell the signals to offset from the clock... Looking now.
 
Actually I think the timing is OK. hitting limits of my older Logic16 with 14/16 channels active can only go 16mhz samples. If I pull back to 8 pins sampled can go up to 32mhz...

If I hook up my Pro 8, can go a lot faster but obviously can still only see 8 channels. ;)

screenshot.jpg

So now back to seeing what data it is actually receiving and work from there.

Edit: Meant to mention that the PIXCLK is offset from the Master clock by half, so the samples look better.
 
Another interesting point, is when I am using the ILI9488 display to display things, it does not make much sense to try to push the camera that much as it takes longer for the image to write to the display than it does to read from the camera.

That is with my faster LA, and 8 channels, I repurposed the one I have for the MCLK to have my code set an IO pin high when I have a new image to display and low again when I complete the writeRect from the high memory

Which you can see in the image below, the yellow (Channel 4 is the time it took after I received a new page until it writes to the display. The bottom channel is the one that shows when it is starting a new image. So the update of the ILI9488 is taking about the time it takes to read in 4 images.
screenshot.jpg

I am thinking about hacking the code to have a third buffer, that when I am ready to work with an image, it swaps the spare buffer into the camera buffer register to, such that the new pages don't overwrite the one I am working with...
 
T_4.1 CSI OV camera board soldered here with fresh T_4.1 - should take apart after FLASH/PSRAM test and wash the PCB and Teensy before more running ...
 
CSI PCB working with OV CAMERA display to ILI9341 - using EXTMEM - so PCB and the soldering checkout here.

First 2k2 resistors I found were 402 sized - barely crossed the gap but working.

And that PCB can run display to new ILI9488 as well - but not seeing camera code with that.
 
Morning all,

I pushed up more current stuff, that was/is working off of interrupts up at: https://github.com/KurtE/Arduino_OV767X/tree/Teensy_t4_dma-hold
Still a WIP, that I am not sure where I will go with it. The DMA part is currently busted especially for my own board as the way I was generating a DMA clock was using XBar pin on PixCLK, but new pixclk is not XBAR... So could jumper again or punt. Punt is looking somewhat probable. Also learning more with other code and going more through it plus PDF files and finding things like, this code is currently checking pixclk and only processing when in HSYNC set. But there is camera option to only have the clock signal when the hsync, which the other code uses. But it is nice to be able to check against two different ways.

The current CSI one I included in this message. Again it is just a snapshot,

Note at the start of the INO file about line 6 is a USE_ILI9488 define which controls if 9488 or 9341 is used.

Now back to coffee and waking up.
 

Attachments

  • CSI_41_main-201115a.zip
    25.6 KB · Views: 53
I extracted the setup and send_tft function from the 201112a version of CSI_41_main and plugged it into my library test program. It worked right out of the zip file! The only change I had to make to the setup was to change the rotation to match the mounting of my TFT screen relative to my camera.

One thing I found is that when I set up the camera to QVGA, the software reshaping of the image is not necessary. All I have to do is send the bitmap and the ILI9341 handles the rotation.

The Image on the TFT looks like an exact match to the image transmitted to the PC.

ILI9341_test.jpg

My next tasks are to add video capability at about 15FPS and add the ILI9341 capability to some example programs in OV7670 library.
 
Here's a simple piece of code that does 15FPS video with the OV7670 library (posted in Project Guidance here: https://forum.pjrc.com/threads/64132-CSI-Camera-Library) and the ILI9341_T3n library from GitHub.

Code:
/****************************************************

    QVGA and ILI9341 test program for OV7670 library

    Transfers a QVGA image to ILI9341 TFT screen
    Capable of 15.0 Frames per second

    This code takes advantage of the fact that the
    OV7670 in QVGA mode matches the 320x240 pixels
    of the ILI9341

    requires T4.1  and saves frames in  DMAMEM
    which means that you do not need PSRAM
 ******************************************************** */

#include <OV7670.h>
#include <ILI9341_t3n.h>

//Specify the pins used for Non-SPI functions
#define TFT_CS   10  // AD_B0_02
#define TFT_DC   9  // AD_B0_03
#define TFT_RST  8


ILI9341_t3n tft = ILI9341_t3n(TFT_CS, TFT_DC, TFT_RST);

const int imarkpin = 32;  // used for o'scope frame rate timing

#define  IMARKHI digitalWriteFast(imarkpin, HIGH); // Also marks IRQ handler timing
#define  IMARKLO digitalWriteFast(imarkpin, LOW);
#define  IMARKTOGGLE digitalToggleFast(imarkpin);

//  Put all the buffers in DMAMEM
uint8_t image[320l * 240l * 2] DMAMEM;
uint8_t cbuff1[320l * 240l * 2] DMAMEM;
uint8_t cbuff2[320l * 240l * 2] DMAMEM;


const char compileTime [] = " Compiled on " __DATE__ " " __TIME__;

const int pinCamReset = 14;

void setup() {
  Serial.begin(9600);
  delay(200);
  Wire.begin();

  pinMode(pinCamReset, OUTPUT);
  pinMode(imarkpin, OUTPUT);

  digitalWriteFast(pinCamReset, LOW);
  delay(10);
  digitalWriteFast(pinCamReset, HIGH);  // subsequent resets via SCB


  if (OV7670.begin(QVGA, cbuff1, cbuff2)) {
    Serial.println("OV7670 camera initialized.");
    Serial.printf("cbuff1 at   %p\n", cbuff1);
    Serial.printf("cbuff2 at    %p\n", cbuff2);
  } else {
    Serial.println("Error initializing OV7670");
  }
  // 12 MHz gives 15FPS.  16MHz will do 20FPS, but leaves little time
  // for anything but video display.
  OV7670.SetCamClock(12);  
  // Start ILI9341
  tft.begin();
  tft.setRotation(3);  // Need rot = 3 for right-side up on MJB proto board

  CMSI();
  CMGV();  //  Jump right into video, so you don't even need a terminal!
}

void loop() {
// Only 3 choices:  's' System Info  'f' capture single frame  'v' video to TFT
  char ch;
  if (Serial.available()) {
    ch = Serial.read();
    if (ch == 's') CMSI();
    if (ch == 'v') CMGV();
    if (ch == 'f') CMGF();
  }

}

void CMSI(void) {
  Serial.printf("\n\nOV7670 Camera and ILI9341  QVGA Test 3 %s\n", compileTime);
  OV7670.ShowCamConfig();
}


// display QVGA video.  CSI and ILI9341_t3n.h can keep up at 15 Frames Per Second
void CMGV(void){
    Serial.println("Press any key to halt video");
    do{
      CMGF(); 
      delayMicroseconds(10);
    }while(!Serial.available());  // Serial input sends you back to loop()
}

// Capture and display a single frame from OV7670
void CMGF(void) {
  uint16_t readyframe;
  uint32_t imagesize;
  imagesize = OV7670.ImageSize();
  uint16_t lastframe = OV7670.FrameReady();
  do {
    readyframe = OV7670.FrameReady();
  } while (readyframe == lastframe); // wait until a frame just completed
  // copy the frame to image buffer to prevent tearing and glitches
  if (readyframe == 2) memcpy(image, cbuff2, imagesize);
  if (readyframe == 1) memcpy(image, cbuff1, imagesize);

  // display the frame.  Seems simple---until you look behind the curtain!
  // That's where KurtE does the hard work!
  IMARKHI
  tft.writeRect(0, 0, tft.width(), tft.height(), (uint16_t *)&image);
  // This takes about 42mSec-thus the 15FPS setting
  IMARKLO
}
 
Morning all,

I pushed up more current stuff, that was/is working off of interrupts up at: https://github.com/KurtE/Arduino_OV767X/tree/Teensy_t4_dma-hold
...
Note at the start of the INO file about line 6 is a USE_ILI9488 define which controls if 9488 or 9341 is used.
...

No luck ...

Using the github sketch : ...\Arduino_OV767X\examples\CameraDisplay_ili9341_t3n\CameraDisplay_ili9341_t3n.ino
No errors are shown on I2C writes:
Code:
I2C Write: reg: 0x12(COM7), value = 0x80
I2C Write: reg: 0x11(CLKRC), value = 0x01
I2C Write: reg: 0x3a(TSLB), value = 0x04
I2C Write: reg: 0x12(COM7), value = 0x00
I2C Write: reg: 0x17(HSTART), value = 0x13
I2C Write: reg: 0x18(HSTOP), value = 0x01
I2C Write: reg: 0x32(HREF), value = 0xb6
I2C Write: reg: 0x19(VSTART), value = 0x02
I2C Write: reg: 0x1a(VSTOP), value = 0x7a
...
Only compiles with #define USE_ILI9488, and only white display - or SerMon dump of chars? No ili image?

Using code in p#189 zip Either display starts up with color fill screens - I read the PCB as CS as 7 but the code has cs=10? Though it seems to work with either?
Here are first fails on writes:
Code:
OV7670 Camera Test 3  Compiled on Nov 15 2020 19:52:01
EXT Memory size: 8
Read Register(Write): 11 error: 2
Read Register(request): 11 data:ff error: 2
I2C Write: reg: 0x11(CLKRC), value = 0x41
Write Register:11 = 41 failed: 2
Read Register(Write): 6b error: 2
Read Register(request): 6b data:ff error: 2
I2C Write: reg: 0x6b(DBLV), value = 0x7f
Write Register:6b = 7f failed: 2
cam clock  12.00 with pll=1 and div=1
I2C Write: reg: 0x12(COM7), value = 0x80
Write Register:12 = 80 failed: 2
...
 
Confirmed - p#191 code in a Sketch works on the KurtE PCB to an ILI9341! Live Video.

Cannot sync camera rotation to the display as it is off a quarter turn - as designed?

I suspect that it is tied to the orientation of the camera module to the ILI9341. My camera is mounted so that the horizontal axis (320 pixels) is aligned with the long axis of the ILI9341. If the long axis of your IL9341 is at right angles to the long axis of your camera, you may not be able to get the display to follow the camera motion. That seems to be the case in the earlier photos in this thread.

That misalignment may be why the earlier code had to morph the data. It seems obvious that if you are going to mount a camera and a display, the longer axes of the two should be aligned.
 
I suspect that it is tied to the orientation of the camera module to the ILI9341. My camera is mounted so that the horizontal axis (320 pixels) is aligned with the long axis of the ILI9341. If the long axis of your IL9341 is at right angles to the long axis of your camera, you may not be able to get the display to follow the camera motion. That seems to be the case in the earlier photos in this thread.

That misalignment may be why the earlier code had to morph the data. It seems obvious that if you are going to mount a camera and a display, the longer axes of the two should be aligned.

That was my suspicion running your code - the long axis have a 90 deg offset. It tears on two of the rotations given the physical alignment.

Was nice that p#191 example ran - not sure what is off when using KurtE's code on a copy of his PCB ... I have a bad library or something out of sync.
 
Yep - I probably should make a version with the camera rotated 90 degrees, but was doing quick and dirty and seeing if I could make it same size as ILI9341...

But I also was/is not sure how I might use this. Would I really want the camera and display on directly opposite sides? Or would I want them to be more independent and maybe use a cable,
maybe like: https://smile.amazon.com/Pc-Accesso...PK/dp/B01DIXOBEQ/ref=sr_1_5?crid=IJFHVO673HPO

Or could do a simple Adapter board 18x18 like connectors, that maybe rotate the camera 90 degrees with option of forward or reverse one setup to point away from board and one pointing back out over top of display...


All suggestions are welcome.
 
Yep - I probably should make a version with the camera rotated 90 degrees, but was doing quick and dirty and seeing if I could make it same size as ILI9341...

But I also was/is not sure how I might use this. Would I really want the camera and display on directly opposite sides? Or would I want them to be more independent and maybe use a cable,
maybe like: https://smile.amazon.com/Pc-Accesso...PK/dp/B01DIXOBEQ/ref=sr_1_5?crid=IJFHVO673HPO

Or could do a simple Adapter board 18x18 like connectors, that maybe rotate the camera 90 degrees with option of forward or reverse one setup to point away from board and one pointing back out over top of display...


All suggestions are welcome.

Like the cable idea. Then you can move the camera a bit without moving the whole unit. That was one of the drawbacks of the Arducam shield where the camera is mounted on the display board.
 
I did order a set of cables... Will be curious if the extra lengths will cause delay issues...

Also did a quick and dirty board to swap orientations...

screenshot.jpg

Not sure if I will order any. May from OSHPark,

If I do may add ability for PU resistors. On that board. Probably not much else. Thought about maybe adding some way for other external IO pins to connect to jumper to show an LED or the like but... KISS

Forgot to mention, I hopefully set it up to allow the camera to point either forward or backward. I was worrying about if rotating it left or right was better but figured we can handle that easily by setting the rotation of the display
 
Quick update:

I hacked up the CSI version, still needs some cleanup. Right now configured for ILI9341 and I believe that it won't copy the memory out of the camera buffer to tft buffer in this case. Will actually make that part of the code #ifdef controlled.
As for this display we can use the 320x240 of both to make it work directly, although I may play with using different camera resolution and rotate...

As for ILI9488, it does extract copy the image as there is no setup for 480x320, although one might be able to set one up.

I am curious to go back and look at the read IO port version with a few changes to see what it does. DMA was busted, again for first pass may jumper to another pin which is an XBAR pin. But I am curious how much impact to the code might happen if I turn off the PIXCLK when not in the horizontal blanking period...
 

Attachments

  • CSI_41_main-201118a.zip
    26.1 KB · Views: 46
Been playing around some more.

Found more information in the Adafruit_OV7670 library about the HSTART, HSTOP...

So I noticed they had a sketch to play some with it, so I added something similar, such that I can set the window parameters and then look at LA data to see if it makes sense.

Got closer. I have a version now that will retrieve a 480x320 image directly from the camera, such that I don't have to then later extract the partial image. Also saves memory and ...

Got a set of setting that worked. I added a new image type to extract and the code is directly outputting to the ILI9488 display...

But it would be fun to have have another version that instead of 480x320 it retrieves a 320x480 image. I think I have setting that the camera should return that, (maybe?) Then am playing with a writeRectRot function in the ILI9488_t3 library, such that you can draw that retrieved image rotated image 90 degrees rotated and not have do it manually and maybe allow you to setup your camera that way it is ordered and compensate that the camera is rotated 90 degrees from orientation of screen. So far not working right yet.
Not sure yet if is is issue with reading from camera or with the drawing.

So I updated the save bitmap function. It was hard coded in the header for 640x480, so it now updates the header data for the current camera image size... That worked for the non-rotated version.

Now back to the working on the rotated version. But thought I would put up the current stuff in case anyone is interested in WIP...

EDIT :D :D - Fixed the rotation version :D Also did not need a writeRectRot (Dah) just change the screen rotation...

So now have the ILI9488 display showing the image in the right orientation... Maybe did not need my rotation cable or board after all...

EDIT: Should have mentioned the zip file was updated here.
 

Attachments

  • CSI_41_main-201119b.zip
    27 KB · Views: 48
Last edited:
Status
Not open for further replies.
Back
Top