FLIR Camera with Teensy 4.1

DroneOverLord

Active member
I have a project reading a 320x356 16bit image from a FLIR Boson Camera. It uses a standard CSI type interface (V Sync, H Sync, Pixel Clock and Data Valid) but for the life of me I can't seem to figure out how to get good data out of the camera with a Teensy 4.1.

I designed a board to interface with it and the camera is functional by hooking it up via USB.
IMG_20220713_114923260_HDR.jpg

I've also scooped all the pins and the signals are clean. V Sync is running at 60Hz and is detected without any issues with correct timing. Now Data Valid is running at 15.7Khz, and the signal is clean but there seems to be an issue with the pin...it should only be triggered 320 times a second but it's being triggered 1k-3k times a second....

Looking at the data sheet the Teensy 4.1 should be able to handle a transition rate of 1.06ns in fast mode...I can't figure out how to see if that's the default mode for the Teensy.

The Pixel Clock runs at 27Mhz, I'm reading the port directly to read the 16 parrel bits so I should be able to read them fast enough right?

Code:
/*
 June 2022
 Don Yates
 This Driver takes a raw 16bit image from the BOSON
 FLIR camera and stores it on an SD card
 */
#include <avr/io.h>
#include <avr/interrupt.h>

//Defining PINS
const int LED = 13;
#define D0 19   //GPIO_AD_B1_00
#define D1 18   //GPIO_AD_B1_01
#define D2 14   //GPIO_AD_B1_02
#define D3 15   //GPIO_AD_B1_03
#define D4 40   //GPIO_AD_B1_04
#define D5 41   //GPIO_AD_B1_05
#define D6 17   //GPIO_AD_B1_06
#define D7 16   //GPIO_AD_B1_07
#define D8 22   //GPIO_AD_B1_08
#define D9 23   //GPIO_AD_B1_09
#define D10 20  //GPIO_AD_B1_10
#define D11 21  //GPIO_AD_B1_11
#define D12 38  //GPIO_AD_B1_12
#define D13 39  //GPIO_AD_B1_13
#define D14 26  //GPIO_AD_B1_14
#define D15 27  //GPIO_AD_B1_15
#define PIXCLK 32
#define V_SYNC 6
#define H_SYNC  2 
#define DATA_VALID 9

#define  LEDON digitalWriteFast(LED, HIGH); 
#define  LEDOFF digitalWriteFast(LED, LOW);

/****************************       Setup     ***************************/
void setup() {
  //initialize the digital pin as an output.
  pinMode(LED, OUTPUT);
  pinMode(D0, INPUT);
  pinMode(D1, INPUT);
  pinMode(D2, INPUT);
  pinMode(D3, INPUT);
  pinMode(D4, INPUT);
  pinMode(D5, INPUT);
  pinMode(D6, INPUT);
  pinMode(D7, INPUT);
  pinMode(D8, INPUT);
  pinMode(D9, INPUT);
  pinMode(D10, INPUT);
  pinMode(D11, INPUT);
  pinMode(D12, INPUT);
  pinMode(D13, INPUT);
  pinMode(D14, INPUT);
  pinMode(D15, INPUT);
  pinMode(PIXCLK, INPUT);
  pinMode(V_SYNC, INPUT);
  //pinMode(V_SYNC, INPUT_PULLUP);
  pinMode(H_SYNC, INPUT);
  pinMode(DATA_VALID, INPUT);  
  // Open serial communications and wait for port to open:
  Serial.begin(115200);
  while (!Serial) {
   ;// wait for serial port to connect. Needed for native USB port only
  }
  
  //Setup Interrupts
 // attachInterrupt(digitalPinToInterrupt(V_SYNC), ISR_V_SYNC, FALLING);

}/************************************************************************/

//*** Global Variables ***
//Camera Resolution 320/256
uint16_t g_Frame[320 * 256]; 
static int g_FrameWidth() { return 320; }
static int g_FrameHeight() { return 256; }
unsigned int g_FrameIndex=0;
int g_FrameRate = 1;

volatile unsigned int g_Count=0;
unsigned int LowLastTime = 0;
unsigned int LowLastTwoTime = 0;
unsigned int LoopCount = 0;

// Global timing variables
volatile uint32_t g_subcnt, g_subcnt_start, g_subcnt_end;   
volatile uint32_t g_cnt, g_cnt_start, g_cnt_end;
volatile unsigned int g_ISR_Triggered = 0;
float     g_ns;
/*************************************************************************/
/*                                 Main Loop                             */
/*************************************************************************/
void loop() {
  uint16_t Frames = 0;
  uint16_t Rows = 0;
  uint16_t Pixels = 0;
  int temp; 
  
  while(1){
    //Check if V_SYNC is low and transitions to high
    if(!digitalReadFast(V_SYNC)){//If Sync Low
      g_FrameIndex = 0;// Reset the Frame Index
      /********************* Getting Frame ***************************/
      while(!digitalReadFast(V_SYNC)){ //Wait till Sync goes high, START OF NEW FRAME
      delayNanoseconds(1); 
      }
      g_cnt_start = ARM_DWT_CYCCNT;//Get current processor clock cycle number
      Frames++;//New Frame on Rising V_Sync
      Rows = 0;
      Pixels = 0;
      /********************* Getting Row ***************************/
      while(digitalReadFast(V_SYNC)){ 
        Rows++;  //Rising Data Valid means new Row
        //temp = digitalReadFast(DATA_VALID);
        while(digitalReadFast(DATA_VALID)){                    
          if(digitalReadFast(PIXCLK)){
            Pixels++;
            //g_subcnt_start = ARM_DWT_CYCCNT; //Get current processor clock cycle number 
            g_Frame[g_FrameIndex++] = cameraReadPixels();
            //g_subcnt_end = ARM_DWT_CYCCNT; //Get current processor clock cycle number
            }
          else{
            delayNanoseconds(1);
            }
          }
      }
    }
    
    if(Frames >=60){    
      g_cnt_end = ARM_DWT_CYCCNT;//Get new processor clock cycle number
      Serial.print("Number of Frames  = ");
      Serial.println(Frames);
      Serial.print("Number of Rows  = ");
      Serial.println(Rows);
      Serial.print("Number of Pixels  = ");
      Serial.println(Pixels);
      g_FrameIndex = 0;
      Frames = 0;
      Rows = 0;
      Pixels = 0;
      //g_cnt = g_subcnt_end - g_subcnt_start;
      //g_ns = g_cnt*1E9f/F_CPU;
      //Serial.printf("Subroutine time: %5u (%0.6g ns)\n",g_cnt, g_ns);
      g_cnt = g_cnt_end - g_cnt_start;
      g_ns = g_cnt*1E9f/F_CPU;
      Serial.printf("Total Run Time: %10u (%0.6g ns)\n",g_cnt, g_ns);   
      Serial.print("Frame Data: ");
      Serial.println(g_Frame[10]);      
      Serial.println("**************************************");  
        LEDON;
        delay(500);                // wait for a second/2
        LEDOFF;
        delay(500);                // wait for a second/2   
    }           
  }
}/************************************************************************/



/*************************************************************************/
// Read a uint8_t of the pixel data
static inline uint16_t cameraReadPixels() 
{
  uint16_t b = 0;
  uint32_t pword= GPIO6_DR >> 16; // get the port bits
  b = (uint16_t)pword;
  return b;
}/************************************************************************/
 
We have played around with a few other camera types that you might want to search on.

Like the OV7670 [/url]

Thanks for the reply.

Yes, I've read that thread and code, that's what gave me the idea but the OV7670 is slower. I'm wondering what the fastest input speed anyone has reliable captured and is there anything more than digitalReadFast to capture a pin at the 1.06ns speed? There is not pinModeFast that I can tell.
 
I am not sure... Sorry,

I know with the OV7670, I was able to read it reasonably well using DMA, Where the DMA read was setup to use the pixel clock as the input clock to drive the DMA operation.

It took some experimenting with things like there was a setting on the camera to only output the pixel clock when in the area being returned, such that we did not have to then decipher which things in the DMA were valid pixels and the like.

If I remember correctly I had DMA save away the whole 32 bits of the register, where I believe I also had some of the additional signals as well. And I had the DMA operation setup to chain some buffers, and when one buffer was filled and it went on to the next one, I setup an interrupt, that then post processed that buffer to extract the pixel data...

But it took a lot of trial and error to get things working. It has also been awhile, so my memory is a bit fuzzy. Things like having the PIXCLK on an XBAR pin (I think)...

The camera Looks like fun, but a little higher priced..

Does this camera allow it to be configured for 4 bit transfers? If so you might look at the CSI chapter and see if that would work for you.

Good luck

EDIT: Side note - Try using GPIO6_PSR instead of GPIO6_DR
and see if that helps.
 
Last edited:
KurtE, yea it's a bit pricey but has great image quality. It's not configurable to 4-bit transfers but could do 8bits. I have it wired for 16-bit data on GPIO6_AD_B1_00-15 so I can do a quick DR. The problem I'm having is not reading the 16-bits it's something with the Sync bits.


It's almost as if I'm reading the pins too fast.

For example, the following code hangs for seconds at a time waiting for DATA_VALID to go high, even though I know it's toggling at 15.7khz with my scope:
Code:
 while(!DATA_VALID_PIN){
 delayNanoseconds(1);}

but if I check the pin using:

Code:
if(!DATA_VALID_PIN){
 delayNanoseconds(1000);} 
if(!DATA_VALID_PIN){
 delayNanoseconds(1000);} 
if(!DATA_VALID_PIN){
 delayNanoseconds(1000);} 
if(!DATA_VALID_PIN){
 delayNanoseconds(1000);} 
if(!DATA_VALID_PIN){
 delayNanoseconds(1000);}

The code will run at the ~15.7Khz rate that Data Valid toggles.

This is driving me mad; it shouldn't be this hard to simply check when a digital line gets toggled from low to high. I have to believe I'm missing something in the setup, but I used pinMode(DATA_VALID, INPUT).

I thought the pins default to Fast mode on reset? In any case I'm reading and re-reading chapters 10- 12 of the IMXRT1060RM. I'm sure I'm missing something simple
 
My bad. It defined as:

Code:
#define IMXRT_GPIO7_DIRECT  (*(volatile uint32_t *)0x42004000)
#define DATA_VALID_PIN (IMXRT_GPIO7_DIRECT & 0b00000000000000000000100000000000) //Direct read of Data Valid pin

Combing through the reference manual, I don't understand how to ensure ALT5 mode is set for GPIO. Maybe that's the problem?
 
Why not first do pinMode(DATA_VALID, INPUT);
And use: digitalReadFast(DATA_VALID )

Which for pin9 as a constant
Code:
		} else if (pin == 9) {
			return (CORE_PIN9_PINREG & CORE_PIN9_BITMASK) ? 1 : 0;
And the compiler optimization will throw the rest of the code of that function away and leave you with just
(CORE_PIN9_PINREG & CORE_PIN9_BITMASK) ? 1 : 0;
 
I tried pinMode(DATA_VALID, INPUT) and digitalReadFast(DATA_VALID). You can see that in my first post. It's not acting as it should (hanging in the while(!DATA_VALID) even though the pin is toggling).

I'm trying to read the port directly, just to see what that does. I'm now thinking I need to use the Pad Sample Register PSR instead of the Data Register DR. If you look at page 952 of the Ref manual it shows PSR as input and the other registers as output. You pointed that out in a post above, but I didn't understand that till reading the manual.
 
After reading, re-reading the ref manual I've come to the conclusion the problem has to be with the Teensy interrupts.

My while(data_valid) loop needs 12 usec to execute uninterrupted, does anyone know how often the Teensy 4.1 triggers the Serial ISR? Is there some documentation somewhere for all the ISR routines that get called with the Teensy bootloader?
 
Which Serial? HardwareSerial, like Serial1, Serial2... only when there is something to input or output.

USB - Again maybe only where there is USB data being sent or received...

As for other interrupts - probably mainly the system tic is going on, unless there is other stuff you are doing.

I was a little curious and was wondering if there was some document that showed the timings of a frame...
So downloaded FLIR Boson Thermal Imaging Core Product Datasheet...

And did not see a full like one picture shows the data frame...
But It looks like pages: 120-121 sort of show it.
And on 120 it looks like cmos_data_valid looks like it is valid for a whole horizontal row... So should have plenty of time?

But was curious about voltages...

Looks like device you give it 3.3v supply. But what is the logic level? some things sound like 3.3v

But then I see P149(12.4 Absolute Maximum Ratings)
Voltage on any GPIO pin 1.98V

So wondering if the signals are 1.8V and maybe not typically registering as a high value?
 
Yes, USB serial is what I'm using to debug. That and system tic are the only things I could think that would be running. BUT again, I don't know if there is documentation other than the source code for how the Teensy (RTOS/BOOTLOADER/Whatever) handles code real time.

The Boson Datasheet took me awhile to understand. Yes, the Data Valid signal is high for the entire row transfer. About ~12 microseconds and then is low for ~50 microseconds. I would think 12 microseconds would be plenty of time for the Teensy.

You are correct as well about the voltage. The CMOS channels are 1.72volt, measured by my O-scope. I know the Teensy handles speeds differently based on voltage. There are typically two voltages listed in the Ref Manual 3.3 and 1.8v. Maybe 1.72volts is too low??? That would royally suck.
 
Sorry I know nothing about running the Teensy with GPIO running at 1.8v mode.

I do see a few things in the RM about fuse settings.

If it were me, I would try to hook up a level shifter from 1.8v to 3.3v and see if it makes a difference

But hopefully others will know if there is a way to force the Teensy IO pin to work at 1.8v levels.
 
Upon further reading the 1.8v and 3.3v pin is set based on the voltage supplied to NVCC_GPIO pins. According to the Teensy 4.1 schematic those pins are wired to 3.3v. So that's not software programable.

The e-fuses look like they are for setting the bootloader, but I haven't deep dived into that.

Look at the MIMXRT1060 Data sheet page 36, table 22. It shows the High-Level input voltage min is 0.7xNVCC_GPIO which, in the Teensy 4.1, is 2.31 volts. SO it looks like I will have to add a level shifter to get this to work. Hope this helps others down the road.
 
Back
Top