Animated images on Teensy 3.5 & ILI9341

Status
Not open for further replies.

NaokiS

Member
Hi all,

First post here and probably jumping in the deep end with my current skill level. However, that being said, I'm working on a TFT project which is going to be used in a vehicle to display information like speed, tachometer and the likes. I chose the Teensy 3.5 because I wanted a good amount of performance with a fairly nice GUI interface too. I also wanted it to have the ability to display an animation after initialisation mainly for look and feel reasons.

Now to do this, my logical assumption was to do a quick check to see if even using some code to display standard GIFs, however I saw people saying to instead load the animation frames as BMP and display them sequentially instead. So taking a video I wanted to use, I converted it to BMP frames at around 20 FPS and named the frames 0.bmp, 1.bmp etc... And put them on the SD card under a folder to keep them together.

Using the example code spitftbitmap.ino, I ran the images through without delays and it end up displaying at about what I consider 3 FPS. I wasn't expecting high FPS but it's incredibly slow at about 400ms per frame. From what I've read from previous topics it could just be loading it from SD is slowing it down. I've tried to use USE_TEENSY3_CODE to no avail (compiler says that "BUILTIN_SDCARD" wasn't previously declared).

So, being I'm very much out of my comfort depth, I'd thought Id ask here incase it's been done or I'm doing it entirely wrong.

The end goal is to have the frames displayed as best they can to acheive the look, essentially this video but at a correct frame rate:

To answer some inevitable questions,
  • Yes, I could forego the use of animations and just display a static image, but I would like to try to see if this even works first
  • I am loading them from SD card so that I can change out the GUI files and update them without hardboiled code. (Using JSON to define file names, where, how many and how fast in future)
  • Screen is using a different set of SPI pins because I would be using the Audio sheild aswell
  • I have limited knowledge on the DMAs or Async of the teensy 3.5 even though I've tried to follow it best I can.

I have all the original images that i've use and the barely modified and thus perhaps poor implementation code is here:
Code:
/**************************************************  *
  This is our Bitmap drawing example for the Adafruit ILI9341 Breakout and Shield
  ----> http://www.adafruit.com/products/1651

  Check out the links above for our tutorials and wiring diagrams
  These displays use SPI to communicate, 4 or 5 pins are required to
  interface (RST is optional)
  Adafruit invests time and resources providing this open source code,
  please support Adafruit and open-source hardware by purchasing
  products from Adafruit!

  Written by Limor Fried/Ladyada for Adafruit Industries.
  MIT license, all text above must be included in any redistribution
 **************************************************  **/


#include <ILI9341_t3n.h> // Hardware-specific library
#include <SPI.h>
#include <SD.h>

// TFT display and SD card will share the hardware SPI interface.
// Hardware SPI pins are specific to the Arduino board type and
// cannot be remapped to alternate pins.  For Arduino Uno,
// Duemilanove, etc., pin 11 = MOSI, pin 12 = MISO, pin 13 = SCK.

#define TFT_DC      20
#define TFT_CS      21
#define TFT_RST    255  // 255 = unused, connect to 3.3V
#define TFT_MOSI     7
#define TFT_SCLK    14
#define TFT_MISO    12
ILI9341_t3n tft = ILI9341_t3n(TFT_CS, TFT_DC, TFT_RST, TFT_MOSI, TFT_SCLK, TFT_MISO);
#define SD_CS BUILTIN_SDCARD

void setup(void) {
  // Keep the SD card inactive while working the display.
  pinMode(SD_CS, INPUT_PULLUP);
  delay(200);

  tft.begin();
  tft.fillScreen(ILI9341_BLUE);
  tft.useFrameBuffer(1);
  tft.updateScreenAsync();
  Serial.begin(9600);
  tft.setTextColor(ILI9341_WHITE);
  tft.setTextSize(2);
  tft.println(F("Waiting for Arduino Serial Monitor..."));
  //while (!Serial) {
  //if (millis() > 8000) break;
  //}
  Serial.println(SD_CS);
  Serial.print(F("Initializing SD card..."));
  tft.println(F("Init SD card..."));
  while (!SD.begin(SD_CS)) {
    Serial.println(F("failed to access SD card!"));
    tft.println(F("failed to access SD card!"));
    delay(2000);
  }
  Serial.println("OK!");
}

void loop() {
  tft.fillScreen(ILI9341_GREEN);
  // Draw a few frames only to compare load time
  bmpDraw("ford/0.bmp", 0, 0);
  delay(5000);
  bmpDraw("ford/25.bmp", 0, 0);
  delay(5000);
  bmpDraw("ford/50.bmp", 0, 0);
  delay(5000);
}

// This function opens a Windows Bitmap (BMP) file and
// displays it at the given coordinates.  It's sped up
// by reading many pixels worth of data at a time
// (rather than pixel by pixel).  Increasing the buffer
// size takes more of the Arduino's precious RAM but
// makes loading a little faster.  20 pixels seems a
// good balance for tiny AVR chips.

// Larger buffers are slightly more efficient, but if
// the buffer is too large, extra data is read unnecessarily.
// For example, if the image is 240 pixels wide, a 100
// pixel buffer will read 3 groups of 100 pixels.  The
// last 60 pixels from the 3rd read may not be used.

#define BUFFPIXEL 240


//==================================================  =========
// Try Draw using writeRect
void bmpDraw(const char *filename, uint8_t x, uint16_t y) {

  File     bmpFile;
  int      bmpWidth, bmpHeight;   // W+H in pixels
  uint8_t  bmpDepth;              // Bit depth (currently must be 24)
  uint32_t bmpImageoffset;        // Start of image data in file
  uint32_t rowSize;               // Not always = bmpWidth; may have padding
  uint8_t  sdbuffer[3 * BUFFPIXEL]; // pixel buffer (R+G+B per pixel)
  uint16_t buffidx = sizeof(sdbuffer); // Current position in sdbuffer
  boolean  goodBmp = false;       // Set to true on valid header parse
  boolean  flip    = true;        // BMP is stored bottom-to-top
  int      w, h, row, col;
  uint8_t  r, g, b;
  uint32_t pos = 0, startTime = millis();

  uint16_t awColors[320];  // hold colors for one row at a time...

  if ((x >= tft.width()) || (y >= tft.height())) return;

  Serial.println();
  Serial.print(F("Loading image '"));
  Serial.print(filename);
  Serial.println('\'');

  // Open requested file on SD card
  if (!(bmpFile = SD.open(filename))) {
    Serial.print(F("File not found"));
    return;
  }

  // Parse BMP header
  if (read16(bmpFile) == 0x4D42) { // BMP signature
    Serial.print(F("File size: ")); Serial.println(read32(bmpFile));
    (void)read32(bmpFile); // Read & ignore creator bytes
    bmpImageoffset = read32(bmpFile); // Start of image data
    Serial.print(F("Image Offset: ")); Serial.println(bmpImageoffset, DEC);
    // Read DIB header
    Serial.print(F("Header size: ")); Serial.println(read32(bmpFile));
    bmpWidth  = read32(bmpFile);
    bmpHeight = read32(bmpFile);
    if (read16(bmpFile) == 1) { // # planes -- must be '1'
      bmpDepth = read16(bmpFile); // bits per pixel
      Serial.print(F("Bit Depth: ")); Serial.println(bmpDepth);
     if ((bmpDepth == 24) && (read32(bmpFile) == 0)) { // 0 = uncompressed

        goodBmp = true; // Supported BMP format -- proceed!
        Serial.print(F("Image size: "));
        Serial.print(bmpWidth);
        Serial.print('x');
        Serial.println(bmpHeight);

        // BMP rows are padded (if needed) to 4-byte boundary
        rowSize = (bmpWidth * 3 + 3) & ~3;

        // If bmpHeight is negative, image is in top-down order.
        // This is not canon but has been observed in the wild.
        if (bmpHeight < 0) {
          bmpHeight = -bmpHeight;
          flip      = false;
        }

        for (row = 0; row < h; row++) { // For each scanline...

          // Seek to start of scan line.  It might seem labor-
          // intensive to be doing this on every line, but this
          // method covers a lot of gritty details like cropping
          // and scanline padding.  Also, the seek only takes
          // place if the file position actually needs to change
          // (avoids a lot of cluster math in SD library).
          if (flip) // Bitmap is stored bottom-to-top order (normal BMP)
            pos = bmpImageoffset + (bmpHeight - 1 - row) * rowSize;
          else     // Bitmap is stored top-to-bottom
            pos = bmpImageoffset + row * rowSize;
          if (bmpFile.position() != pos) { // Need seek?
            bmpFile.seek(pos);
            buffidx = sizeof(sdbuffer); // Force buffer reload
          }

          for (col = 0; col < w; col++) { // For each pixel...
            // Time to read more pixel data?
            if (buffidx >= sizeof(sdbuffer)) { // Indeed
              bmpFile.read(sdbuffer, sizeof(sdbuffer));
              buffidx = 0; // Set index to beginning
            }

            // Convert pixel from BMP to TFT format, push to display
            b = sdbuffer[buffidx++];
            g = sdbuffer[buffidx++];
            r = sdbuffer[buffidx++];
            awColors[col] = tft.color565(r, g, b);
          } // end pixel
          tft.writeRect(0, row, w, 1, awColors);
        } // end scanline
        Serial.print(F("Loaded in "));
        Serial.print(millis() - startTime);
        Serial.println(" ms");
      } // end goodBmp
    }
  }

  bmpFile.close();
  tft.updateScreen();
  if (!goodBmp) Serial.println(F("BMP format not recognized."));
}



// These read 16- and 32-bit types from the SD card file.
// BMP data is stored little-endian, Arduino is little-endian too.
// May need to reverse subscript order if porting elsewhere.

uint16_t read16(File &f) {
  uint16_t result;
  ((uint8_t *)&result)[0] = f.read(); // LSB
  ((uint8_t *)&result)[1] = f.read(); // MSB
  return result;
}

uint32_t read32(File &f) {
  uint32_t result;
  ((uint8_t *)&result)[0] = f.read(); // LSB
  ((uint8_t *)&result)[1] = f.read();
  ((uint8_t *)&result)[2] = f.read();
  ((uint8_t *)&result)[3] = f.read(); // MSB
  return result;
}
 
I got video running on a 3.6, but never on a 3.5 - there was a DMA problem I never solved (I must confess, I not tried it very intensively, because at that time T3.5 had less RAM than a 3.6 :)
https://www.youtube.com/watch?v=cyGIW3KFrtw

I did see that which is what gave me some inspiration to do this stuff in honesty! I went with a 3.5 as it seemed to fit the bill and not be over specced for what was needed. I'm assuming the SD isn't being read fast enough which is causing the problem, but can't fully figure out how to speed it up.
 
I got video running on a 3.6, but never on a 3.5 - there was a DMA problem I never solved (I must confess, I not tried it very intensively, because at that time T3.5 had less RAM than a 3.6 :)
https://www.youtube.com/watch?v=cyGIW3KFrtw

Anyone have schematic for pinouts etc for this? Also, what's the format for the video file? I took a look on the drive share, and seems like some .bin files there, but I don't know how they're storing the frames. Is it just raw frame data (RGB bytes?) Any help appreciated!
 
@NaokiS - sorry I did not see this thread back in February,

Are you still active on it?

As @Frank B mentioned - The T3.6 DMA is a lot nicer than it is on the 3.5. But I was able to get some of it to work in the ili9341_t3n library, after we figured out that it did have more memory.

I don't know enough about your desires here, or limitations... There are a few things about the ILI9341_t3n library and the use of a frame buffer that might help (or not)...

When you turn on using a frame buffer, by default it will use malloc to allocate a buffer...
Alternatively you can pre allocate the buffer your self and set it using the call: tft.setFrameBuffer(my_buffer);
Which needs to be a uint32_t buffer of size 320*240...

Now after you call useFrameBuffer(1);
All of the internal methods will write to this buffer and by default will NOT update the screen, until you call something like:

tft.updateScreen() - will update the screen once, Not using DMA and will not return until the full screen has been updated.

tft.updateScreenAsync() - Will update the screen once USING DMA, it will return just after starting the transfer to the display. You can call functions like
asynchUpdateActive - which will return true if the asynch update is still active.
waitUpdateAsyncComplete - Will wait until that update completes

But you can also set the update to happen all of the time:
tft.updateScreenAsync(true); - Which will start up the DMA operation, and continue to blast pixels out the display, when it completes the end of the display it will continue back to the start of the display again.

When it completes a "Frame" it updates a frame count, which you can retrieve using frameCount().
You can end the continuous updates by calling endUpdateAsync() - which will stop once the current frame completes.

What I honestly don't know is how long does it take to update the display versus how long does it take for you to read in the next picture...

There are several things that one could try:

You could see what happens if you turn on using updateScreenAsync(1);
And then change the bmpDraw function not to call updateSceen() at end...
This might do everything you need. Although you might see some potential tearing or ... as you see part of one frame drawn will still part of another frame...

Or if you wish to be more in control, you could change the: updateScreen call at the end of bmpDraw to be an updateScreenAsync();
Which will start the output of that one frame.

Now if your load of the next frame is slower than how long it takes to load the next frame, it should just show one frame after another without partial frames...
Note, If the updateScreenAsync() is very close to the speed of the load bitmap, you may want to add a waitUpdateAsyncComplete() call just before the updateScreenAsync() as the updateScreenAsync() will fail if one is already running.

Now if the load bitmap function is faster than the screen update... We may want to play around in the library to get some additional information about where the system is updating in the frame...

Likewise one could optimize the loading of bitmaps in this case to write directly to frame buffer memory instead of other memory and use drawRect which then copies the memory...

Hope that helps.

Kurt
 
Thanks for the reply Kurt!

I've only just got back to looking into the project again and thank you for your suggestions. I have started another version without the video to get the actual project going but will look back into your suggestions. Altough, currently I'm getting it where my sketch is loading both t3 and t3n libraries. I havent a clue why
 
I've tried your suggestions but it's clearly not a display issue. The issue, upon further reading, seems to be with the SD loading routines. Even loading one entire line into RAM doesn't seem to make a difference.

As to my desire and/or limitations, I'm essentially just trying to get this logo to display at around 50ms per image with each image about 170kb in size at 24-bit depth. Ideally I would like that to be best case but I can settle for 100ms if needed. I currently load all the images in a sequence of bmps starting at 0 and going up to 60.bmp. I'm considering either preformatting the images in 565 format (which brings the bmps to 113kb) or even have them in one contiguous file that would contain the whole animation with frames stored after the other since then I could read in the whole image and push the colours to the display ready to format. I'm not sure which would help. I'm not even sure if the files can be displayed that quickly on teensy 3.5 altough I'd hope so as 5V compatibility would be a godsend in the design I'm using.
 
Code:
// Seek to start of scan line.  It might seem labor-
          // intensive to be doing this on every line, but this
          // method covers a lot of gritty details like cropping
          // and scanline padding.  Also, the seek only takes
          // place if the file position actually needs to change
          // (avoids a lot of cluster math in SD library).
          if (flip) // Bitmap is stored bottom-to-top order (normal BMP)
            pos = bmpImageoffset + (bmpHeight - 1 - row) * rowSize;
          else     // Bitmap is stored top-to-bottom
            pos = bmpImageoffset + row * rowSize;
           
          if (bmpFile.position() != pos) { // Need seek?
            bmpFile.seek(pos);
            buffidx = sizeof(sdbuffer); // Force buffer reload
          }

So, this is the snippet of code that causes up to 200ms of extra execution time. I'm guessing that the 200ms is because it does it every line and there's 240 lines. More specifically it's bmpFile.seek(pos) which is the worst offender. Commenting it out will *massively* speed up the code to run at only 29ms per image which is faster than my desired execution time but now the image is flipped, red and rendered wrong. I will have to look and hope i can seek at the start of the file but without knowing specifics on BMP file format ordering I'm not sure if I can yet. I'm hoping I may even be able to remove the conversion from 24-bit to RGB565 by just loading the 565-encoded bmp files aswell. Will update later if I get this working.
 
Status
Not open for further replies.
Back
Top