Accessing PROGMEM correctly with Structs

[Ein]

I am a miserable beginner, so please be patient with me, but I would very much appreciate help. I have a complex and growing program that displays information on a TFT display using an Arduino (Teensy 3.2, more specifically). I've been trying to tame the situation with some structs, but I'm not sure I'm doing this correctly in order to keep as much as possible in PROGMEM and out of RAM.

Some pseudo-code below to explain what I'm doing:

I have a file I have split all of my graphical data off into called "graphics.h" which I am #including where necessary to utilize its contents. This file has a number of data sets:

• Multiple sets of 4-bit-per-pixel bitmap data. These are stored in PROGMEM in uint8_t arrays. EX:
  const uint8_t bitmap1[] PROGMEM = { 0x00, 0x00... etc ... };
  const uint8_t bitmap2[] PROGMEM = { 0xFF, 0XFF... etc ... };
  ... and so on, and so on. A bunch of these.
  
• Sets of color palettes, stored in uint32_t arrays, which correlate with the bitmap sets. EX:
  const uint32_t colorSet1[] PROGMEM = { 0xFF5A737E, 0XFF64808C... etc ... };
  

These palettes don't correlate 1:1 to the bitmaps, meaning I use one palette for multiple images, rather than one palette per image.

Now, I have been using the data in this file without issue. I am using a variant of the ILI9341_t3 library, which has a 4-bit-per-pixel bitmap writing function that works with the above data types. I have an object from this library that I have named 'tft' and if I want to draw something, I just have to call the writeRect4BPP() function, EX:

tft.writeRect4BPP(x, y, width, height, bitmap2, colorSet1);

The writeRect4BPP function is itself looking for pointers to the bitmap and colorSet:

void ILI9341_t3::writeRect4BPP(int16_t x, int16_t y, int16_t w, int16_t h, const uint8_t *pixels, const uint32_t * palette ){}

I set up a struct to organize all of the data sets necessary in my graphics.h file.

struct imageSet{
uint8_t *bitmap1;
uint8_t *bitmap2;
uint8_t *bitmap3;
uint8_t *bitmap4;
uint8_t *colorSet};

const imageSet set1 PROGMEM = {bitmap1, bitmap2, bitmap3, bitmap4, colorSet};

I have a function that picks one of these bitmaps from the set based on other logic in the code (the value of an int, basically) and draws that image accordingly:

if(int == 0){
tft.writeRect4BPP(x, y, width, height, set1.bitmap1, set1.colorSet);
} else if(int == 1){
tft.writeRect4BPP(x, y, width, height, set1.bitmap2, set1.colorSet);
} else if(int == 2){
tft.writeRect4BPP(x, y, width, height, set1.bitmap3, set1.colorSet);
} else {
tft.writeRect4BPP(x, y, width, height, set1.bitmap4, set1.colorSet);
}

The thing I'm noticing is that as I increase this in complexity, my program is starting to hang, freeze, lock up, or otherwise show garbage data in other unrelated areas (text output, for example). If I get rid of the conditional statements above and just always write one bitmap from the set (i.e. set1.bitmap1), there are no issues, but once I start adding the conditionals in things get weird.

I'm almost positive this weirdness is resulting from my RAM getting used up. This suggests to me that I may be doing the PROGMEM thing wrong, and that it may be inadvertently grabbing all of the separate bitmap data arrays for each image and trying to hold them in RAM. The actual flash memory is only seeing about 23% usage when I compile.

Can anyone tell me if/how I'm approaching this wrong?

 

[defragster]

The ARM T_3.2 has a flat 32 bit memory space versus the AVR RAM and FLASH. "const" items will go to FLASH and are usable from there Use of PROGMEM isn't needed, perhaps certain uses may not map away. Make a version without PROGMEM usage and if the problem persists it is due to other usage factors. If it doesn't resolve it - reduce the problem to a simple sample if possible and post the source per the forum rule.

 

[KurtE]

As @defragster mentioned hard to say what the issue might be without seeing more information.

As he mentioned, PROGMEM means nothing on ARM processors, the more important thing is to make sure they are marked as const.

When you build your program how much program space (23%) and memory space does the system say you used? Close to the limits?

Could be lots of things that might corrupt memory. Like maybe you have a function that has an array that you over run the bounds of it. Or maybe you have a function that you think you are passing a pointer to a bitmap, but instead you are trying to pass the whole bitmap on the stack... Maybe you are doing some recursive calls that go deep enough to corrupt the memory.

Maybe just simple program bug like an infinite loop. I have seen things like: uint8_t i = something.... while(i-- >= 0) {...}
since i is unsigned it will always be >= 0... Normally will get compiler warning... So compile verbose and look at all compiler warnings.

With hangs and the like, I try doing some of the default debugging things, like put in Serial.print statements in key places like before and after calls, maybe show more information, like if you are going to draw a bitmap, show it's address, index, ... Also as I have a Logic Analyzer, I use unused IO pins to tell me what is going on. Like do a digitalWriteFast(DEBUG_PIN1, HIGH); callMyFunction(); digitalWriteFast(DEBUG_PIN1, LOW);
So if I hang and my debug pin 1 is HIGH, I know I am still in callMyFunction.... Obviously you needed to previously set the debug IO pin to OUTPUT.

Again without seeing additional information, can not give any more specific advice.

Good luck

 

[Ein]

Man oh man. That's an epiphany moment if ever there was one.

My project scope started on a Pro Trinket - AVR architecture - but I rapidly outgrew the capabilities of the device while I was working on the screen, so I switched to the Teensy 3.2, which had the same footprint but way more metaphorical horsepower, and just kept trucking. Until these responses, I had no idea the PROGMEM thing was moot, and trying to set up what I thought were working pgm_read_byte/pgm_read_word lines to pull data from that memory space.

Once I read these responses, I facepalmed mightily, stripped out all the PROGMEM-related stuff, replaced all my pgm_read stuff with straight variables, ran the program, and now everything seems to be working beautifully.

Thanks guys. I'd have posted the full code but it is getting long, split across many files, and I assumed it'd be too burdensome to review. These answers alone were enough.

 

[KurtE]

Glad it is working

 

[PaulStoffregen]

Indeed today PROGMEM does nothing in Teensy LC & 3.x. But that won't necessarily always be the case in future ARM-based Teensy.

With the upcoming iMX RT chips, we'll have a 2 Mbyte external flash chip and 512K internal RAM. On those boards, variables will default to RAM, even if const. But if they are also PROGMEM, then they'll be located only in the external flash chip.

The good news is they'll still be memory mapped, so you can read them normally. The macros like pgm_read_byte() won't be needed, unless you wish to make your code able to run on 8 bit AVR. Accessing the external flash chip is slower than internal RAM, at least for cache misses. But there is a large 32K cache in those new chips (able to run at the full 600 MHz), so repetitive access to external flash will usually be very fast.