issues on program with large dataset

Status
Not open for further replies.

cyrille

Well-known member
Hello,

I am working on a program that has around 600KB worth of const data.
These are variables defined as static VarType const var[]={...}; with a whole bunch of const data in them.

When compiling, I get linker errors, basically telling me that there is not enough space...
However, they should be enough place in the flash (which is where I wold have expected my code/const to reside)...

ld.exe: C:\Users\cydeb\AppData\Local\Temp\arduino_build_180041/arduViseur.ino.elf section `.data' will not fit in region `DTCM'
ld.exe: address 0x2009b2c0 of C:\Users\cydeb\AppData\Local\Temp\arduino_build_180041/arduViseur.ino.elf section `.bss' is not within region `DTCM'
ld.exe: address 0x2009b2c0 of C:\Users\cydeb\AppData\Local\Temp\arduino_build_180041/arduViseur.ino.elf section `.bss' is not within region `DTCM'
ld.exe: region `DTCM' overflowed by 111296 bytes

I have tried defining my variables with PROGMEM as in the doc, but PROGMEM seemed to be undefined (missing a #include in my .h file?)



Is there a way to see the linker file to gain more info as to what is happening?

Another question:
The following code yeilds a warning
struct Str { uint16_t ra; int16_t dec; uint16_t tpe: 3, mag:4, szeM:7, szee:2; uint16_t dstM:7, dste:3, strindex: 6; };
Str ska ={ 11417u , 5636u , 11u , 7u , 63u , 3u , 5u , 0u , 1u };
stardisp.cpp:728: warning: large integer implicitly truncated to unsigned type
Str ska ={ 11417u , 5636u , 11u , 7u , 63u , 3u , 5u , 0u , 1u };
But all my ints have a u sufix... Does anyone have a clue as to why I am getting this warning (in fact, I get around 100K of them:)...

Thanks,
Cyrille
 
Hello,

Well, I kind of knew all of that...
What I do not seem to know is the syntax.

I have tried
static TStar const stars[] PROGMEM ={...};
and
static PROGMEM TStar const stars[] ={...};

but both generated errors

stars.h:3: error: expected initializer before 'PROGMEM'
static TStar const stars[] PROGMEM ={

stars.h:3: error: 'PROGMEM' does not name a type
static PROGMEM TStar const stars[] ={

What is the syntax? or do I need some type of #include at the top (the data is in a .h file included from a .cpp file)...

Cyrille
 
Also I have a thread: https://forum.pjrc.com/threads/57326-T4-0-Memory-trying-to-make-sense-of-the-different-regions
That tries to describe the different memory regions.

In there I talk about a program that FrankB started that I made my own changes to, that I add into my builds which give me additional information.
Example from a run building an example in FlexIO (fixed a bug yesterday)
Code:
cmd /c "C:\\arduino-1.8.13\\hardware\\teensy\\..\\tools\\arm\\bin\\arm-none-eabi-gcc-nm -n C:\\Users\\kurte\\AppData\\Local\\Temp\\arduino_build_353997\\FlexSerial_Both.ino.elf | D:\\GITHUB\\imxrt-size\\Debug\\imxrt-size.exe"

FlexRAM section ITCM+DTCM = 512 KB
    Config : aaaaaaab (DDDDDDDDDDDDDDDI)
    ITCM :  27928 B	(85.23% of   32 KB)
    DTCM :  12992 B	( 2.64% of  480 KB)
    Available for Stack: 478528
OCRAM: 512KB
    DMAMEM:  12384 B	( 2.36% of  512 KB)
    Available for Heap: 511904 B	(97.64% of  512 KB)
Flash:  40280 B	( 0.50% of 7936 KB)
Using library FlexIO_t4 in folder: C:\Users\kurte\Documents\Arduino\libraries\FlexIO_t4 (legacy)
"C:\\arduino-1.8.13\\hardware\\teensy/../tools/arm/bin/arm-none-eabi-size" -A "C:\\Users\\kurte\\AppData\\Local\\Temp\\arduino_build_353997/FlexSerial_Both.ino.elf"
Sketch uses 40272 bytes (0%) of program storage space. Maximum is 8126464 bytes.
Global variables use 45748 bytes (8%) of dynamic memory, leaving 478540 bytes for local variables. Maximum is 524288 bytes.

Paul has also added more description to the product page describing the different memory regions... https://www.pjrc.com/store/teensy40.html

So for example if you just do something like: static VarType const var[]={...};

By default they will still be copied at startup into the DTCM (faster) memory... but if you add the PROGMEM to it they will be left up in flash:

Code:
static VarType const var[] PROGMEM ={...}
;

Note: again by default your program code is also copied into this 512kb region of memory. Where it takes 32kb slots up a time. So if your code is 40K it will take 2slots or 64KB away from Data space.
You can also mark code to stay up in flash

@PaulStoffregen - I think the page: https://www.pjrc.com/store/teensy40.html
Needs to be updated:
Code:
PROGMEM Code - Functions defined with "PROGMEM" executed directly from Flash. If the Cortex-M7 cache is not already holding a copy of the function, a delay results while the Flash memory is read into the M7's cache. PROGMEM should be used on startup code and other functions where speed is not important.
should be changed to:
Code:
PROGMEM Code - Functions defined with "FLASHMEM" executed directly from Flash. If the Cortex-M7 cache is not already holding a copy of the function, a delay results while the Flash memory is read into the M7's cache. PROGMEM should be used on startup code and other functions where speed is not important.
 
Hello,

I understand the memory structure and all of that...

My problem is one of syntax or I should say compiler error...
my code:
static TStar const stars[] PROGMEM ={...};
yields this error:
stars.h:3: error: expected initializer before 'PROGMEM'


Since my data here is in a .h file which is included in a .cpp file in my project, I am assuming that some needed #include are missing... But I have no clue/idea which one? Can you help me here?

Cyrille
 
Does your .cpp file include <Arduino.h>

Or Alternately:
#include "avr/pgmspace.h" // for PROGMEM, DMAMEM, FASTRUN
 
Hello,

Exactly the info I was missing!!! THANK YOU!!!

Another question, is there a ARDUINO define or anything like that that would allow me to know that I am being compiled for arduino as my file is also compiled using visual C++ and requires slightly different includes?

Cyrille
 
You are welcome,

I don't know about your Visual C++ setup, so can not directly give answer. As some setups with Visual Micro allow you to build with the Arduino build... Or others may not...

But what I would look at is, what defines do you get when Arduino builds...
Note: I always turn on Verbose compile options, in Arduino (preferences).

So for example when I compiled a sketch this morning for T4.1, the command line to compile the sketch was:
Code:
"C:\\arduino-1.8.13\\hardware\\teensy/../tools/arm/bin/arm-none-eabi-g++" -c -O2 -g -Wall -ffunction-sections -fdata-sections -nostdlib -MMD -std=gnu++14 -fno-exceptions -fpermissive -fno-rtti -fno-threadsafe-statics -felide-constructors -Wno-error=narrowing -mthumb -mcpu=cortex-m7 -mfloat-abi=hard -mfpu=fpv5-d16 -D__IMXRT1062__ -DTEENSYDUINO=153 -DARDUINO=10813 -DARDUINO_TEENSY41 -DF_CPU=600000000 -DUSB_SERIAL -DLAYOUT_US_ENGLISH "-IC:\\Users\\kurte\\AppData\\Local\\Temp\\arduino_build_955672/pch" "-IC:\\arduino-1.8.13\\hardware\\teensy\\avr\\cores\\teensy4" "C:\\Users\\kurte\\AppData\\Local\\Temp\\arduino_build_955672\\sketch\\SerialHalfDuplexEcho_SERIAL_HALF_DUPLEX.ino.cpp" -o "C:\\Users\\kurte\\AppData\\Local\\Temp\\arduino_build_955672\\sketch\\SerialHalfDuplexEcho_SERIAL_HALF_DUPLEX.ino.cpp.o"
And here you see several defines that your code could look at like:
TEENSYDUINO, or ARDUINO_TEENSY41 or ARDUINO or ...

EDIT: That is anything in command line that has -D<symbol>..
So not sure if any of these would work for you.
 
Hello,

Yep, perfect, thanks... I had not seen the options in preferences...

I have one last off topic question:

Cyrille
 
hello,

Sorry about that.. I erased the previous question as I found the answer (and feel dumb about it!)...

Bus since there was an answer, I will put it back here:

The problem was a "warning: large integer implicitly truncated to unsigned type " on this lines
struct Str { uint16_t ra; int16_t dec; uint16_t tpe: 3, mag:4, szeM:7, szee:2; uint16_t dstM:7, dste:3, strindex: 6; };
Str ska ={ 11417u , 5636u , 11u , 7u , 63u , 3u , 5u , 0u , 1u };

The error is NOT on the 5636u, but on the 11u, which needs to go on a 3 bit number... Took me a while to find it...

Cyrille
 
Status
Not open for further replies.
Back
Top