Forum Rule: Always post complete source code & details to reproduce any issue!
Results 1 to 11 of 11

Thread: issues on program with large dataset

  1. #1

    issues on program with large dataset

    Hello,

    I am working on a program that has around 600KB worth of const data.
    These are variables defined as static VarType const var[]={...}; with a whole bunch of const data in them.

    When compiling, I get linker errors, basically telling me that there is not enough space...
    However, they should be enough place in the flash (which is where I wold have expected my code/const to reside)...

    ld.exe: C:\Users\cydeb\AppData\Local\Temp\arduino_build_18 0041/arduViseur.ino.elf section `.data' will not fit in region `DTCM'
    ld.exe: address 0x2009b2c0 of C:\Users\cydeb\AppData\Local\Temp\arduino_build_18 0041/arduViseur.ino.elf section `.bss' is not within region `DTCM'
    ld.exe: address 0x2009b2c0 of C:\Users\cydeb\AppData\Local\Temp\arduino_build_18 0041/arduViseur.ino.elf section `.bss' is not within region `DTCM'
    ld.exe: region `DTCM' overflowed by 111296 bytes

    I have tried defining my variables with PROGMEM as in the doc, but PROGMEM seemed to be undefined (missing a #include in my .h file?)



    Is there a way to see the linker file to gain more info as to what is happening?

    Another question:
    The following code yeilds a warning
    struct Str { uint16_t ra; int16_t dec; uint16_t tpe: 3, mag:4, szeM:7, szee:2; uint16_t dstM:7, dste:3, strindex: 6; };
    Str ska ={ 11417u , 5636u , 11u , 7u , 63u , 3u , 5u , 0u , 1u };
    stardisp.cpp:728: warning: large integer implicitly truncated to unsigned type
    Str ska ={ 11417u , 5636u , 11u , 7u , 63u , 3u , 5u , 0u , 1u };
    But all my ints have a u sufix... Does anyone have a clue as to why I am getting this warning (in fact, I get around 100K of them:-)...

    Thanks,
    Cyrille

  2. #2
    Senior Member
    Join Date
    Jul 2020
    Posts
    460
    This thread should explain things: https://forum.pjrc.com/threads/57269...y-4-0-s-memory

  3. #3
    Hello,

    Well, I kind of knew all of that...
    What I do not seem to know is the syntax.

    I have tried
    static TStar const stars[] PROGMEM ={...};
    and
    static PROGMEM TStar const stars[] ={...};

    but both generated errors

    stars.h:3: error: expected initializer before 'PROGMEM'
    static TStar const stars[] PROGMEM ={

    stars.h:3: error: 'PROGMEM' does not name a type
    static PROGMEM TStar const stars[] ={

    What is the syntax? or do I need some type of #include at the top (the data is in a .h file included from a .cpp file)...

    Cyrille

  4. #4
    Senior Member+ KurtE's Avatar
    Join Date
    Jan 2014
    Posts
    7,871
    Also I have a thread: https://forum.pjrc.com/threads/57326...ferent-regions
    That tries to describe the different memory regions.

    In there I talk about a program that FrankB started that I made my own changes to, that I add into my builds which give me additional information.
    Example from a run building an example in FlexIO (fixed a bug yesterday)
    Code:
    cmd /c "C:\\arduino-1.8.13\\hardware\\teensy\\..\\tools\\arm\\bin\\arm-none-eabi-gcc-nm -n C:\\Users\\kurte\\AppData\\Local\\Temp\\arduino_build_353997\\FlexSerial_Both.ino.elf | D:\\GITHUB\\imxrt-size\\Debug\\imxrt-size.exe"
    
    FlexRAM section ITCM+DTCM = 512 KB
        Config : aaaaaaab (DDDDDDDDDDDDDDDI)
        ITCM :  27928 B	(85.23% of   32 KB)
        DTCM :  12992 B	( 2.64% of  480 KB)
        Available for Stack: 478528
    OCRAM: 512KB
        DMAMEM:  12384 B	( 2.36% of  512 KB)
        Available for Heap: 511904 B	(97.64% of  512 KB)
    Flash:  40280 B	( 0.50% of 7936 KB)
    Using library FlexIO_t4 in folder: C:\Users\kurte\Documents\Arduino\libraries\FlexIO_t4 (legacy)
    "C:\\arduino-1.8.13\\hardware\\teensy/../tools/arm/bin/arm-none-eabi-size" -A "C:\\Users\\kurte\\AppData\\Local\\Temp\\arduino_build_353997/FlexSerial_Both.ino.elf"
    Sketch uses 40272 bytes (0%) of program storage space. Maximum is 8126464 bytes.
    Global variables use 45748 bytes (8%) of dynamic memory, leaving 478540 bytes for local variables. Maximum is 524288 bytes.
    Paul has also added more description to the product page describing the different memory regions... https://www.pjrc.com/store/teensy40.html

    So for example if you just do something like: static VarType const var[]={...};

    By default they will still be copied at startup into the DTCM (faster) memory... but if you add the PROGMEM to it they will be left up in flash:

    Code:
    static VarType const var[] PROGMEM ={...}
    ;

    Note: again by default your program code is also copied into this 512kb region of memory. Where it takes 32kb slots up a time. So if your code is 40K it will take 2slots or 64KB away from Data space.
    You can also mark code to stay up in flash

    @PaulStoffregen - I think the page: https://www.pjrc.com/store/teensy40.html
    Needs to be updated:
    Code:
    PROGMEM Code - Functions defined with "PROGMEM" executed directly from Flash. If the Cortex-M7 cache is not already holding a copy of the function, a delay results while the Flash memory is read into the M7's cache. PROGMEM should be used on startup code and other functions where speed is not important.
    should be changed to:
    Code:
    PROGMEM Code - Functions defined with "FLASHMEM" executed directly from Flash. If the Cortex-M7 cache is not already holding a copy of the function, a delay results while the Flash memory is read into the M7's cache. PROGMEM should be used on startup code and other functions where speed is not important.

  5. #5
    Hello,

    I understand the memory structure and all of that...

    My problem is one of syntax or I should say compiler error...
    my code:
    static TStar const stars[] PROGMEM ={...};
    yields this error:
    stars.h:3: error: expected initializer before 'PROGMEM'


    Since my data here is in a .h file which is included in a .cpp file in my project, I am assuming that some needed #include are missing... But I have no clue/idea which one? Can you help me here?

    Cyrille

  6. #6
    Senior Member+ KurtE's Avatar
    Join Date
    Jan 2014
    Posts
    7,871
    Does your .cpp file include <Arduino.h>

    Or Alternately:
    #include "avr/pgmspace.h" // for PROGMEM, DMAMEM, FASTRUN

  7. #7
    Hello,

    Exactly the info I was missing!!! THANK YOU!!!

    Another question, is there a ARDUINO define or anything like that that would allow me to know that I am being compiled for arduino as my file is also compiled using visual C++ and requires slightly different includes?

    Cyrille

  8. #8
    Senior Member+ KurtE's Avatar
    Join Date
    Jan 2014
    Posts
    7,871
    You are welcome,

    I don't know about your Visual C++ setup, so can not directly give answer. As some setups with Visual Micro allow you to build with the Arduino build... Or others may not...

    But what I would look at is, what defines do you get when Arduino builds...
    Note: I always turn on Verbose compile options, in Arduino (preferences).

    So for example when I compiled a sketch this morning for T4.1, the command line to compile the sketch was:
    Code:
    "C:\\arduino-1.8.13\\hardware\\teensy/../tools/arm/bin/arm-none-eabi-g++" -c -O2 -g -Wall -ffunction-sections -fdata-sections -nostdlib -MMD -std=gnu++14 -fno-exceptions -fpermissive -fno-rtti -fno-threadsafe-statics -felide-constructors -Wno-error=narrowing -mthumb -mcpu=cortex-m7 -mfloat-abi=hard -mfpu=fpv5-d16 -D__IMXRT1062__ -DTEENSYDUINO=153 -DARDUINO=10813 -DARDUINO_TEENSY41 -DF_CPU=600000000 -DUSB_SERIAL -DLAYOUT_US_ENGLISH "-IC:\\Users\\kurte\\AppData\\Local\\Temp\\arduino_build_955672/pch" "-IC:\\arduino-1.8.13\\hardware\\teensy\\avr\\cores\\teensy4" "C:\\Users\\kurte\\AppData\\Local\\Temp\\arduino_build_955672\\sketch\\SerialHalfDuplexEcho_SERIAL_HALF_DUPLEX.ino.cpp" -o "C:\\Users\\kurte\\AppData\\Local\\Temp\\arduino_build_955672\\sketch\\SerialHalfDuplexEcho_SERIAL_HALF_DUPLEX.ino.cpp.o"
    And here you see several defines that your code could look at like:
    TEENSYDUINO, or ARDUINO_TEENSY41 or ARDUINO or ...

    EDIT: That is anything in command line that has -D<symbol>..
    So not sure if any of these would work for you.

  9. #9
    Hello,

    Yep, perfect, thanks... I had not seen the options in preferences...

    I have one last off topic question:

    Cyrille

  10. #10
    Member
    Join Date
    Aug 2018
    Location
    Brisbane, Australia
    Posts
    30
    You are getting the warning because its converting 5636u to int16_t

  11. #11
    hello,

    Sorry about that.. I erased the previous question as I found the answer (and feel dumb about it!)...

    Bus since there was an answer, I will put it back here:

    The problem was a "warning: large integer implicitly truncated to unsigned type " on this lines
    struct Str { uint16_t ra; int16_t dec; uint16_t tpe: 3, mag:4, szeM:7, szee:2; uint16_t dstM:7, dste:3, strindex: 6; };
    Str ska ={ 11417u , 5636u , 11u , 7u , 63u , 3u , 5u , 0u , 1u };

    The error is NOT on the 5636u, but on the 11u, which needs to go on a 3 bit number... Took me a while to find it...

    Cyrille

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •