Section type conflict and other memory error

Lesept · Wednesday at 6:03 AM

Hi
I have a Teensy 4.1 with 8MB PSRAM and 256 MB Flash.
I have a large code that uses large arrays declared in .h files. I try to manage the storage between RAM1 & 2, PSRAM and Flash, depending on their size.

For example, the file classifier_2_weight.h contains:

Code:

#include <stdint.h>
PROGMEM const float classifier_2_weight_output_0[128000] =
    {
        -0.0524069182574749, -0.03734472393989563, -0.04629778116941452, -0.0400727204978466, -0.02280816249549389,
        -0.02316550724208355, 0.002435319824144244, 0.0019113686867058277, -0.015797318890690804, -0.04310063645243645,
        -0.011440770700573921, 0.03244912996888161, -0.008675714023411274, -0.022800328209996223, -0.010684050619602203,
...
};

I use this array in a function called model_forward, in a forward.cpp file, which declares and includes some arrays as well:

Code:

// Memory block
#include "..\include\parameters\onnx__Conv_95.h"
#include "..\include\parameters\onnx__Conv_98.h"
#include "..\include\parameters\onnx__Conv_101.h"
#include "..\include\parameters\onnx__Conv_104.h"
#include "..\include\parameters\onnx__Conv_107.h"
#include "..\include\parameters\onnx__Conv_110.h"
#include "..\include\parameters\onnx__Conv_113.h"
#include "..\include\parameters\classifier_2_weight.h" // <--- Here
EXTMEM float input[131072];
DMAMEM float input2[32768];
EXTMEM float output[131072];
DMAMEM float weightsRAM2[73728];
EXTMEM float weightsPSRAM[921600];
PROGMEM float weightsFlash[921600];

When I compile the code I get this error message:

Code:

In file included from C:\Users\fa125436\Documents\Arduino\Aidge_Teensy_ResNet\src\dnn\src\forward.cpp:54:
C:\Users\fa125436\Documents\Arduino\Aidge_Teensy_ResNet\src\dnn\include\parameters\classifier_2_weight.h: In function 'void model_forward(float**)':
C:\Users\fa125436\Documents\Arduino\Aidge_Teensy_ResNet\src\dnn\include\parameters\classifier_2_weight.h:3:21: error: 'classifier_2_weight_output_0' causes a section type conflict with 'weightsFlash'
    3 | PROGMEM const float classifier_2_weight_output_0[128000] =
      |                     ^~~~~~~~~~~~~~~~~~~~~~~~~~~~
C:\Users\fa125436\Documents\Arduino\Aidge_Teensy_ResNet\src\dnn\src\forward.cpp:60:15: note: 'weightsFlash' was declared here
   60 | PROGMEM float weightsFlash[921600];
      |               ^~~~~~~~~~~~

exit status 1
Compilation error: 'classifier_2_weight_output_0' causes a section type conflict with 'weightsFlash'

These 2 arrays seem to have coliving problems. They interact here:

Code:

memcpy(weightsFlash, classifier_2_weight_output_0, 512000);

If I remove the PROGMEM statement in the .H file, keeping only:

Code:

#include <stdint.h>
const float classifier_2_weight_output_0[128000] =

then the compilation continues and I get another error:

Code:

c:/users/fa125436/appdata/local/arduino15/packages/teensy/tools/teensy-compile/11.3.1/arm/bin/../lib/gcc/arm-none-eabi/11.3.1/../../../../arm-none-eabi/bin/ld.exe: C:\Users\fa125436\AppData\Local\arduino\sketches\EE9AC2D007C02D446B7F6F7B98C17DC1/Aidge_Teensy_ResNet.ino.elf section `.data' will not fit in region `DTCM'
c:/users/fa125436/appdata/local/arduino15/packages/teensy/tools/teensy-compile/11.3.1/arm/bin/../lib/gcc/arm-none-eabi/11.3.1/../../../../arm-none-eabi/bin/ld.exe: C:\Users\fa125436\AppData\Local\arduino\sketches\EE9AC2D007C02D446B7F6F7B98C17DC1/Aidge_Teensy_ResNet.ino.elf section `.data' will not fit in region `FLASH'
c:/users/fa125436/appdata/local/arduino15/packages/teensy/tools/teensy-compile/11.3.1/arm/bin/../lib/gcc/arm-none-eabi/11.3.1/../../../../arm-none-eabi/bin/ld.exe: address 0x20cf3de0 of C:\Users\fa125436\AppData\Local\arduino\sketches\EE9AC2D007C02D446B7F6F7B98C17DC1/Aidge_Teensy_ResNet.ino.elf section `.bss' is not within region `DTCM'
c:/users/fa125436/appdata/local/arduino15/packages/teensy/tools/teensy-compile/11.3.1/arm/bin/../lib/gcc/arm-none-eabi/11.3.1/../../../../arm-none-eabi/bin/ld.exe: address 0x20cf3de0 of C:\Users\fa125436\AppData\Local\arduino\sketches\EE9AC2D007C02D446B7F6F7B98C17DC1/Aidge_Teensy_ResNet.ino.elf section `.bss' is not within region `DTCM'
c:/users/fa125436/appdata/local/arduino15/packages/teensy/tools/teensy-compile/11.3.1/arm/bin/../lib/gcc/arm-none-eabi/11.3.1/../../../../arm-none-eabi/bin/ld.exe: section .bss.dma VMA [20200000,2026b07f] overlaps section .data VMA [20000000,20cf1ebf]
c:/users/fa125436/appdata/local/arduino15/packages/teensy/tools/teensy-compile/11.3.1/arm/bin/../lib/gcc/arm-none-eabi/11.3.1/../../../../arm-none-eabi/bin/ld.exe: region `DTCM' overflowed by 13057504 bytes
c:/users/fa125436/appdata/local/arduino15/packages/teensy/tools/teensy-compile/11.3.1/arm/bin/../lib/gcc/arm-none-eabi/11.3.1/../../../../arm-none-eabi/bin/ld.exe: region `FLASH' overflowed by 9455616 bytes
collect2.exe: error: ld returned 1 exit status

exit status 1
Compilation error: exit status 1

This is more complex to understand.
I have read somewhere that if I don't declare arrays as PROGMEM, they are first stored in RAM1 then put in Flash. But if their size is larger than the RAM1 size, maybe this results in section `.bss' is not within region `DTCM' ?

Can anyone explain these error messages and help me fix them?
Thanks for your help.

PaulStoffregen · Wednesday at 6:18 AM

Lesept said:
PROGMEM float weightsFlash[921600];

You need const when using PROGMEM.

If your weightsFlash[] array is actually initialized by a different .cpp or .ino file, but you want to give access to it without creating a duplicate (and conflicting) copy, use "extern const float weightsFlash[921600];" (without PROGMEM). But it must be created somewhere, and in that place it must have PROGMEM and const.

Lesept · Wednesday at 6:51 AM

Thanks Paul.

PaulStoffregen said:
You need const when using PROGMEM.

Should I do the same for EXTMEM and DMAMEM?

Code:

EXTMEM const float input[131072];
DMAMEM const float input2[32768];
EXTMEM const float output[131072];
DMAMEM const float weightsRAM2[73728];
EXTMEM const float weightsPSRAM[921600];
PROGMEM const float weightsFlash[921600];

When I do this, I get a warning, such as:

Code:

In file included from c:\users\fa125436\appdata\local\arduino15\packages\teensy\tools\teensy-compile\11.3.1\arm\arm-none-eabi\include\c++\11.3.1\cstring:42,
                 from C:\Users\fa125436\Documents\Arduino\Aidge_Teensy_ResNet\src\dnn\src\forward.cpp:1:
c:\users\fa125436\appdata\local\arduino15\packages\teensy\tools\teensy-compile\11.3.1\arm\arm-none-eabi\include\string.h:31:18: note:   initializing argument 1 of 'void* memcpy(void*, const void*, size_t)'
   31 | void *   memcpy (void *__restrict, const void *__restrict, size_t);
      |                  ^~~~~~
C:\Users\fa125436\Documents\Arduino\Aidge_Teensy_ResNet\src\dnn\src\forward.cpp:123:24: warning: invalid conversion from 'const void*' to 'void*' [-fpermissive]
  123 |                 memcpy(weightsRAM2, onnx__Conv_95_output_0, 294912);
      |                        ^~~~~~~~~~~
      |                        |
      |                        const void*

Lesept · Wednesday at 7:45 AM

Actually, input and output cannot be const as they are changed during the execution.
I changed my code like this:

Code:

// Memory block
#include "..\include\parameters\onnx__Conv_95.h"
#include "..\include\parameters\onnx__Conv_98.h"
#include "..\include\parameters\onnx__Conv_101.h"
#include "..\include\parameters\onnx__Conv_104.h"
#include "..\include\parameters\onnx__Conv_107.h"
#include "..\include\parameters\onnx__Conv_110.h"
#include "..\include\parameters\onnx__Conv_113.h"
#include "..\include\parameters\classifier_2_weight.h"
#include "..\..\inputs.h"
EXTMEM float input[131072];               // PSRAM
DMAMEM const float input2[32768];         // RAM2
EXTMEM float output[131072];              // PSRAM
DMAMEM const float weightsRAM2[73728];    // RAM2
EXTMEM const float weightsPSRAM[921600];  // PSRAM
PROGMEM const float weightsFlash[921600]; // Flash
FASTRUN void model_forward(float **net_output)
{
    /*
    Input  stored in PSRAM
    Output stored in PSRAM
    */
    memcpy(input, inputs, 12288);
...

with the inputs.h file as follows:

Code:

#include <stdint.h>
PROGMEM const float inputs[3072] =
    {
        1.4285330772399902,
        2.087620258331299,
...
};

But I still get an error message:

Code:

C:\Users\fa125436\Documents\Arduino\Aidge_Teensy_ResNet\src\dnn\src\forward.cpp:60:20: error: 'weightsPSRAM' causes a section type conflict with 'input'
   60 | EXTMEM const float weightsPSRAM[921600];  // PSRAM
      |                    ^~~~~~~~~~~~
C:\Users\fa125436\Documents\Arduino\Aidge_Teensy_ResNet\src\dnn\src\forward.cpp:56:14: note: 'input' was declared here
   56 | EXTMEM float input[131072];                               // PSRAM
      |              ^~~~~

What is wrong?

Edit: maybe the sum of the arrays does not fit into my PSRAM...?

PaulStoffregen · Wednesday at 10:03 AM

Can you compose a single small program I can copy into Arduino IDE and click Verify to reproduce the error?

defragster · Wednesday at 12:04 PM

Lesept said:
C:\Users\fa125436\Documents\Arduino\Aidge_Teensy_ResNet\src\dnn\src\forward.cpp:60:20: error: 'weightsPSRAM' causes a section type conflict with 'input' 60 | EXTMEM const float weightsPSRAM[921600]; // PSRAM | ^~~~~~~~~~~~

EXTMEM won't work with const? It is not initialized and that is the area to be written from FLASH after startup correct?

Lesept said:
C:\Users\fa125436\Documents\Arduino\Aidge_Teensy_ResNet\src\dnn\src\forward.cpp:56:14: note: 'input' was declared here 56 | EXTMEM float input[131072]; // PSRAM

This may be a problem that 'input' is a name conflict defined elsewhere? try :: inputPSRAM or other

Lesept · Thursday at 12:58 AM

Thanks for your answers.

defragster said:
This may be a problem that 'input' is a name conflict defined elsewhere? try :: inputPSRAM or other

I have tried but no improvement.

PaulStoffregen said:
Can you compose a single small program I can copy into Arduino IDE and click Verify to reproduce the error?

Not easy, I'll try...

Lesept · Thursday at 2:53 AM

I did my best to reduce the size and complexity of the code while keeping the error. You can find a smaller program here.
Please tell me if you can donwload it and if it gives the same error I get:

Code:

C:\Users\fa125436\Documents\Arduino\Aidge_Teensy_test\src\dnn\src\forward.cpp:14:14: error: 'input' causes a section type conflict with 'weightsFlash'
   14 | EXTMEM float input[131072];                        // PSRAM
      |              ^~~~~
C:\Users\fa125436\Documents\Arduino\Aidge_Teensy_test\src\dnn\src\forward.cpp:19:20: note: 'weightsFlash' was declared here
   19 | EXTMEM const float weightsFlash[921600]; // PSRAM
      |                    ^~~~~~~~~~~~


exit status 1


Compilation error: 'input' causes a section type conflict with 'weightsFlash'

One additionnal information: if I comment out line 104 of the file convolution.hpp, I do not get the error.

PaulStoffregen · Thursday at 4:26 AM

Looked at your code. Took some time because you have many Windows-only style pathnames. Please use forward slash, so we non-Windows users can help you without this time-consuming step!

The fix for your problem is exactly as Defragster said. Just delete the "const" from the arrays in EXTMEM and DMAMEM.

In forward.cpp, change this:

Code:

#include "../include/parameters/onnx__Conv_101.h"
EXTMEM float input[131072];                        // PSRAM
DMAMEM const float input2[32768];          // RAM2
EXTMEM float output[131072];               // PSRAM
DMAMEM const float weightsRAM2[73728]; // RAM2
// EXTMEM const float weightsPSRAM[921600];  // PSRAM
EXTMEM const float weightsFlash[921600]; // PSRAM

to this:

Code:

#include "../include/parameters/onnx__Conv_101.h"
EXTMEM float input[131072];                        // PSRAM
DMAMEM float input2[32768];        // RAM2
EXTMEM float output[131072];               // PSRAM
DMAMEM float weightsRAM2[73728]; // RAM2
// EXTMEM const float weightsPSRAM[921600];  // PSRAM
EXTMEM float weightsFlash[921600]; // PSRAM

Lesept · Thursday at 6:11 AM

Sorry about that, and thanks for your help. It fixes the problem indeed, but there is still another error message:

Code:

c:/users/fa125436/appdata/local/arduino15/packages/teensy/tools/teensy-compile/11.3.1/arm/bin/../lib/gcc/arm-none-eabi/11.3.1/../../../../arm-none-eabi/bin/ld.exe: C:\Users\fa125436\AppData\Local\arduino\sketches\EE9AC2D007C02D446B7F6F7B98C17DC1/Aidge_Teensy_ResNet.ino.elf section `.text.progmem' will not fit in region `FLASH'
c:/users/fa125436/appdata/local/arduino15/packages/teensy/tools/teensy-compile/11.3.1/arm/bin/../lib/gcc/arm-none-eabi/11.3.1/../../../../arm-none-eabi/bin/ld.exe: region `FLASH' overflowed by 5769216 bytes
collect2.exe: error: ld returned 1 exit status

exit status 1
Compilation error: exit status 1

My board has 256MB Flash, how is it supposed to be overflowed? Should I declare this size somewhere?
Of course, this is with the entire program, not the smaller one I provided earlier.

PaulStoffregen · Thursday at 6:20 AM

Teensy 4.1 has 8 MB flash, not 256 MB.

Lesept · Thursday at 7:34 AM

Yes, but I bought it at Protosupplies:

Teensy 4.1 Fully Loaded For Prototyping System - ProtoSupplies

Fully assembled Teensy 4.1 with optional 8MB or 16MB PSRAM and 16MB, 128MB or 256MB Flash installed and tested for Prototyping System.

protosupplies.com

DEV-26-2G (8MB PSRAM/256MB Flash) has an 8MB (64M-bit) volatile PSRAM installed along with a 256MB (2G-bit) non-volatile NAND Flash memory as shown below for those that need an even larger amount of non-volatile storage such as for data logging, image files or audio files. The NAND Flash is the 8-pad WSON 8x6mm package.

The Flash chip will be a Winbond W25N02KVZEIR. The PSRAM will be the AP 6404L-3SQR or equivalent.

Edit:
After double thinking about what you said, I went back to the prosupplies site and read this:

Prototyping System for Teensy 4.1 - Working With Teensy 4.1 Memory - ProtoSupplies

Prototyping System for Teensy 4.1 example software tutorial covers the operation of the example software preloaded on Fully Stuffed systems.

protosupplies.com

Optional Flash Memory
The optional Flash memory is also QSPI like the Teensy 4.1 built-in Flash with similar performance. Both smaller NOR and larger NAND Flash chips are supported in sizes as large as 256MB/2Gb using the built-in LittleFS library.

The main thing to know about the optional Flash memory is that it cannot be used to simply increase the built-in 8MB program space. While the optional Flash memory cannot be used to directly increase program space on the Teensy 4.1, it can sometimes be used to stretch the program space by off-loading large data structures and the like from program space and putting it into the optional Flash chip.

If I understand well, I have to use LittleFS to address this memory. That's not what I was expecting at the beginning, so I need further reading...

defragster · Thursday at 9:25 AM

That OPTIONAL FLASH is on QSPI underside for ADDED storage of 256MB. Available for use with LittleFS.

The PJRC installed Primary Boot FLASH "PROGMEM" storage is a fixed 8MB on the T_4.1.

Lesept · Thursday at 12:57 PM

defragster said:
That OPTIONAL FLASH is on QSPI underside for ADDED storage of 256MB. Available for use with LittleFS.

Sorry, I don't understand. Does it mean I have 8 or 16MB PSRAM?
This piece of code:

Code:

// Check for PSRAM chip(s) installed
  uint8_t size = external_psram_size;
  Serial.printf("PSRAM Memory Size = %d Mbyte\n", size);

prints 8 MB.

KenHahn · Thursday at 2:24 PM

You have 8MB PSRAM and 256MB of NAND Flash that have been added to the bottom of the Teensy which sit on a QSPI bus. To have 16MB PSRAM, a 2nd 8MB PSRAM chip would be installed instead of the 256MB Flash chip. The Teensy 4.1 by itself does not have any PSRAM.

The main Teensy 4.1 Flash used for program memory is fixed at 8MB and is a separate small part that sits on the top side of the Teensy on a different QSPI bus.

As you have discovered, the 256MB Flash that has been added cannot be used as additional program space. It must be accessed via the LittleFS library. The usage of it basically parallels that of using something like an SD card.

Section type conflict and other memory error

Lesept

Active member

PaulStoffregen

Well-known member

Lesept

Active member

Lesept

Active member

PaulStoffregen

Well-known member

defragster

Senior Member+

Lesept

Active member

Lesept

Active member

PaulStoffregen

Well-known member

Lesept

Active member

PaulStoffregen

Well-known member

Lesept

Active member

Teensy 4.1 Fully Loaded For Prototyping System - ProtoSupplies

Prototyping System for Teensy 4.1 - Working With Teensy 4.1 Memory - ProtoSupplies

Optional Flash Memory

defragster

Senior Member+

Lesept

Active member

KenHahn

Well-known member

Section type conflict and other memory error

Active member

Well-known member

Active member

Active member

Well-known member

Senior Member+

Active member

Active member

Well-known member

Active member

Well-known member

Active member

Optional Flash Memory​

Senior Member+

Active member

Well-known member

Optional Flash Memory