Teensyduino 1.55 Beta #1

[KurtE]

Again I could be completely out to lunch

But I am wondering if this code in usb.c may have something to do with it:

FLASHMEM void _reboot_Teensyduino_(void)
{
	if (!(HW_OCOTP_CFG5 & 0x02)) {
		asm("bkpt #251"); // run bootloader
	} else {
		__disable_irq(); // secure mode NXP ROM reboot
		USB1_USBCMD = 0;
		IOMUXC_GPR_GPR16 = 0x00200003;
		// TODO: wipe all RAM for security
		[COLOR="#FF0000"]__asm__ volatile("mov sp, %0" : : "r" (0x20201000) : );[/COLOR]
		__asm__ volatile("dsb":::"memory");
		volatile uint32_t * const p = (uint32_t *)0x20208000;
		*p = 0xEB120000;
		((void (*)(volatile void *))(*(uint32_t *)(*(uint32_t *)0x0020001C + 8)))(p);
	}
	__builtin_unreachable();
}

Again maybe totally off, but...
It looks like it is setting up a pointer to allow for a 4K stack...

 

[KurtE]

Again for what it is worth, I added two buffers... And second one was not mucked with...

#include <MemoryHexDump.h>  // https://github.com/KurtE/MemoryHexDump
#include "debug/printf.h"


DMAMEM char buf[ 256 * 1024 ];  // DMAMEM Uses RAM2 :: Lost on any restart
DMAMEM char buf2[224*1024]; 

void setup() {
  while (!Serial) ; // needs serial
  Serial.begin(115200);

  Serial.println("\n\nSimple DMAMEM startup memory test");
  MemoryHexDump(Serial, buf , sizeof(buf), true, "*** At Startup ***\n", 100);

  memset(buf, 0x2a, sizeof(buf));
  MemoryHexDump(Serial, buf , sizeof(buf), true, "\n*** After memset ***\n", 100);
  arm_dcache_flush(buf , sizeof(buf));

  // Try second buffer higher up...
  Serial.println("\n\nTry second buffer");
  MemoryHexDump(Serial, buf2, sizeof(buf2), true, "*** At Startup Buf2 ***\n", 100);

  memset(buf2, 0xc5, sizeof(buf2));
  MemoryHexDump(Serial, buf2 , sizeof(buf2), true, "\n*** After memset Buf2 ***\n", 100);
  arm_dcache_flush(buf2 , sizeof(buf2));


  Serial.println("Press any Key to reboot the Teensy");
  while (Serial.read() == -1) ;
  Serial.println("Now Resetting");
    for (uint32_t i = 0; i < sizeof(buf); i++) {
      if (buf[i] != 0x2a) {
        Serial.print("Found something changed before reset: ");
        Serial.print(i, DEC);
        break;
      }
    }
  
  delay(100);

  SCB_AIRCR = 0x05FA0004;

}

void loop() {
  // put your main code here, to run repeatedly:

}
extern "C" {
  void startup_early_hook(void) {
    printf("*** startup_early_hook ***\n");

    // See how many bytes are our 2a
    uint32_t index_first_none_2a = (uint32_t) - 1;
    uint32_t count_none_2a = 0;
    for (uint32_t i = 0; i < sizeof(buf); i++) {
      if (buf[i] != 0x2a) {
        if (count_none_2a == 0) index_first_none_2a = i;
        count_none_2a++;
      }
    }
    printf("buff size: %u first none 2a: %u, count: %u\n", sizeof(buf), index_first_none_2a, count_none_2a);

  }
}

Press any Key to reboot the Teensy
Now Resetting


Simple DMAMEM startup memory test
*** At Startup ***
20200000 - 2A 2A 2A 2A 2A 2A 2A 2A  2A 2A 2A 2A 2A 2A 2A 2A  : ******** ********
...	 151 duplicate line(s) removed.
20200980 - 2A 2A 2A 2A 2A 2A 2A 2A  2A 2A 2A 2A 2A 2A 2A 2A  : ******** ********
20200990 - 67 E6 09 6A 85 AE 67 BB  72 F3 6E 3C 3A F5 4F A5  : g..j..g. r.n<:.O.
202009A0 - 7F 52 0E 51 8C 68 05 9B  48 A8 AF 05 C0 5F 51 36  : .R.Q.h.. H...._Q6
202009B0 - 8C 85 BC B7 96 3F A9 22  C3 78 CB DF 58 7E F6 5A  : .....?." .x..X~.Z
202009C0 - 5B 5F 95 95 94 D7 6D 18  58 7E F6 5A F6 23 19 D0  : [_....m. X~.Z.#..
202009D0 - BC 4B 3B BD EC 63 AC B0  75 E0 5B E2 45 54 9B 91  : .K;..c.. u.[.ET..
202009E0 - C0 68 57 25 31 1D B0 FD  96 EB D0 19 0D F7 86 48  : .hW%1... .......H
202009F0 - 31 04 09 01 43 20 04 22  84 99 71 C0 DE 50 F9 13  : 1...C ." ..q..P..
20200A00 - 94 FA 30 5A 9B FD 38 59  9F F5 F2 A9 C9 16 AA 28  : ..0Z..8Y .......(
20200A10 - 33 9B 46 C6 80 91 04 A5  00 00 00 00 00 00 00 00  : 3.F..... ........
20200A20 - 00 00 00 00 00 00 00 00  58 03 00 00 0C 05 FD 31  : ........ X......1
20200A30 - 7A DA E8 31 E2 E1 1F BC  C7 1A 5F 21 7D C7 32 3E  : z..1.... .._!}.2>
20200A40 - DC D5 57 CB 85 06 A2 2B  BD 80 4D FF 28 79 B4 77  : ..W....+ ..M.(y.w
20200A50 - 96 C2 67 24 79 84 10 7A  CC 96 43 57 47 EC 04 1E  : ..g$y..z ..CWG...
20200A60 - 31 78 4C E5 23 6F D0 9B  65 70 FB 89 AC 58 78 55  : 1xL.#o.. ep...XxU
20200A70 - 09 D7 98 AC ED 0B 93 87  E5 3F 18 FD 6C 3F B5 5D  : ........ .?..l?.]
20200A80 - 54 9A DB 0D D2 93 22 F2  53 C5 7D 53 F0 BE 5E 18  : T.....". S.}S..^.
20200A90 - 26 5A BA E2 0D BD BE 77  A5 2D 55 D2 C7 EF 8A 16  : &Z.....w .-U.....
20200AA0 - 00 00 00 00 1A AE 20 2C  D8 0A 20 20 19 74 20 00  : ...... , ..  .t .
20200AB0 - 3D F6 B3 4D 00 10 20 20  7F 1B 20 20 1A AE 20 2C  : =..M..   ..  .. ,
20200AC0 - F0 0A 20 20 20 80 0F 40  09 00 00 00 20 0B 20 20  : ..   ..@ .... .  
20200AD0 - 00 00 00 00 D3 77 20 00  80 1B 20 20 1A AE 20 2C  : .....w . ..  .. ,
20200AE0 - 00 00 00 00 20 80 0F 40  F0 C7 21 00 1D 7F 20 00  : .... ..@ ..!... .
20200AF0 - 01 00 00 00 1A AE 20 2C  28 0B 20 20 19 74 20 00  : ...... , (.  .t .
20200B00 - 20 80 0F 40 80 1B 20 20  3F 37 20 20 1A AE 20 2C  :  ..@..   ?7  .. ,
20200B10 - CC 0B 20 20 20 80 0F 40  01 00 00 00 1A AE 20 2C  : ..   ..@ ...... ,
20200B20 - 50 0B 20 20 19 74 20 00  40 37 20 20 80 1B 20 20  : P.  .t . @7  ..  
20200B30 - 3F 37 20 20 1A AE 20 2C  CC 0B 20 20 20 80 0F 40  : ?7  .. , ..   ..@
20200B40 - F8 1C 20 20 CC 0B 20 20  C8 0C 20 20 73 74 20 00  : ..  ..   ..  st .
20200B50 - 40 37 20 20 1A AE 20 2C  01 00 00 00 1A AE 20 2C  : @7  .. , ...... ,
20200B60 - 90 0B 20 20 19 74 20 00  98 0B 20 20 80 1B 20 20  : ..  .t . ..  ..  
20200B70 - 01 00 00 00 1A AE 20 2C  A8 0B 20 20 19 74 20 00  : ...... , ..  .t .
20200B80 - 01 00 00 00 1A AE 20 2C  B8 0B 20 20 19 74 20 00  : ...... , ..  .t .
20200B90 - 40 37 20 20 80 1B 20 20  3F 37 20 20 1A AE 20 2C  : @7  ..   ?7  .. ,
20200BA0 - 00 00 00 00 20 80 0F 40  10 1C 20 20 00 00 00 00  : .... ..@ ..  ....
20200BB0 - 08 53 00 60 73 74 20 00  40 37 20 20 1A AE 20 2C  : .S.`st . @7  .. ,
20200BC0 - 20 80 0F 40 20 80 0F 40  00 00 00 00 2D 77 20 00  :  ..@ ..@ ....-w .
20200BD0 - 00 00 00 00 33 00 00 2C  00 00 00 00 00 00 00 00  : ....3.., ........
20200BE0 - 00 00 00 00 1A AE 20 2C  00 00 00 00 00 00 00 00  : ...... , ........
20200BF0 - 0C 54 00 60 CD 9B 20 00  C8 0C 20 20 80 78 C3 84  : .T.`.. . ..  .x..
20200C00 - 83 93 3F F1 8E 52 00 60  0C 54 00 60 A4 10 20 20  : ..?..R.` .T.`..  
20200C10 - F9 52 00 60 00 00 00 00  78 0D 20 20 3E 96 DF 9A  : .R.`.... x.  >...
20200C20 - BC BD 1E 89 00 00 00 00  01 00 00 00 1A F0 F0 F0  : ........ ........
20200C30 - F0 F0 F0 F0 A2 01 00 00  69 00 00 00 2F 00 00 00  : ........ i.../...
20200C40 - 22 00 00 00 22 00 00 00  20 00 00 00 20 00 00 00  : "..."...  ... ...
20200C50 - 6B 00 00 00 00 00 00 00  00 00 00 00 33 00 00 00  : k....... ....3...
20200C60 - 00 00 00 00 1A AE 20 2C  98 0C 20 20 19 74 20 00  : ...... , ..  .t .
20200C70 - 00 00 00 00 00 10 20 20  7F 1B 20 20 1A AE 20 2C  : ......   ..  .. ,
20200C80 - 00 00 00 00 20 80 0F 40  0C 00 00 00 00 00 00 00  : .... ..@ ........
20200C90 - 00 00 00 00 D3 77 20 00  80 1B 20 20 1A AE 20 2C  : .....w . ..  .. ,
20200CA0 - 1A AE 20 2C 20 80 0F 40  20 80 0F 40 51 7D 20 00  : .. , ..@  ..@Q} .
20200CB0 - 00 10 20 20 1A AE 20 2C  10 00 00 00 7D 75 20 00  : ..  .. , ....}u .
20200CC0 - 80 1B 20 20 33 00 00 20  01 00 00 00 1A AE 20 2C  : ..  3..  ...... ,
20200CD0 - 00 0D 20 20 19 74 20 00  0C 00 00 00 80 1B 20 20  : ..  .t . ......  
20200CE0 - 3F 37 20 20 1A AE 20 2C  08 00 00 00 20 80 0F 40  : ?7  .. , .... ..@
20200CF0 - B8 1B 20 20 C0 1B 20 20  20 80 0F 40 73 74 20 00  : ..  ..    ..@st .
20200D00 - 40 37 20 20 1A AE 20 2C  20 80 0F 40 C0 1B 20 20  : @7  .. ,  ..@..  
20200D10 - C0 1B 20 20 2D 77 20 00  B8 1B 20 20 1A AE 20 2C  : ..  -w . ..  .. ,
20200D20 - F0 00 00 00 D7 6E 20 00  78 0D 20 20 1A AE 20 2C  : .....n . x.  .. ,
20200D30 - 30 4C 00 60 60 00 10 00  00 00 3C 00 33 00 00 2C  : 0L.``... ..<.3..,
20200D40 - 00 00 00 00 1A AE 20 2C  78 0D 20 20 19 74 20 00  : ...... , x.  .t .
20200D50 - 00 00 00 00 1A AE 20 2C  88 0D 20 20 19 74 20 00  : ...... , ..  .t .
20200D60 - 00 00 00 00 00 10 20 20  7F 1B 20 20 1A AE 20 2C  : ......   ..  .. ,
20200D70 - 00 00 00 00 1A AE 20 2C  A8 0D 20 20 19 74 20 00  : ...... , ..  .t .
20200D80 - B4 1B 20 20 00 10 20 20  7F 1B 20 20 1A AE 20 2C  : ..  ..   ..  .. ,
20200D90 - 1A AE 20 2C 20 80 0F 40  09 00 00 00 F0 0D 20 20  : .. , ..@ ......  
20200DA0 - 0C 00 00 00 D3 77 20 00  00 00 00 00 1A AE 20 2C  : .....w . ...... ,
20200DB0 - E0 0D 20 20 19 74 20 00  00 00 00 00 1A AE 20 2C  : ..  .t . ...... ,
20200DC0 - F0 0D 20 20 19 74 20 00  20 80 0F 40 00 10 20 20  : ..  .t .  ..@..  
20200DD0 - 7F 1B 20 20 1A AE 20 2C  A0 00 00 00 20 80 0F 40  : ..  .. , .... ..@
20200DE0 - DB 00 00 00 00 00 00 00  00 00 00 00 D3 77 20 00  : ........ .....w .
20200DF0 - 80 1B 20 20 07 00 00 00  CC 00 00 00 03 00 00 00  : ..  .... ........
20200E00 - 00 00 00 00 1A AE 20 2C  38 0E 20 20 19 74 20 00  : ...... , 8.  .t .
20200E10 - 00 AE 20 2C 00 10 20 20  7F 1B 20 20 1A AE 20 2C  : .. ,..   ..  .. ,
20200E20 - 4B 0E 20 20 20 80 0F 40  DB 00 00 00 33 00 00 00  : K.   ..@ ....3...
20200E30 - 00 00 00 00 1A AE 20 2C  68 0E 20 20 19 74 20 00  : ...... , h.  .t .
20200E40 - 1A AE 20 2C 00 10 20 20  7F 1B 20 20 1A AE 20 2C  : .. ,..   ..  .. ,
20200E50 - 00 00 00 00 20 80 0F 40  DB 00 00 00 33 00 00 00  : .... ..@ ....3...
20200E60 - 00 00 00 00 D3 77 20 00  80 1B 20 20 1A AE 20 2C  : .....w . ..  .. ,
20200E70 - 1A AE 20 2C 20 80 0F 40  00 00 00 00 2D 7C 20 00  : .. , ..@ ....-| .
20200E80 - 00 00 00 00 1A AE 20 2C  00 00 00 00 68 0F 20 20  : ...... , ....h.  
20200E90 - E8 0E 20 20 20 80 0F 40  C4 0E 20 20 8F 7D 20 00  : ..   ..@ ..  .} .
20200EA0 - C4 0E 20 20 1A AE 20 2C  00 00 00 00 20 80 0F 40  : ..  .. , .... ..@
20200EB0 - 1A AE 20 2C F0 00 00 00  00 03 20 00 A5 23 20 00  : .. ,.... .. ..# .
20200EC0 - 00 00 00 00 33 00 00 20  00 00 00 00 00 00 00 00  : ....3..  ........
20200ED0 - 00 00 00 00 1A AE 20 2C  08 0F 20 20 1A AE 20 2C  : ...... , ..  .. ,
20200EE0 - 71 23 20 00 01 09 21 00  7F 1B 20 20 1A AE 20 2C  : q# ...!. ..  .. ,
20200EF0 - 1A AE 20 2C 20 80 0F 40  DB 00 00 00 00 00 00 00  : .. , ..@ ........
20200F00 - 00 00 00 00 1A AE 20 2C  38 0F 20 20 19 74 20 00  : ...... , 8.  .t .
20200F10 - 1A AE 20 2C 07 00 00 00  CC 00 00 00 03 00 00 00  : .. ,.... ........
20200F20 - 1A AE 20 2C 00 80 0F 40  21 48 21 00 01 00 00 00  : .. ,...@ !H!.....
20200F30 - 00 AE 20 2C 07 00 00 00  CC 00 00 00 03 00 00 00  : .. ,.... ........
20200F40 - 1A AE 20 2C F0 00 00 00  21 48 21 00 01 00 00 00  : .. ,.... !H!.....
20200F50 - 20 C0 3E 40 70 3A 20 20  1A AE 20 2C F0 00 00 00  :  .>@p:   .. ,....
20200F60 - 00 03 20 00 00 00 00 00  00 00 00 00 9F 10 00 60  : .. ..... .......`
20200F70 - 70 3A 20 20 41 10 00 60  1A AE 20 2C 17 75 21 00  : p:  A..` .. ,.u!.
20200F80 - 1A AE 20 2C F0 00 00 00  00 00 00 00 00 03 20 00  : .. ,.... ...... .
20200F90 - 00 80 0F 40 70 44 1F 40  1A AE 20 2C 27 72 21 00  : ...@pD.@ .. ,'r!.
20200FA0 - 00 00 00 00 CC 00 00 00  1A AE 20 2C 1A AE 20 2C  : ........ .. ,.. ,
*** Max Count of Output Lines Reached ***

*** After memset ***
20200000 - 2A 2A 2A 2A 2A 2A 2A 2A  2A 2A 2A 2A 2A 2A 2A 2A  : ******** ********
...	 16382 duplicate line(s) removed.
2023FFF0 - 2A 2A 2A 2A 2A 2A 2A 2A  2A 2A 2A 2A 2A 2A 2A 2A  : ******** ********


Try second buffer
*** At Startup Buf2 ***
20240000 - C5 C5 C5 C5 C5 C5 C5 C5  C5 C5 C5 C5 C5 C5 C5 C5  : ........ ........
...	 14334 duplicate line(s) removed.
20277FF0 - C5 C5 C5 C5 C5 C5 C5 C5  C5 C5 C5 C5 C5 C5 C5 C5  : ........ ........

*** After memset Buf2 ***
20240000 - C5 C5 C5 C5 C5 C5 C5 C5  C5 C5 C5 C5 C5 C5 C5 C5  : ........ ........
...	 14334 duplicate line(s) removed.
20277FF0 - C5 C5 C5 C5 C5 C5 C5 C5  C5 C5 C5 C5 C5 C5 C5 C5  : ........ ........
Press any Key to reboot the Teensy

So again it may be some range of memory we know is not usable... But parts that are...

 

[defragster]

Doing the { optional '2' } before "R" on restart shows - AFTER multiple RESTARTS!:

	 ----->>>>> BUF REAL != BACK 
202069A0 - 00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  : ........ ........
202069B0 - 00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  : ........ ........
202069C0 - 00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  : ........ ........
202069D0 - 00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  : ........ ........
	  BUF BACK <<<<<-----
20243DA0 - 65 65 45 45 45 45 45 45  45 45 45 45 45 45 45 65  : eeEEEEEE EEEEEEEe
20243DB0 - 65 65 65 65 65 65 65 65  65 65 65 65 45 45 45 45  : eeeeeeee eeeeEEEE
20243DC0 - 45 45 45 45 45 45 45 45  45 65 65 65 65 65 65 65  : EEEEEEEE Eeeeeeee
20243DD0 - 65 65 65 65 65 65 45 45  45 45 45 45 45 45 45 45  : eeeeeeEE EEEEEEEE

So the DISK image there bufREAL shows it has been ZERO's where before "R"estart that area held contents of an "E_file.txt".

All 'L'etter files are filled with that letter alternating {UPPER,lower,...} case of that letter in groups of 13.

Doing a "D"ir w/verify shows : File Size Error:: E_file.txt found 8065 Bytes for Size 8064
>> looking at DirectoryVerify() - I see there was CODER negligence {blame:defragster} - it should also be testing for the 'L'etter value in the read characters - ideally in the UP/low 13 char sequence

ERROR is caught on next loop # to touch that file ::
>> :: /4_dir/E_file.txt <Bad Byte! e! = ? [0x3] @1
or
>> :: /4_dir/A_file.txt <Bad Byte! a! = A [0x41] @260

Above didn't catch error until after MULTIPLE restarts.

Next :" F, m, 2, 2, R" - failed on first Restart and the disk was 'Formatted'

I suppose for a complete EXAM of this I should return to TD 1.54 and see if it acts the same ...

 

[defragster]

UPDATED : github.com/Defragster/T4LockBeta/tree/main/RAM_SPLIT

"D"ir Verify watches 'L'etter files for Case Change as expected, AND verifies #BigFiles to have their content which is '#' of the file { 0,1,2,... } { not sure what happens on "S" - those can't create on RAM DISK }

Multiple TyComm Ctrl+R / RESET : No "D" issues show.

Then a couple "R" software reset okay : No "D" issues show.

BUT 't' buf[] array compares finds bad compare???

 

[defragster]

On Win 10 laptop with TD 1.54 and T_4.0 :: It is showing the same behavior as with TD 1.55b1.

The RAM_SPLIT sketch is catching it - as noted either or both buf REAL or BACK could be losing it. It can work at times ... others crash ... this 'expecting DMAMEM stability' isn't good for TD 1.54 or 1.55 ... should this go back to the LittleFS thread?

<edit>: back on desktop a fresh IDE 1.8.15 with TD 1.54 beta 11 :: same looking loss of integrity of DMAMEM on Reset

 

[Frank B]

Again I could be completely out to lunch

But I am wondering if this code in usb.c may have something to do with it:

FLASHMEM void _reboot_Teensyduino_(void)
{
    if (!(HW_OCOTP_CFG5 & 0x02)) {
        asm("bkpt #251"); // run bootloader
    } else {
        __disable_irq(); // secure mode NXP ROM reboot
        USB1_USBCMD = 0;
        IOMUXC_GPR_GPR16 = 0x00200003;
        // TODO: wipe all RAM for security
        [COLOR=#FF0000]__asm__ volatile("mov sp, %0" : : "r" (0x20201000) : );[/COLOR]
        __asm__ volatile("dsb":::"memory");
[B]        volatile uint32_t * const p = (uint32_t *)0x20208000;
        *p = 0xEB120000;
        ((void (*)(volatile void *))(*(uint32_t *)(*(uint32_t *)0x0020001C + 8)))(p);[/B]
    }
    __builtin_unreachable();
}

After that is a call to the i.mx.rt bootloader.. whatever this does. Possible, that this call configures or deletes something.

 

[Frank B]

The NXP flashloader API is described in a document in a zipfile named FLASHLOADER-RT106x-1-GA.zip (it has some tool also, i.e. for encryption (windows, linx, mac))
(edit: the sourcecode for blhost is here: https://www.nxp.com/webapp/sps/down...e=blhost_2.6.7&appType=file1&DOWNLOAD_ID=null)



But I couldn't find a source where the API<--> adresses are described. It must exist somewhere..

 

[mjs513]

Again I could be completely out to lunch

But I am wondering if this code in usb.c may have something to do with it:

FLASHMEM void _reboot_Teensyduino_(void)
{
    if (!(HW_OCOTP_CFG5 & 0x02)) {
        asm("bkpt #251"); // run bootloader
    } else {
        __disable_irq(); // secure mode NXP ROM reboot
        USB1_USBCMD = 0;
        IOMUXC_GPR_GPR16 = 0x00200003;
        // TODO: wipe all RAM for security
        [COLOR=#FF0000]__asm__ volatile("mov sp, %0" : : "r" (0x20201000) : );[/COLOR]
        __asm__ volatile("dsb":::"memory");
[B]        volatile uint32_t * const p = (uint32_t *)0x20208000;
        *p = 0xEB120000;
        ((void (*)(volatile void *))(*(uint32_t *)(*(uint32_t *)0x0020001C + 8)))(p);[/B]
    }
    __builtin_unreachable();
}

After that is a call to the i.mx.rt bootloader.. whatever this does. Possible, that this call configures or deletes something.

@Frank B - @KurtE - @defragster - @Paul

That code snippet to reboot the T4 is only in TD1.55b1.

Its not in TD1.54 which I am testing using a Production T4 which data doesn't persist across Restarts (using tycommander for resets).

I also just tested with a T4.1 using DMAMEM and TD1.54 and same thing. So not 100% sure that the root cause is what is in that reboot code?

 

[KurtE]

Good Early Morning all (at least in my time zone):

I am trying out yet another simple DMAMEM see what changes sketch:

#define FILL_CHAR 0xc5
#define MIN_REPORT_SIZE 1024
DMAMEM uint8_t dma_mem_buf[MIN_REPORT_SIZE * 32];
extern uint8_t _heap_start[], _heap_end[];

void setup() {
  while (!Serial) ; // needs serial
  Serial.begin(115200);
  uint32_t heap_size = (uint32_t)_heap_end - (uint32_t)_heap_start;
  Serial.println("\n\nSimple DMAMEM startup memory test");
  Serial.printf("buf addr range: %x - %x (%u)\n", (uint32_t)dma_mem_buf,
                (uint32_t)dma_mem_buf + sizeof(dma_mem_buf) - 1, sizeof(dma_mem_buf));
  Serial.printf("Heap Range: %x - %x (%u)\n", (uint32_t)_heap_start, (uint32_t)_heap_end, heap_size);

  uint8_t *p = dma_mem_buf; // try to walk all of this memory
  bool output_range_in_heap = false;
  while (p < _heap_end) {
    // find next char that is the fill char
    while ((p < _heap_end) && (*p != FILL_CHAR)) p++;
    uint8_t *first_match = p;
    if (!output_range_in_heap && (p >= _heap_start)) {
      Serial.println("  -------");
      output_range_in_heap = true;
    }
    while ((p < _heap_end) && (*p == FILL_CHAR)) p++;
    uint32_t match_size = (uint32_t)p - (uint32_t)first_match;
    if (match_size >= MIN_REPORT_SIZE)
      Serial.printf("  %x - %x: %u\n", (uint32_t)first_match, (uint32_t)p - 1, match_size);
  }

  // fill memory.
  memset(_heap_start, FILL_CHAR, heap_size);
  arm_dcache_flush(_heap_start , heap_size);
  memset(dma_mem_buf, FILL_CHAR, sizeof(dma_mem_buf));
  arm_dcache_flush(dma_mem_buf, sizeof(dma_mem_buf));

  Serial.println("Press any Key to reboot the Teensy");
  while (Serial.read() == -1) ;
  Serial.println("Now Resetting"); Serial.flush();

  delay(100);

  SCB_AIRCR = 0x05FA0004;
}

void loop() {
  // put your main code here, to run repeatedly:

}

Now if you look at the symbols defined in the range of DMAMEM which I sorted by address. We see that there are other things up in DMAMDM...

20200000 g     O .bss.dma	00008000 dma_mem_buf
20200000 l    d  .bss.dma	00000000 .bss.dma
20208000 g     O .bss.dma	0000004b usb_descriptor_buffer
20208060 l     O .bss.dma	00001000 rx_buffer
20209060 l     O .bss.dma	00002000 txbuffer
2020b060 g       .bss.dma	00000000 _heap_start
20280000 g       .text.csf	00000000 _heap_end

So with Locked teensy so far, I find interesting things.

If I use the Reset button within TyCommander, the Teensy tool does stuff. You can see it flash like doing something...
Log shows it:

06:38:42.504 (ports 5): WM_DEVICECHANGE DBT_DEVICEREMOVECOMPLETE
06:38:42.507 (ports 5): remove: loc=usb:0/140000/0/1/1/1
06:38:42.507 (ports 5): usb_remove: usb:0/140000/0/1/1/1
06:38:42.507 (ports 5): nothing new, skipping HID & Ports enum
06:38:42.522 (ports 5): WM_DEVICECHANGE DBT_DEVNODES_CHANGED
06:38:42.522 (ports 5): WM_DEVICECHANGE DBT_DEVICEREMOVECOMPLETE
06:38:42.523 (ports 5): nothing new, skipping HID & Ports enum
06:38:42.681 (ports 5): WM_DEVICECHANGE DBT_DEVICEARRIVAL
06:38:42.683 (ports 5): nothing new, skipping HID & Ports enum
06:38:42.718 (ports 5): WM_DEVICECHANGE DBT_DEVNODES_CHANGED
06:38:42.719 (ports 5): nothing new, skipping HID & Ports enum
06:38:42.723 (loader): handle 548
06:38:42.723 (loader): HID/win32: HidD_GetPreparsedData ok
06:38:42.733 (loader):  security: 03  12 34 34 12
06:38:42.733 (loader):  response: 04  12 8A 8A 12
06:38:42.733 (loader): nxp_write_register32 success, 12343412 128a8a12
06:38:42.733 (loader): Device came online, code_size = 100
06:38:42.733 (loader): Board is: NXP IMXRT1062 ROM
06:38:42.733 (loader): begin operation
06:38:42.753 (loader): File "C:\Users\kurte\AppData\Local\Temp\arduino_build_893127\zzz.ino.hex", 37888 bytes
06:38:42.753 (loader): File "C:\Users\kurte\AppData\Local\Temp\arduino_build_893127\zzz.ino.ehex", 37888 bytes, 4960 extra
06:38:42.763 (loader): ehex is valid, key hash: FB63C934 9683889D 3A20DB52 811A0593 83BEA33F 0F810CDF D6AD3C83 5DF1DA1A
06:38:42.763 (loader): set background IMG_ONLINE
06:38:42.773 (loader): state STATE_NXP_BEGIN
06:38:42.773 (loader):  security: 03  12 34 34 12
06:38:42.773 (loader):  response: 04  12 8A 8A 12
06:38:42.773 (loader): HAB locked secure mode
06:38:42.784 (loader): state STATE_NXP_CLOSED
06:38:42.784 (loader): sending ehex access, 4960 bytes
06:38:42.793 (loader):  security: 03  12 34 34 12
06:38:42.798 (loader):  response: 04  88 88 88 88
06:38:42.798 (loader): run it..
06:38:42.803 (loader):  security: 03  12 34 34 12
06:38:42.813 (loader): end operation, total time = 0.070 seconds
06:38:42.821 (ports 5): WM_DEVICECHANGE DBT_DEVICEREMOVECOMPLETE
06:38:42.822 (ports 5): nothing new, skipping HID & Ports enum
06:38:42.823 (loader): redraw timer set, image 80 to show for 2000 ms
06:38:42.837 (ports 5): WM_DEVICECHANGE DBT_DEVNODES_CHANGED
06:38:42.838 (ports 5): nothing new, skipping HID & Ports enum
06:38:42.869 (ports 5): WM_DEVICECHANGE DBT_DEVNODES_CHANGED
06:38:42.870 (ports 5): nothing new, skipping HID & Ports enum
06:38:42.973 (loader): HID/win32:  vid:058F pid:9410 ver:0122
06:38:42.984 (loader): HID/win32:  vid:058F pid:9410 ver:0122
06:38:42.993 (loader): HID/win32:  vid:045E pid:07A5 ver:0797
06:38:42.996 (loader): HID/win32:  vid:045E pid:07A5 ver:0797
06:38:43.003 (loader): HID/win32:  vid:045E pid:07A5 ver:0797
06:38:43.007 (loader): HID/win32:  vid:045E pid:07A5 ver:0797
06:38:43.195 (ports 5): WM_DEVICECHANGE DBT_DEVICEARRIVAL
06:38:43.197 (ports 5): found_usb_device, id=\\?\usb#vid_16c0&pid_0478#000fd6ab#{a5dcbf10-6530-11d2-901f-00c04fb951ed}
06:38:43.197 (ports 5): found_usb_device, loc=usb:0/140000/0/1/1/1    Port_#0001.Hub_#0007
06:38:43.197 (ports 5): found_usb_device, hwid=USB\VID_16C0&PID_0478&REV_0107
06:38:43.197 (ports 5): found_usb_device, devinst=00000017
06:38:43.197 (ports 5): add: loc=usb:0/140000/0/1/1/1, class=HID, vid=16C0, pid=0478, ver=0107, serial=000fd6ab, dev=\\?\usb#vid_16c0&pid_0478#000fd6ab#{a5dcbf10-6530-11d2-901f-00c04fb951ed}
06:38:43.197 (ports 5): hiddev_from_devinst_list: iface=0
06:38:43.198 (ports 5): found_usb_device complete
06:38:43.199 (ports 5): hid, found devinst=00000018
06:38:43.199 (ports 5): hid, path=\\?\hid#vid_16c0&pid_0478#7&1b4a35bb&0&0000#{4d1e55b2-f16f-11cf-88cb-001111000030}
06:38:43.199 (ports 5): hid,  opened handle
06:38:43.199 (ports 5):  devinst=00000018, location=usb:0/140000/0/1/1/1
06:38:43.199 (ports 5):  vid=16C0, pid=0478, ver=0107, usepage=FF9C, use=0024
06:38:43.199 (ports 5):  devpath=\\?\hid#vid_16c0&pid_0478#7&1b4a35bb&0&0000#{4d1e55b2-f16f-11cf-88cb-001111000030}
06:38:43.199 (ports 5): usb_add: usb:0/140000/0/1/1/1  hid#vid_16c0&pid_0478 (Teensy 4.0) Bootloader
06:38:43.263 (loader): encryption is required, public key hash: FB63C934 9683889D 3A20DB52 811A0593 83BEA33F 0F810CDF D6AD3C83 5DF1DA1A
06:38:43.263 (loader): Device came online, code_size = 2031616
06:38:43.273 (loader): Board is: Teensy 4.0 (IMXRT1062), version 1.07
06:38:43.285 (loader): File "C:\Users\kurte\AppData\Local\Temp\arduino_build_893127\zzz.ino.hex", 37888 bytes
06:38:43.293 (loader): File "C:\Users\kurte\AppData\Local\Temp\arduino_build_893127\zzz.ino.ehex", 37888 bytes, 4960 extra
06:38:43.293 (loader): ehex is valid, key hash: FB63C934 9683889D 3A20DB52 811A0593 83BEA33F 0F810CDF D6AD3C83 5DF1DA1A
06:38:43.303 (loader): set background IMG_ONLINE
06:38:43.313 (ports 5): WM_DEVICECHANGE DBT_DEVICEREMOVECOMPLETE
06:38:43.314 (ports 5): remove: loc=usb:0/140000/0/1/1/1
06:38:43.314 (ports 5): usb_remove: usb:0/140000/0/1/1/1
06:38:43.314 (ports 5): nothing new, skipping HID & Ports enum
06:38:43.350 (ports 5): WM_DEVICECHANGE DBT_DEVNODES_CHANGED
06:38:43.351 (ports 5): nothing new, skipping HID & Ports enum
06:38:43.373 (ports 5): WM_DEVICECHANGE DBT_DEVNODES_CHANGED
06:38:43.374 (ports 5): nothing new, skipping HID & Ports enum
06:38:43.473 (loader): HID/win32: HidD_GetPreparsedData failed, device assumed disconnected
06:38:43.488 (loader): Device went offline
06:38:43.591 (ports 5): WM_DEVICECHANGE DBT_DEVICEARRIVAL
06:38:43.592 (ports 5): found_usb_device, id=\\?\usb#vid_16c0&pid_0483#10379950#{a5dcbf10-6530-11d2-901f-00c04fb951ed}
06:38:43.592 (ports 5): found_usb_device, loc=usb:0/140000/0/1/1/1    Port_#0001.Hub_#0007
06:38:43.592 (ports 5): found_usb_device, hwid=USB\VID_16C0&PID_0483&REV_0279
06:38:43.592 (ports 5): found_usb_device, devinst=00000005
06:38:43.592 (ports 5): add: loc=usb:0/140000/0/1/1/1, class=USB, vid=16C0, pid=0483, ver=0279, serial=10379950, dev=\\?\usb#vid_16c0&pid_0483#10379950#{a5dcbf10-6530-11d2-901f-00c04fb951ed}
06:38:43.593 (ports 5):   comport_from_devinst_list attempt
06:38:43.593 (ports 5):   found Ports in classguid_list at index=1
06:38:43.593 (ports 5):   port COM74 found from devnode
06:38:43.593 (ports 5): found_usb_device complete
06:38:43.594 (ports 5): usb_add: usb:0/140000/0/1/1/1  COM74 (Teensy 4.0) Serial
06:38:43.660 (ports 5): WM_DEVICECHANGE DBT_DEVICEARRIVAL
06:38:43.660 (ports 5): WM_DEVICECHANGE DBT_DEVNODES_CHANGED
06:38:43.662 (ports 5): nothing new, skipping HID & Ports enum
06:38:43.723 (loader): HID/win32:  vid:058F pid:9410 ver:0122
06:38:43.733 (loader): HID/win32:  vid:058F pid:9410 ver:0122
06:38:43.733 (loader): HID/win32:  vid:045E pid:07A5 ver:0797
06:38:43.753 (loader): HID/win32:  vid:045E pid:07A5 ver:0797
06:38:43.753 (loader): HID/win32:  vid:045E pid:07A5 ver:0797
06:38:43.763 (loader): HID/win32:  vid:045E pid:07A5 ver:0797
06:38:43.775 (ports 5): WM_DEVICECHANGE DBT_DEVNODES_CHANGED
06:38:43.776 (ports 5): nothing new, skipping HID & Ports enum

And output from this test sketch looks like:

Simple DMAMEM startup memory test
buf addr range: 20200000 - 20207fff (32768)
Heap Range: 2020b060 - 20280000 (479136)
  20200000 - 20200927: 2344
  202011c8 - 20201b7f: 2488
  202027ec - 202037ff: 4116
  -------
  2020b060 - 2020ffaf: 20304
  20210000 - 2027ffff: 458752
Press any Key to reboot the Teensy

If I have program reboot itself, the teensy program does some stuff but just seeing the device go away and come back...
And program output is different:

Simple DMAMEM startup memory test
buf addr range: 20200000 - 20207fff (32768)
Heap Range: 2020b060 - 20280000 (479136)
  20200000 - 2020098f: 2448
  202011c8 - 20201b7f: 2488
  202027ec - 20203a5f: 4724
  -------
  2020b060 - 2027ffff: 479136
Press any Key to reboot the Teensy

Next up try it on different T4.xs to see if locked makes differences

 

[KurtE]

Follow on to previous:

Standard normal T4 (also not new one): output did not differ between TyCommander and software reset:

Simple DMAMEM startup memory test
buf addr range: 20200000 - 20207fff (32768)
Heap Range: 2020b060 - 20280000 (479136)
  20200000 - 20200c67: 3176
  202010ec - 20201b7f: 2708
  20201c38 - 20203801: 7114
  -------
  2020b060 - 2027ffff: 479136
Press any Key to reboot the Teensy
Now Resetting


Simple DMAMEM startup memory test
buf addr range: 20200000 - 20207fff (32768)
Heap Range: 2020b060 - 20280000 (479136)
  20200000 - 20200c67: 3176
  202010ec - 20201b7f: 2708
  20201c38 - 20203a5f: 7720
  -------
  2020b060 - 2027ffff: 479136
Press any Key to reboot the Teensy

Running on Micromod: first one is reset using reset button in TyCommander second one software.

Simple DMAMEM startup memory test
buf addr range: 20200000 - 20207fff (32768)
Heap Range: 2020b060 - 20280000 (479136)
  20200000 - 20200c67: 3176
  202010ec - 20201b7f: 2708
  20201c38 - 20203801: 7114
  -------
  2020b060 - 2027ffff: 479136
Press any Key to reboot the Teensy
Now Resetting


Simple DMAMEM startup memory test 
buf addr range: 20200000 - 20207fff (32768)
Heap Range: 2020b060 - 20280000 (479136)
  20200000 - 20200c67: 3176
  202010ec - 20201b7f: 2708
  20201c38 - 20203a5f: 7720
  -------
  2020b060 - 2027ffff: 479136
Press any Key to reboot the Teensy

Note: a couple of times when I hit the reset button, I would see the teensy app sort of hang and show a Erasing that failed...

 

[KurtE]

Another quick follow up:

If I run same (slightly edited) sketch again on T4 with security enabled but not locked, it acts same as other T4 family that is not locked)
And I get the same output from either pressing reset button on TyCommander, or using software reset or something else I just tried:
Doing: teensy_reboot.exe from a command prompt.

Simple DMAMEM startup memory test
buf addr range: 20200000 - 20207fff (32768)
Heap Range: 2020b060 - 20280000 (479136)
  20200000 - 2020098f: 2448 0
  202011c8 - 20201b7f: 2488 2104
  202027ec - 20203801: 4118 3180
  -------
  2020b060 - 2027ffff: 479136 30814
Press any Key to reboot the Teensy

The difference in code is that in the range printings added another number which is the number of bytes between the range printed and the one above it... Plus or minus one...

Other Side note: pressing the reset button in TyCommander or running the command: teensy_reboot.exe
On the locked T4, causes the T4 / Teensy app to try to reprogram the T4. In case of TyCOmmander running it fails as it was holding the Serial port. But if Serial port is not active. the command line did reprogram the locked T4

Probably all for now, until...

 

[PaulStoffregen]

Sorry, I've not kept up with every message these last 2 days. Hope to read them all this evening.

But regarding data persistent in RAM2 / DMAMEM across rebooting, NXP's ROM code which starts up the chip and checks the data in flash (regardless of whether secure mode is locked) is documented on page 209 in the reference manual as potentially using 32K, from 2020,0000 to 2020,7FFF.



In practice probably less than all 32K is really used. But if you wanted to store data in DMAMEM and have it persist across rebooting, you should avoid using anything in this 32K range.

The linker script has a special section called "hab_log" which is meant for this purpose. Well, at least for preventing use of the first part of this 32K range, where NXP's ROM stores the HAB event log. Here's how it's used in the LockSecureMode sketch:

// Reserve the HAB memory, so USB buffers don't overwrite logged HAB events
__attribute__ ((section(".hab_log"), used)) volatile uint8_t HAB_logfile[8192];

The linker will omit this array, even though it has the "used" attribute, unless something in the program accesses it. You'll see the LockSecureMode program also has this:

  HAB_logfile[sizeof(HAB_logfile)-1]; // do not delete this line!

You could put this into your program and change it to 32768 bytes, so no part of the ROM's usage of RAM is used by your program. Or maybe a smaller amount, so the USB buffers and other stuff normally allocated in DMAMEM overlaps some of that 32K, but your allocation for persistent data starts after 2020,8000.

However, if you are using locked secure mode, Teensy Loader and a loader utility which is part of the EHEX file (mentioned previously on this thread, and in the documentation I plan to upload later today) uses 2020,8000 to 2020,FFFF. So to be sure to persist across uploads even in secure mode, you need to avoid using any of the first 64K of DMAMEM.

Again, I've been working heavily on the documentation yesterday and Friday (also took a little time off Friday). Here's an image I made yesterday afternoon for the upcoming documentation page.

 

[defragster]

Sorry, I've not kept up with every message these last 2 days. Hope to read them all this evening.

But regarding data persistent in RAM2 / DMAMEM across rebooting, NXP's ROM code which starts up the chip and checks the data in flash (regardless of whether secure mode is locked) is documented on page 209 in the reference manual as potentially using 32K, from 2020,0000 to 2020,7FFF.
...

In practice probably less than all 32K is really used. But if you wanted to store data in DMAMEM and have it persist across rebooting, you should avoid using anything in this 32K range.

The linker script has a special section called "hab_log" which is meant for this purpose. Well, at least for preventing use of the first part of this 32K range, where NXP's ROM stores the HAB event log. Here's how it's used in the LockSecureMode sketch:

// Reserve the HAB memory, so USB buffers don't overwrite logged HAB events
__attribute__ ((section(".hab_log"), used)) volatile uint8_t HAB_logfile[8192];

The linker will omit this array, even though it has the "used" attribute, unless something in the program accesses it. You'll see the LockSecureMode program also has this:
...
You could put this into your program and change it to 32768 bytes, so no part of the ROM's usage of RAM is used by your program. Or maybe a smaller amount, so the USB buffers and other stuff normally allocated in DMAMEM overlaps some of that 32K, but your allocation for persistent data starts after 2020,8000.

However, if you are using locked secure mode, Teensy Loader and a loader utility which is part of the EHEX file (mentioned previously on this thread, and in the documentation I plan to upload later today) uses 2020,8000 to 2020,FFFF. So to be sure to persist across uploads even in secure mode, you need to avoid using any of the first 64K of DMAMEM.

Again, I've been working heavily on the documentation yesterday and Friday (also took a little time off Friday). Here's an image I made yesterday afternoon for the upcoming documentation page.

...

Well - that explains that ... matches the post I made about it being the low end getting hit.

Also as seen the build process seems to CHOOSE what DMAMEM hunk gets allocated LOW/HIGH and used for LFS_RAM.begiin() - not by order in the source file and even on one build declare Abuf then Bbuf - using Abuf is LOW address and Bbuf is HIGH address, and Abuf used for LFS_RAM. Then rebuild using Bbuf, it becomes the USED LOW address.

Is there a way to make a PROGMEM, FLASHMEM like build clue that would put the "DMAMEM" to 'originate' over that 64K boundary?

 

[PaulStoffregen]

Is there a way to make a PROGMEM, FLASHMEM like build clue that would put the "DMAMEM" to 'originate' over that 64K boundary?

Do you consider "a way" to involve editing the linker scripts?

 

[mjs513]

@PaulStoffregen

Thanks for the detailed explanation. Going to give your suggestion a try and see what happens. While working on new examples for LittleFS I added this code to Program_datalogger sketch that seems to work well:

  // see if the Flash is present and can be initialized:
  // lets check to see if the T4 is setup for security first
  #if ARDUINO_TEENSY40
    if ((IOMUXC_GPR_GPR11 & 0x100) == 0x100) {
      //if security is active max disk size is 960x1024
      if(PROG_FLASH_SIZE > 960*1024){
        diskSize = 960*1024;
        Serial.printf("Security Enables defaulted to %u bytes\n", diskSize);  
      } else {
        diskSize = PROG_FLASH_SIZE;
        Serial.printf("Security Not Enabled using %u bytes\n", diskSize);
      }
    }
  #else
    diskSize = PROG_FLASH_SIZE;
  #endif

Should be easy enough to adapt for using

// Reserve the HAB memory, so USB buffers don't overwrite logged HAB events
__attribute__ ((section(".hab_log"), used)) volatile uint8_t HAB_logfile[32768];

hopefully this is what you meant

EDIT: Ok guess I am not sure how it works

 

[KurtE]

Hi @Paul and @defragster ...

I think you already mentioned one way to get all of your stuff farther up in memory:

// Reserve the HAB memory, so USB buffers don't overwrite logged HAB events
__attribute__ ((section(".hab_log"), used)) volatile uint8_t HAB_logfile[8192];

void setup() {
  HAB_logfile[sizeof(HAB_logfile) - 1]; // do not delete this line!

Where you might change the 8192 to 32K or 64K...

However maybe the simplest and most direct answer is just say no!

That is I believe this memory usage (and or corruption) depending on how you look at it. That is I don't want to necessarily lose 32K or 64K of memory buffers for things that don't need it.
For example might cause us to no longer be able to create frame buffers for some displays...

Also from the comments above, you may wipe all memory anyway...


At times what I have wondered is assuming no memory wipe... Would it make sense to add some thing like _sbrk and modify _sbrk

char *__brkval = (char *)&_heap_start;

void * _sbrk(int incr)
{
        char *prev = __brkval;
        if (incr != 0) {
                if (prev + incr > (char *)&_heap_end) {
                        errno = ENOMEM;
                        return (void *)-1;
                }
                __brkval = prev + incr;
        }
        return prev;
}

Like a method lets say: void( _ebrk(int incr)
which does the same as ebrk but tries to get the memory from the end?
You would then want to use variable instead of hard coded _heap_end...

But again not sure if it is worth it

EDIT:
Alternative: not sure if can define new region like:
PDMAMEM uint8_t our_file_buffer[1024*256];

And somehow the linker script would setup that that section must be at least beyond the 32K or 64K section?

 

[defragster]

Do you consider "a way" to involve editing the linker scripts?

That was the expectation - so it could be managed by the build process and linker for allocation to a specific region.

I found this to work - fixed ALLOC that persists without corruption across restarts and Upload:

// This declares the LittleFS Media type and gives a text name to Identify in use
LittleFS_RAM myfs;  // CAN use either RAM1 or RAM2 as available

[B]#define LFS_BUF_SIZE 430*1024
void *LFS_BUF = (void*)(0x2027ff00-LFS_BUF_SIZE);
[/B]
//char buf[ 390 * 1024 ];	// BUFFER in RAM1 :: Lost on any restart
//DMAMEM char buf[ 390 * 1024 ];	// DMAMEM Uses RAM2 :: Typically survives Restart/Upload

and setup():

void setup() {
	while (!Serial) ; // wait
   MemoryHexDump(Serial, [B]LFS_BUF[/B], LFS_BUF_SIZE, true, "*** LFS_BUF[] At Startup ***\n", 100);
	Serial.println("\n" __FILE__ " " __DATE__ " " __TIME__);
	Serial.println("LittleFS Test : File Integrity"); delay(5);

	if (!myfs.begin([B]LFS_BUF[/B], LFS_BUF_SIZE)) {
		Serial.printf("Error starting %s\n", szDiskMem);
		while( 1 );
	}
   MemoryHexDump(Serial, [B]LFS_BUF[/B], LFS_BUF_SIZE, true, "*** LFS_BUF[] At BEGUN ***\n", 100);

Walk all Files verify Read Size:	
0	D1 DE
	D2 A
	D3 
	D4 BCD
BC	0 Errors found

Bytes Used: 186880, Bytes Total:440320

But is in UNCONTROLLED memory - not known by Linker/Build:

Memory Usage on Teensy 4.0:
  FLASH: code:78464, data:10264, headers:8548   free for files:1934340
   RAM1: variables:21184, code:73624, padding:24680   free for local variables:404800
[B]   RAM2: variables:12384  free for malloc/new:511904[/B]
Using library LittleFS at version 1.0.0 in folder: T:\Arduino_1.8.15td155\hardware\teensy\avr\libraries\LittleFS 
Using library SPI at version 1.0 in folder: T:\Arduino_1.8.15td155\hardware\teensy\avr\libraries\SPI 
Using library MemoryHexDump at version 1.0.1 in folder: T:\tCode\libraries\MemoryHexDump 
Using library SdFat at version 2.0.5-beta.1 in folder: T:\Arduino_1.8.15td155\hardware\teensy\avr\libraries\SdFat

some DUMP:

*** LFS_BUF[] At Startup ***
20214700 - 05 00 00 00 F0 0F FF F7  6C 69 74 74 6C 65 66 73  : ........ littlefs
20214710 - 2F E0 00 10 00 00 02 00  00 01 00 00 B8 06 00 00  : /....... ........
20214720 - 27 00 00 00 FF FF FF 7F  FE 03 00 00 20 30 04 1D  : '....... .... 0..
20214730 - 31 5F 64 69 72 00 00 0C  00 32 5F 64 69 72 20 20  : 1_dir... .2_dir  
20214740 - 00 0D F4 02 00 00 62 00  00 00 00 00 0C 00 DF 00  : ......b. ........
20214750 - 00 00 60 00 00 00 40 1F  F8 00 4A 02 00 00 6D 00  : ..`...@. ..J...m.
20214760 - 00 00 30 10 00 12 C1 E7  B6 E8 FF FF FF FF FF FF  : ..0..... ........
20214770 - FF FF FF FF FF FF FF FF  FF FF FF FF FF FF FF FF  : ........ ........
20214780 - 30 10 00 12 25 01 00 00  4A 02 00 00 30 10 00 38  : 0...%... J...0..8
20214790 - FC 22 B4 D8 FF FF FF FF  FF FF FF FF FF FF FF FF  : ."...... ........
202147A0 - FF FF FF FF FF FF FF FF  FF FF FF FF FF FF FF FF  : ........ ........
202147B0 - FF FF FF FF FF FF FF FF  FF FF FF FF FF FF FF FF  : ........ ........
202147C0 - 10 1F F8 30 40 00 00 0D  30 5F 62 69 67 66 69 6C  : ...0@... 0_bigfil
202147D0 - 65 2E 74 78 74 20 00 00  0D 70 1F F8 23 2A 60 99  : e.txt .. .p..#*`.
202147E0 - 29 FF FF FF FF FF FF FF  FF FF FF FF FF FF FF FF  : )....... ........
202147F0 - FF FF FF FF FF FF FF FF  FF FF FF FF FF FF FF FF  : ........ ........
20214800 - 06 00 00 00 F0 0F FF F7  6C 69 74 74 6C 65 66 73  : ........ littlefs
20214810 - 2F E0 00 10 00 00 02 00  00 01 00 00 B8 06 00 00  : /....... ........
20214820 - 27 00 00 00 FF FF FF 7F  FE 03 00 00 20 30 08 1D  : '....... .... 0..
20214830 - 31 5F 64 69 72 00 00 04  00 32 5F 64 69 72 20 20  : 1_dir... .2_dir  
20214840 - 00 0D F4 02 00 00 62 00  00 00 00 00 04 00 DF 00  : ......b. ........
20214850 - 00 00 60 00 00 00 20 10  0C 05 30 5F 62 69 67 66  : ..`... . ..0_bigf
20214860 - 69 6C 65 2E 74 78 74 20  30 00 05 26 03 00 00 00  : ile.txt  0..&....
20214870 - 30 02 00 40 3F F8 00 25  01 00 00 4A 02 00 00 30  : 0..@?..% ...J...0
20214880 - 10 00 35 77 FB 70 B5 FF  FF FF FF FF FF FF FF FF  : ..5w.p.. ........
20214890 - FF FF FF FF FF FF FF FF  FF FF FF FF FF FF FF FF  : ........ ........
...	 5 duplicate line(s) removed.
202148F0 - FF FF FF FF FF FF FF FF  FF FF FF FF FF FF FF FF  : ........ ........
20214900 - B7 06 00 00 B6 06 00 00  B4 06 00 00 B0 06 00 00  : ........ ........
20214910 - 61 61 61 61 61 61 61 61  41 41 41 41 41 41 41 41  : aaaaaaaa AAAAAAAA
20214920 - 41 41 41 41 41 61 61 61  61 61 61 61 61 61 61 61  : AAAAAaaa aaaaaaaa
20214930 - 61 61 41 41 41 41 41 41  41 41 41 41 41 41 41 61  : aaAAAAAA AAAAAAAa
20214940 - 61 61 61 61 61 61 61 61  61 61 61 61 41 41 41 41  : aaaaaaaa aaaaAAAA
20214950 - 41 41 41 41 41 41 41 41  41 61 61 61 61 61 61 61  : AAAAAAAA Aaaaaaaa
20214960 - 61 61 61 61 61 61 41 41  41 41 41 41 41 41 41 41  : aaaaaaAA AAAAAAAA
20214970 - 41 41 41 61 61 61 61 61  61 61 61 61 61 61 61 61  : AAAaaaaa aaaaaaaa

 

[defragster]

@Paul - BTW - back to Locked T_4.0 beta.

Have been testing on production T_4.0 for past days.

Just hit program Button on LOCKED T_4.0 beta :: 10:55:52.585 (loader): using encrypted ehex (required - secure mode is locked)

It is working as above p#242 on that too with 'Reset' and Re-Upload.

 

[defragster]

Looks like the 'D'irVerify code did what was promised before :: 'D' Walk all Dir Files verify Read Size ... it just didn't do ALL it could do.

I have the updated DirVerify() code I can do a PR on ... for all Integrity Sketches functions.ino ... that is programmed to test the expected Content of files IT creates.

DirVerify was a late add ... on delete of any file during 'loop's it was doing the right thing to catch errors - but the "D" walks All files on command to Verify.

Post #242 method seems to work and show integrity of the LFS_RAM in DMAMEM across warm restarting ...

And it looks like room was even left to allow CrashReport at : 0x2027FF80 with :: void *LFS_BUF = (void*)(0x2027ff00-LFS_BUF_SIZE);

*** LFS_BUF[] At BEGUN ***
20214700 - 01 00 00 00 F0 0F FF F7  6C 69 74 74 6C 65 66 73  : ........ littlefs

...

2027FEF0 - FF FF FF FF FF FF FF FF  FF FF FF FF FF FF FF FF  : ........ ........

 

[mjs513]

Hi @Paul and @defragster ...

I think you already mentioned one way to get all of your stuff farther up in memory:

// Reserve the HAB memory, so USB buffers don't overwrite logged HAB events
__attribute__ ((section(".hab_log"), used)) volatile uint8_t HAB_logfile[8192];

void setup() {
  HAB_logfile[sizeof(HAB_logfile) - 1]; // do not delete this line!

Where you might change the 8192 to 32K or 64K...

However maybe the simplest and most direct answer is just say no!

That is I believe this memory usage (and or corruption) depending on how you look at it. That is I don't want to necessarily lose 32K or 64K of memory buffers for things that don't need it.
For example might cause us to no longer be able to create frame buffers for some displays...

Also from the comments above, you may wipe all memory anyway...


At times what I have wondered is assuming no memory wipe... Would it make sense to add some thing like _sbrk and modify _sbrk

char *__brkval = (char *)&_heap_start;

void * _sbrk(int incr)
{
        char *prev = __brkval;
        if (incr != 0) {
                if (prev + incr > (char *)&_heap_end) {
                        errno = ENOMEM;
                        return (void *)-1;
                }
                __brkval = prev + incr;
        }
        return prev;
}

Like a method lets say: void( _ebrk(int incr)
which does the same as ebrk but tries to get the memory from the end?
You would then want to use variable instead of hard coded _heap_end...

But again not sure if it is worth it

EDIT:
Alternative: not sure if can define new region like:
PDMAMEM uint8_t our_file_buffer[1024*256];

And somehow the linker script would setup that that section must be at least beyond the 32K or 64K section?

Thanks Kurt - guess what line I forgot to add - the one that said do not delete

It does work - just tested with the datalogger sketch using a 32k buffer on a production T4. Did confirm that Tim's method works on a Production T4 as well.

But have to remember it only works for DMAMEM so whatever we do we need to make it clear to a user.

EDIT: It seems cleaner if you use Tim;s method but think that always uses DMAMEM so you are no longer supporting RAM1 - not sure that matters 100% but somebody may want to use it.

 

[Frank B]

Sorry for the dumb question - what are possible use-cases for littlFS in RAM ? And why does it need to survive a reset, and has to be rather large the same time?

As it seems to an issue with the HAB log only, anway - which means it is an issue only if encryption is enabled - which hardly anyone will use (besides some commercial apps) i'd prefer not to change anything.
- And to have the memory occupied by HAB available if no encryption is enabled.
In no way it is desireable to have that memory blocked for the 99.9999% of users who will not encrypt. And the remaining rest will not use littlFS there anyway..

 

[defragster]

This was a problem in TD 1.54 as well with the HAB stuff starting ... or maybe just a natural process in startup.

The Value of DMAMEM could be: LittleFS LOG to DMAMEM - FASTER than FLASH at 6.5 MB /sec to create files. Then those files could be copied to FLASH ... SPI LittleFS for long term storage, or SD CARD like in the one sample I wrote @ LittleFS-port-to-Teensy-SPIFlash.

Like CrashReport - if something goes wrong - the data survives a reset to resume, recover, or interpret the data. If it is lost on restart - then any logged data is destroyed,

It can be easily modified - hardcoded - a bit cleaner to the Integrity/RAM.ino as:

// This declares the LittleFS Media type and gives a text name to Identify in use
LittleFS_RAM myfs;  // CAN use either RAM1 or RAM2 as available
[B]#define LFS_BUF_SIZE 430*1024[/B]
//char LFS_BUF[ LFS_BUF_SIZE ];	// BUFFER in RAM1 :: Lost on any restart
[B]void *LFS_BUF = (void*)(0x2027ff00-LFS_BUF_SIZE); // DMAMEM Uses RAM2 :: This region typically survives Restart/Upload[/B]

...

void setup() {
...
	if (!myfs.begin(LFS_BUF, LFS_BUF_SIZE)) {
...

The user can set the LFS_BUF_SIZE just like before and leave one of those two lines uncommented.

If OTHER usage of DMAMEM pushes too HIGH the drive will be destroyed at runtime startup.

Not as clean as PSRAM usage :: EXTMEM char buf[MYPSRAM * 1024 * 1024]; // Contents preserved with Power on Restart and Upload

It doesn't WASTE memory - except using dynamic HEAP Allocs would trash the LFS_RAM too.

<edit> :: Heap - Memory allocated by C++ "new" and C malloc(), and Arduino String variables are placed in RAM2, starting immediately after the DMAMEM variables.

 

[Frank B]

Some mentioned to mark it "used" with an attribute. This way it would be wasted.

 

[KurtE]

@all - @Frank - I personally totally agree with you, that I don't see a very strong usage case on when I would want to use a LittleFS in RAM and expect and prepare for it to work sometimes but not others.

But others may find usage. So for me most of this was for a learning exercise and a way to test some differences for the new lockable T4...

As I think I mentioned above (or in other conversations), I don't think one should reserve some major pieces of memory from sketches and/or cause them to fragment.
Yes I was going to say you could just grab a random memory location up high enough and as long as nobody calls malloc or new would work... But I can imagine someone doing this in some library and then some one decides to use a TFT display to display the contents which uses a Frame buffer and all heck breaks lose.

Also problematic is understanding that every time you update something that you wish to be preserved, you are probably going to have to do a: arm_dcache_flush of the data as the DMAMEM may or may not write through to the actual memory.

But if we were to want memory that has a reasonable chance to survive a reboot. Then as I mentioned earlier we could extend the void * _sbrk(int incr) stuff to have another function that allocates from the end instead of the start... Both would need to check they don't collide.

Or as another option: We could maybe? create another memory region like DMAMEM, maybe something like HDMAMEM (high) or PDMAMEM(preserved (not overwriten? ) or the like that maybe could be added to
linker script? where somehow you add that section
To end at the current _heap_end type definition. _heap_end = ORIGIN(RAM) + LENGTH(RAM);
and have _heap_end be something like _heap_end = ORIGIN(RAM) + LENGTH(RAM) - sizeof(.hbss);
Warning not an expert in these scripts!

 

[Frank B]

Yup, that would work if you'd tell the linker the size of the memory you want to preserve.
The question is, how that all works on 1170.
You could provide an optional linker script, that moves the heap back to the DTCM, were it was some time back, too. But I guess (no.. I hope) a commercial user is smart enough to do things like this himself.