Call to arms | Teensy + SDRAM = true

[sbfreddie]

Gentleman:
I hope you understand how little 1.1 to 10 PicoFarads at 233MHz really is. This value could very easily be created by simply routing an etch close to power or ground. In the pictures above I see at least 10 to 30 pF of capacitance floating around with all the lead lengths and bits of 30 ga. wire.
MarkT I think it is about time you chimed in here to give some details on capacitance on PCB's. I know that Paul is very aware of this issue as he has a great deal of experience designing PCB's.

It been a very interesting conversation to read.

Thanks,
Ed

 

[Dogbone06]

@sbfreddie I just want to give you a little information to add to what you wrote.
The routing for the SDRAM to the 1062, all traces were copied straight over from the NXP EVAL board to minimize errors. Just so you know. Thanks for your input!

 

[shawn]

In SDRAM_t4.h, I'd recommend declaring all the functions as static, and add an empty constexpr constructor. Calling class member functions from the startup hooks is technically not correct, but like our Serial, HardwareSerial, SPI, Wire classes this ought to be enough for it to work anyway. Would be nice if the SDRAM could be initialized from the middle or late startup hook, so a C++ class needing large memory could access it from its constructor.

Got and done will push it in a few minutes but here is what is in it:

class SDRAM_t4 {
public:
    constexpr SDRAM_t4() {};
    static bool init();
 
private:
    static unsigned int ns_to_clocks(float ns, float freq);
    static void configure_sdram_pins();
    static bool SendIPCommand(uint32_t address, uint16_t command, uint32_t write, uint32_t *read);
    static bool IPCommandComplete();
 
    //set NOCAP to 1 if cap C29 is removed
    uint8_t NOCAP = 0;
 
};

@defragster - note the change where i put NOCAP setting.

Misc. ramblings:

Using a namespace for what you’re doing here is actually more appropriate. But, if you use a class, do these instead:
1. Make the constructor private.
2. The constructor does not have to be constexpr because theres no non-static (well, I’m not sure how NOCAP is being used) data members. (There’s a bit more to this whole constexpr constructor business, actually, and it does not mean “make the class a constant.”)
3. Change NOCAP to static.
4. Access all things inside SDRAM_t41 using “::” instead of “.”

Having said all that, the “Arduino” style is to use a “.”, which means creating objects. This actually makes things a lot more complicated because now you have to use singletons for everything. That is hard because there’s no guarantee, without special handling, that these objects get created before you use them. I’ll repeat a phrase that I’ve used before: “constexpr for constructors does not solve this problem.”
Specifically: constexpr does nothing (that is, not specified to do anything) unless you also declare the variable constexpr. Note that it might prevent compilation if the constructor does not satisfy certain criteria, but it still does nothing unless using a declared variable as constexpr.

So, my suggestion is to do one of these things:
1. Use the “everything is a static member” approach with a private constructor, or
2. Dive into how things like cout and the like (for example, notice that the ostream, the type of cout, doesn't have a constexpr constructor) get created such that they’re guaranteed to be initialized before anything in a program uses them. I use the “Nifty Counter” idiom to accomplish this. (See also solutions to the “static initialization order fiasco”) —Note: see the STATIC_INIT_DECL and STATIC_INIT_DEFN macros in my QNEthernet library; I use them to declare and define all the statically-available objects such as Ethernet and MDNS.

I can’t reiterate strongly enough that simply using constexpr constructors doesn’t solve this problem, and does not guarantee the object will be created before needing to use it.

Thanks for listening.

One of these days, I’ll host a little workshop or something on how to solve this (my preference right now vs. writing something up due to time constraints). Most Arduino libraries that rely on a pre-created object get it wrong, and just because the compiler does it correctly in this version, doesn’t mean that any compiler updates won’t change anything. (This is actually what happened when Teensyduino switched to the v11.3.1 compiler: a bunch of stuff stopped working for this reason.)

 

[mjs513]

@shawn

Lib was written based on @Paul's advice in msg #88:

In SDRAM_t4.h, I'd recommend declaring all the functions as static, and add an empty constexpr constructor. Calling class member functions from the startup hooks is technically not correct, but like our Serial, HardwareSerial, SPI, Wire classes this ought to be enough for it to work anyway. Would be nice if the SDRAM could be initialized from the middle or late startup hook, so a C++ class needing large memory could access it from its constructor.

But will keep what you said in mind if we go another way.

 

[defragster]

With the Fixed Pattern added back in here are results of runs with SDRAM at 166, 198, 221 with that Cap array wired to the pads - ALL worked for 4 reRead passes of ALL expected values The '#' shows the pattern tests as they pass, then dots for PRand tests:

--- START ------ with 4 reReads ... wait ...
#############............................................
No Errors. All memory tests passed (time 118.05 secs)

--- START ------ with 4 reReads ... wait ...
#############............................................
No Errors. All memory tests passed (time 110.50 secs)

--- START ------ with 4 reReads ... wait ...
#############............................................
No Errors. All memory tests passed (time 106.20)

Starting 3,000 reRead at 198 MHz now ... update later ... {wondered where this post was - unsent} : started at 3AM - more to go:

--- START ------ with 3000 reReads ... wait ...
#############................... { No errors - halting this long running test }

verbose USB1 showing - 24+ mins per PRandtest - on #9 out of #[0-43] - No errors expected and long run time:

>>**>> PseudoRand(9) Seed 74539250 with readRepeat 3000 ...
Test pseudo-random sequence Compare time secs 1461.08 with Error Cnt 0

 

[shawn]

Lib was written based on @Paul's advice in msg #88:

@shawn




But will keep what you said in mind if we go another way.

I'm not disagreeing that this might work now for many cases, I'm just pointing out that it relies on something that may or may not work later, or with all code written to use it. The advice is not correct from a strict C++ standpoint. It's important to keep in mind.

 

[defragster]

Gentleman:
... In the pictures above I see at least 10 to 30 pF of capacitance floating around with all the lead lengths and bits of 30 ga. wire.
MarkT I think it is about time you chimed in here to give some details on capacitance on PCB's. I know that Paul is very aware of this issue as he has a great deal of experience designing PCB's.

It been a very interesting conversation to read.
...
Ed

Interesting @sbfreddie! I wondered how much those flying wires added - and hoped for an EE'ducated guess/idea.
@MarkT / All - there is about 4" of 30 ga WW and parts of the legs of 4 PTH caps - and 13 soldered joints.
For scale that is the 10mm 1062 package. And soldering 4 wrapped joints seemed to change the result to more errors.

 

[defragster]

@jmarsh
The MPU entry is causing problems for any board without SDRAM (just hangs it someplace - assuming hang since nothing appears in SerMon) and the TMM with SDRAM installed and initialized I2C-SPI so far does not work (does the same thing as non-sdram boards - hangs it).

Now with all that said if I add one statement after Serial.begin();

if(CrashReport) {}

Everything works even on non-SDRAM boards like the T4.1. Just finished verifing it.

That is interesting - just as well I didn't try i2c/SPI here as this is present, as doing a memcmp( SDRAM,SDRAM) test faulted once:

void setup()
{
while (!Serial) ; // wait
pinMode(13, OUTPUT);
if ( CrashReport ) Serial.print( CrashReport );

 

[defragster]

RE: p#228 @shawn

The SDRAM_t4.h had to be adjusted and updated to this on first use:
//set NOCAP to 1 if cap C29 is removed static const uint8_t NOCAP = 1;

 

[shawn]

RE: p#228 @shawn

The SDRAM_t4.h had to be adjusted and updated to this on first use:
//set NOCAP to 1 if cap C29 is removed static const uint8_t NOCAP = 1;

Oh, good. you changed this to static. (See also my post.)
Try with constexpr too instead of const...

 

[defragster]

Oh, good. you changed this to static. (See also my post.)
Try with constexpr too instead of const...

@mjs513 wrote it before he had the board - and on first build I had to edit as the compile was telling me there was NO NOCAP ...

 

[PaulStoffregen]

I hope you understand how little 1.1 to 10 PicoFarads at 233MHz really is. This value could very easily be created by simply routing an etch close to power or ground.

Yes, exactly.

Earlier I've suggested starting with testing 4.7pF to 15pF without any explanation, other than using my imagination because we really don't have any documentation from NXP and just scant descriptions from their people answering questions on their own forum (see msg #49). But I didn't really say much about what I was imagining and why this capacitance range...

I'm imagining the SEMC hardware probably shares IP with the external memory interfaces of NXP's other iMX chips. Circumstantial evidence suggests that IP might be from their iMX6 chips, which support DDR (double data rate) memory. DDR has a DQS signal. My hunch is the mysterious DQS pin is playing a role similar to what it would do with DDR, even though we're using plain old "single data rate" SDRAM which lacks DQS.

DDR added a DQS signal because it cuts the allowed timing in half by transferring data on both clock edges, and it also supports higher clock rates. At those speed, the propagation delay just from PCB traces becomes a problem. DQS solves this problem by transmitting a strobe signal together with the data, where the strobe can be understood as pretty much a copy of the clock. If the data suffers minor delay in PCB traces or output pins driving whatever capacitive load is connected, hopefully the strobe gets the same delay. The receiver uses the edges on the strobe, rather than the original clock.

So my guess is IMXRT's SEMC for (single data rate) SDRAM might internally be using IP designed for DDR and it creates a false DQS signal because that IP wants to receive the incoming data with DQS rather than the original SDRAM clock. NXP gives us a register bit to configure whether DQS is routed fully within the chip, or routed to the EMC_39 pad. We know the internal routing works for 133 MHz but not 166 MHz. So there's good reason to believe the approx 5pF load of the EMC_39 pin delays the DQS signal slightly.

I recommended testing such small capacitors because 1pF to 22pF is about the amount of capacitive load expected from a PCB trace (1-3pF depending on PCB stackup) and 1 or 2 pins on a memory chip (likely ~5pF each).



Caveats.... This is all just guesswork about how NXP might have designed IMXRT. I have no direct knowledge of what's really inside the chip. So far, I've not personally designed anything yet with DDR memory, and I'm trying to keep this brief and informal, so please forgive me if I get some details wrong. Hopefully the idea and rationale are clear, even if I said something not perfectly correct.

 

[mjs513]

Ok guys - bad news.

Was testing the fix that worked for scanner and Kurts library with SDFat and Pauls ILI9341 both failed and hung the board so has to be something really subtle going on thats beyond me. Goning to try a different I2c test as well.

 

[defragster]

Now that CAP/SDRAM Test sketch usable - https://github.com/Defragster/EVKB_1060/tree/main/SDRAM_t4/examples/CapReadSDRAM and CAP cluster removed looking to start UART test with big buffers Serial#.addMemory with EXTSDMEM smalloc.

Simple UART CROSS Serial 1 and 2 to themselves works and then crossed to each other running fine with simple message each second the last 2 hours.

Pulled CAP array off DevBoard - put single leg trimmed 22pF on just shorter WW30ga and it passed the FIXED pattern tests - but failed all PRand with large % of errors. - then removed all to NO_CAP and clean pads on Devboard to move on.

 

[defragster]

Got twin UART Serial 1&2 test underway - they are connected to each other 1<>2. Each loop() manages one transfer on both UARTS.

Serial1.begin(6000000);
Serial2.begin(6000000);

Using DTCM buffers for 2KB transfers gives: Loops/sec=270 Bytes/sec=1,101,060 @XFEREACH 2000
> Using 4 buffers with ".addMemoryFor[Write,Read]() on both UARTS
> UARTS use lots of interrupts for 1.1 MB/sec transfers OUT and then IN and not affecting SDRAM integrity/utility
> Four other buffers: xfer, both reads, and another xfer copy for comparing
Display of content looks good on TyCommander, and after the 'variable lead text' the 2,000 char payload is doing NO ERROR memcmp()
>> for REF using DMAMEM for the buffers matches this speed of 270 or 271 loops/sec

No CrashReport - and no Cores edit since early [MPU in startup?] and SDRAM_t4 : Not using mallocSDRAM WIP in yet
With only .addMemoryFor[W,R] buffers in SDRAM the speed matches
Manually placing all 8 buffers in SDRAM results in a one loop/sec drop: Loops/sec=269 Bytes/sec=1096982
> NO ERRORS { except when I built at 221 MHz } :: Now running at SPEC 166 MHz
> quick test at 198 MHz still same speed

Transfer SPEW looks like (RED is part of the transmitted message to get stats - and skipped for memcmp():

1 Here ONE 746365 xb B[2032 2047] xb ERR[0 0] // milliseconds xferbuffer lengths[1 2] Xfer Errs [0 0]
!"#$%&'()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz
... // 20 lines removed
!"#$%&'()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxy

2 Here TWO L/s=269 B/s=1097251 // loop/sec and Bytes/sec
!"#$%&'()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz
... // 20 lines removed
!"#$%&'()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxy

> Last 'z' was missing with 91 char lines - it is NOW there with 2001 byte xfer
> Adding one xfer byte to 2001 dropped speed to 268/2 loops/sec

Code posted: https://github.com/Defragster/EVKB_1060/tree/main/mallocSDRAM/examples/UART_extmem
At ONLY 1.1 MB/sec not a real throughput test. 6Mbaud is only max 600KB/sec.
> Actual memory rate with Read and Write buffer in SDRAM is 2.2 MB/sec, still slow.
> could repeat on Added UARTS?
> compare of prior buffer done after 'Next' send to that should be 'free' while the UARTS work
> USB prints don't seem to slow anything
> Could use PJRC:UART page trick for Higher baud rate
> Since DMAMEM runs as well for buffers the size could increase into the 512MB area
> 8 hours ago I was falling asleep watching TV - now at 4AM time for Zzzz's
> Added MEM locate in the #ifdef: 2 Here TWO L/s=268 B/s=1097192 MEM=IN SDRAM

And Happy New Year to All.

 

[mjs513]

HAPPY NEW YEAR ALL

Just tested I2C with MPU changes I made and I can confirm that they are working on Wire and Wire1

With the following output

I2C Scanner
Scanning(Wire)...
Device found at address 0x29  (TSL2561,VL6180,TSL2561,TSL2591,BNO055,CAP1188)
Device found at address 0x3C  (SSD1306,DigisparkOLED)
Device found at address 0x62  (LidarLite,MCP4725,AtlasEzoORP,Qwiic Button)
done

Scanning(Wire1)...
Device found at address 0x4B  (ADS1115,TMP102,BNO080,Qwiic Keypad)
Device found at address 0x50  (EEPROM,FRAM)
Device found at address 0x68  (DS1307,DS3231,MPU6050,MPU9050,MPU9250,ITG3200,ITG3701,LSM9DS0,L3G4200D,Qwiic Button)
Device found at address 0x7C  (FRAM_ID)
done

Scanning(Wire2)...
No I2C devices found

Starting MB85...
Data_0: true
Data_1: 1.3574999571
Data_2: 314159
Data_3: 142
Data_4: 0x50
The Quick Brown Fox Jumped
...... ...... ......
Init Done - array loaded
...... ...... ......
Write Done - array loaded in FRAM chip
...... ...... ......

Hello

9250 Initialized
055 Initialized
Current Temperature: 24 C

Calibration status values: 0=uncalibrated, 3=fully calibrated
080 InitRotation vector enabled
Output in form i, j, k, real, accuracy
ialized
LL3 Init
ialized C+
MPU9250: 0.062250, 0.119712, -11.044621, 0.002131, -0.017311, -0.013050, 0.000000, 0.000000, 0.000000

BNO055: X: 359.94 Y: -2.44 Z: -1.19


LidarLiteV3: 178, 178, 174, 177, 174, 177, 174, 177, 173, 177

MB85 Read Done - array loaded with read data
...... ...... ......
Data: true, 1.3574999571,  314159, 142, 0x50
Data_5: The Quick Brown Fox

MPU9250: 0.062250, 0.134077, -11.054197, 0.002397, -0.017044, -0.011718, 0.000000, 0.000000, 0.000000

BNO055: X: 359.94 Y: -2.44 Z: -1.19

BNO080: -0.00,0.00,-1.00,0.08,1.58,

LidarLiteV3: 175, 174, 177, 174, 177

MB85 Read Done - array loaded with read data
...... ...... ......
Data: true, 1.3574999571,  314159, 142, 0x50
Data_5: The Quick Brown Fox

MPU9250: 0.057462, 0.119712, -11.049409, 0.001598, -0.017843, -0.012517, 11.549399, 34.500603, -0.176881

BNO055: X: 359.94 Y: -2.44 Z: -1.19


LidarLiteV3: 175, 174, 177, 174

MB85 Read Done - array loaded with read data
...... ...... ......
Data: true, 1.3574999571,  314159, 142, 0x50
Data_5: The Quick Brown Fox

Now to play with SPI again in case I messed it up. But it is strange that SPI works with @KurtE's ILI9341_t3n library but not the ILI9341 Highly Optimized version or SDFat. Confused

 

[KurtE]

with @KurtE's ILI9341_t3n library but not the ILI9341 Highly Optimized

WAGs - Timing differences? Memory locations touched?

 

[mjs513]

Could very well be. But I did notice that when I power on I get 9 blinks:

9 Blinks = ARM JTAG DAP Init ErrorThe ARM JTAG DAP was detected (4 blinks) but could not be initialized. This error is rather unlikely by hardware! However, a software crash resulting in hard fault or CPU lockup (typically very early in startup) can result in this error.

So looks like startup error. But if I add back in

if(CrashReport) {}

it works again - so something getting shifted around.

EDIT: The plot thickens; If I change the nocache to 4mb instead of 2m it SPI seems to be working again:

SCB_MPU_RBAR = 0x81C00000 | REGION(i++); // SEMC: SDRAM, NAND, SRAM, etc
    SCB_MPU_RASR = MEM_NOCACHE | READWRITE | NOEXEC | SIZE_4M;

Still have to see if that works on a t4.1

EDIT2: Just tested on a T4.1 and I get those 9 blinks again so something going on thats beyond me I think but at least have a work around to at least test with for now. Yes it works with if(CrashReport) is included.

 

[defragster]

WAGs - Timing differences? Memory locations touched?

Very Odd! Does a LST file show anything interesting with CrashReport moving things around?
The startup.c here should be checked/changed in case it isn't what it was a week ago.

UART UPDATE about 6 hours later { 22886 seconds 6.357 hrs) - msg size right and ERR count ZERO's
> Loop reate steady 268/sec and 1,097,728 B/sec using 8 SDRAM buffers.:

1 Here ONE 22886492 xb B[2046 2050] xb ERR[0 0]
!"#$%&'()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz

2 Here TWO L/s=268 B/s=1097728 MEM=IN SDRAM
!"#$%&'()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz

<edit> To add = using only 8 2K buffers (16KB) - the 32KB cache might be helping here - keeping DMAMEM times the same as DTCM.

With the memcmp() working the debug SPEW can go away and can use larger buffers and transfers and will array the UARTS and add more.

 

[mjs513]

@defragster
Here are the 2 lsts, There are differences but not smart enough to know where the issue is. Not seeing it.

 

[BriComp]

If it helps here are the (EDIT) PARTIAL differences found by a batch file compare:

Comparing files graphicstestnw.ino.lst and GRAPHICSTEST.INO.LST
***** graphicstestnw.ino.lst
C:\Users\Merli\AppData\Local\Temp\arduino\sketches\6E5F598F2A7229BC99A0A74D12499BEA/graphicstest.ino.elf:     file format elf32
-littlearm
***** GRAPHICSTEST.INO.LST
C:\Users\Merli\AppData\Local\Temp\arduino\sketches\E10A26BDED997C2D7261962E20CF77E0/graphicstest.ino.elf:     file format elf32
-littlearm
*****

***** graphicstestnw.ino.lst
60001426:       bf00            nop
60001428:       20004ac0        .word   0x20004ac0
6000142c:       20004f20        .word   0x20004f20
60001430 <ResetHandler2>:
***** GRAPHICSTEST.INO.LST
60001426:       bf00            nop
60001428:       200052c0        .word   0x200052c0
6000142c:       20005720        .word   0x20005720
60001430 <ResetHandler2>:
*****

***** graphicstestnw.ino.lst
        startup_early_hook(); // must be in FLASHMEM, as ITCM is not yet initialized!
60001442:       f000 f943       bl      600016cc <startup_default_early_hook>
        PMU_MISC0_SET = 1<<3; //Use bandgap-based bias currents for best performance (Page 1175)
***** GRAPHICSTEST.INO.LST
        startup_early_hook(); // must be in FLASHMEM, as ITCM is not yet initialized!
60001442:       f000 fd41       bl      60001ec8 <startup_default_early_hook>
        PMU_MISC0_SET = 1<<3; //Use bandgap-based bias currents for best performance (Page 1175)
*****

***** graphicstestnw.ino.lst
        configure_cache();
60001516:       f000 f8e1       bl      600016dc <configure_cache>
        _VectorsRam[14] = pendablesrvreq_isr;
***** GRAPHICSTEST.INO.LST
        configure_cache();
60001516:       f000 fcdf       bl      60001ed8 <configure_cache>
        _VectorsRam[14] = pendablesrvreq_isr;
*****

***** graphicstestnw.ino.lst
        usb_pll_start();    
6000154e:       f000 f953       bl      600017f8 <usb_pll_start>
        CCM_ANALOG_PFD_528_SET = (1 << 31) | (1 << 23) | (1 << 15) | (1 << 7);
***** GRAPHICSTEST.INO.LST
        usb_pll_start();    
6000154e:       f000 fd51       bl      60001ff4 <usb_pll_start>
        CCM_ANALOG_PFD_528_SET = (1 << 31) | (1 << 23) | (1 << 15) | (1 << 7);
*****

***** graphicstestnw.ino.lst
        set_arm_clock(F_CPU);
60001564:       f000 fae4       bl      60001b30 <__set_arm_clock_veneer>
        CCM_CCGR1 |= CCM_CCGR1_PIT(CCM_CCGR_ON);
***** GRAPHICSTEST.INO.LST
        set_arm_clock(F_CPU);
60001564:       f000 fedc       bl      60002320 <__set_arm_clock_veneer>
        CCM_CCGR1 |= CCM_CCGR1_PIT(CCM_CCGR_ON);
*****

***** graphicstestnw.ino.lst
    if(!sdram_init()) {
600015ac:       f000 fad4       bl      60001b58 <__sdram_init_veneer>
600015b0:       b988            cbnz    r0, 600015d6 <ResetHandler2+0x1a6>
***** GRAPHICSTEST.INO.LST
    if(!sdram_init()) {
600015ac:       f000 fec4       bl      60002338 <__sdram_init_veneer>
600015b0:       b988            cbnz    r0, 600015d6 <ResetHandler2+0x1a6>
*****

***** graphicstestnw.ino.lst
600015ec:       4829            ldr     r0, [pc, #164]  ; (60001694 <ResetHandler2+0x264>)
600015ee:       f000 fab7       bl      60001b60 <__sm_set_pool_veneer>
        analog_init();
600015f2:       f000 fa65       bl      60001ac0 <analog_init>
        pwm_init();
600015f6:       f000 faab       bl      60001b50 <__pwm_init_veneer>
        tempmon_init();
600015fa:       f000 f92b       bl      60001854 <tempmon_init>
        startup_middle_hook();
600015fe:       f000 f867       bl      600016d0 <startup_default_middle_hook>
// Returns the number of milliseconds since your program started running.
***** GRAPHICSTEST.INO.LST
600015ec:       4829            ldr     r0, [pc, #164]  ; (60001694 <ResetHandler2+0x264>)
600015ee:       f000 fe9b       bl      60002328 <__sm_set_pool_veneer>
        analog_init();
600015f2:       f000 fe63       bl      600022bc <analog_init>
        pwm_init();
600015f6:       f000 febf       bl      60002378 <__pwm_init_veneer>
        tempmon_init();
600015fa:       f000 fd29       bl      60002050 <tempmon_init>
        startup_middle_hook();
600015fe:       f000 fc65       bl      60001ecc <startup_default_middle_hook>
// Returns the number of milliseconds since your program started running.
*****

***** graphicstestnw.ino.lst
        usb_init();
60001608:       f000 f9bc       bl      60001984 <usb_init>
6000160c:       6823            ldr     r3, [r4, #0]
***** GRAPHICSTEST.INO.LST
        usb_init();
60001608:       f000 fdba       bl      60002180 <usb_init>
6000160c:       6823            ldr     r3, [r4, #0]
*****

***** graphicstestnw.ino.lst
        startup_debug_reset();
60001614:       f000 f860       bl      600016d8 <startup_debug_reset>
        startup_late_hook();
60001618:       f000 f85c       bl      600016d4 <startup_default_late_hook>
        __libc_init_array();
6000161c:       f000 fa94       bl      60001b48 <____libc_init_array_veneer>
        main();
60001620:       f000 fa7e       bl      60001b20 <__main_veneer>
        while (1) asm("WFI");
***** GRAPHICSTEST.INO.LST
        startup_debug_reset();
60001614:       f000 fc5e       bl      60001ed4 <startup_debug_reset>
        startup_late_hook();
60001618:       f000 fc5a       bl      60001ed0 <startup_default_late_hook>
        __libc_init_array();
6000161c:       f000 fe98       bl      60002350 <____libc_init_array_veneer>
        main();
60001620:       f000 fe92       bl      60002348 <__main_veneer>
        while (1) asm("WFI");
*****

***** graphicstestnw.ino.lst
60001628:       400d8000        .word   0x400d8000
6000162c:       00008fb0        .word   0x00008fb0
60001630:       60001c78        .word   0x60001c78
60001634:       00000000        .word   0x00000000
60001638:       20004ac0        .word   0x20004ac0
6000163c:       6000ac2c        .word   0x6000ac2c
60001640:       20000000        .word   0x20000000
60001644:       20004800        .word   0x20004800
60001648:       000072cd        .word   0x000072cd
6000164c:       e000e400        .word   0xe000e400
***** GRAPHICSTEST.INO.LST
60001628:       400d8000        .word   0x400d8000
6000162c:       00008fe0        .word   0x00008fe0
60001630:       60002498        .word   0x60002498
60001634:       00000000        .word   0x00000000
60001638:       200052c0        .word   0x200052c0
6000163c:       6000b47c        .word   0x6000b47c
60001640:       20000000        .word   0x20000000
60001644:       20005000        .word   0x20005000
60001648:       000075e5        .word   0x000075e5
6000164c:       e000e400        .word   0xe000e400
*****

***** graphicstestnw.ino.lst
60001654:       400ac000        .word   0x400ac000
60001658:       00006cc1        .word   0x00006cc1
6000165c:       20004e70        .word   0x20004e70
60001660:       00006cc5        .word   0x00006cc5
60001664:       20200000        .word   0x20200000
***** GRAPHICSTEST.INO.LST
60001654:       400ac000        .word   0x400ac000
60001658:       00006ccd        .word   0x00006ccd
6000165c:       20005670        .word   0x20005670
60001660:       00006cd1        .word   0x00006cd1
60001664:       20200000        .word   0x20200000
*****

***** graphicstestnw.ino.lst
60001684:       80000000        .word   0x80000000
60001688:       2000459a        .word   0x2000459a
6000168c:       80000000        .word   0x80000000
60001690:       20004e74        .word   0x20004e74
60001694:       20004dfc        .word   0x20004dfc
60001698:       2018101b        .word   0x2018101b
***** GRAPHICSTEST.INO.LST
60001684:       80000000        .word   0x80000000
60001688:       20004e56        .word   0x20004e56
6000168c:       80000000        .word   0x80000000
60001690:       20005674        .word   0x20005674
60001694:       200055fc        .word   0x200055fc
60001698:       2018101b        .word   0x2018101b
*****

Resync Failed.  Files are too different.
***** graphicstestnw.ino.lst
600016c8:       20070000        .word   0x20070000
600016cc <startup_default_early_hook>:
FLASHMEM void startup_default_early_hook(void) {}
600016cc:       4770            bx      lr
600016ce:       bf00            nop
600016d0 <startup_default_middle_hook>:
600016d0:       4770            bx      lr
600016d2:       bf00            nop
600016d4 <startup_default_late_hook>:
600016d4:       4770            bx      lr
600016d6:       bf00            nop
600016d8 <startup_debug_reset>:
FLASHMEM void startup_debug_reset(void) { __asm__ volatile("nop"); }
600016d8:       bf00            nop
600016da:       4770            bx      lr
600016dc <configure_cache>:
{
600016dc:       b410            push    {r4}
        SCB_MPU_CTRL = 0; // turn off MPU
600016de:       f04f 23e0       mov.w   r3, #3758153728 ; 0xe000e000
600016e2:       2100            movs    r1, #0
        SCB_MPU_RBAR = 0x00000000 | REGION(i++); //https://developer.arm.com/docs/146793866/10/why-does-the-cortex-m7-initiate-
axim-read-accesses-to-memory-addresses-that-do-not-fall-under-a-defined-mpu-region
600016e4:       2410            movs    r4, #16
        SCB_MPU_RASR = SCB_MPU_RASR_TEX(0) | NOACCESS | NOEXEC | SIZE_4G;
600016e6:       4833            ldr     r0, [pc, #204]  ; (600017b4 <configure_cache+0xd8>)
        SCB_MPU_CTRL = 0; // turn off MPU
600016e8:       f8c3 1d94       str.w   r1, [r3, #3476] ; 0xd94
        SCB_MPU_RBAR = 0x00000000 | REGION(i++); //https://developer.arm.com/docs/146793866/10/why-does-the-cortex-m7-initiate-
axim-read-accesses-to-memory-addresses-that-do-not-fall-under-a-defined-mpu-region
600016ec:       f8c3 4d9c       str.w   r4, [r3, #3484] ; 0xd9c
        SCB_MPU_RBAR = 0x00000000 | REGION(i++); // ITCM
600016f0:       2411            movs    r4, #17
        SCB_MPU_RASR = SCB_MPU_RASR_TEX(0) | NOACCESS | NOEXEC | SIZE_4G;
600016f2:       f8c3 0da0       str.w   r0, [r3, #3488] ; 0xda0
        SCB_MPU_RASR = MEM_NOCACHE | READONLY | SIZE_512K;
600016f6:       4830            ldr     r0, [pc, #192]  ; (600017b8 <configure_cache+0xdc>)
        SCB_MPU_RBAR = 0x00000000 | REGION(i++); // ITCM
600016f8:       f8c3 4d9c       str.w   r4, [r3, #3484] ; 0xd9c
        SCB_MPU_RBAR = 0x00000000 | REGION(i++); // trap NULL pointer deref
600016fc:       2412            movs    r4, #18
        SCB_MPU_RASR = MEM_NOCACHE | READONLY | SIZE_512K;
600016fe:       f8c3 0da0       str.w   r0, [r3, #3488] ; 0xda0
        SCB_MPU_RASR =  DEV_NOCACHE | NOACCESS | SIZE_32B;
60001702:       482e            ldr     r0, [pc, #184]  ; (600017bc <configure_cache+0xe0>)
        SCB_MPU_RBAR = 0x00000000 | REGION(i++); // trap NULL pointer deref
60001704:       f8c3 4d9c       str.w   r4, [r3, #3484] ; 0xd9c
        SCB_MPU_RBAR = ((uint32_t)&_ebss) | REGION(i++); // trap stack overflow
60001708:       4a2d            ldr     r2, [pc, #180]  ; (600017c0 <configure_cache+0xe4>)
        SCB_MPU_RASR =  DEV_NOCACHE | NOACCESS | SIZE_32B;
6000170a:       f8c3 0da0       str.w   r0, [r3, #3488] ; 0xda0
        SCB_MPU_RBAR = 0x00200000 | REGION(i++); // Boot ROM
6000170e:       4c2d            ldr     r4, [pc, #180]  ; (600017c4 <configure_cache+0xe8>)
        SCB_MPU_RBAR = ((uint32_t)&_ebss) | REGION(i++); // trap stack overflow
60001710:       f042 0215       orr.w   r2, r2, #21
        SCB_MPU_RASR = MEM_CACHE_WT | READONLY | SIZE_128K;
60001714:       482c            ldr     r0, [pc, #176]  ; (600017c8 <configure_cache+0xec>)
        SCB_MPU_RBAR = 0x00200000 | REGION(i++); // Boot ROM
60001716:       f8c3 4d9c       str.w   r4, [r3, #3484] ; 0xd9c
        SCB_MPU_RASR = MEM_CACHE_WT | READONLY | SIZE_128K;
6000171a:       f8c3 0da0       str.w   r0, [r3, #3488] ; 0xda0
        SCB_MPU_RBAR = 0x20000000 | REGION(i++); // DTCM
6000171e:       4c2b            ldr     r4, [pc, #172]  ; (600017cc <configure_cache+0xf0>)
        SCB_MPU_RASR = MEM_NOCACHE | READWRITE | NOEXEC | SIZE_512K;
60001720:       482b            ldr     r0, [pc, #172]  ; (600017d0 <configure_cache+0xf4>)
        SCB_MPU_RBAR = 0x20000000 | REGION(i++); // DTCM
60001722:       f8c3 4d9c       str.w   r4, [r3, #3484] ; 0xd9c
        SCB_MPU_RASR = MEM_NOCACHE | READWRITE | NOEXEC | SIZE_512K;
60001726:       f8c3 0da0       str.w   r0, [r3, #3488] ; 0xda0
        SCB_MPU_RASR = SCB_MPU_RASR_TEX(0) | NOACCESS | NOEXEC | SIZE_32B;
6000172a:       4c2a            ldr     r4, [pc, #168]  ; (600017d4 <configure_cache+0xf8>)
        SCB_MPU_RBAR = ((uint32_t)&_ebss) | REGION(i++); // trap stack overflow
6000172c:       f8c3 2d9c       str.w   r2, [r3, #3484] ; 0xd9c
        SCB_MPU_RBAR = 0x20200000 | REGION(i++); // RAM (AXI bus)
60001730:       4829            ldr     r0, [pc, #164]  ; (600017d8 <configure_cache+0xfc>)
        SCB_MPU_RASR = MEM_CACHE_WBWA | READWRITE | NOEXEC | SIZE_1M;
60001732:       4a2a            ldr     r2, [pc, #168]  ; (600017dc <configure_cache+0x100>)
        SCB_MPU_RASR = SCB_MPU_RASR_TEX(0) | NOACCESS | NOEXEC | SIZE_32B;
60001734:       f8c3 4da0       str.w   r4, [r3, #3488] ; 0xda0
        SCB_MPU_RBAR = 0x20200000 | REGION(i++); // RAM (AXI bus)
60001738:       f8c3 0d9c       str.w   r0, [r3, #3484] ; 0xd9c
        SCB_MPU_RASR = MEM_CACHE_WBWA | READWRITE | NOEXEC | SIZE_1M;
6000173c:       f8c3 2da0       str.w   r2, [r3, #3488] ; 0xda0
        SCB_MPU_RASR = DEV_NOCACHE | READWRITE | NOEXEC | SIZE_64M;
60001740:       f502 22a0       add.w   r2, r2, #327680 ; 0x50000
        SCB_MPU_RBAR = 0x40000000 | REGION(i++); // Peripherals
60001744:       4c26            ldr     r4, [pc, #152]  ; (600017e0 <configure_cache+0x104>)
        SCB_MPU_RASR = DEV_NOCACHE | READWRITE | NOEXEC | SIZE_64M;
60001746:       320c            adds    r2, #12
        SCB_MPU_RBAR = 0x60000000 | REGION(i++); // QSPI Flash
60001748:       4826            ldr     r0, [pc, #152]  ; (600017e4 <configure_cache+0x108>)
        SCB_MPU_RBAR = 0x40000000 | REGION(i++); // Peripherals
6000174a:       f8c3 4d9c       str.w   r4, [r3, #3484] ; 0xd9c
        SCB_MPU_RASR = DEV_NOCACHE | READWRITE | NOEXEC | SIZE_64M;
6000174e:       f8c3 2da0       str.w   r2, [r3, #3488] ; 0xda0
        SCB_MPU_RASR = MEM_CACHE_WBWA | READONLY | SIZE_16M;
60001752:       4a25            ldr     r2, [pc, #148]  ; (600017e8 <configure_cache+0x10c>)
        SCB_MPU_RBAR = 0x60000000 | REGION(i++); // QSPI Flash
60001754:       f8c3 0d9c       str.w   r0, [r3, #3484] ; 0xd9c
        SCB_MPU_RBAR = 0x70000000 | REGION(i++); // FlexSPI2
***** GRAPHICSTEST.INO.LST
600016c8:       20070000        .word   0x20070000
600016cc <CrashReportClass::clear()>:
}
FLASHMEM
static void cleardata(struct arm_fault_info_struct *info)
{
        info->len = 0;
600016cc:       4b20            ldr     r3, [pc, #128]  ; (60001750 <CrashReportClass::clear()+0x84>)
600016ce:       2000            movs    r0, #0
{
600016d0:       b430            push    {r4, r5}
        info->ipsr  = 0;
600016d2:       e9c3 0020       strd    r0, r0, [r3, #128]      ; 0x80
        info->cfsr  = 0;
        info->hfsr  = 0;
600016d6:       e9c3 0022       strd    r0, r0, [r3, #136]      ; 0x88
        info->mmfar = 0;
        info->bfar  = 0;
600016da:       e9c3 0024       strd    r0, r0, [r3, #144]      ; 0x90
        info->ret = 0;
        info->xpsr  = 0;
600016de:       e9c3 0026       strd    r0, r0, [r3, #152]      ; 0x98
        info->crc = 0;
600016e2:       f8c3 00a8       str.w   r0, [r3, #168]  ; 0xa8
static inline void arm_dcache_flush_delete(void *addr, uint32_t size)
{
        uint32_t location = (uint32_t)addr & 0xFFFFFFE0;
        uint32_t end_addr = (uint32_t)addr + size;
        asm volatile("": : :"memory");
        asm("dsb");
600016e6:       f3bf 8f4f       dsb     sy
        do {
                SCB_CACHE_DCCIMVAC = location;
600016ea:       f04f 23e0       mov.w   r3, #3758153728 ; 0xe000e000
600016ee:       4919            ldr     r1, [pc, #100]  ; (60001754 <CrashReportClass::clear()+0x88>)
600016f0:       4a19            ldr     r2, [pc, #100]  ; (60001758 <CrashReportClass::clear()+0x8c>)
600016f2:       f8c3 1f70       str.w   r1, [r3, #3952] ; 0xf70
600016f6:       f8c3 2f70       str.w   r2, [r3, #3952] ; 0xf70
                location += 32;
        } while (location < end_addr);
        asm("dsb");
600016fa:       f3bf 8f4f       dsb     sy
        asm("isb");
600016fe:       f3bf 8f6f       isb     sy
        arm_dcache_flush_delete(info, sizeof(*info));
        SRC_SRSR = SRC_SRSR; // zeros all write-1-to-clear bits
60001702:       4b16            ldr     r3, [pc, #88]   ; (6000175c <CrashReportClass::clear()+0x90>)
                        arm_dcache_flush((void *)bc, sizeof(struct crashreport_breadcrumbs_struct));
                }
        }
        static uint32_t checksum(volatile const void *data, int len) {
                volatile const uint16_t *p = (volatile const uint16_t *)data;
                uint32_t a=1, b=0; // Adler Fletcher kinda, len < 720 bytes
60001704:       4601            mov     r1, r0
  *(volatile uint32_t *)(&bc->bitmask) = 0;
60001706:       4c16            ldr     r4, [pc, #88]   ; (60001760 <CrashReportClass::clear()+0x94>)
60001708:       2201            movs    r2, #1
        SRC_SRSR = SRC_SRSR; // zeros all write-1-to-clear bits
6000170a:       689d            ldr     r5, [r3, #8]
                while (len > 0) {
6000170c:       f604 74dc       addw    r4, r4, #4060   ; 0xfdc
60001710:       609d            str     r5, [r3, #8]
        SRC_GPR5 = 0;
60001712:       6318            str     r0, [r3, #48]   ; 0x30
                volatile const uint16_t *p = (volatile const uint16_t *)data;
60001714:       4b13            ldr     r3, [pc, #76]   ; (60001764 <CrashReportClass::clear()+0x98>)
  *(volatile uint32_t *)(&bc->bitmask) = 0;
60001716:       f844 0c1c       str.w   r0, [r4, #-28]
                        a += *p++;
6000171a:       4618            mov     r0, r3
6000171c:       3302            adds    r3, #2
6000171e:       8800            ldrh    r0, [r0, #0]
                while (len > 0) {
60001720:       42a3            cmp     r3, r4
                        a += *p++;
60001722:       fa12 f280       uxtah   r2, r2, r0
                        b += a;
60001726:       4411            add     r1, r2
                while (len > 0) {
60001728:       d1f7            bne.n   6000171a <CrashReportClass::clear()+0x4e>
                        len -= 2;
                }
                a = a & 65535;
6000172a:       b292            uxth    r2, r2
  *(volatile uint32_t *)(&bc->checksum) = checksum(bc, 28);
6000172c:       4b0c            ldr     r3, [pc, #48]   ; (60001760 <CrashReportClass::clear()+0x94>)
                b = b & 65535;
                return a | (b << 16);
6000172e:       ea42 4201       orr.w   r2, r2, r1, lsl #16
60001732:       f8c3 2fdc       str.w   r2, [r3, #4060] ; 0xfdc
        asm("dsb");
60001736:       f3bf 8f4f       dsb     sy
                SCB_CACHE_DCCMVAC = location;
6000173a:       f04f 23e0       mov.w   r3, #3758153728 ; 0xe000e000
6000173e:       4a09            ldr     r2, [pc, #36]   ; (60001764 <CrashReportClass::clear()+0x98>)
60001740:       f8c3 2f68       str.w   r2, [r3, #3944] ; 0xf68
        asm("dsb");
60001744:       f3bf 8f4f       dsb     sy
        asm("isb");
60001748:       f3bf 8f6f       isb     sy
*****

 

[defragster]

@defragster
Here are the 2 lsts, There are differences but not smart enough to know where the issue is. Not seeing it.

Thx - looking - with codecompare:
> lots of tiny offset diffs as expected
> CrashReport is of course FLASHMEM resident - so that unused code in FLASH is made present - but does nothing - except maybe check/init the HIGH 128 DWORDS in DMAMEM to start.

I wonder if DEBUG build works differently - or the LST's might compare better. There are some big code pieces shifted that makes it hard to see what might be real changes.

 

[KurtE]

@defragster
Here are the 2 lsts, There are differences but not smart enough to know where the issue is. Not seeing it.

Maybe try doing some of the !crashreport stuff in the sketch directly...
That is:

FLASHMEM
CrashReportClass::operator bool()
{
    struct arm_fault_info_struct *info = (struct arm_fault_info_struct *)0x2027FF80;
    if (isvalid(info)) return true;
    return false;
}


FLASHMEM
static int isvalid(const struct arm_fault_info_struct *info)
{
    uint32_t i, crc;
    const uint32_t *data, *end;


    if (info->len != sizeof(*info) / 4) return 0;
    data = (uint32_t *)info;
    end = data + (sizeof(*info) / 4 - 1);
    crc = 0xFFFFFFFF;
    while (data < end) {
        crc ^= *data++;
        for (i=0; i < 32; i++) crc = (crc >> 1) ^ (crc & 1)*0xEDB88320;
    }
    if (crc != info->crc) return 0;
    return 1;
}

Like what any of this changes that make a difference?
Just have your main do:

struct arm_fault_info_struct *info = (struct arm_fault_info_struct *)0x2027FF80;
    if (info->len != sizeof(*info) / 4) {}

Does it change the behavior...
If not add more... Maybe it is the FLASHMEM?

 

[mjs513]

@PaulStoffregen - @defragster - @Rezo - @Dogbone06
Think I found the solution that works for the Teensy 4.1, Teensy 4.0 and the dev board as well. Used alot of asm volatile("NOP")'s. Also put an ifdef around that 2mb region.

Pushed changes to the repo.

Have to test if lib still works - probably not.

 

[Dogbone06]

@PaulStoffregen - @defragster - @Rezo - @Dogbone06
Think I found the solution that works for the Teensy 4.1, Teensy 4.0 and the dev board as well. Used alot of asm volatile("NOP")'s. Also put an ifdef around that 2mb region.

Pushed changes to the repo.

Have to test if lib still works - probably not.

Good work finding a workaround!

Do we have any idea at all to why this issue occurs? Is it like a memory issue or what exactly may it be?