Forum Rule: Always post complete source code & details to reproduce any issue!
Page 6 of 6 FirstFirst ... 4 5 6
Results 126 to 141 of 141

Thread: Teensy 4.1 Freeze (~1786ms)

  1. #126
    Senior Member+ Frank B's Avatar
    Join Date
    Apr 2014
    Location
    Germany
    Posts
    8,281
    Still no freezes. Startet the game now. It is incredible fast. I can't remember that it was that fast on my 286 or 386.
    I miss the sound..

  2. #127
    Senior Member
    Join Date
    Apr 2020
    Location
    DFW area in Texas
    Posts
    244
    I can confirm that the freeze in my updated TeensyMIDIPolySynth (both the full version & the pared down version) no longer occurs after making Frank B's code change (as posted/clarified by defragster) !!

    Thanks & congratulations for successfully finding the apparent cause/fix . . . well done, Frank B !!

    Mark J Culross
    KD5RXT

  3. #128
    Senior Member
    Join Date
    Jul 2014
    Posts
    3,154
    out of curiosity, what is FlexSPI2 used for? only psram?
    Yes, I use EXTMEM in my code.

  4. #129
    Senior Member+ defragster's Avatar
    Join Date
    Feb 2015
    Posts
    13,915
    The QSPI pads for PSRAM on the bottom are mapped to the MCU with those FlexSPI2 commands for MPU (protection ) control. Their privilege's and purpose.

    From the .ld:
    Code:
    	FLASH (rwx): ORIGIN = 0x60000000, LENGTH = 7936K
    	ERAM (rwx):  ORIGIN = 0x70000000, LENGTH = 16384K
    The FlexSPI handles the address mapping and the interface - and that gives the rules for the MCU to enforce:
    Code:
    	SCB_MPU_RBAR = 0x60000000 | REGION(i++); // QSPI Flash
    	SCB_MPU_RASR = MEM_CACHE_WBWA | READONLY | SIZE_16M;
    
    	SCB_MPU_RBAR = 0x70000000 | REGION(i++); // FlexSPI2
    	SCB_MPU_RASR = MEM_CACHE_WBWA | READWRITE | NOEXEC | SIZE_16M;

  5. #130
    Senior Member
    Join Date
    Jul 2014
    Posts
    3,154
    Quote Originally Posted by defragster View Post
    The QSPI pads for PSRAM on the bottom are mapped to the MCU with those FlexSPI2 commands for MPU (protection ) control. Their privilege's and purpose.
    Question now is, is 1.7s freeze related with use of PSRAM or with doubling (or 'wrong'/huge size) in startup.c ?
    IOW, do all freeze code use PSRAM?
    Is access to PSRAM cached?, does this cache then be cleared as with DMAMEM?

  6. #131
    Senior Member+ Frank B's Avatar
    Join Date
    Apr 2014
    Location
    Germany
    Posts
    8,281
    Good Morning

    @Mark: Glad that is works now for you!
    @Tim thank you.
    Quote Originally Posted by WMXZ View Post
    Question now is, is 1.7s freeze related with use of PSRAM or with doubling (or 'wrong'/huge size) in startup.c ?
    In Doom, yes, it uses PSRAM (but I don't think it's used in the menu, where the freeze happened) and the doubling was problem.
    Quote Originally Posted by WMXZ View Post
    IOW, do all freeze code use PSRAM?
    Good question. Let's wait what Udo says.
    There are a few more questionable MPU settings - maybe we have to modify them too.

    My current theory is, there is no need for accesses of the FlexSPI2 memory for the 1.7 sec. freezes to occur. It's just the MPU setting. Perhaps it leads to some wrong bits, MPU-internally.
    If Udo says he has *still* freezes, it may be a good Idea to review the other settings, too.

    Quote Originally Posted by WMXZ View Post
    Is access to PSRAM cached?,
    does this cache then be cleared as with DMAMEM?
    Yes and yes. If there are accesses by the hardware (i.e. DMA) you need the same cache handling as with DMAMEM.
    If you write to the flash on FlexSPI2, you need it there, too.
    Last edited by Frank B; 03-17-2021 at 07:23 AM.

  7. #132
    Senior Member+ Frank B's Avatar
    Join Date
    Apr 2014
    Location
    Germany
    Posts
    8,281
    Walter, you were right when you said that we don't know the chip.
    ARM updated a pretty hidden detail in the MPU.
    It looks like the Cortex-M7 does not have the priority regions the M4 had, and uses subregion-bits instead.

  8. #133
    Senior Member+ defragster's Avatar
    Join Date
    Feb 2015
    Posts
    13,915
    When QSPI work first started it seems it was wrongly perceived to be working before it really was - thanks to the cache.

    When the MCU is told to cache a region it does - it can be written and read back - even if not backed by any real resource. Not sure what happens when MCU needs to flush the cache for new data?

    The Frank commented lines marked out a big region - was wondering if that overlapped anything - or it anything doing access there might be timing out doing I/O?
    > not sure what the 256M includes starting at :: 0x70000000

    Could be interesting to LA or Scope those pins if there was a 'Freeze' sketch that didn't use PSRAM with the old lines in place. Or maybe the DOOM sketch while not or before the PSRAM is used.

  9. #134
    Member
    Join Date
    Mar 2021
    Location
    Germany
    Posts
    39
    Heureka, Frank's solution seems to work

    Using Frank's development IDE, including his fix, proposed and described above yesterday, my sketch runs since 16.5 hours without Freezes. I am using a version (v161) which has all features and functions re-enabled, which were one by one disabled in the sequence of fault finding over the last few days. That means all libraries and user ISRs, output is on USBSerial too.

    Well, I guess we should still be a little cautious, because we have seen before that there can be many hours between Freezes and therefore I assume, we could only safely call this issue resolved, after we understood, why not only Frank's solution but also disabling any use of USBSerial are/were fixes.

    I will continue to look out for Freezes during the next few days, just in case there still some "hiding". I will report back, hopefully just with good news.

    Thank you Frank, well done!! Your hard work is truly appreciated!!!

  10. #135
    Senior Member+ Frank B's Avatar
    Join Date
    Apr 2014
    Location
    Germany
    Posts
    8,281
    Quote Originally Posted by UdoZ View Post

    Well, I guess we should still be a little cautious, because we have seen before that there can be many hours between Freezes and therefore I assume, we could only safely call this issue resolved, after we understood, why not only Frank's solution but also disabling any use of USBSerial are/were fixes.
    I think it's because USB uses the cache-functions of the core which cause the cache-contents to be written to the RAM.
    Somewhere here in this operation the MPU checks if everything is OK and it *somehow* controls the cache. But it could not do its job right because it was somehow "confused".

    I whish I knew it more exactly, but there is no documentation about the inner working of the MPU.

  11. #136
    Senior Member+ Frank B's Avatar
    Join Date
    Apr 2014
    Location
    Germany
    Posts
    8,281
    Indeed I spent several hours to identify the problem.
    BIG thanks to Jean-Marc Harvengt who told me that it happens in Doom. It was luck that I had the VGA board "just in time"..

    @Udo: yes, more testing is good.
    You can use your version of Arduino/Teensyduino now, if you coment out these two lines in startup.c

    Paul told me that a Teensyduino Beta 8 with this fix is comming, too.

    I still find it very fascinating that the freeze lasts so exactly (exact cylce-count) the same time. This can only be explained if the MPU has a counter built in somehow. But what is it for?
    And, I wish to know why it - first - seemed to be fixed with GCC9 - and later not... I can't remember what I did...
    Last edited by Frank B; 03-17-2021 at 10:14 AM. Reason: CPU->MPU

  12. #137
    Senior Member+ Frank B's Avatar
    Join Date
    Apr 2014
    Location
    Germany
    Posts
    8,281
    I investigated this further:
    It sems not to be the 2nd region.
    It even freezes if I set the size of the remaining single region to a value larger then 16M

    Code:
        SCB_MPU_RBAR = 0x70000000 | REGION(i++); // FlexSPI2
        SCB_MPU_RASR = MEM_CACHE_WBWA | READWRITE | NOEXEC | SIZE_32M;
    freezes.

    @Paul, any Idea?
    Last edited by Frank B; 03-18-2021 at 02:40 PM. Reason: fix sizes

  13. #138
    This may or may not relate:

    I have been fighting an issue with a program that uses the SmartMatrix and Adafruit_GFX libraries with Teensy4.1, and if you are looking at pins via a scope or equivalent, needs no other hardware or mods to the teensy4.1 (or presumably 4.0). It is very repeatable, as the (I'm assuming) DMA that is refreshing the led panel freezes (the LED panel goes blank) at pretty much the same points in a 33.9 second long loop when the problem is not seen, and one time I measured with a stopwatch 41.1 seconds when the problem is seen. I am temporarily bedridden so I have not been able to put a scope on it, but it is in a relatively simple test loop of the LED display. ANYTHING seems to make it go away, including your fix in startup.c, as well as adding even a DigitalWriteFast(14,LOW) anywhere in loop(), or most any other change in loop(), much less adding timing routines to it. Other changes, like adding in InternalTemperature.h, and initializing it but nothing else changes the places where the blanking occurs, and causes it after 20 minutes to do something (change the DMA update frequency?) so the display looks like it's shimmering a bit. Again, sorry for the lack of specificity.

    Note: The problem is sudden blanking of the screen, however the program does dim out to a blank screen and holds that blank for 1 full second.

    In any case, it seems to be an easier program to debug this problem on, if it is the same, because it happens so regularly. On the other hand, it might be a problem, in that it is 'delicate', in that even adding a single writeDigitalFast() in loop() makes it go away.

    Here is a link to the thread on the PixelMatrix site, although it has no answers except the one you have identified above: https://community.pixelmatix.com/t/w...ng-problem/951

    Here is a link to a zip of the source code (two files) https://www.dropbox.com/s/aosoyiqor9...moMod.zip?dl=0

    Here is a link to a video showing the problem: https://www.dropbox.com/s/iimsnh7hi3...oblem.mp4?dl=0
    And a link to a video showing what it should be: https://www.dropbox.com/s/bjtabbfkfs...added.mp4?dl=0

    Here is a link to the basic info on the product, with a diagram of what pins are used: http://docs.pixelmatix.com/SmartMatrix/shield-t4.html
    and a link to github for source code, and schematics under extras/hardware/teensy4:https://github.com/pixelmatix/SmartMatrix

    If this isn't the same issue, it certainly is an odd issue, that seems to be compiler related.
    Last edited by CoreyCoop; 04-05-2021 at 11:14 PM.

  14. #139
    BTW: Does the 'fix' in startup.c break anything else?

  15. #140
    Senior Member
    Join Date
    Apr 2020
    Location
    DFW area in Texas
    Posts
    244
    Quote Originally Posted by CoreyCoop View Post
    BTW: Does the 'fix' in startup.c break anything else?
    @CoreyCoop:

    My evidence is only circumstantial, but after adding the fix to startup.c manually (to remedy freezes on the latest version of my TeensyMIDIPolySynth that I'm tinkering with), I have not seen any detectable adverse effects from the addition/fix.

    Mark J Culross
    KD5RXT

  16. #141
    Member
    Join Date
    Feb 2017
    Location
    Chicago, IL
    Posts
    31
    Quote Originally Posted by Frank B View Post
    @Paul:
    Is this correct (?)
    Code:
        SCB_MPU_RBAR = 0x70000000 | REGION(i++); // FlexSPI2
        SCB_MPU_RASR = MEM_CACHE_WBWA | READONLY | NOEXEC | SIZE_256M;
    
        SCB_MPU_RBAR = 0x70000000 | REGION(i++); // FlexSPI2
        SCB_MPU_RASR = MEM_CACHE_WBWA | READWRITE | NOEXEC | SIZE_16M;
    The 2nd has higher priority - so the first is not active. But can the MPU handle this correctly?

    Don't want produce the next false positive here...take this with care: Doom is running for 30minutes now, with the first commented.-out, and without freeze. That's unusual..
    Thanks Frank! Wish I had seen this thread earlier Last week, I was experiencing similar issues inside a loop that "normally" took around 22ms for ~500 iterations, experiencing "freezes" every 50 or 60th iteration for around 1.6 seconds in duration (I never timed it precisely as it was the sheer size of the freeze that was of concern, plus I could never reliably isolate which piece of code caused it). My app was large and I normally use platformIO / VSCode, but I was able to move a small subset of my code and reproduce in the Arduino IDE.

    What many hours of testing and attempted changes showed me was, it *only* happened if I compiled with Optimize: Fastest. At all other optimize levels, the freezes *never* occurred, but with Fastest they always occurred, immediately. I had been tempted to try GCC9, so I did - and the issue went away, I no longer experienced these freezes. Given other needs to move forward, I noted it as an anomaly to recall in the future, but moved on.

    Having now seen this thread, I just went back to my test sketch, confirmed I still have the "freeze" issue under GCC5 optimize: Fastest, then made the changes mentioned here, recomplied, and everything is fine. Great job! I'm not having issues with GCC9 and am definitely liking the improved compiler, warnings, etc. but it's nice to have the option vs. being forced down that path. Thanks again for discovering the root cause!

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •