Teensy 4.1 ST7789 Uncanny Eyes works great for 20sec then DACCVIOL MMARVALID Crash

Status
Not open for further replies.

rcouineau

Member
Hi,

This is my first post here. I bought my first Teensy, a 4.1, to do a Uncanny Eyes pumpkin for Halloween. I now realize it might easier to go with a 3.X version but that's the one I bought. I already had a 240x240 ST7789 display (https://www.aliexpress.com/item/33038943606.html?spm=a2g0s.9042311.0.0.27424c4dkuECJo). The display seems fully functional as I succeed in running various examples with it, including with the Teensy 4.1. Interestingly enough, the display requires SPI_MODE3 to function. After research I installed the PaulStoffregen/ST7735_t3 library that has in the examples a version of the Uncanny Eyes supposed to work with Teensy 4 and ST7789 (code attached View attachment uncannyEyes_async_st7789_240x240.ino View attachment config.h).

I was able to make the whole thing work in commenting out the DumpMemoryInfo() that made the whole thing crash. The eye is successfully starring at me for about 20 seconds then the program crashes and debug mentions DACCVIOL MMARVALID and a few other errors

Here's the dump of my serial debug window:


Code:
M:\Robotics\Arduino\Sketches\Uncanny_Eyes-master\uncannyEyes_async_st7789_240x240\uncannyEyes_async_st7789_240x240.ino Aug 13 2020 13:38:01


********
 T4 connected Serial_tt ******* debug_tt port


>>> Reason for 'reset': 1 IPP_RESET_B :: done Reason

F_CPU==600000000   F_BUS==150000000 FreeMem(); 4293316428

Init

Create display #0

ST7789_t3::init mode: c
Init ST77xx display #0

Rotate

done

Display logo

$0: Using Frame buffer
 700 :     76 105202    0: 120  255: 472
 699 :     76 105202    0: 120    0: 488
 698 :     76 105202    0: 120    0: 488
 701 :     75 106605    0: 120    0: 488
 703 :     75 106605    0: 120    0: 488
 702 :     75 106605    0: 120    0: 488
 704 :     75 106605    0: 120    0: 488
 706 :     74 108045    0: 120    0: 488
 708 :     74 108045    0: 120    0: 488
 710 :     73 109525    0: 120    0: 488
 711 :     73 109525    0: 120    0: 488
 713 :     73 109525    0: 120    0: 488
 714 :     72 111047    0: 120    0: 488
 716 :     72 111047    0: 120    0: 488
 718 :     71 112611    0: 120    0: 488
 720 :     71 112611    0: 120    0: 488
 723 :     70 114219    0: 120  247: 484
 725 :     70 114219  255:   0  255:   0
 727 :     69 115875    0: 120    0: 488
 728 :     69 115875    0: 120    0: 488
 729 :     69 115875    0: 120    0: 488
 730 :     69 115875    0: 120    0: 488
 732 :     68 117579    0: 120    0: 488
 733 :     68 117579    0: 120    0: 488
 735 :     68 117579    0: 120    0: 488
 737 :     67 119334    0: 120    0: 488
 738 :     67 119334    0: 120    0: 488
 739 :     67 119334    0: 120    0: 488
 734 :     68 117579    0: 120    0: 488
 736 :     67 119334    0: 120    0: 488
 741 :     66 121142    0: 120    0: 488
 742 :     66 121142    0: 120    0: 488
 743 :     66 121142    0: 120    0: 488
 745 :     65 123006    0: 120    0: 488
 746 :     65 123006    0: 120    0: 488
 747 :     65 123006    0: 120    0: 488
 749 :     64 124928    0: 120    0: 488
 751 :     64 124928    0: 120    0: 488
 752 :     64 124928    0: 120    0: 488
 750 :     64 124928    0: 120    0: 488
 724 :     70 114219    0: 120    0: 488
 722 :     71 112611    0: 120    0: 488
 721 :     71 112611    0: 120    0: 488
 719 :     71 112611  255:   0  255:   0
 717 :     72 111047    0: 120  244: 488
 715 :     72 111047    0: 120    0: 488
 705 :     75 106605    0: 120    0: 488
 694 :     77 103836    0: 120    0: 488
 690 :     78 102505    0: 120    0: 488
 687 :     79 101207    0: 120    0: 488
 685 :     79 101207    0: 120    0: 488
 683 :     80  99942    0: 120    0: 488
 681 :     80  99942    0: 120    0: 488
 679 :     81  98708    0: 120    0: 488
 677 :     81  98708    0: 109  255: 464
 675 :     82  97504  255:   0  255:   0
 684 :     79 101207    0: 120    0: 488
 682 :     80  99942    0: 120    0: 488
 680 :     80  99942    0: 120    0: 488
 678 :     81  98708    0: 120    0: 488
 676 :     81  98708    0: 120    0: 488
 674 :     82  97504    0: 120    0: 488
 672 :     82  97504    0: 120    0: 488
 671 :     83  96330    0: 120    0: 488
 669 :     83  96330    0: 120    0: 488
 667 :     83  96330    0: 120    0: 488
 666 :     84  95183    0: 120    0: 488
 664 :     84  95183    0:   1  255: 368
 663 :     84  95183  255:   0  255:   0
 661 :     85  94063    0:  91  255: 447
 658 :     86  92969    0: 120    0: 488
 655 :     86  92969    0: 120    0: 488
 652 :     87  91901    0: 120    0: 488
 648 :     88  90856    0: 120    0: 488
 645 :     89  89835    0: 120    0: 488
 642 :     89  89835    0: 120    0: 488
 639 :     90  88837    0: 120    0: 488
 637 :     90  88837    0: 120    0: 488
 636 :     91  87861    0: 120    0: 488
 635 :     91  87861    0: 120    0: 488
 633 :     91  87861  255:   0  255:   0
 632 :     92  86906    0: 115  255: 462
 631 :     92  86906    0: 120    0: 488
 629 :     92  86906  255:   0  255:   0
 625 :     93  85971  255:   0  255:   0
 621 :     94  85057    0:  85  255: 448
 616 :     95  84162    0: 120    0: 488
 610 :     97  82426    0: 120    0: 488
 605 :     98  81585    0: 120    0: 488
 601 :     99  80761    0: 120    0: 488
 597 :    100  79953    0: 120    0: 488
 594 :    101  79162    0: 120    0: 488
 591 :    101  79162    0: 120    0: 488
 588 :    102  78386    0: 120    0: 488
 584 :    103  77625    0: 120    0: 488
 583 :    103  77625    0: 120    0: 488
 582 :    103  77625    0: 120    0: 488
 580 :    104  76878    0: 120    0: 488
 577 :    105  76146    0: 120    0: 488
 574 :    105  76146    0: 120    0: 488
 572 :    106  75428    0: 120    0: 488
 571 :    106  75428    0: 120    0: 488
 570 :    106  75428    0: 120    0: 488
 568 :    107  74723    0: 120    0: 488
 566 :    107  74723    0: 120    0: 488
 564 :    108  74031    0: 120    0: 488
 563 :    108  74031    0: 120    0: 488
 565 :    107  74723    0: 120    0: 488
 569 :    106  75428    0: 120    0: 488
 573 :    105  76146    0: 120    0: 488
 575 :    105  76146    0: 120    0: 488
 576 :    105  76146    0: 120    0: 488
 578 :    104  76878    0: 120    0: 488
 581 :    104  76878    0: 103  251: 478
 585 :    103  77625    0: 120    0: 488
 586 :    102  78386    0: 120    0: 488
 589 :    102  78386    0: 120    0: 488
 592 :    101  79162    0: 120    0: 488
 595 :    100  79953    0: 120    0: 488
 598 :    100  79953    0: 120    0: 488
 603 :     98  81585    0: 120    0: 488
 608 :     97  82426    0: 120    0: 488
 611 :     97  82426    0: 120    0: 488
 614 :     96  83285    0: 120    0: 488
 618 :     95  84162    0: 120    0: 488
 620 :     94  85057    0: 120    0: 488
 622 :     94  85057    0: 120    0: 488
 624 :     94  85057    0: 120    0: 488
 627 :     93  85971    0: 120    0: 488
 630 :     92  86906    0: 120    0: 488
 634 :     91  87861    0: 120    0: 488
 641 :     90  88837    0: 120    0: 488
 643 :     89  89835    0: 120    0: 488
 651 :     87  91901    0: 120    0: 488
 653 :     87  91901    0: 120    0: 488
 654 :     86  92969    0: 120    0: 488
 657 :     86  92969    0: 120    0: 488
 659 :     85  94063    0: 120    0: 488
 660 :     85  94063    0: 120    0: 488
 662 :     85  94063    0: 120    0: 488
 665 :     84  95183    0: 120    0: 488
19



Fault irq 3

 stacked_r0 ::  00000002

 stacked_r1 ::  C0000000

 stacked_r2 ::  01020304

 stacked_r3 ::  200062C0

 stacked_r12 ::  20201062

 stacked_lr ::  000006BD

 stacked_pc ::  0000062C

 stacked_psr ::  210E0000

 _CFSR ::  00000082

      (DACCVIOL) Data Access Violation

      (MMARVALID) MemMange Fault Address Valid

 _HFSR ::  40000000

      (FORCED) Forced Hard Fault

 _DFSR ::  00000000

 _AFSR ::  00000000

 _BFAR ::  200062C4

 _MMAR ::  200062C4

 debug_tt (weak) default :: customize with 'void Debug_Dump()' in user code.

 userDebugDumptt() in debug_tt  ___ 



 F_CPU=300000000

 >>>> Debug Fault   >>>> debug_fault   >>>> TYPE:T_4

debug_tt Info:



 >>>> Debug Fault   >>>> TYPE:T_4

--- Faulted >>>>  Execution Halted.



For more info - print it in sketch :: Debug_Dump(void)

I may be doing something obviously wrong but your help would be highly appreciated.

Thanks!
 
I may have to see if I can find both of my 240x240 displays to see if I can reproduce. Defragster - do you recognize this error traceback?
May have to find my ones without CS pins. Not sure how well it could work without CS pins to select between. But will take a look.

Also debug_tt.h - I am not sure if I have the most recent copy?

What version of Arduino, Teensyduino... are you using?
 
That just enabled fault handler - something faulting memory it seems:
Code:
 _CFSR ::  00000082
      (DACCVIOL) Data Access Violation
      (MMARVALID) MemMange Fault Address Valid

 _HFSR ::  40000000
      (FORCED) Forced Hard Fault

I came and went with debug_tt for lack of traction and evolving new hardware ... then GDB support popped up ... though that is more complex and limiting to start ...

Anyhow - setting up for the GDB support might allow back trace with live CODE to see the offending code?

Or just look for a creeping or abused pointer or array reference in the code ...
 
Thanks for the help! I have the same error without any display attached...

It seems to have something to do with memory. DumpMemoryInfo works only if I comment out the part to "fill rest of DTCM with known pattern".

Code:
void DumpMemoryInfo() {
Serial.println("Start DumpMemoryInfo");
#if defined(__IMXRT1062__)
  Serial.println("IMXRT1062 Defined");
  uint32_t flexram_config = IOMUXC_GPR_GPR17;
  Serial.printf("IOMUXC_GPR_GPR17:%x IOMUXC_GPR_GPR16:%x IOMUXC_GPR_GPR14:%x\n", flexram_config, IOMUXC_GPR_GPR16, IOMUXC_GPR_GPR14);
  Serial.printf("Initial Stack pointer: %x\n", &_estack);
  uint32_t dtcm_size = 0;
  uint32_t itcm_size = 0;
  for (; flexram_config; flexram_config >>= 2) {
    if ((flexram_config & 0x3) == 0x2) dtcm_size += 32768;
    else if ((flexram_config & 0x3) == 0x3) itcm_size += 32768;
  }
  Serial.printf("ITCM allocated: %u  DTCM allocated: %u\n", itcm_size, dtcm_size);
  Serial.printf("ITCM init range: %x - %x Count: %u\n", &_stext, &_etext, (uint32_t)&_etext - (uint32_t)&_stext);
  Serial.printf("DTCM init range: %x - %x Count: %u\n", &_sdata, &_edata, (uint32_t)&_edata - (uint32_t)&_sdata);
  Serial.printf("DTCM cleared range: %x - %x Count: %u\n", &_sbss, &_ebss, (uint32_t)&_ebss - (uint32_t)&_sbss);
  Serial.println("Now fill rest of DTCM with known pattern"); Serial.flush(); //
  //Guess of where it is safe to fill memory... Maybe address of last variable we have defined - some slop...
  //for (uint32_t *pfill = (&_ebss + 1); pfill < (&itcm_size - 10); pfill++) {
    //*pfill = 0x01020304;  // some random value
  //}
#endif
}

The serial debug output is:

Code:
Serial started

Start DumpMemoryInfo

IMXRT1062 Defined

IOMUXC_GPR_GPR17:aaaaaaaf IOMUXC_GPR_GPR16:200007 IOMUXC_GPR_GPR14:aa0000
Initial Stack pointer: 20070000
ITCM allocated: 65536  DTCM allocated: 458752
ITCM init range: 0 - dab8 Count: 55992
DTCM init range: 20000000 - 20002820 Count: 10272
DTCM cleared range: 20002820 - 200062c0 Count: 15008
Now fill rest of DTCM with known pattern

I'm running Arduino 1.8.12 and Teensy Loader 1.53.
I REALLY appreciate your help :)
 
Just to clarify, when I comment out the filling of the balance of the DTCM, DumpMemoryInfo runs but the Teensy 4.1 still crashes after about 20 seconds with the mentioned errors, with or without the display.
 
something odd that the DumpMemoryInfo() DTCM fill not working as coded and used for test by author [ KurtE or mjs513 ? ]
 
Evening All
Code:
_CFSR ::  00000082
      (DACCVIOL) Data Access Violation
      (MMARVALID) MemMange Fault Address Valid
Usually means some sort of subscript or array out of range - haven't seen that for awhile. Funny that DumpMemoryInfo() isn't working.

As @KurtE said would have to find and pull out the displays to see what is happening but not sure I have 2 without CS pins. Sounds vaguely familar.
 
I have it running on my machine now.

Comment out the code or call to EstimateStackUsage and as you already have seen and likewise my fill of memory
Code:
#if 0
  // Guess of where it is safe to fill memory... Maybe address of last variable we have defined - some slop...
  for (uint32_t *pfill = (&_ebss + 1); pfill < (&itcm_size - 10); pfill++) {
    *pfill = 0x01020304;  // some random value
  }
#endif
It was working on T4, but it faults when I try to access the memory just above the last variable zeroed...

What I was trying to do is fill all memory above the un init variables (we zero) and up to about 10 words before a stack variable in the function... So later I can walk back down and find the first place that is not that value...

Looks like something in memory map is different now...
 
FWIW, I tried it on my T4.0 setup, and it hangs there as well until I commented out the DumpMemoryInfo using Teensydunio 1.52 or 1.53, but if I fall back to Teensydunio 1.50 or 1.51, DumpMemoryInfo runs, but the display does not work:

  • Teensydunio 1.50, Ardunio 1.811: DumpMemoryInfo works, but ST7789 library does not work;
  • Teensydunio 1.51, Arduino 1.8.12: DumpMemoryInfo works, but ST7789 library does not work;
  • Teensydunio 1.52, Arduino 1.8.12: DumpMemoryInfo hangs, bug ST778 library works if DumpMemoryInfo commented out; (and)
  • Teensydunio 1.53, Arduino 1.8.13: DumpMemoryInfo hangs, bug ST778 library works if DumpMemoryInfo commented out.

For this particular setup, I am using the Adafruit display, but I have used the no-name displays without a CS pin in the past:

I have changed some of the pin defaults, to match the setup for Teensy 3.x on the left screen and avoid the Audio Shield CS pins (6, 10):
  • Left CS: 22/A8 (was -1)
  • Right CS: 0 (was -1)
  • Right DC: 25 (was 6)
  • Blink pin 3 (was not defined)
  • IRIS_MIN 500 (was 120)
  • IRIS_MAX 700 (was 720)
 
Last edited:
By way of rehash I repeated what has been done with one a ST7789 240x240 Waveshare display with a CS pin (https://www.amazon.com/gp/product/B07MH93747/ref=ppx_yo_dt_b_search_asin_title?ie=UTF8&psc=1) on a T4.1 using the default pins for SPI.

Results the same as previously posted.

I then hacked up the non-asynch uncannyeyes sketch to use a ST7789 and other than having the logo and eye displayed on the screen didn't see any hangs, i.e., memory faults. I did let run for about 15 minutes.
 
That works now! I indeed upgraded to Teensydunio 1.53 and Arduino 1.8.13, commented out the call to EstimateStackUsage and to the fill of memory. I didn't need to change the pin defaults. You folks rock :)
 
That works now! I indeed upgraded to Teensydunio 1.53 and Arduino 1.8.13, commented out the call to EstimateStackUsage and to the fill of memory. I didn't need to change the pin defaults. You folks rock :)

Just commented out the calls to dumpmemoryinfo and estimated stack usage as well with my display and seems to work with no issue.

@KurtE - just remembered something with a problem that I was having with @Frank B's T4_PowerButton lib - it was hanging when i was calling flexRamInfo but Frank made a correction for the T4.1. Can't remember what it was though :)

EDIT: Here is where the discussion started: https://forum.pjrc.com/threads/59875-TeensyDuino-Updates?p=235651&viewfull=1#post235651
 
Last edited:
I am sort of curious why it crashes. I ran it under GDB and it died accessing the memory location right after the last zeroed memory location, which I would think was still valid... May have to look to see if we some of the startup code changed to mark these locations as not valid...
 
Morning all,

Yes looked like @FrankB did change his library to start looking at memory after some gap... So that works now.
BUT I also made some new #define at start (commented out), which enables this code on T4.x.
I also turned off the debug info showing in the eyes (by default - It is under a #if defined)

Also removed the debug_tt stuff that was commented out.

Note: I also changed the default back to the version that has CS pins...

Issued PR: https://github.com/PaulStoffregen/ST7735_t3/pull/20
 
Morning all,

Yes looked like @FrankB did change his library to start looking at memory after some gap... So that works now.
BUT I also made some new #define at start (commented out), which enables this code on T4.x.
I also turned off the debug info showing in the eyes (by default - It is under a #if defined)

Also removed the debug_tt stuff that was commented out.

Note: I also changed the default back to the version that has CS pins...

Issued PR: https://github.com/PaulStoffregen/ST7735_t3/pull/20

@KurtE
Sorry for the delay had to run out. Anyway ran UncannyEyes with and without defining DEBUG_MEMORY both cases worked like a charm. So fix is confirmed.

Saw that you changed memory start from:
Code:
(&_ebss + 1)
to
Code:
(&_ebss + 32)
which I also saw in Frank's code.
 
@KurtE
Sorry for the delay had to run out. Anyway ran UncannyEyes with and without defining DEBUG_MEMORY both cases worked like a charm. So fix is confirmed.

Saw that you changed memory start from:
Code:
(&_ebss + 1)
to
Code:
(&_ebss + 32)
which I also saw in Frank's code.
Thansks, always good to have things like this double checked.

Probably may be awhile before next teensyduino beta and hopefully by then Paul will pick up some of the current PRs...

And yep I went to the link you mentioned and noticed he had made that change and since it worked.... Figured good enough :D
 
Cool that GDB was useful in getting into the code! As noted - very odd tested and used and shipped code that worked triggering to fail? Weird that it fails on the END, but the edit is change to START *pointer?

@MichaelMeissner - TD 1.50 was before the MemoryFault was enabled in startup. Interesting it fails though - something did shift.

@rcouineau: If you can point to the source used for debug_tt it might inspire my updating a version for quick testing.
 
Cool that GDB was useful in getting into the code! As noted - very odd tested and used and shipped code that worked triggering to fail? Weird that it fails on the END, but the edit is change to START *pointer?

@MichaelMeissner - TD 1.50 was before the MemoryFault was enabled in startup. Interesting it fails though - something did shift.

@rcouineau: If you can point to the source used for debug_tt it might inspire my updating a version for quick testing.
Note, I have my own version of the source, so that I can change pins, and possibly add other things. I just went back to the previous teensy installations I have on the system to see where the breakage was.
 
As a way to "give back", I translated into 240x240 a bunch of 128x128 eyes that I found as well as the python script to create them. Here it is View attachment convert.zip . With upscaling the eyeMap from 128x128 to 240x240, it grows by more than 350% to the 400kB range and it takes quite some finessing with the image sizes to make it fit in the DTCM. Is there anyway to store it at least in part in RAM2 and/or Flash? Thanks
 
As a way to "give back", I translated into 240x240 a bunch of 128x128 eyes that I found as well as the python script to create them. Here it is View attachment 21407 . With upscaling the eyeMap from 128x128 to 240x240, it grows by more than 350% to the 400kB range and it takes quite some finessing with the image sizes to make it fit in the DTCM. Is there anyway to store it at least in part in RAM2 and/or Flash? Thanks

If the images are constant, you can use 'FLASHMEM' macro as part of the declaration. Note however, you may need to use a bounce buffer or flush the cache if the display function tries to use DMA to update the image directly. A bounce buffer is a buffer in the memory that DMA doesn't need to be careful with the flash. You would first copy part of the image into the bounce buffer, and then do the DMA operation on that buffer. When the DMA is done, you reuse the buffer to do the next part. I would imagine you might want 2 bounce buffers, so you can fill one while the DMA engine is using the other.

The Teensy 4.0 has 2M of flash memory minus the 64K used for EEPROM and restoring the blink image and minus the size of the initialized text and data that is copied into SRAM at sketch startup. The Teensy 4.1 has 8M of flash memory.

If buffer is dynamic, on the Teensy 4.1 you could use one or two 4M psram chips if they are soldered onto the Teensy 4.1, but you will need to consider the memory speed and caching behavior (i.e. you might need to do more of a deep dive into the internals of the 1062). Note, the psram is volatile, and it is reset upon power cycles. On the Teensy 4.1, you can also solder a persistent flash memory (usually 8M) instead of one of the psram chips. This memory is preserved across power cycles.

The newer Adafruit boards (M4 Express, Hallowing M4, Monster M4SK, etc.) that support the 240x240 uncanny eyes variants provide a persistent flash memory as a removable disk drive, and you can change the eyes and adjust the configurations without having to reprogram the board. The Circuit Python that runs on Teensy 4.0/4.1 allows a small area of the flash memory to be used in this fashion.
 
Last edited:
This is beyond my level of expertise and it works like that so that's good enough for me for now LOL However this seems to be the bottleneck for more elaborate graphic designs or bigger screens... You may add the 240x240 eye maps and the Python script in your library examples if you want. Thanks again for all the help!
 
Status
Not open for further replies.
Back
Top