Forum Rule: Always post complete source code & details to reproduce any issue!
Results 1 to 11 of 11

Thread: T4.1 - using PSRAM chip as RAM

  1. #1
    Senior Member
    Join Date
    Oct 2019
    Posts
    146

    T4.1 - using PSRAM chip as RAM

    I've tried searching the forum on this specific question for the last 40 mins but didn't come to an answer...

    I was wondering if it would be possible to allocate the 8Mb PSRAM chip to the system RAM without having to actually define in code where the variables are initialised?
    I'm using the LVGL graphics library and it's a bit heavy on RAM with what I am trying to do with it, and when I compile it, it require 620kb of RAM. Is there a way for the complier to automatically overflow RAM requirement onto the PSRAM chip?

    btw I mentioned RAM in this post 7 times

  2. #2
    Senior Member+ defragster's Avatar
    Join Date
    Feb 2015
    Posts
    12,224
    NINE - if you count the RAM in PSRAM.

    There is no provision to initialize or extend 'onboard RAM' directly onto the PSRAM chip. It is off chip and in non-contiguous address space - and runs not at CPU clock speed - but some clock cycles slower unless it is cached.

    Using EXTMEM will provide a pointer to PSRAM RAM - search for that perhaps to see examples, like this post :: psram-with-qspi-flash-on-teensy-4-1

  3. #3
    Senior Member
    Join Date
    Oct 2019
    Posts
    146
    Thanks!!

    I opened a thread over at the LVGL forum and after some help there we realised the images I was loading was being stored in RAM and not in Flash, so I am making a modification to change that - which should free up a lot of the RAM I was using.
    But, I will utilise the post you linked me to in order to increase the memory buffer LVGL uses and point it to the PSRAM - it might help with loading more objects on to make transitions on the screen smoother.

  4. #4
    Senior Member+ defragster's Avatar
    Join Date
    Feb 2015
    Posts
    12,224
    Moving that image data from RAM to FLASH would be using:
    Code:
    PROGMEM Variables - Variables defined with "PROGMEM" are placed only in the flash memory. They can be accessed normally.

  5. #5
    Senior Member
    Join Date
    Oct 2019
    Posts
    146
    Quote Originally Posted by defragster View Post
    Moving that image data from RAM to FLASH would be using:
    Code:
    PROGMEM Variables - Variables defined with "PROGMEM" are placed only in the flash memory. They can be accessed normally.
    Yep, that's exactly what was suggested to me. I applied that to both the image and to all of my font files. I went from 126% usage of RAM down to 60%!
    It was also suggested that I could move some heavy functions into flash, and I might do so if performance isn't impacted much:

    Code:
    myReturnType PROGMEM MyFunction( MyParameters ) {
    
         Code for Function....
    
    }

    If I want to address my code to PSRAM, so I just tell it to start from address 0x70000000? Or do I need to initialise and allocate a buffer first to external RAM such as
    Code:
    uint8_t gui_buff[1024 * 1024] EXTMEM;

  6. #6
    Senior Member+ defragster's Avatar
    Join Date
    Feb 2015
    Posts
    12,224
    For CODE run from FLASH best to use :: FLASHMEM
    Code:
    myReturnType FLASHMEM MyFunction( MyParameters ) {
    
         Code for Function....
    
    }
    That is new and not yet shown on the T_4.0 memory page linked above - Compile/Link has a problem when FLASHMEM and PROGMEM are used in the same compile unit when one is CODE and the other DATA

    <edit> : EXTMEM will never be initialized - it will just be a 'pointer' to space for use. Do a forum search and there should be other examples of use. I played with it before that was a thing and had to manually do the address. The EXTMEM is now in the linker '.ld' file and AFAIK allows allocation based on var/struct sizes to allocate sequentially/incrementally just like normal RAM/FLASH - but it will always be uninitialized as there is no copy from flash or other mechanism for that.

  7. #7
    Senior Member+ KurtE's Avatar
    Join Date
    Jan 2014
    Posts
    7,461
    I would never say never

    There are comments in cores\teeny4\startup.c:
    Code:
    		// TODO: zero uninitialized EXTMEM variables
    		// TODO: copy from flash to initialize EXTMEM variables
    		// TODO: set up for malloc_extmem()
    @PaulStoffregen - It one point we were going to add something like the malloc like you show in startup.c...

    Awhile ago I experimented some with doing some of this.. Example sketch:
    Code:
    #define COUNT_ALLOC 1250000
    
    
    //===============================================================================
    #if defined(ARDUINO_TEENSY41)
    extern "C" {
      extern uint8_t external_psram_size;
    }
    uint8_t *extmem_next_free = nullptr;  // set in startup NULL if we don't have external memory
    
    uint8_t *malloc_extmem(size_t cb_alloc) {
      if (!extmem_next_free) extmem_next_free = 0x70000000l + (uint32_t)(external_psram_size * 1048576);
            // lets allocate and return on 16 byte boundaries.
            uint8_t *ret_val = (uint8_t *)((uint32_t)(extmem_next_free - cb_alloc) & 0xfffffff0);
    
      // BUGBUG only comparing that we run of EXTERNAL memory not that we overrun pre allocated
      if (ret_val < 0x70000000l) return nullptr;
      extmem_next_free = ret_val;
      return ret_val;
    }
    #else
    uint8_t *malloc_extmem(size_t cb_alloc) {
      return nullptr;
    }
    #endif
    void free_extmem(char *p) {
      // we don't do any free yet
    }
    //===============================================================================
    
    
    extern uint8_t *malloc_extmem(size_t cb_alloc);
    
    uint16_t *buffer1;
    uint16_t *buffer2;
    uint16_t *buffer3;
    
    uint8_t *buffer_eat_upper_8mb = nullptr;
    void setup() {
      pinMode(13, OUTPUT);
      while (!Serial && millis() < 4000) ;
      Serial.begin(115200);
    
    #if defined(ARDUINO_TEENSY41)
      Serial.printf("External PSRAM size: %d\n", external_psram_size);
    //  if (external_psram_size == 16) buffer_eat_upper_8mb = malloc_extmem(1024*1024*8);
    #endif
      buffer1 = (uint16_t*)malloc_extmem(COUNT_ALLOC * sizeof(uint16_t));
      buffer2 = (uint16_t*)malloc_extmem(COUNT_ALLOC * sizeof(uint16_t));
      buffer3 = (uint16_t*)malloc_extmem(COUNT_ALLOC * sizeof(uint16_t));
      Serial.printf("Eat: %x B1:%x B2:%x B3: %x\n", (uint32_t)buffer_eat_upper_8mb, (uint32_t)buffer1, (uint32_t)buffer2, (uint32_t)buffer3);
      if (!buffer1 || !buffer2 || !buffer3) {
        Serial.println("Failed to allocate external memory");
        for (;;) {
          digitalWrite(13, !digitalRead(13));
          delay(250);
        }
      }
    }
    
    uint32_t loop_count = 0;
    void loop() {
      digitalWrite(13, !digitalRead(13));
      loop_count++;
      for (uint32_t i = 0; i < COUNT_ALLOC; i++) {
        buffer1[i] = i & 0xffff;
        buffer2[i] = loop_count & 0xffff;
        buffer3[i] = buffer1[i] ^ buffer2[i];
        if (buffer_eat_upper_8mb)buffer_eat_upper_8mb[i] = loop_count + i ;
      }
      uint32_t error_count = 0;
      for (uint32_t i = 0; i < COUNT_ALLOC; i++) {
        if ((buffer1[i] != (i & 0xffff)) || (buffer2[i] != (loop_count & 0xffff)) || (buffer3[i] != (buffer1[i] ^ buffer2[i]))) {
          error_count++;
          if (error_count < 5) Serial.printf("%u %x %x %x\n", i, buffer1[i], buffer2[i], buffer3[i]);
        }
      }
      Serial.printf("Pass %d error count: %d\n", loop_count, error_count);
    }
    Again as mentioned in previous threads, it would be also good to have the ability to malloc (or heapAlloc or ???) also out of DTCM for those sketches that don't use all of it as it is faster and for DMA makes it nice that you don't have to worry about flush/delete... the cache to work well with DMA...

  8. #8
    Senior Member+ defragster's Avatar
    Join Date
    Feb 2015
    Posts
    12,224
    Yes, Controlled heap alloc from DTCM and ERAM would both be cool.

    For :: // TODO: set up for malloc_extmem()

    With edit to imxrt1062_t42.ld could the compile allocated size of EXTMEM be published like for itcm, then the ERAM_heap size would be known:

    Using .bss.extram from:
    Code:
    	.bss.extram (NOLOAD) : {
    		*(.externalram)
    	} > ERAM
    Code:
    	_seram = ADDR(.bss.extram);
    	_eeram = ADDR(.bss.extram) + SIZEOF(.bss.extram);

  9. #9
    Senior Member+ KurtE's Avatar
    Join Date
    Jan 2014
    Posts
    7,461
    @Paul and @defragster -

    As you mentioned, we could probably know the size of what was previously allocated... But at the time I did the above sketch with a quick and dirty implementation of malloc_extmem(), we were in the release cycle for Teensyduino, and I knew the chances of getting a change to a load script was not high... I figured we might do that if/when we do the zero and copy portions of the todo...

    But I also wondered if we are talking about initialized portions of EXTMEM, of why we don't do that today with DMAMEM section?

  10. #10
    Senior Member+ defragster's Avatar
    Join Date
    Feb 2015
    Posts
    12,224
    Seems to work adding to ...\hardware\teensy\avr\cores\teensy4\imxrt1062_t4 1.ld:
    Code:
    	_seram = ADDR(.bss.extram);
    	_eeram = ADDR(.bss.extram) + SIZEOF(.bss.extram);
    And edited code p#7 below shows either with #if 0 small EXTMEM:
    Code:
    T:\tCode\RAM\KurtE_ERAM_Alloc\KurtE_ERAM_Alloc.ino Sep 15 2020 13:27:40
    ERAM start==0x70000000	 end==0x70000800
    EXTMEM 	 fluff==0x70000000[size=400]	 fluff2==0x70000400[size=400]
    External PSRAM size: 16
    Eat: 70800000 B1:7059da60 B2:7033b4c0 B3: 700d8f20
    Pass 1 error count: 0
    Pass 2 error count: 0
    or #if 1 larger EXTMEM:
    Code:
    T:\tCode\RAM\KurtE_ERAM_Alloc\KurtE_ERAM_Alloc.ino Sep 15 2020 13:27:56
    ERAM start==0x70000000	 end==0x70002000
    EXTMEM 	 fluff==0x70000000[size=1000]	 fluff2==0x70001000[size=1000]
    External PSRAM size: 16
    Eat: 70800000 B1:7059da60 B2:7033b4c0 B3: 700d8f20
    Pass 1 error count: 0
    Pass 2 error count: 0
    > Not getting error in code - not sure when that might happen
    > this was just proof of concept tracking the EXTMEM alloc - got rid of warnings on fixed 0x70000000l usage - no promise it is right
    Here with edits:
    Code:
    #define COUNT_ALLOC 1250000
    
    
    //===============================================================================
    #if defined(ARDUINO_TEENSY41)
    extern "C" {
      extern uint8_t external_psram_size;
    }
    uint8_t *extmem_next_free = nullptr;  // set in startup NULL if we don't have external memory
    extern unsigned long _seram;
    extern unsigned long _eeram;
    uint8_t *malloc_extmem(size_t cb_alloc) {
      if (!extmem_next_free) extmem_next_free = (uint8_t *)&_seram + (uint32_t)(external_psram_size * 1048576);
            // lets allocate and return on 16 byte boundaries.
            uint8_t *ret_val = (uint8_t *)((uint32_t)(extmem_next_free - cb_alloc) & 0xfffffff0);
    
      // BUGBUG only comparing that we run of EXTERNAL memory not that we overrun pre allocated
      if (ret_val < (uint8_t *)&_eeram) return nullptr;
      extmem_next_free = ret_val;
      return ret_val;
    }
    #else
    uint8_t *malloc_extmem(size_t cb_alloc) {
      return nullptr;
    }
    #endif
    void free_extmem(char *p) {
      // we don't do any free yet
    }
    //===============================================================================
    
    
    extern uint8_t *malloc_extmem(size_t cb_alloc);
    
    uint16_t *buffer1;
    uint16_t *buffer2;
    uint16_t *buffer3;
    
    #if 1
    EXTMEM uint32_t fluff[1024];
    EXTMEM uint32_t fluff2[1024];
    #else
    EXTMEM uint32_t fluff[256];
    EXTMEM uint32_t fluff2[256];
    #endif
    
    uint8_t *buffer_eat_upper_8mb = nullptr;
    void setup() {
      pinMode(13, OUTPUT);
      Serial.begin(115200);
      while (!Serial && millis() < 4000 );
      Serial.println("\n" __FILE__ " " __DATE__ " " __TIME__);
      Serial.printf( "ERAM start==0x%x\t end==0x%x\n", &_seram, &_eeram );
      Serial.printf( "EXTMEM \t fluff==0x%x[size=%x]\t fluff2==0x%x[size=%x]\n", fluff, sizeof(fluff), fluff2, sizeof(fluff2) );
    
    #if defined(ARDUINO_TEENSY41)
      Serial.printf("External PSRAM size: %d\n", external_psram_size);
      if (external_psram_size == 16) buffer_eat_upper_8mb = malloc_extmem(1024*1024*8);
    #endif
      buffer1 = (uint16_t*)malloc_extmem(COUNT_ALLOC * sizeof(uint16_t));
      buffer2 = (uint16_t*)malloc_extmem(COUNT_ALLOC * sizeof(uint16_t));
      buffer3 = (uint16_t*)malloc_extmem(COUNT_ALLOC * sizeof(uint16_t));
      Serial.printf("Eat: %x B1:%x B2:%x B3: %x\n", (uint32_t)buffer_eat_upper_8mb, (uint32_t)buffer1, (uint32_t)buffer2, (uint32_t)buffer3);
      if (!buffer1 || !buffer2 || !buffer3) {
        Serial.println("Failed to allocate external memory");
        for (;;) {
          digitalWrite(13, !digitalRead(13));
          delay(250);
        }
      }
    }
    
    uint32_t loop_count = 0;
    void loop() {
      digitalWrite(13, !digitalRead(13));
      loop_count++;
      for (uint32_t i = 0; i < COUNT_ALLOC; i++) {
        buffer1[i] = i & 0xffff;
        buffer2[i] = loop_count & 0xffff;
        buffer3[i] = buffer1[i] ^ buffer2[i];
        if (buffer_eat_upper_8mb)buffer_eat_upper_8mb[i] = loop_count + i ;
      }
      uint32_t error_count = 0;
      for (uint32_t i = 0; i < COUNT_ALLOC; i++) {
        if ((buffer1[i] != (i & 0xffff)) || (buffer2[i] != (loop_count & 0xffff)) || (buffer3[i] != (buffer1[i] ^ buffer2[i]))) {
          error_count++;
          if (error_count < 5) Serial.printf("%u %x %x %x\n", i, buffer1[i], buffer2[i], buffer3[i]);
        }
      }
      Serial.printf("Pass %d error count: %d\n", loop_count, error_count);
    }
    With two tiny single byte fluffs:
    ERAM start==0x70000000 end==0x70000002
    EXTMEM fluff==0x70000000[size=1] fluff2==0x70000001[size=1]
    With no fluff:
    T:\tCode\RAM\KurtE_ERAM_Alloc\KurtE_ERAM_Alloc.ino Sep 15 2020 14:07:40
    ERAM start==0x70000000 end==0x70000000
    Last edited by defragster; 09-15-2020 at 09:14 PM. Reason: Add smaller and NO EXTMEM compile allocs

  11. #11
    Senior Member+ defragster's Avatar
    Join Date
    Feb 2015
    Posts
    12,224
    Realized any big failure would render the rest of EXTMEM LOST after a single large failure, so when nullptr is returned on large alloc, undo the ptr* change that alloc would have used.

    Added some test in loop() to ask for block4's in decreasing size until they fail.
    output looks like this - numbers look funny?:
    Code:
    T:\tCode\RAM\KurtE_ERAM_Alloc\KurtE_ERAM_Alloc.ino Sep 15 2020 17:17:36
    ERAM start==0x70000000	 end==0x70002000
    EXTMEM 	 fluff==0x70000000[size=1000]	 fluff2==0x70001000[size=1000]
    External PSRAM size: 16
    Eat: 70800000 B1:7059da60 B2:7033b4c0 B3: 700d8f20
    
    Pass 1 error count: 0
     + + + + + + + + + + + + +
    	EXTMEM exhausted at Pass 15 alloc size: 0x8000 	 was @ 0x70008f20
    Try smaller size:0x4000	0x2000	 +!
    	EXTMEM exhausted at Pass 16 alloc size: 0x2000 	 was @ 0x70004f20
    Try smaller size:0x1000	 +!
    	EXTMEM exhausted at Pass 17 alloc size: 0x1000 	 was @ 0x70002f20
    Try smaller size:0x800	0x400	 +!
    	EXTMEM exhausted at Pass 18 alloc size: 0x400 	 was @ 0x70002720
    Try smaller size:0x200	 +!
    	EXTMEM exhausted at Pass 19 alloc size: 0x200 	 was @ 0x70002320
    Try smaller size:0x100	 +!
    	EXTMEM exhausted at Pass 20 alloc size: 0x100 	 was @ 0x70002120
    Try smaller size:0x80	 +!
    	EXTMEM exhausted at Pass 21 alloc size: 0x80 	 was @ 0x70002020
    Try smaller size:0x40	0x20	0x10	 +!
    	EXTMEM exhausted at Pass 22 alloc size: 0x10 	 was @ 0x70002000
    Try smaller size:0x8	0x4	0x2	0x1	 . . . . . . . . . . . . . . . . . . . . . . . . . . . .
    Pass 51 error count: 0
    Code for that
    Code:
    #define COUNT_ALLOC 1250000
    
    
    //===============================================================================
    #if defined(ARDUINO_TEENSY41)
    extern "C" {
    	extern uint8_t external_psram_size;
    }
    uint8_t *extmem_next_free = nullptr;  // set in startup NULL if we don't have external memory
    extern unsigned long _seram;
    extern unsigned long _eeram;
    uint8_t *malloc_extmem(size_t cb_alloc) {
    	uint8_t *extmem_free_last = extmem_next_free;  // Record Prior value in case last alloc fails
    
    	if (!extmem_next_free) {
    		extmem_next_free = (uint8_t *)&_seram + (uint32_t)(external_psram_size * 1048576);
    		extmem_free_last = extmem_next_free;
    	}
    	// lets allocate and return on 16 byte boundaries.
    	uint8_t *ret_val = (uint8_t *)((uint32_t)(extmem_next_free - cb_alloc) & 0xfffffff0);
    
    	// comparing _eeram to assure that we not overrun pre allocated
    	if (ret_val < (uint8_t *)&_eeram) {
    		extmem_next_free = extmem_free_last; // On this fail undo last attempt alloc
    		return nullptr;
    	}
    	extmem_next_free = ret_val;
    	return ret_val;
    }
    #else
    uint8_t *malloc_extmem(size_t cb_alloc) {
    	return nullptr;
    }
    #endif
    void free_extmem(char *p) {
    	// we don't do any free yet
    }
    //===============================================================================
    
    
    extern uint8_t *malloc_extmem(size_t cb_alloc);
    
    uint16_t *buffer1;
    uint16_t *buffer2;
    uint16_t *buffer3;
    
    #if 1
    EXTMEM uint32_t fluff[1024];
    EXTMEM uint32_t fluff2[1024];
    #else
    EXTMEM uint32_t fluff[256];
    EXTMEM uint32_t fluff2[256];
    #endif
    
    uint8_t *buffer_eat_upper_8mb = nullptr;
    void setup() {
    	pinMode(13, OUTPUT);
    	Serial.begin(115200);
    	while (!Serial && millis() < 4000 );
    	Serial.println("\n" __FILE__ " " __DATE__ " " __TIME__);
    	Serial.printf( "ERAM start==0x%x\t end==0x%x\n", &_seram, &_eeram );
    	Serial.printf( "EXTMEM \t fluff==0x%x[size=%x]\t fluff2==0x%x[size=%x]\n", fluff, sizeof(fluff), fluff2, sizeof(fluff2) );
    
    #if defined(ARDUINO_TEENSY41)
    	Serial.printf("External PSRAM size: %d\n", external_psram_size);
    	if (external_psram_size == 16) buffer_eat_upper_8mb = malloc_extmem(1024 * 1024 * 8);
    #endif
    	buffer1 = (uint16_t*)malloc_extmem(COUNT_ALLOC * sizeof(uint16_t));
    	buffer2 = (uint16_t*)malloc_extmem(COUNT_ALLOC * sizeof(uint16_t));
    	buffer3 = (uint16_t*)malloc_extmem(COUNT_ALLOC * sizeof(uint16_t));
    	Serial.printf("Eat: %x B1:%x B2:%x B3: %x\n", (uint32_t)buffer_eat_upper_8mb, (uint32_t)buffer1, (uint32_t)buffer2, (uint32_t)buffer3);
    	if (!buffer1 || !buffer2 || !buffer3) {
    		Serial.println("Failed to allocate external memory");
    		for (;;) {
    			digitalWrite(13, !digitalRead(13));
    			delay(250);
    		}
    	}
    }
    
    uint32_t loop_count = 0;
    uint32_t loop_alloc = 8192 * 4;
    uint16_t *buffer4 = nullptr; // waste and realloc memory test
    void loop() {
    	digitalWrite(13, !digitalRead(13));
    	loop_count++;
    	for (uint32_t i = 0; i < COUNT_ALLOC; i++) {
    		buffer1[i] = i & 0xffff;
    		buffer2[i] = loop_count & 0xffff;
    		buffer3[i] = buffer1[i] ^ buffer2[i];
    		if (buffer_eat_upper_8mb)buffer_eat_upper_8mb[i] = loop_count + i ;
    	}
    	uint32_t error_count = 0;
    	for (uint32_t i = 0; i < COUNT_ALLOC; i++) {
    		if ((buffer1[i] != (i & 0xffff)) || (buffer2[i] != (loop_count & 0xffff)) || (buffer3[i] != (buffer1[i] ^ buffer2[i]))) {
    			error_count++;
    			if (error_count < 5) Serial.printf("%u %x %x %x\n", i, buffer1[i], buffer2[i], buffer3[i]);
    		}
    	}
    	if ( error_count == 0 && 1 != (loop_count % 50) ) {
    		if ( loop_count == 2 || buffer4 != nullptr ) {
    			uint16_t *before4 = buffer4;
    			buffer4 = (uint16_t*)malloc_extmem(loop_alloc * sizeof(uint16_t));
    			if ( buffer4 == nullptr ) {
    				Serial.printf("\n\tEXTMEM exhausted at Pass %d alloc size: 0x%x \t was @ 0x%x\nTry smaller size:", loop_count, loop_alloc, before4 );
    				loop_alloc = loop_alloc / 2;
    				while ( loop_alloc > 0 && buffer4 == nullptr ) {
    					Serial.printf("0x%x\t", loop_alloc);
    					buffer4 = (uint16_t*)malloc_extmem(loop_alloc * sizeof(uint16_t));
    					if ( buffer4 == nullptr )
    						loop_alloc = loop_alloc / 2;
    					else
    						Serial.printf(" +!");
    				}
    			}
    			else
    				Serial.printf(" +");
    		}
    		else
    			Serial.printf(" .");
    	}
    	else
    		Serial.printf("\nPass %d error count: %d\n", loop_count, error_count);
    }

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •