Forum Rule: Always post complete source code & details to reproduce any issue!
Page 3 of 3 FirstFirst 1 2 3
Results 51 to 66 of 66

Thread: malloc/free for EXTMEM and DTCM

  1. #51
    Senior Member PaulStoffregen's Avatar
    Join Date
    Nov 2012
    Posts
    23,084
    Quote Originally Posted by luni View Post
    Looks like the standard Serial port is now a interface 0 from a compound device, where as the old version generates the usual stand alone device. Is this intended?
    Yes, planning to switch to composite device for all USB serial.

    The CDC class at device level was needed to support Windows XP-SP2, Windows Vista (no service packs) and Macintosh OS-X 10.5 & 10.6.

  2. #52
    Senior Member
    Join Date
    Apr 2014
    Location
    Germany
    Posts
    1,224
    Ok, got it. If this is permanent I'll log an issue at TyCommander. For the time being I can work with the dual serial.

  3. #53
    Senior Member
    Join Date
    Apr 2014
    Location
    Germany
    Posts
    1,224
    Looks like you allocate all of the remaining free space in EXTRAM for the EXTRAM heap. This is a fantastic idea. I was only using user defined heap buffers so far but this makes it much more easy to use of course


    Here's a quick test which queries the free space, then tries to allocate it all at once. Fails.
    The reported free space is the total free space not the largest free chunk. Don't know if you saw my video here https://www.youtube.com/watch?v=s3U5QSO7Rd8 It prints out the reported stats on the right side.

  4. #54
    Senior Member
    Join Date
    Apr 2014
    Location
    Germany
    Posts
    1,224
    BTW: it is still possible to use the full functionality of sm_alloc. E.g. Add an additional Heap on DMAMEM:

    Code:
    DMAMEM char dmaBuffer[10000];
    smalloc_pool dmaHeap;
    
    void setup()
    {
      while (!Serial){}
    
      sm_set_pool(&dmaHeap, dmaBuffer, 10000, false, nullptr);
    
      char *extMemChunk = (char *)extmem_malloc(100);
      char *dmaMemChunk = (char *)sm_malloc_pool(&dmaHeap,100);
    
      Serial.printf("extMemPtr: %p dmaMemPtr %p\n", extMemChunk, dmaMemChunk);
    }
    
    void loop()
    {
    }
    Which prints:
    Code:
    extMemPtr: 0x7000000c dmaMemPtr 0x2020000c
    Maybe do the same pre allocation for DMAMEM as for EXTMEM to make usage simpler?

  5. #55
    Senior Member+ mjs513's Avatar
    Join Date
    Jul 2014
    Location
    New York
    Posts
    5,909
    @luni

    Ok think you need to give me another lesson here. Using your printInfo function which boils down to basically:
    Code:
        size_t total, totalUser, totalFree;
        int nrBlocks;
        sm_malloc_stats(&total, &totalUser, &totalFree, &nrBlocks);
        Serial.printf(" %u %u %u %u\n",  total, totalUser, totalFree, nrBlocks);
    should return bytes for total, totalUser and totalFree?

    So if I add it to my example along with some of Paul's example I get:
    Code:
    {total, totalUser, totalFree, nrBlocks
     1028 1074774016 352 536874412
    {pauls example"}
    Free Space: 8388552
    Total: 48
    Initializing values...
    
    Initialized values
    1
    3
    5
    7
    9
    Here is what I am using:
    Code:
    #include "Streaming.h"
    #define cout Serial
    #include "smalloc.h"
    
    void setup() {
      while(!Serial)
        ;
      int *ptr;
      ptr = (int*) extmem_malloc(5*sizeof(int));
        size_t total, totalUser, totalFree;
        int nrBlocks;
        sm_malloc_stats(&total, &totalUser, &totalFree, &nrBlocks);
        Serial.printf(" %u %u %u %u\n",  total, totalUser, totalFree, nrBlocks);
    
        Serial.print("Free Space: ");
      size_t total1 = 0, freespace = 0;
      sm_malloc_stats_pool(&extmem_smalloc_pool, &total1, NULL, &freespace, NULL);
      Serial.println(freespace);
      Serial.print("Total: "); Serial.println(total);
      if(!ptr)
      {
        cout << "Memory Allocation Failed"  << endl;
        exit(1);
      }
      cout << "Initializing values..." << endl << endl;
    
      for (int i=0; i<5; i++)
      {
        ptr[i] = i*2+1;
      }
      cout << "Initialized values" << endl;
      for (int i=0; i<5; i++)
      {
        /* ptr[i] and *(ptr+i) can be used interchangeably */
        cout << *(ptr+i) << endl;
      }
    
      extmem_free(ptr);
    
    }
    
    void loop() {
      // put your main code here, to run repeatedly:
    
    }

  6. #56
    Senior Member PaulStoffregen's Avatar
    Join Date
    Nov 2012
    Posts
    23,084
    Quote Originally Posted by luni View Post
    Maybe do the same pre allocation for DMAMEM as for EXTMEM to make usage simpler?
    Regular newlib malloc() already puts its heap on all the unused DMAMEM (the other 512K which isn't ITCM / DTCM).

    Yeah, you could create a static array in DMAMEM and use it with smalloc, but what's the point when regular malloc already uses it?

  7. #57
    Senior Member
    Join Date
    Apr 2014
    Location
    Germany
    Posts
    1,224
    Sorry, I ment DTCM of course

    Code:
    #include "smalloc.h"
    
    char dtcmBuffer[100000];
    smalloc_pool dmaHeap;
    
    void setup()
    {
      while (!Serial){}
    
      sm_set_pool(&dmaHeap, dtcmBuffer, 10000, false, nullptr);
    
      char *extMemChunk = (char *)extmem_malloc(100);
      char *dtcmMemChunk = (char *)sm_malloc_pool(&dmaHeap,100);
    
      Serial.printf("extMemPtr: %p dtcmMemPtr %p\n", extMemChunk, dtcmMemChunk);
    }
    Code:
    extMemPtr: 0x7000000c dtcmMemPtr 0x20000d50

  8. #58
    Senior Member
    Join Date
    Apr 2014
    Location
    Germany
    Posts
    1,224
    Quote Originally Posted by mjs513 View Post
    @luni

    Ok think you need to give me another lesson here. Using your printInfo function which boils down to basically:
    Code:
        size_t total, totalUser, totalFree;
        int nrBlocks;
        sm_malloc_stats(&total, &totalUser, &totalFree, &nrBlocks);
        Serial.printf(" %u %u %u %u\n",  total, totalUser, totalFree, nrBlocks);
    should return bytes for total, totalUser and totalFree?
    Paul didn't set up the default pool, so sm_malloc_stats won't give correct values. You need to use the pool version:

    Code:
    void setup()
    {
      while (!Serial);
    
      int* ptr = (int*)extmem_malloc(5 * sizeof(int));
    
      size_t total, totalUser, free;
      int blocks;
      sm_malloc_stats_pool(&extmem_smalloc_pool, &total, &totalUser, &free, &blocks);
    
      Serial.printf(
          "Total used:       %u bytes\n"
          "Total user vars:  %u bytes\n"
          "Free space:       %.4f MB\n"
          "Allocated blocks: %u\n\n",
          total, totalUser, free / 1024 / 1024.0f, blocks);
    }
    Which prints the correct values:

    Code:
    Total used:       48 bytes
    Total user vars:  20 bytes
    Free space:       15.9990 MB
    Allocated blocks: 1
    Each allocated chunk of memory is contained in one block. The block starts with a 12byte header, followed by the user data + some fill bytes and a closing 12byte tag.

    Click image for larger version. 

Name:	Anmerkung 2020-10-26 210335.jpg 
Views:	8 
Size:	26.2 KB 
ID:	22210

    The fill bytes will fill up the user data to n*12 bytes. The returned pointer points to the beginning of the user data of course. If you want to get a pointer to the block you can use the macro USER_TO_HEADER as shown in #11.

    Hope that answers your question somehow?

  9. #59
    Senior Member PaulStoffregen's Avatar
    Join Date
    Nov 2012
    Posts
    23,084
    Maybe we should have a way to pad allocations to 32-byte cache row boundaries?

  10. #60
    Senior Member+ mjs513's Avatar
    Join Date
    Jul 2014
    Location
    New York
    Posts
    5,909
    @luni

    Yep that answers my question including my unasked question about what is a chunk - thanks for explanation!

  11. #61
    Senior Member
    Join Date
    Apr 2014
    Location
    Germany
    Posts
    1,224
    Quote Originally Posted by PaulStoffregen View Post
    Maybe we should have a way to pad allocations to 32-byte cache row boundaries?
    From the readme:
    Searches are done by shifting a header-wide pointer across the pool.
    Allocated block is found by testing each possible header for validity.
    So, as it is now the block size needs to be a multiple of he header size. If I understand the readme correctly the header has 4bytes for the block size, 4bytes for the size of the user memory and 4bytes for a hash -> My gut feeling is that 32bit blocks will need some major rewriting of the library. Might be interesting of course...

  12. #62
    Senior Member PaulStoffregen's Avatar
    Join Date
    Nov 2012
    Posts
    23,084
    Indeed, seems likely we'll eventually have to make substantial changes to smalloc, or replace it with some other memory management scheme. For now (version 1.54) this is probably good enough. Long-term, we probably do need alignment to cache rows to avoid thorny issues when people try to use DMA on their allocated memory.

    I've added a comment in smalloc.h to advise against using its API directly from Arduino sketches & libraries.

    https://github.com/PaulStoffregen/co...e918ce498a7df3

    If we do end up changing the underlying memory management, at least there's a warning that smalloc.h may change in future versions.

  13. #63
    Senior Member+ KurtE's Avatar
    Join Date
    Jan 2014
    Posts
    7,935
    Sounds good. My guess is that DMA from EXTMEM should also be 32 byte aligned as well? Note: Currently I don't think malloc is 32 byte aligned either.

    Which is why in some of our display drivers that do DMA we have things like:
    Code:
    			_we_allocated_buffer = (uint8_t *)malloc(CBALLOC+32);
    			if (_we_allocated_buffer == NULL)
    				return 0;	// failed 
    			_pfbtft = (RAFB*) (((uintptr_t)_we_allocated_buffer + 32) & ~ ((uintptr_t) (31)));
    I was playing around with reading the OV... Camera (640x480 2 bytes) and trying to output to ILI9488 (480x320);

    I had the Camera buffer allocated using EXTMEM and I allocated the frame buffer using new memory allocater ...
    Will probably move the frame buffer back to DMAMEM

  14. #64
    Senior Member+ defragster's Avatar
    Join Date
    Feb 2015
    Posts
    12,752
    Re: Loss of Teensy with current cores about 10pm 10/26? was 4 hours old then ...

    Sent over code and tyComm could not see them.

    Switched IDE to TeensyLoader

    Did 15s Restore and the Red LED for Button bootloader no longer appears.

    They boot and blink - button stops Orange blink - but no RED led appears

    Cannot program T_4.1 or T_4.0

    USBView from MSFT and nirsoft USBDevView don't indicate anything Teensy.

    Rebooted just 38 hours ago - suppose will do it again to see ...

  15. #65
    Senior Member+ defragster's Avatar
    Join Date
    Feb 2015
    Posts
    12,752
    Moved the T_4.0 and 4.1 in turn to cable on USB Port - Both act normally there

    But it seems the HUB is Offline????

    Umplugged and moved HUB to another port and it seems okay now ???

  16. #66
    Junior Member
    Join Date
    Sep 2020
    Posts
    8
    Ive had success using tinyalloc before https://github.com/thi-ng/tinyalloc

    Initialised something like this. Chunk sizes etc could be adjusted to suit your application.

    Code:
    //Init external RAM and memory heap
    EXTMEM uint8_t ext_ram[1]; //Just to get EXTMEM pointer
    extern uint8_t external_psram_size; //in MB. Set in startup.c
    
    void init_memory()
    {
        uint32_t psram_bytes = 1024 * 1024 * external_psram_size;
        ta_init((void *)(ext_ram),               //Base of heap
                (void *)(ext_ram + psram_bytes), //End of heap
                psram_bytes / 32768,             //Number of memory chunks (32k/per chunk)
                16,                              //Smaller chunks than this won't split
                32);                             //32 word size alignment
    }
    This allows use of ta_alloc, ta_free, ta_calloc etc. directly into external RAM.

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •