Using USB host port to read/write USB memory (MSC)

sjm-mcl · Apr 6, 2023

Hi, I'm investigating using the Teensy4.1 in a project which must essentailly copy/delete jpg image files from a usb memory stick (that gets randomly inserted and removed to/from the Teensy's USB host port) to a local SD card archive and then uploaded via a WiFi connection to the cloud. I would use an airLift module to do the WiFi bit. What concerns me is the simultaneous use of FatFs on both the usb host port and the on-board sd card - is this achievable in an Arduino sketch? If so any pointers on how to do it much apreciated. Thanks for your time.

wwatson · Apr 7, 2023

sjm-mcl said:
Hi, I'm investigating using the Teensy4.1 in a project which must essentailly copy/delete jpg image files from a usb memory stick (that gets randomly inserted and removed to/from the Teensy's USB host port) to a local SD card archive and then uploaded via a WiFi connection to the cloud. I would use an airLift module to do the WiFi bit. What concerns me is the simultaneous use of FatFs on both the usb host port and the on-board sd card - is this achievable in an Arduino sketch? If so any pointers on how to do it much apreciated. Thanks for your time.

Sorry about the late response...
It kind of sounds like you want to copy files to SD card from a USB stick with the T4.1, Correct? That is doable but not with FatFS. You will need to use the new SD library for SD card and USBHost_t36 library which supports MSC drives like a USB memory stick. They both use the built in SdFat library. You can copy and delete files between both devices.

With all of the recent changes a lot of the original examples have become obsolete. I am working on updating these examples to be compatible with Teensyduino 1.58. For now I have put together an example sketch you can test with. It creates a test file on the USB drive and fills it with 32k bytes of data. Then it copies it to another file ("copy.txt") on the USB drive. Next we do a file read from the SD card. During these operations read/write/copy times will be displayed.
The sketch tested with Arduino 1.8.19 and Arduino IDE 2.0.4 with Tennsyduino 1.58:

Code:

/*
  MSC Drive and SD card read/write copy file.
 
 This example shows how to read and write data to and from an SD card file
 and USB MSC drive. 	

 The SD card circuit:
 * SD card attached to SPI bus as follows:
 ** MOSI - pin 11, pin 7 on Teensy with audio board
 ** MISO - pin 12
 ** CLK - pin 13, pin 14 on Teensy with audio board
 ** CS - pin 4, pin 10 on Teensy with audio board
 
 created   Nov 2010
 by David A. Mellis
 modified 9 Apr 2012
 by Tom Igoe
 Modified 2017-2023 Warren Watson
 This example code is in the public domain.
 	 
 */
 
#include <USBHost_t36.h>
#include <SD.h>

// Setup USBHost_t36 and as many HUB ports as needed.
USBHost myusb;
USBHub hub1(myusb);
USBHub hub2(myusb);
USBHub hub3(myusb);
USBHub hub4(myusb);
// Setup MSC for the number of USB Drives you are using. (Two for this example)
// Mutiple  USB drives can be used. Hot plugging is supported. There is a slight
// delay after a USB MSC device is plugged in. This is waiting for initialization
// but after it is initialized ther should be no delay.

// mscController is now USBDrive. 
USBDrive msDrive1(myusb);
USBDrive msDrive2(myusb);

// USBFilesystem is a class based on claimed partitions discovered during
// initialization. We are using the first discovered partition here. More
// discovered partitions can be used by adding more instances of the
// 'USBFilesystem' class.
USBFilesystem partition1(myusb);

const int chipSelect = BUILTIN_SDCARD;

File myFile;
File myFile1;

// Size of read/write.
// Play with with this to see affect on read/write speeds.(8192 - 65536)
const size_t BUF_SIZE = 16384;
// File size in MB where MB = 1,024,000 bytes.
const uint32_t FILE_SIZE_MB = 32;

// File size in bytes.
const uint32_t FILE_SIZE = 1024000UL*FILE_SIZE_MB;
FASTRUN uint8_t* buf[BUF_SIZE];
uint32_t t;
uint32_t flSize = 0;
float MBs = 1.0f;

void setup()
{
 // Open serial communications and wait for port to open:
  Serial.begin(9600);
   while (!Serial) {
    ; // wait for serial port to connect.
  }

  // Start USBHost_t36, HUB(s) and USB devices.
  myusb.begin();

  Serial.print("\nInitializing USB MSC drive...");
  while (!partition1) {
    myusb.Task();
  }
  Serial.println("initialization done.\n");
  File root = partition1.open("/");
  Serial.println("Directory listing for USB drive:");
  printDirectory(root, 0);

  Serial.print("\nInitializing SD card...");
  if (!SD.begin(chipSelect)) {
    Serial.println("initialization failed!");
    return;
  }
  Serial.println("initialization done.\n");
  File root1 = SD.open("/");
  Serial.println("Directory listing for SD card:");
  printDirectory(root1, 0);

  // fill buf with known data
  if (BUF_SIZE > 1) {
    for (size_t i = 0; i < (BUF_SIZE - 2); i++) {
      buf[i] = (uint8_t *)('A' + (i % 26));
    }
    buf[BUF_SIZE-2] = (uint8_t *)'\r';
  }
  buf[BUF_SIZE-1] = (uint8_t *)'\n';
  
  uint32_t n = FILE_SIZE/BUF_SIZE;

  waitforInput(); // Wait for a keypress.

  //*******************************
  // If "test.txt" exists delete it.
  // (partition1 is USB drive).
  //*******************************
  if(partition1.exists("test.txt")) {
	Serial.println("\ntest.txt exists, removing...");
	partition1.remove("test.txt");
  }
  //********************************************************
  // Open test.txt file for write.
  // (FILE_WRITE_BEGIN) starts writing at the begining
  // of the file. (FILE_WRITE) appends the end of the file.
  // Write data and close the file.
  //********************************************************
  myFile = partition1.open("test.txt", FILE_WRITE_BEGIN);
  // if the file opened okay, write to it:
  if (myFile) {
    Serial.print("\nWriting test.txt to USB drive...");
  t = millis();
  for (uint32_t i = 0; i < n; i++) {
    if (myFile.write(buf, BUF_SIZE) != BUF_SIZE) {
      Serial.printf("Write Failed: Stopping Here...");
      while(1);
    }
  }
  t = millis() - t;
  flSize = myFile.size();
  MBs = (float)flSize / t;
  Serial.printf("Wrote %lu bytes %f seconds. Speed : %f MB/s\n",
                      flSize, (1.0 * t)/1000.0f, MBs / 1000.0f);
  // close the file:
  myFile.close();
  } else {
    // if the file didn't open, print an error:
    Serial.println("Error opening test.txt: Write Failed: Stoppiing Here...");
    while(1);
  }

  //**************************************************************
  // Re-open "test.txt" file for reading. Defaults to (FILE_READ)
  // if not specified. If "copy.txt" exists, delete it.
  // Open "copy.txt" for write. (FILE_WRITE_BEGIN).
  //**************************************************************
  myFile = partition1.open("test.txt");
  if (myFile) {
	if(partition1.exists("copy.txt")) {
      Serial.println("\ncopy.txt exists, removing...");
	  partition1.remove("copy.txt");
	}
    // open the second file for writing seek to beginning of file. 
    myFile1 = partition1.open("copy.txt", FILE_WRITE_BEGIN);
    // if the file opened okay, write to it:
    if (myFile1) {
      Serial.printf("Copying USB drive test.txt to USB drive copy.txt...");
	  t = millis();
      // Read from "test.txt and write to "copy.txt".
      while(myFile.read(buf, BUF_SIZE) == BUF_SIZE) {
        if (myFile1.write(buf, BUF_SIZE) != BUF_SIZE) {
          Serial.printf("Write Failed: Stoppiing Here...");
          while(1);
        }
      }
    }
    //*********************************
    // Print out some copy time stats. 
    //*********************************
    t = millis() - t;
    flSize = myFile.size();
    MBs = flSize / t;
    Serial.printf("Copied %lu bytes %f seconds. Speed : %f MB/s\n",
                         flSize, (1.0 * t)/1000.0f, MBs/1000.0f);
    // close the files:
    myFile.close();
    myFile1.close();
  } else {
  	// if the file didn't open, print an error:
    Serial.println("error opening test.txt");
  }

  //***************************************
  // Re-open "copy.txt" file for reading.
  // Calculate read time stats.
  //***************************************
  myFile1 = partition1.open("copy.txt");
  if (myFile1) {
    // open the file for a read. 
    myFile1 = partition1.open("copy.txt");
    // if the file opened okay, write to it:
    if (myFile1) {
      Serial.printf("Reading File from USB drive copy.txt...");
	  t = millis();
      while(myFile1.read(buf, BUF_SIZE) == BUF_SIZE);
    }
    t = millis() - t;
    flSize = myFile1.size();
    MBs = flSize / t;
    Serial.printf("Read %lu bytes %f seconds. Speed : %f MB/s\n",
                       flSize, (1.0 * t)/1000.0f, MBs/1000.0f);
    // close the files:
    myFile1.close();
  } else {
  	// if the file didn't open, print an error:
    Serial.println("Error opening copy.txt");
  }

  Serial.print("\nNow lets copy test.txt from the USB drive to the built in SD card.\n");
  waitforInput(); // Wait for a keypress.

  //**************************************************************
  // Open "test.txt" file on USB drive for reading. Defaults to (FILE_READ)
  // Defaults to (FILE_READ) if not specified. If "copy.txt" exists
  // delete it.
  // Open "copy.txt" for write to SD card. (FILE_WRITE_BEGIN).
  //**************************************************************
  myFile = partition1.open("test.txt"); 
  if (myFile) {
	if(SD.exists("copy.txt")) {// If it exists on SD card
    	Serial.println("\ncopy.txt exists, removing...");
		SD.remove("copy.txt");// delete it...
	}
    // Open "copy.txt" file on SD card for writing.
    // seek to beginning of file. 
    myFile1 = SD.open("copy.txt", FILE_WRITE_BEGIN);
    // if the file opened okay, write to it:
    if (myFile1) {
      Serial.printf("Copying USB drive test.txt to SD card copy.txt...");
	  t = millis();
      // Read from "test.txt and write to "copy.txt".
      while(myFile.read(buf, BUF_SIZE) == BUF_SIZE) {
        if (myFile1.write(buf, BUF_SIZE) != BUF_SIZE) {
          Serial.printf("Write Failed: Stoppiing Here...");
          while(1);
        }
      }
    }
    //*********************************
    // Print out some copy time stats. 
    //*********************************
    t = millis() - t;
    flSize = myFile.size();
    MBs = flSize / t;
    Serial.printf("Copied %lu bytes %f seconds. Speed : %f MB/s\n",
                         flSize, (1.0 * t)/1000.0f, MBs/1000.0f);
    // close the files:
    myFile.close();
    myFile1.close();
  } else {
  	// if the file didn't open, print an error:
    Serial.println("error opening test.txt");
  }

  //***************************************
  // Re-open "copy.txt" file for reading.
  // Calculate read time stats. (SD card)
  //***************************************
  myFile1 = SD.open("copy.txt");
  if (myFile1) {
    // open the file for a read. 
    myFile1 = SD.open("copy.txt");
    // if the file opened okay, write to it:
    if (myFile1) {
      Serial.printf("Reading File SD card copy.txt...");
	  t = millis();
      while(myFile1.read(buf, BUF_SIZE) == BUF_SIZE);
    }
    t = millis() - t;
    flSize = myFile1.size();
    MBs = flSize / t;
    Serial.printf("Read %lu bytes %f seconds. Speed : %f MB/s\n",
                       flSize, (1.0 * t)/1000.0f, MBs/1000.0f);
    // close the files:
    myFile1.close();
  } else {
  	// if the file didn't open, print an error:
    Serial.println("Error opening copy.txt on SD card");
  }
  Serial.print("Done..\n");
}

void loop()
{
	// nothing happens after setup
}

void printDirectory(File dir, int numSpaces) {
   while(true) {
     File entry = dir.openNextFile();
     if (! entry) {
       //Serial.println("** no more files **");
       break;
     }
     printSpaces(numSpaces);
     Serial.print(entry.name());
     if (entry.isDirectory()) {
       Serial.println("/");
       printDirectory(entry, numSpaces+2);
     } else {
       // files have sizes, directories do not
       printSpaces(48 - numSpaces - strlen(entry.name()));
       Serial.print("  ");
       Serial.println(entry.size(), DEC);
     }
     entry.close();
   }
}

void printSpaces(int num) {
  for (int i=0; i < num; i++) {
    Serial.print(" ");
  }
}

// This was taken from LittleFS.
void waitforInput()
{
  Serial.println("Press anykey to continue");
  while (Serial.read() == -1) ;
  while (Serial.read() != -1) ;
}

Displayed results on my USB and SD devices:

Code:

Initializing USB MSC drive...initialization done.

Directory listing for USB drive:
32MEGfile.dat                                     32768000
test.txt                                          32768000
System Volume Information/
  WPSettings.dat                                  12
  IndexerVolumeGuid                               76
copy.txt                                          32768000

Initializing SD card...initialization done.

Directory listing for SD card:
32MEGfile.dat                                     32768000
MSCtesting.bas                                    534
copy.txt                                          32768000
Press anykey to continue

test.txt exists, removing...

Writing test.txt to USB drive...Wrote 32768000 bytes 4.189000 seconds. Speed : 7.822392 MB/s

copy.txt exists, removing...
Copying USB drive test.txt to USB drive copy.txt...Copied 32768000 bytes 5.971000 seconds. Speed : 5.487000 MB/s
Reading File from USB drive copy.txt...Read 32768000 bytes 1.765000 seconds. Speed : 18.565001 MB/s

Now lets copy test.txt from the USB drive to the built in SD card.
Press anykey to continue

copy.txt exists, removing...
Copying USB drive test.txt to SD card copy.txt...Copied 32768000 bytes 3.563000 seconds. Speed : 9.196000 MB/s
Reading File SD card copy.txt...Read 32768000 bytes 1.447000 seconds. Speed : 22.645000 MB/s
Done..

Changing the buffer size will vary the read/write/copy times. You will find other examples in the SdFat, LittleFS and SD libraries as well.
Enough of this

Hope it helps...

Edit: Forgot to mention hot plugging is supported with USB MSC devices. You might also want to checkout the Teensy MTP library here.

sjm-mcl · Apr 8, 2023

@wwatson: Thanks so much for your incredibly helpful response. The timing data you supplied is amazing ~8MB/s - I tried something similar on a RPi Pico and clocked just 64KB/s. Brilliant!

wwatson · Apr 8, 2023

sjm-mcl said:
@wwatson: Thanks so much for your incredibly helpful response. The timing data you supplied is amazing ~8MB/s - I tried something similar on a RPi Pico and clocked just 64KB/s. Brilliant!

Awesome

I hope it all works out. The speeds really depend on the USB and SD devices you are using and the amount of buffer size you are willing to give. Effective buffer sizes vary depending on the design of the devices you are using. Beyond a certain buffer size you will not gain much more speed. I will usually experiment with buffer sizes on a device until I find the optimal speed gain. Then see if I can spare that much memory.

sjm-mcl · Apr 8, 2023

Sound advice, thanks. Just to confirm: you use the standard USB port for comms with the Aduino IDE and the external 5-pin USB port connector for the USB memory stick?

wwatson · Apr 8, 2023

sjm-mcl said:
Sound advice. Can I just ask: how do you develop / download code that uses the USB port in host mode? I kind of assumed the USB would be used for downloading code from the IDE so it would get rather clunky to have to keep swapping the usb port over for download/testing. Or is there an alternate route for dev/download?

I see. Using a USB stick will need the host USB port on the T4x not the USB port used for program the Teensy from the IDE on a PC.
Info on the host USB port is here and here. With the second link scroll down to the "USB Host" section. You would plug the USB stick into the host port. Hopefully I understand what you are after

Also, as mentioned in a previous post you can turn the T41 into an external drive using MTP which allows you to read/write to the SD card or USB stick connected to the USB host port...

sjm-mcl · Apr 8, 2023

Thanks again for the clarification, I realised I had asked a daft question but wasn't quick enough to edit it into a more sensible one ;-) You are too efficient!

wwatson · Apr 8, 2023

sjm-mcl said:
Thanks again for the clarification, I realised I had asked a daft question but wasn't quick enough to edit it into a more sensible one ;-) You are too efficient!

HA! I thought it was me giving silly advice

sjm-mcl · Apr 12, 2023

My brand new Teensy4.1 board should be arriving tomorrow & I'm looking forward to testing out your example code! Thinking further ahead, is there a recommended way of adding BlueTooth and WiFi capabilities to the Teensy - perhaps the ESP32-Airlift breakout board from Adafruit? My application must wake up an external box with a bluetooth signal (to be defined) then set-up a WiFi access point (with DHCP server) and open a TCP server socket for comms with the external box over which I'll send the images previously read from the usb msc memory. Thanks again for your time.

wwatson · Apr 12, 2023

@sjm-mci - Myself and a few of the others were playing around with an Adafruit AirLift a while back. Here is the link:
https://forum.pjrc.com/threads/70062-Teensy-4-1-Adafruit-Airlift-Featherwing-Co-Processor-FTP-Server-not-opening-Port-21?highlight=AirLift

You might also do a forum search for WiFiNINA. Not sure about BlueTooth though. Have never played with it...

KurtE · Apr 12, 2023

sjm-mcl said:
My brand new Teensy4.1 board should be arriving tomorrow & I'm looking forward to testing out your example code! Thinking further ahead, is there a recommended way of adding BlueTooth and WiFi capabilities to the Teensy - perhaps the ESP32-Airlift breakout board from Adafruit? My application must wake up an external box with a bluetooth signal (to be defined) then set-up a WiFi access point (with DHCP server) and open a TCP server socket for comms with the external box over which I'll send the images previously read from the usb msc memory. Thanks again for your time.

wwatson said:
@sjm-mci - Myself and a few of the others were playing around with an Adafruit AirLift a while back. Here is the link:
https://forum.pjrc.com/threads/70062-Teensy-4-1-Adafruit-Airlift-Featherwing-Co-Processor-FTP-Server-not-opening-Port-21?highlight=AirLift

You might also do a forum search for WiFiNINA. Not sure about BlueTooth though. Have never played with it...

WiFi - I have not played much, except for something like mentioned or ESPxx hooked up mostly by Serial port.

As for USBHost for this:

So far the only thing I know of, is AntNet was added, which I have never used before.

BlueTooth: We have played with two varieties of this.

We do have a bluetooth object that supports a set of USB BT Dongles, where we can pair to a few times of BT devices, I believe all of them are HID, like Mouse, Keyboard and Joystick.

A while ago I was playing with the ArduinoBLE library. I made a version of it that used the USBHost code for the lower level stuff and was able get it to talk to some different BLE devices using it. Not sure if anything will happen with that code. but was fun to play with

sjm-mcl · Apr 14, 2023

OK, thanks again for your valuable WiFi help, that thread made fascinating reading, great to see how you guys operate! Afraid I've hit a bit a problem with your USB/SD-card example code, it starts off well, correctly listing the files on the USB memory stick but then I get:

Code:

Initializing SD card...initialization failed!

I have a SanDisk 32GB uSD plugged into the on-board SD slot, it's FAT32 formatted with a few small files on it, reads fine in a PC.
Am I missing something obvious? Sorry but I'm very new to Arduino etc.

BTW I get no compile-time error messages.
Sorry if I should have posted this in a more appropriate section of the forum.

wwatson · Apr 14, 2023

sjm-mcl said:
OK, thanks again for your valuable WiFi help, that thread made fascinating reading, great to see how you guys operate! Afraid I've hit a bit a problem with your USB/SD-card example code, it starts off well, correctly listing the files on the USB memory stick but then I get:

Code:

Initializing SD card...initialization failed!

I have a SanDisk 32GB uSD plugged into the on-board SD slot, it's FAT32 formatted with a few small files on it, reads fine in a PC.
Am I missing something obvious? Sorry but I'm very new to Arduino etc.

BTW I get no compile-time error messages.
Sorry if I should have posted this in a more appropriate section of the forum.

I will double check on my side tomorrow. It's the wife's birthday

sjm-mcl · Apr 15, 2023

Well top marks for remembering that!
FYI, here's a couple of snippets from the verbose compile-time output, just to confirm which SD lib is being used:

Code:

Using board 'teensy41' from platform in folder: C:\Users\Steve\AppData\Local\Arduino15\packages\teensy\hardware\avr\1.58.0
Using core 'teensy4' from platform in folder: C:\Users\Steve\AppData\Local\Arduino15\packages\teensy\hardware\avr\1.58.0

Code:

Memory Usage on Teensy 4.1:
  FLASH: code:164056, data:10216, headers:9020   free for files:7943172
   RAM1: variables:28064, code:161400, padding:2440   free for local variables:332384
   RAM2: variables:12416  free for malloc/new:511872
"C:\\Users\\Steve\\AppData\\Local\\Arduino15\\packages\\teensy\\tools\\teensy-tools\\1.58.0/stdout_redirect" "C:\\Users\\Steve\\AppData\\Local\\Temp\\arduino\\sketches\\449EEAF9E55E5C8FC947F5DCA3B86F57/HtheP.ino.lst" "C:\\Users\\Steve\\AppData\\Local\\Arduino15\\packages\\teensy\\tools\\teensy-compile\\11.3.1/arm/bin/arm-none-eabi-objdump" -d -S -C "C:\\Users\\Steve\\AppData\\Local\\Temp\\arduino\\sketches\\449EEAF9E55E5C8FC947F5DCA3B86F57/HtheP.ino.elf"
Multiple libraries were found for "SD.h"
  Used: C:\Users\Steve\AppData\Local\Arduino15\packages\teensy\hardware\avr\1.58.0\libraries\SD
  Not used: C:\Users\Steve\AppData\Local\Arduino15\libraries\SD
Using library USBHost_t36 at version 0.2 in folder: C:\Users\Steve\AppData\Local\Arduino15\packages\teensy\hardware\avr\1.58.0\libraries\USBHost_t36 
Using library SdFat at version 2.1.2 in folder: C:\Users\Steve\AppData\Local\Arduino15\packages\teensy\hardware\avr\1.58.0\libraries\SdFat 
Using library SPI at version 1.0 in folder: C:\Users\Steve\AppData\Local\Arduino15\packages\teensy\hardware\avr\1.58.0\libraries\SPI 
Using library SD at version 2.0.0 in folder: C:\Users\Steve\AppData\Local\Arduino15\packages\teensy\hardware\avr\1.58.0\libraries\SD 
Using library EEPROM at version 2.0 in folder: C:\Users\Steve\AppData\Local\Arduino15\packages\teensy\hardware\avr\1.58.0\libraries\EEPROM

I'm using Aduino IDE 2.0.4

sjm-mcl · Apr 15, 2023

Some more info: the CardInfo.ino example sketch - _with_

Code:

const int chipSelect = BUILTIN_SDCARD;

edited in - also fails to initialise the SD card.
So I tried another card, a Kingston 64GB formatted with exFAT / FAT64 and it worked! Both the Kingston (good) and SanDisk (bad) cards are Class10 U1 spec. I reformatted the SanDisk to exFAT and it still didn't work, I re-reformatted back to FAT32 - no joy. Please note the SanDisk card works fine with both Windows and Linux PCs, the Windows disk-check function found no errors. I can continue work with the Kingston card but will need to know what's going on with the SanDisk, my application requires maximum card compatibility. Be keen to know if you can find any cards that don't work for you. As ever thanks so much for your time.

wwatson · Apr 15, 2023

sjm-mcl said:
Some more info: the CardInfo.ino example sketch - _with_

Code:

const int chipSelect = BUILTIN_SDCARD;

edited in - also fails to initialise the SD card.
So I tried another card, a Kingston 64GB formatted with exFAT / FAT64 and it worked! Both the Kingston (good) and SanDisk (bad) cards are Class10 U1 spec. I reformatted the SanDisk to exFAT and it still didn't work, I re-reformatted back to FAT32 - no joy. Please note the SanDisk card works fine with both Windows and Linux PCs, the Windows disk-check function found no errors. I can continue work with the Kingston card but will need to know what's going on with the SanDisk, my application requires maximum card compatibility. Be keen to know if you can find any cards that don't work for you. As ever thanks so much for your time.

I tested on my end and could not get a failure. I am not that familiar with SD cards but it seems to me you can adjust the SPI speeds. Can't remember if it 's possible with QSPI which the built in sdcard IF uses. Will have to check that out. Also need to see if I have a SanDisk card to use. Maybe somebody else on the forum that knows can jump in.

EDIT: Ok, found 3 SanDisk SD cards a 8G, 32G and 64G. The 8G failed because it was formatted EXT4. reformatted to FAT32. All cards tested ok. Not sure why yours are failing. I also have tested two PNY32's and one Samsong 32G. All work. The 8G and 32G SanDisk cards are very old HC's. Not sure what's going on with yours. Sorry...

sjm-mcl · Apr 23, 2023

FYI I replaced this line:

Code:

FASTRUN uint8_t* buf[BUF_SIZE];

with just

Code:

 uint8_t buf[BUF_SIZE];

and it improved all the speeds, especially the USB -> SD transfer speed (which is what concerns me) from 6.6 to 7.4 MBps.
Not sure what FASTRUN is supposed to do here?

defragster · Apr 23, 2023

See pjrc.com/store/teensy41.html#memory

Code:

FASTRUN - Functions defined with "FASTRUN" are allocated in the beginning of RAM1. A copy is also stored in Flash and copied to RAM1 at startup. These functions are accessed by the Cortex-M7 ITCM bus, for the fastest possible performance. By default, functions without any memory type defined are treated as FASTRUN. A small amount of memory is typically unused, because the ITCM bus must access a memory region which is a multiple of 32K.

That is for code - that by default on T_4.1 is all FASTRUN.

Was the '*' actually different between the two as shown?
One allocated array BUF_SIZE pointers to uint8_t's? The other allocates array BUF_SIZE of uint8_t's - and all volatile memory not otherwise specified is in fast RAM1 where code also resides.

sjm-mcl · Apr 23, 2023

Yes the star _was_ there in @wwatson's original code (see this thread post#2), by mistake I surmise. I just tried it again with FASTRUN but without the star: the USB -> SD-card transfer speed has improved to 7.1, but is still shy of the non-FASTRUN speed of 7.4MBps.

wwatson · Apr 23, 2023

sjm-mcl said:
Yes the star _was_ there in @wwatson's original code (see this thread post#2), by mistake I surmise. I just tried it again with FASTRUN but without the star: the USB -> SD-card transfer speed has improved to 7.1, but is still shy of the non-FASTRUN speed of 7.4MBps.

My results with the Sandisk 32G Dual.

With FASTRUN:

Code:

Initializing SD card...initialization done.

Directory listing for SD card:
EnvironmentalData/
32MEGfile.dat                                     32768000
bench.dat                                         32000000
armaged.wav                                       56863276
copy.txt                                          32768000
SdioLogger.csv                                    18427909
IsrLoggerTest.bin                                 261120
Press anykey to continue

test.txt exists, removing...

Writing test.txt to USB drive...Wrote 32768000 bytes 3.655000 seconds. Speed : [COLOR="#FF0000"]8.965253 MB/s[/COLOR]

copy.txt exists, removing...
Copying USB drive test.txt to USB drive copy.txt...Copied 32768000 bytes 4.537000 seconds. Speed : 7.222000 MB/s
Reading File from USB drive copy.txt...Read 32768000 bytes 1.254000 seconds. Speed : 26.129999 MB/s

Now lets copy test.txt from the USB drive to the built in SD card.
Press anykey to continue

copy.txt exists, removing...
Copying USB drive test.txt to SD card copy.txt...Copied 32768000 bytes 2.873000 seconds. Speed : 11.405000 MB/s
Reading File SD card copy.txt...Read 32768000 bytes 1.430000 seconds. Speed : 22.914000 MB/s
Done..

Without FASTRUN:

Code:

Initializing SD card...initialization done.

Directory listing for SD card:
EnvironmentalData/
32MEGfile.dat                                     32768000
bench.dat                                         32000000
armaged.wav                                       56863276
copy.txt                                          32768000
SdioLogger.csv                                    18427909
IsrLoggerTest.bin                                 261120
Press anykey to continue

test.txt exists, removing...

Writing test.txt to USB drive...Wrote 32768000 bytes 3.132000 seconds. Speed : [COLOR="#00FF00"]10.462324 MB/s[/COLOR]

copy.txt exists, removing...
Copying USB drive test.txt to USB drive copy.txt...Copied 32768000 bytes 4.587000 seconds. Speed : 7.143000 MB/s
Reading File from USB drive copy.txt...Read 32768000 bytes 1.254000 seconds. Speed : 26.129999 MB/s

Now lets copy test.txt from the USB drive to the built in SD card.
Press anykey to continue

copy.txt exists, removing...
Copying USB drive test.txt to SD card copy.txt...Copied 32768000 bytes 2.905000 seconds. Speed : 11.279000 MB/s
Reading File SD card copy.txt...Read 32768000 bytes 1.430000 seconds. Speed : 22.914000 MB/s
Done..

You are correct. There is a difference. I ran the test with and without FASTRUN three times results were escentually the same.
As for:

Code:

FASTRUN uint8_t buf[BUF_SIZE];

It generates these warnings due to code later on in the sketch:

Code:

/home/wwatson/Arduino/examples/copyFileUSBSD/copyFileUSBSD.ino:98:16: warning: invalid conversion from 'uint8_t*' {aka 'unsigned char*'} to 'uint8_t' {aka 'unsigned char'} [-fpermissive]
   98 |       buf[i] = (uint8_t *)('A' + (i % 26));
      |                ^~~~~~~~~~~~~~~~~~~~~~~~~~~
      |                |
      |                uint8_t* {aka unsigned char*}
/home/wwatson/Arduino/examples/copyFileUSBSD/copyFileUSBSD.ino:100:23: warning: invalid conversion from 'uint8_t*' {aka 'unsigned char*'} to 'uint8_t' {aka 'unsigned char'} [-fpermissive]
  100 |     buf[BUF_SIZE-2] = (uint8_t *)'\r';
      |                       ^~~~~~~~~~~~~~~
      |                       |
      |                       uint8_t* {aka unsigned char*}
/home/wwatson/Arduino/examples/copyFileUSBSD/copyFileUSBSD.ino:102:21: warning: invalid conversion from 'uint8_t*' {aka 'unsigned char*'} to 'uint8_t' {aka 'unsigned char'} [-fpermissive]
  102 |   buf[BUF_SIZE-1] = (uint8_t *)'\n';
      |                     ^~~~~~~~~~~~~~~
      |                     |
      |                     uint8_t* {aka unsigned char*}

Initializing SD card...initialization done.

Seems to just be when writing to the USB drive. Curious...

jmarsh · Apr 23, 2023

FASTRUN is meant to be for functions only, it would put the data buffer in the ITCM region... which probably isn't optimized for data access (I'm surprised it's not read-only).

wwatson · Apr 23, 2023

jmarsh said:
FASTRUN is meant to be for functions only, it would put the data buffer in the ITCM region... which probably isn't optimized for data access (I'm surprised it's not read-only).

Thanks for the explanation

Makes sense.

sjm-mcl · Apr 23, 2023

Glad to have piqued your curiosity!

wwatson · Apr 23, 2023

sjm-mcl said:
Glad to have piqued your curiosity!

It's always a good way to learn

By the way, here is the speed results of a Kingston 120G SSD drive used with a StarTech USB to SATA adapter:

Code:

Press anykey to continue

test.txt exists, removing...

Writing test.txt to USB drive...Wrote 32768000 bytes 1.539000 seconds. Speed : [COLOR="#FF0000"]21.291748 MB/s[/COLOR]

copy.txt exists, removing...
Copying USB drive test.txt to USB drive copy.txt...Copied 32768000 bytes 3.107000 seconds. Speed : 10.546000 MB/s
Reading File from USB drive copy.txt...Read 32768000 bytes 1.531000 seconds. Speed : 21.403000 MB/s

Now lets copy test.txt from the USB drive to the built in SD card.
Press anykey to continue

copy.txt exists, removing...
Copying USB drive test.txt to SD card copy.txt...Copied 32768000 bytes 2.872000 seconds. Speed : 11.409000 MB/s
Reading File SD card copy.txt...Read 32768000 bytes 1.439000 seconds. Speed : 22.771000 MB/s
Done..

Definitely a lot faster. I'm sure there are newer USB sticks that are faster as well and more expensive...

Using USB host port to read/write USB memory (MSC)

Active member

Well-known member

Active member

Well-known member

Active member

Well-known member

Active member

Well-known member

Active member

Well-known member

Senior Member+

Active member

Well-known member

Active member

Active member

Well-known member

Active member

Senior Member+

Active member

Well-known member

Well-known member

Well-known member

Active member

Well-known member