USBHost_t36 USB Mass Storage Driver Experiments

@mjs513 - with SerMon offline - can you light the LED during the wait here in setup?:
Code:
void setup()
{
[B]  // wait for serial line to come up
  pinMode(13,OUTPUT);
  pinMode(13,HIGH);

  while(!Serial);
  Serial.println("Test logger_RawWrite");[/B]
  Serial.print("uSDFS_VER:"); Serial.println(uSDFS_VER);
  Serial.print("BUFFSIZE :");  Serial.println(BUFFSIZE);
  Serial.print("Dev Type :");  Serial.println(Dev);
  if((rc = f_mount (&fatfs, Dev, 1))) die("Mount",rc);      /* Mount/Unmount a logical drive */

Don't see that I have time to do it with clearing the decks for a week away trip. But would be interesting to do an IDE 1.8.10 fresh install then TD 1.48 { since it won't downgrade from 1.49 }
 
@defragster - @wwatson

Just downloaded fresh copy of 1.8.10 and a fresh copy the beta4. The test sketch now runs without a problem and the LED it blinking properly. For the Cruz disk:
Code:
Test logger_RawWrite

uSDFS_VER:30_Jun_19_07_17

BUFFSIZE :8192

Dev Type :2:/

Test logger_RawWriteRead>Write

Test logger_RawWrite

uSDFS_VER:30_Jun_19_07_17

BUFFSIZE :8192

Dev Type :2:/

File System FS_EXFAT
Free Disk Size -1 clusters
Cluster Size 64 sectors
Sector Size 512 bytes

Change drive

TESING DRIVE WRITE SPEED

A_00001.dat

stat FR_OK 0
 opened FR_OK 0

................................................................

.................................... (7436973 - 4.406094 MB/s)
 (open: 3016 us; close: 11000 us; write: min,max: 6977 131979 us)

A_00002.dat

stat FR_OK 1463
 opened FR_OK 0

................................................................

.................................... (7416981 - 4.417970 MB/s)
 (open: 6005 us; close: 9000 us; write: min,max: 6977 122959 us)

A_00003.dat

stat FR_OK 3f0
 opened FR_OK 0

................................................................

.................................... (7315981 - 4.478962 MB/s)
 (open: 15006 us; close: 11000 us; write: min,max: 6959 128979 us)

A_00004.dat

stat FR_OK 7d8
 opened FR_OK 0

................................................................

.................................... (7458994 - 4.393085 MB/s)
 (open: 50993 us; close: 65000 us; write: min,max: 6977 134978 us)

A_00005.dat

stat FR_OK bc0
 opened FR_OK 0

................................................................

.................................... (7328981 - 4.471017 MB/s)
 (open: 18006 us; close: 11000 us; write: min,max: 6959 131979 us)

openDir FR_OK
A_00001.dat 32768000 2019-01-01 00:00:08
A_00002.dat 32768000 2019-01-01 00:00:16
A_00003.dat 32768000 2019-01-01 00:00:24
A_00004.dat 32768000 2019-01-01 00:00:30
A_00005.dat 32768000 2019-01-01 00:00:38
unmount FR_OK

Code:
TESING DRIVE READ SPEED

A_00001.dat

stat FR_OK 0
 opened FR_OK 1463

................................................................

.................................... (3877977 - 8.449766 MB/s)
 (open: 3012 us; close: 1 us; write: min,max: 2998 4000 us)

A_00002.dat

stat FR_OK 1463
 opened FR_OK 3f0

................................................................

.................................... (3890949 - 8.421596 MB/s)
 (open: 39 us; close: 1 us; write: min,max: 2998 4000 us)

A_00003.dat

stat FR_OK 3f0
 opened FR_OK 7d8

................................................................

.................................... (3849994 - 8.511182 MB/s)
 (open: 8996 us; close: 1 us; write: min,max: 2998 4000 us)

A_00004.dat

stat FR_OK 7d8
 opened FR_OK bc0
................................................................
.................................... (3956991 - 8.281040 MB/s)
 (open: 11998 us; close: 1 us; write: min,max: 2998 7000 us)

A_00005.dat

stat FR_OK bc0
 opened FR_OK 107b

................................................................

.................................... (3851989 - 8.506774 MB/s)
 (open: 12000 us; close: 1 us; write: min,max: 2998 4000 us)

openDir FR_OK
A_00001.dat 32768000 2019-01-01 00:00:08
A_00002.dat 32768000 2019-01-01 00:00:16
A_00003.dat 32768000 2019-01-01 00:00:24
A_00004.dat 32768000 2019-01-01 00:00:30
A_00005.dat 32768000 2019-01-01 00:00:38
unmount FR_OK

/CODE]
 
@ all - Back home now. I was able to duplicate this error:
Code:
Test logger_RawWrite
uSDFS_VER:30_Jun_19_07_17
BUFFSIZE :32768
Dev Type :2:/
File System FS_FAT32
Free Disk Size 3183446 clusters
Cluster Size 64 sectors
Sector Size 512 bytes

Change drive

TESING DRIVE READ SPEED

A_00001.dat
stat FR_OK 0
 opened FR_OK 1371

................................................................
.................................... (12258994 - 10.691905 MB/s)
 (open: 2997 us; close: 0 us; write: min,max: 11999 101000 us)

A_00002.dat
stat FR_OK 1371
 opened FR_OK 3bc7

................................................................
.................................... (12270994 - 10.681449 MB/s)
 (open: 2996 us; close: 0 us; write: min,max: 11999 73000 us)

A_00003.dat
stat FR_NO_FILE 3bc7
 opened FR_NO_FILE 3bc7

write: Failed with rc=FR_INVALID_OBJECT.

This happened because it was trying to read a non exsistent file 'A_00003.dat'. The files must be written first.
After changing:
Code:
#define MXFN 3 // maximal number of files //was 100

Do a write test fisrt then a read test:)
 
@wwatson
Yep that's what I was doing when I got the error. However the Issue seems to be cleared up when I reinstalled the IDE and Beta4. So its is working on the t4.
 
That's good to here. I'm just setting up on the T4. I have been working on an updated version of MSC and am trying to figure out the best way to buffer up reads and writes. (mostly writes). Been checking out using LRU queuing or maybe a linked list of pointers to msCommandBlockWrapper_t structs that can be fed to MSC through callbacks.

Still learning about this stuff.
 
note to self { and whoever I cut/pasted from} - this code will NEVER turn on the LED - and likely takes it out of OUTPUT**:
Code:
  pinMode(13,OUTPUT);
  [U]pinMode[/U](13,HIGH);

this does however turn on the LED:
Code:
  pinMode(13,OUTPUT);
  [U]digitalWriteFast[/U](13,HIGH);

Wiped the 1.8.10 folder and did fresh unzip and install of TD 1.49b4 - referring to the note above I see that light … but still not getting USB data … will have to try other hardware.

**<edit>: nope :: #define OUTPUT 1 == #define HIGH 1
 
Last edited:
note to self { and whoever I cut/pasted from} - this code will NEVER turn on the LED - and likely takes it out of OUTPUT**:
Code:
  pinMode(13,OUTPUT);
  [U]pinMode[/U](13,HIGH);

this does however turn on the LED:
Code:
  pinMode(13,OUTPUT);
  [U]digitalWriteFast[/U](13,HIGH);

Wiped the 1.8.10 folder and did fresh unzip and install of TD 1.49b4 - referring to the note above I see that light … but still not getting USB data … will have to try other hardware.

**<edit>: nope :: #define OUTPUT 1 == #define HIGH 1

I see what you are saying. The amber LED on pin #13 should be turning on when setup is called:
Code:
 pinMode(13,OUTPUT);
  [U]pinMode[/U](13,HIGH);
But it does not turn on unless you use:
Code:
pinMode(13,OUTPUT);
  [U]digitalWriteFast[/U](13,HIGH);

Actually I never paid attention to this before. I wonder how far back in Arduino and TD's this has happened with.

EDIT: To be clear isn't pinMode(13,OUTPUT); and pinMode(13,HIGH); the same?
 
Problem - missing code - add the red code:

Code:
#include "uSDFS.h"
[B][COLOR="#FF0000"]// for use of USB-HUBs
#include <USBHost_t36.h>
extern USBHost myusb;
USBHub hub1(myusb);
USBHub hub2(myusb);[/COLOR][/B]

It worked on SDIO - 18 MB/s to read

On SSD write shows :: 8.173626 MB/s
On SSD read shows :: 8.458544 MB/s

Other HDD still MIA with original and read sketch …

Here is code with USB and start print change:
Code:
//Copyright 2019 by Walter Zimmer
// test_RawWrite.ino
// Version 08-jun-19
// Modified for testing wwatson
// use following lines for early definitions of multiple partition configuration in uSDFS.h
#define MY_VOL_TO_PART
#include "uSDFS.h"
// for use of USB-HUBs
#include <USBHost_t36.h>
extern USBHost myusb;
USBHub hub1(myusb);
USBHub hub2(myusb);
#include "MassStorage.h"

#if FF_MULTI_PARTITION		/* Multiple partition configuration */ 
	PARTITION VolToPart[] = {{DEV_SPI, 0}, //{ physical drive number, Partition: 0:Auto detect, 1-4:Forced partition)} 
							 {DEV_SDHC,0}, 
							 {DEV_USB, 0}, 
							 {DEV_USB, 1}, 
							 {DEV_USB, 2}
							 }; /* Volume - Partition resolution table */
#endif
// end of early definition

#define TEST_DRV 2
//
#define MXFN 5 // maximal number of files //was 100
#define MXRC 1000 // number of records in file // was 1000
const char *fnamePrefix = "A";

//
#if TEST_DRV == 0
  const char *Dev = "0:/";  // SPI
#elif TEST_DRV == 1
  const char *Dev = "1:/";  // SDHC
#elif TEST_DRV == 2
  const char *Dev = "2:/";  // USB
#endif

FRESULT rc;        /* Result code */
FATFS fatfs;      /* File system object */
FIL fil;        /* File object */

#if defined(__MK20DX256__)
  #define BUFFSIZE (2*1024) // size of buffer to be written
#elif defined(__MK66FX1M0__)
  #define BUFFSIZE (32*1024) // size of buffer to be written
#elif defined(__IMXRT1062__)
  #define BUFFSIZE (8*1024) // size of buffer to be written
#endif
// Change the next two lines for writing or reading 0 = read, 1 = write.
// uint8_t buffer[BUFFSIZE];, uint16_t buffer[BUFFSIZE];, uint32_t buffer[BUFFSIZE];
#define WR_RD 0
uint32_t buffer[BUFFSIZE];

UINT wr;

/* Stop with error message */
void die(const char *text, FRESULT rc)
{ Serial.printf("%s: Failed with rc=%s.\r\n", text,FR_ERROR_STRING[rc]);  while(1) asm("wfi"); }

//=========================================================================
void blink(uint16_t msec)
{
  digitalWriteFast(13,!digitalReadFast(13)); delay(msec);
}

#ifndef HAVETM
  #include <time.h>
  #define HAVETM
#endif
extern "C" struct tm seconds2tm(uint32_t tt);
extern "C" struct tm decode_fattime (uint16_t td, uint16_t tt);


void listDir(const char *dirName)
{
	FRESULT res;
	DIR dir;
	FILINFO fno;

	res = f_opendir(&dir, dirName);                       /* Open the directory */
	Serial.printf("openDir %s\n",FR_ERROR_STRING[res]);
	if (res == FR_OK) 
	{
		for (;;) 
		{
			res = f_readdir(&dir, &fno);                   /* Read a directory item */
			if (res != FR_OK || fno.fname[0] == 0) break;  /* Break on error or end of dir */
			if(!strncmp(fno.fname,fnamePrefix,strlen(fnamePrefix)))
			{
			  Serial.printf("%s ", fno.fname);                /* Display the object name */
			  Serial.printf("%d ", fno.fsize);                /* Display the object size */
			  struct tm tx = decode_fattime (fno.fdate, fno.ftime);
			  Serial.printf("%4d-%02d-%02d %02d:%02d:%02d\n",
							tx.tm_year,tx.tm_mon,tx.tm_mday, 
							tx.tm_hour,tx.tm_min,tx.tm_sec);
			}
		}
    }
    f_closedir(&dir);
}

void setup()
{
  // wait for serial line to come up
  pinMode(13,OUTPUT);
  digitalWriteFast(13,HIGH);
  delay(500);
  while(!Serial);
#if WR_RD == 1
  Serial.println("Test logger_RawWriteRead>Write");
#else
  Serial.println("Test logger_RawWriteRead>Read");
#endif
  Serial.print("uSDFS_VER:"); Serial.println(uSDFS_VER);
  if((rc = f_mount (&fatfs, Dev, 1))) die("Mount",rc);      /* Mount/Unmount a logical drive */
//  BUFFSIZE = (fatfs.csize*FF_MIN_SS/2);
  Serial.print("BUFFSIZE :");  Serial.println(BUFFSIZE);
  Serial.print("Dev Type :");  Serial.println(Dev);

  Serial.printf("File System %s\n", fileSystem[fatfs.fs_type]);
  Serial.printf("Free Disk Size %d clusters\n",fatfs.free_clst);
  Serial.printf("Cluster Size %d sectors\n",fatfs.csize);
#if FF_MAX_SS != FF_MIN_SS
  Serial.printf("Sector Size %d bytes\n",fatfs.ssize);
#else
  Serial.printf("Sector Size %d bytes\n",FF_MIN_SS);
#endif
  //-----------------------------------------------------------
  Serial.printf("\nChange drive\n");
  if((rc = f_chdrive(Dev))) die("chdrive",rc);
#if WR_RD == 1
  Serial.printf("\nTESING DRIVE WRITE SPEED\n\n");
#else
  Serial.printf("\nTESING DRIVE READ SPEED\n\n");
#endif
}

void loop()
{
	static uint32_t count=0;
	static int32_t ifn=0;
	static uint32_t isFileOpen=0;
	static char filename[80];
	static uint32_t t0=0;
	static uint32_t t1=0;
  static uint32_t dtwmin=1<<31, dtwmax=0;
  static uint32_t dto=1<<31, dtc=0;

  if(ifn>MXFN) { blink(500); return; }
  
  // stop testing (finish actual file)
  while (Serial.available() > 0) 
  { if ('q' == Serial.read() ) ifn = MXFN+1; }
  
  if(!count)
  {
    // close file
    if(isFileOpen)
    { dtc = micros();
      //close file
      if ((rc = f_close(&fil))) die("close", rc);
      //
      isFileOpen=0;
      t1=micros();
      dtc = t1-dtc;
      float MBs = (MXRC*BUFFSIZE*sizeof(buffer[0])*1.0f)/(1.0f*(t1-t0));
      Serial.printf(" (%d - %f MB/s)\n (open: %d us; close: %d us; write: min,max: %d %d us)\n\r",
                        t1-t0,MBs, dto, dtc, dtwmin,dtwmax);
      dtwmin=1<<31; dtwmax=0;
    }
  }
  //
  if(!isFileOpen)
  {
    // open new file
    ifn++;
    if(ifn>MXFN) 
    { listDir(Dev);
	  rc = f_unmount(Dev);
      Serial.print("unmount "); Serial.println(FR_ERROR_STRING[rc]);
      pinMode(13,OUTPUT); return; 
    } // at end of test: prepare for blinking

    dto=micros();
    sprintf(filename,"%s_%05d.dat",fnamePrefix,(int)ifn);
    Serial.println(filename);
    //
    // check status of file
    rc = f_stat(filename,0);
    Serial.printf("stat %s %x\n",FR_ERROR_STRING[rc],fil.obj.sclust);
    
#if WR_RD == 1
    rc = f_open(&fil, filename, (FA_WRITE | FA_CREATE_ALWAYS));
#else
    rc = f_open(&fil, filename, FA_READ);
#endif
    Serial.printf(" opened %s %x\n\r",FR_ERROR_STRING[rc],fil.obj.sclust);
    // check if file is Good
    if(rc == FR_INT_ERR)
    { // only option is to close file
        rc = f_close(&fil);
        if(rc == FR_INVALID_OBJECT)
        { Serial.println("unlinking file");
          rc = f_unlink(filename);
          if (rc) die("unlink", rc);
        }
        else
          die("close", rc);
      // retry open file
#if WR_RD == 1
      if(rc = f_open(&fil, filename, (FA_WRITE | FA_CREATE_ALWAYS))) die("open", rc);
#else
	  if(rc = f_open(&fil, filename, FA_READ)) die ("open",rc);
#endif
    }
    //
    isFileOpen=1;
    t0=micros();
    dto=t0-dto;
  }
  
  if(isFileOpen)
  {
     // fill buffer
#if WR_RD == 1
     for(int ii=0;ii<BUFFSIZE;ii++) buffer[ii]='0'+(count%10);
#endif
     count++;
     if(!(count%10))Serial.printf(".");
     if(!(count%640)) Serial.println(); Serial.flush();
     uint32_t ta=micros();
#if WR_RD == 1
     //write data to file 
     rc = f_write(&fil, buffer, BUFFSIZE*sizeof(buffer[0]), &wr);
#else
     //read data from file 
     rc = f_read(&fil, buffer, BUFFSIZE*sizeof(buffer[0]), &wr);
#endif
     uint32_t tb=micros();
     if (rc == FR_DISK_ERR) // IO error
     {  Serial.printf(" write FR_DISK_ERR at count # %d\n",count);
        // only option is to close file
        // force closing file
        count=MXRC;
     }
     else if(rc) die("write",rc);
    //    
     uint32_t dt=tb-ta;
     if(dt<dtwmin) dtwmin=dt;
     if(dt>dtwmax) dtwmax=dt;
     //
     count %= MXRC;
  }    
}
 
@defragster
Think you needed that code only when you are using a hub or that Oricon hub with the embedded drive. I just ran my SSD and saw about the same as @KurtE using 8K buffer:
Code:
Writes:  5-9 MBs
Reads:  10.8Mbs
If I changed the buffer to 32:
Code:
Reads: 86 MBs
 
Starting off slightly off-topic. I just received a new Raspberry PI 4, that hopefully tomorrow will set up. Forgot to get a micro HDMI cable. It is now in mail box in town...

Was looking around on how to install Ubuntu on it and ran across a site that shows you how to install it on an SSD: https://jamesachambers.com/raspberry-pi-ubuntu-server-18-04-2-installation-guide/

Included in this are some cheap setups for SSD drives.
One is Do it your self with a cheap SSD: https://jamesachambers.com/raspberry-pi-cheap-ssd-upgrade-30/

cheap SSD Drive: https://www.amazon.com/Kingston-120...53dbebfb93ed94bab4c5b0c7710cdf&language=en_US

Cheap Adapter: https://www.amazon.com/StarTech-com...7793fb63d038ad872536659ae548a8&language=en_US

Total: $19.99+$9.99 or lets say $30

Or a bit more expensive, a plug in thumb drive wheich is SSD: https://www.amazon.com/dp/B01MU8TZR...15cba1635572a5cf7083fdb96238df&language=en_US $46

The reason I mention that here, is maybe it might be fun to try one of these here... And then maybe end up using it for the new RPI4 (or 3)..

Did you ever do any speed testing with the Kingston SSD drive and the StarTech USB to SATA adapter? I bought them and using the latest Arduino 8.10 and TD 1.49b5 I am getting transfer speeds of 10.733134 MB/s for both reads and writes using 8k to 32k buffer sizes. If you write uint32_t BYTE file sizes and read the file with a buffer size less than the buffer size you wrote the file then your speeds will not reflect the true transfer speed. It will show a faster transfer. Still playing with this and other combinations of buffer sizes and word sizes.
 
@defragster
Think you needed that code only when you are using a hub or that Oricon hub with the embedded drive. I just ran my SSD and saw about the same as @KurtE using 8K buffer:
Code:
Writes:  5-9 MBs
Reads:  10.8Mbs
If I changed the buffer to 32:
Code:
Reads: 86 MBs

Indeed - only needed with a HUB - and except FLASH drives or powered external drives - my bare HDD's and SSD's all require a HUB for power.

I just bumped buffer from 8 to 32 with uint32_t array of "#define BUFFSIZE (32*1024)"

On older SSD READ went from 8.5 to :: 74.496475 MB/s


Not sure why but a couple of drives I thought worked before won't work on either sketch now ... will revisit later
 
Indeed - only needed with a HUB - and except FLASH drives or powered external drives - my bare HDD's and SSD's all require a HUB for power.

I just bumped buffer from 8 to 32 with uint32_t array of "#define BUFFSIZE (32*1024)"

On older SSD READ went from 8.5 to :: 74.496475 MB/s


Not sure why but a couple of drives I thought worked before won't work on either sketch now ... will revisit later

Did you write to the disk with "#define BUFFSIZE (32*1024)"? This is what I am trying to figure out. Probably something obvious. If I write to the disk with a 32 bit 32768 byte buffer the SSD write speed is 10.7 MB/s. If I change the bufffer size to (8*1024) and do a read then I get 43.087784 MB/s. If I change the buffer size back to (32*1024) and do a read I get 10.811867 MB/s as read speed. I am not sure where the difference is coming from. Probably something I have done in test_RawWrite.ino. Not sure yet.

The SSD is faster and more consistent with the read and write speeds.

I am testing right now with a T36 at 240 MHz. Not using a hub. Just directly connected to the HOST USB port. Did not need a powered USB hub.
 
I just swapped out 256GB SSD on PC labelled 5V 1.7A. I put in a new 512GB SSD that said writes take 3.3W or 0.66A if at 5V. The 2011 Teensy Test 120GB SSD says 5V at 1A. So none of these would run from Teensy shared USB 5V. That 2011 or prior older dead SSD's had Red LED suggesting is was not happy and never showed as mounting without being on a powered HUB.

Did you write to the disk with "#define BUFFSIZE (32*1024)"? This is what I am trying to figure out. Probably something obvious. If I write to the disk with a 32 bit 32768 byte buffer the SSD write speed is 10.7 MB/s. If I change the bufffer size to (8*1024) and do a read then I get 43.087784 MB/s. If I change the buffer size back to (32*1024) and do a read I get 10.811867 MB/s as read speed. I am not sure where the difference is coming from. Probably something I have done in test_RawWrite.ino. Not sure yet.

The SSD is faster and more consistent with the read and write speeds.

I am testing right now with a T36 at 240 MHz. Not using a hub. Just directly connected to the HOST USB port. Did not need a powered USB hub.

Yes, that was copied from code where 8 was changed to 32 and read speed went way up like in prior post.

32K DWORD buffer: #define BUFFSIZE (32*1024) // size of buffer to be written
FlexRAM section ITCM+DTCM = 512 KB
Config : aaaaaaaf
ITCM : 52784 B (80.54% of 64 KB)
DTCM : 156352 B (34.08% of 448 KB)
Available for Stack: 302400

8K DWORD buffer: #define BUFFSIZE (8*1024) // size of buffer to be written
FlexRAM section ITCM+DTCM = 512 KB
Config : aaaaaaaf
ITCM : 52784 B (80.54% of 64 KB)
DTCM : 58048 B (12.65% of 448 KB)
Available for Stack: 400704
 
@ALL - Here is something I ran across testing changes to MSC. I noticed that the write speeds of drives are different if using USB hub rather than being
directly connected to the T4/3 host port. The write speeds seem to be faster without using a hub. I tried three different hubs one of which was a powered
hub. The results were the same.

SSD - NO HUB:
Code:
Test logger_RawWriteRead>Write
uSDFS_VER:30_Jun_19_07_17
maxLUN = 0
BUFFSIZE :8192
Dev Type :2:/
File System FS_FAT32
Free Disk Size 3649848 clusters
Cluster Size 64 sectors
Sector Size 512 bytes

Change drive

TESING DRIVE WRITE SPEED

A_00001.dat
stat FR_OK 0
 opened FR_OK 0

................................................................
.................................... (3260999 - [COLOR="#FF0000"]10.048454 MB/s[/COLOR])
 (open: 104992 us; close: 15000 us; write: min,max: 2984 20985 us)

Bytes Read or Written: 32768000.
openDir FR_OK
A_00001.dat 32768000 2020-01-25 14:44:18
A_00002.dat 65536000 2020-01-25 13:54:48
unmount FR_OK

SSD - WITH HUB:
Code:
Test logger_RawWriteRead>Write
uSDFS_VER:30_Jun_19_07_17
maxLUN = 0
BUFFSIZE :8192
Dev Type :2:/
File System FS_FAT32
Free Disk Size 3649848 clusters
Cluster Size 64 sectors
Sector Size 512 bytes

Change drive

TESING DRIVE WRITE SPEED

A_00001.dat
stat FR_OK 0
 opened FR_OK 0

................................................................
.................................... (4252999 - [COLOR="#FF0000"]7.704681 MB/s[/COLOR])
 (open: 105992 us; close: 15000 us; write: min,max: 3984 23985 us)

Bytes Read or Written: 32768000.
openDir FR_OK
A_00001.dat 32768000 2020-01-25 14:46:26
A_00002.dat 65536000 2020-01-25 13:54:48
unmount FR_OK

HDD - NO HUB:
Code:
Test logger_RawWriteRead>Write
uSDFS_VER:30_Jun_19_07_17
maxLUN = 0
BUFFSIZE :16384
Dev Type :2:/
File System FS_FAT32
Free Disk Size 7614019 clusters
Cluster Size 64 sectors
Sector Size 512 bytes

Change drive

TESING DRIVE WRITE SPEED

A_00001.dat
stat FR_OK 0
 opened FR_OK 0

................................................................
.................................... (6533999 - [COLOR="#FF0000"]10.029999 MB/s[/COLOR])
 (open: 198992 us; close: 15000 us; write: min,max: 5969 50970 us)

Bytes Read or Written: 65536000.
openDir FR_OK
A_00007.dat 65536000 2020-01-12 20:38:28
A_00002.dat 65536000 2020-01-13 18:27:36
A_00001.dat 65536000 2020-01-25 14:52:52
unmount FR_OK

HDD - WITH HUB:
Code:
Test logger_RawWriteRead>Write
uSDFS_VER:30_Jun_19_07_17
maxLUN = 0
BUFFSIZE :16384
Dev Type :2:/
File System FS_FAT32
Free Disk Size 7615019 clusters
Cluster Size 64 sectors
Sector Size 512 bytes

Change drive

TESING DRIVE WRITE SPEED

A_00001.dat
stat FR_OK 0
 opened FR_OK 0

................................................................
.................................... (8572999 - [COLOR="#FF0000"]7.644466 MB/s[/COLOR])
 (open: 132992 us; close: 15000 us; write: min,max: 7969 99970 us)

Bytes Read or Written: 65536000.
openDir FR_OK
A_00007.dat 65536000 2020-01-12 20:38:28
A_00002.dat 65536000 2020-01-13 18:27:36
A_00001.dat 65536000 2020-01-25 14:57:40
unmount FR_OK

With the SSD I used both StarTech USB to SATA adapters (2.0 and 3.0) with the same results.
Not sure what is causing this difference in write speeds. Read speeds are the same with or without using a hub.
 
@Paul, @All - I have spent the last few months re-writing MSC. Now in a separate
branch. (MSC2). I restructured it to move all but the basic low level functions
from the driver layer to host layer.

Wanted to:
1) Add the ability to queue an array of transfers but not implemented yet.
2) Add Support for more than one USB storage device. This works, but still working
on bullet proofing hot plugging one or more devices. This has taken quite some
time to figure out but I think I have it close to a reliable state.
Still need to be able to auto initialize and de-initialize devices as they are plugged or
unplugged from the Teensy. Using more than two USB storage devices at a time
becomes unstable depending on how many hubs and what devices are plugged in
without a powered USB HUB so I have limited MSC2 to two instances of msController
for now. More can be added. How practical is more than one or two USB storage
devices on the Teensy? I have used up to four USB drives with a powered Hub and
MSC2 recognized them all. MSC2 recognizes the use of multiple HUBS.
3) Reduce redundant code. Lot's of work:) WIP.
4) Add more comprehensive information about connected devices including HUB number,
port number, device address and mounted drive info. All available from function
calls after mounting and initializing the drive. Kept in a separate structure for each
device and accessible from function calls. Hot plugging included.
5) Keep it backwards compatible with the original MSC and uSDFS examples as much
as possible. Due to the ability to use multiple USB devices I had to make changes
to uSDFS. I forked the latest version from WXMZ and updated my fork to match
the changes. MSC2 will need to be used with my fork of uSDFS. The readme file
explains what needs to be done.
6) Put MSC2 in a condition to be used with any upgrade of USBHost_t36. (Maybe)

I have tested MSC2 on the T4 and T36. Both work. Other than initialization I have
not used any delays in any other part of the code (I think).
The transfer speeds are staying the same if not better than the original MSC version.
I am making sure MSC2 is as stable as if not more than the original version of MSC.

USER TESTING NEEDED! Please test.

I have written some example sketches to test different aspects of
MSC2. More to come.

Additional thoughts:
Use with SDFat.
Use the a dedicated small OS. Works with mine:)
Use with MP3. (Waveplayer works)
Use with streaming video.
What would be needed? Thoughts?

These are things I am really curious about.

Here is the link to MSC2:
https://github.com/wwatson4506/MSC/tree/MSC2
The link to my updated version of uSDFS:
https://github.com/wwatson4506/uSDFS
 
All - Finally got back to working with MSC. In particular trying to speed up write speeds. I have been experimenting with queuing an array of transfers and was hitting a road block with queuing the array 'transferIndex'. I spent a whole weekend trying to figure out why when I used the uSDFS RawWrite test sketch with the eventResponder it would only increment the 'transferIndex' value from 0 to 1.

Long story short, copy and paste is fine if you pay attention to what you have just copied and pasted and what functionality you are seeking.
Check this...
Failing code:
Code:
class msController : public USBDriver {
public:
	msController(USBHost &host) { init(); }
	msController(USBHost *host) { init(); }

	volatile bool transferComplete = false; //Unfortunately must be public so mscEventResponder() can set it
	volatile bool transferIndex = 0;           //Unfortunately must be public so mscEventResponder() can set it
'transferIndex' starts at 0. Increments no higher than 1.

Realized what I had done or failed to do, fixed it like this:
Code:
class msController : public USBDriver {
public:
	msController(USBHost &host) { init(); }
	msController(USBHost *host) { init(); }

	volatile bool transferComplete = false; //Unfortunately must be public so mscEventResponder() can set it
	volatile [COLOR="#FF0000"]uint8_t[/COLOR] transferIndex = 0; //Unfortunately must be public so mscEventResponder() can set it

A lot of time wasted trying to rush through this. I have taken time to learn and understand the principals of using eventResponder thanks @KurtE with his example sketch, @Paul for the way he documented the eventResponder library and researching the different types of queuing, linked lists and such.

I am so curious to see if I can speed up MSC transfer rates.
 
Mass Storage Controller for Teensy 3.6 and Teensy 4.x

This the latest version of MSC that I am submitting. It should function the same way as the original MSC did but also has the ability to do USB disk transfers in a non-blocking mode. Using the non-blocking mode is probably only useful with a an RTOS or modified version of FatFS or SDFat. I tried modifying FatFS but that failed due to the fact that it had to wait for the transfers to complete before processing. There will have to be a mechanism to signal completion of data to or from the USB drive. I guess file locking and semaphores. Some improvements have been made. It still retains the blocking mode of MSC. It remains compatible with my updated version of uSDFS. My newer version uSDFS-non-blocking will have to be used when using uSDFS examples.

To really see what is happening in the background with this non-blocking version I have set up two pins on the T4.1 to run a red and green LED. Red for writes and green for reads. Using MSC-non-blocking.ino will give you several options to test with. Pin defines are in the MassStorage.h file so change to suit. I do not have the blocking version of this driver using LED's.

*********** BIG WARNING ************: Because MSC-non-blocking.ino is using direct sector reads and writes any OS formatting to the USB mass storage device will be overwritten with a write to the device requiring it to be reformatted. The start sector address of the reads and writes are set high but still can destroy formatting. Use on USB drives with no volatile data!!!!

Key things you can play with to get various results.

MassStorage.h in this section:

Define queue sizes
#define MAX_TRANSFERS 1024.

read/write queue sizes (must be powers of 2!!) As the queue sizes get Smaller and depending on the size of the read or write buffer the bigger the chance of MSC going into blocking. If the queues are full it will block until space is available in the queues. This can be a memory eater:)

Setup debugging LED pin defs. Used mainly to see the activity of non-blocking reads and writes.
#define WRITE_PIN 33 // Pin number of drive read activity led (RED LED).
#define READ_PIN 34 // Pin number of drive read activity led (GREEN LED).

UnComment the following define for verbose debug output.
//#define DBGprint 1

UnComment to watch read/write queue activity.
//#define DBGqueue 1

Define USE_EXTERNAL_INIT to Initialize MSC Externally when not using uSDFS. If defined, MSC needs the following in the setup section of the ino sketch:

USBHost myusb;
myusb.begin();
mscHost.mscHostInit();

//#define USE_EXTERNAL_INIT

In MSC-non-blocking.ino:

#define FILLCHAR 0xff
Change this to write a different value to sectors. Also used for verifying multi sector writes.

UnComment #define BLOCKING to wait for the transfer to complete (blocking) and the time it took. Reads and writes will block until transfer is complete. Will return to loop() when complete. This will read and write sectors in blocking mode and display read and write times. Setting MSC to non-blocking mode will show how long it takes to queue reads or writes.

//#define BLOCKING

MSC in non-blocking mode is using tonton81's CircularBuffer and luni64's attachYieldFunc libraries. As such, using attachYieldFunc requires any while statements to include the yield() function to keep it from blocking. I have tried to comment the source code heavily to explain what is going on.

THIS IS JUST PROOF OF CONCEPT:)

There are probably better way's to do this. Still learning.

Here are the links.

MSC-non-blocking:
https://github.com/wwatson4506/MSC/tree/MSC-non-blocking

uSDFS-non-blocking:
https://github.com/wwatson4506/uSDFS/tree/uSDFS-non-blocking
 
Hi wwatson
Thanks for your library.
I'm always working with your previous MSC (blocking ?) version, and it work with old USB stick, but not with lots of new one. The library stay lock in f_mount function with this kind of stick.
Do you think your last version can solve this ? I think there is problem during enumerate process but it's out of my competence. Perhapse it's problem with USBHost_T36 and not MSC ?
I have take a look in your mscFS version https://github.com/wwatson4506/mscFSand like you explain, the try to install last version of USBHost_V36. But when i run you example mscTesting but encounter compilation error. After i have install Teensyduino 1.54 rev 5 but dont find your MSC files in USBHost_T36 folder.
Eric
 
Hi wwatson
Thanks for your library.
I'm always working with your previous MSC (blocking ?) version, and it work with old USB stick, but not with lots of new one. The library stay lock in f_mount function with this kind of stick.
Do you think your last version can solve this ? I think there is problem during enumerate process but it's out of my competence. Perhapse it's problem with USBHost_T36 and not MSC ?
I have take a look in your mscFS version https://github.com/wwatson4506/mscFSand like you explain, the try to install last version of USBHost_V36. But when i run you example mscTesting but encounter compilation error. After i have install Teensyduino 1.54 rev 5 but dont find your MSC files in USBHost_T36 folder.
Eric

First of all mscFS is for use with my version of SdFat-parallel which is a version of the Teensy file abstraction 'FS.h' being developed right now and is just an experiment with using MSC and SdFat. This is not usable by itself.

Next, if you are using 'mscTesting.ino' you will need to backup the 'USBHost_t36' library in TD1.54B5 'arduino-1.8.13/hardware/teensy/avr/libraries' folder and then copy my version of USBHost_t36 to the same folder. The library is found here:

https://github.com/wwatson4506/USBHost_t36

As far as the new USB sticks not working I would un-comment the debug print enable in line #62. Then upload 'mscTesting.ino' to the Teensy without a USB stick plugged in. After the Teensy boots plugin the USB stick that is not working. Then post the output from the terminal here. Normally with 'mscTesting.ino' if you press enter it should show the device enumeration.

mscTesting.ino does not use a file system or mount the USB stick. If it has been enumerated and claimed it will give information about it.
 
Last edited:
First of all mscFS is for use with my version of SdFat-parallel which is a version of the Teensy file abstraction 'FS.h' being developed right now and is just an experiment with using MSC and SdFat. This is not usable by itself.

Next, if you are using 'mscTesting.ino' you will need to backup the 'USBHost_t36' library in TD1.54B5 'arduino-1.8.13/hardware/teensy/avr/libraries' folder and then copy my version of USBHost_t36 to the same folder. The library is found here:

https://github.com/wwatson4506/USBHost_t36

As far as the new USB sticks not working I would un-comment the debug print enable in line #62. Then upload 'mscTesting.ino' to the Teensy without a USB stick plugged in. After the Teensy boots plugin the USB stick that is not working. Then post the output from the terminal here. Normally with 'mscTesting.ino' if you press enter it should show the device enumeration.

mscTesting.ino does not use a file system or mount the USB stick. If it has been enumerated and claimed it will give information about it.
Ok i will test during this week and send you result.
But before, one last question.Is there limitations for stick size, Fat format cluster size, or any recommendations of choose of kind USB Stick for the using with your previous MSC library ?
 
Ok i will test during this week and send you result.
But before, one last question.Is there limitations for stick size, Fat format cluster size, or any recommendations of choose of kind USB Stick for the using with your previous MSC library ?

I have just tested a 128GB USB thumb drive without a problem. In the past several people as well as myself have tested with several different types of USB Mass Storage devices. The size of the device really only matters to the program using the USB device like FatFS, SdFAT and of course the microcontroller. I have a USB to IDE/SATA adaptor that I have tested IDE drives, SATA drives (750GB so far) and a 120GB SSD drive with success.

Other devices successfully used:

1) USB SD card readers.

2) SATA and IDE CDROMS. (Only as far as recognizing the CDROM's , mounting and unmounting the media).

As I stated before, I am working heavily with SdFat and MSC right now and I am not sure how or if this will be incorporated into Teensyduino yet.

Hope this helps:)
 
Hi Wwatson
I do what you are suggested in post #394, and now I can compile mscTesting.ino. I have tested my stick and now can enumerate it.
You speak about SdFat, is-it this https://github.com/greiman/SdFat ?
Do you have one simple example where you used MSC and SdFat? for teensy. I have previously used your MSC with µSDFS and I'm not sure that I can modify it simply.
Thanks
 
Hi Wwatson
I do what you are suggested in post #394, and now I can compile mscTesting.ino. I have tested my stick and now can enumerate it.
You speak about SdFat, is-it this https://github.com/greiman/SdFat ?
Do you have one simple example where you used MSC and SdFat? for teensy. I have previously used your MSC with µSDFS and I'm not sure that I can modify it simply.
Thanks

No it is not Bill Greiman's current SdFat-beta it is a modified version of it that I forked from Paul Stoffregen's repository. I am in the process of adapting MSC to use SdFat as well as FatFS. But again this is work in progress and until Teensyduino 1.54B6 or final is released I am not sure which version of SdFat that I am modifying should be used or if Paul has a plan for using MSC with SdFat. In any case it will follow the file access abstraction through FS.h.

For now with all the changes going on I would just stick with the version(s) of MSC and uSDFS that you are currently using if it is working for you:)
 
Okay. Thanks for these clarifications and hoping to soon have a final version integrated in teensyduino.I will continue for the moment with the µSDFS version, but its main problem for me is that it does not work with all USB sticks, or at least in the use I’ve made of it. Thank you again for your help and work.:)
 
@wwatson - I would like to start playing around some as I know these capabilities would be great addition to Teensy world.

Are the instructions up at: https://forum.pjrc.com/threads/5582...er-Experiments?p=231523&viewfull=1#post231523
Maybe the best thing to try things out?

It looks like you had a delta to usbhost_t36? has this already been folded int a Pull Request or do we need to?

Would be fun to try this stuff out, and maybe see if it can feed through to MTP...
 
Back
Top