Teensyduino File System Integration, including MTP and MSC

Back power from Hub is odd/good/bad. Indeed once it latches - it sticks/stays on. The External Hub with inbuilt 2.5" HDD carrier here does that. Nice package and assures HDD has needed power - and 3 USB ports ... but it also backfeeds power for better for worse.

USB Host on Teensy has a controlled switch, or is that just a current limiter?

Sunday and Monday strong wind for sure as I read it. If I get the pickup unloaded I can put a tarp away before the wind takes it.
 
Just pushed up changes...

Will hack some on ExFat, yes I believe it will be very similar with Fat32/16...

Would be curious to see your SSD drive MBR print to see where the extended one was added...
That is are the 4 slots like: FAT32 <extended> <unused> <unused> ?

If we support extended partition, will be interesting to decide on partition numbers to pass in: In theory could have:

<Fat32> <extended with: Fat32 ExFat UNUSED UNUSED> <FAT16> <UNUSED>

So If I pass in 3 do I get the Fat16 of MBR, or should it the ExFat of the Extended partition...

Unfortunately don't have any GPT drives to experiement with. But did set up a 128GB jump drive to have a single exFAT partition and 2 extended partitions. THink I got all the updates installed and this is what I am seeing:
Code:
USB Drive: 0 connected

msc # Partition Table
	part,boot,bgnCHS[3],type,endCHS[3],start,length
exFAT:	1,0,0x20,0x21,0x0,0x7,0xFE,0xFF,0xFF,2048,118290432
Extend:	2,0,0xFE,0xFF,0xFF,0xF,0xFE,0xFF,0xFF,118292480,123730048
pt_#0:	3,0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0,0
pt_#0:	4,0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0,0
addFilesystem: 1 MSC0-p1 20008a70
Which is what I was expecting to see from what I was reading.

So looks like I have to read the 512bytes start at sector 118292480 to 123730048 to actually see the extended partition data. Why do I say this I can see where the 2 actual extend partitions exist in WinHex.

My guess looks like
0 = exFat
1 = extended
2 = exFAT
3 = FAT32
but have to grab those sectors to be sure which I will do in the morning
 
Back power from Hub is odd/good/bad. Indeed once it latches - it sticks/stays on. The External Hub with inbuilt 2.5" HDD carrier here does that. Nice package and assures HDD has needed power - and 3 USB ports ... but it also backfeeds power for better for worse.

USB Host on Teensy has a controlled switch, or is that just a current limiter?

Sunday and Monday strong wind for sure as I read it. If I get the pickup unloaded I can put a tarp away before the wind takes it.

Tim - good luck with the wind - know it can be a issue with roofs and anything loose. Been there done that.
 
Unfortunately don't have any GPT drives to experiement with. But did set up a 128GB jump drive to have a single exFAT partition and 2 extended partitions. THink I got all the updates installed and this is what I am seeing:
Code:
USB Drive: 0 connected

msc # Partition Table
	part,boot,bgnCHS[3],type,endCHS[3],start,length
exFAT:	1,0,0x20,0x21,0x0,0x7,0xFE,0xFF,0xFF,2048,118290432
Extend:	2,0,0xFE,0xFF,0xFF,0xF,0xFE,0xFF,0xFF,118292480,123730048
pt_#0:	3,0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0,0
pt_#0:	4,0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0,0
addFilesystem: 1 MSC0-p1 20008a70
Which is what I was expecting to see from what I was reading.

So looks like I have to read the 512bytes start at sector 118292480 to 123730048 to actually see the extended partition data. Why do I say this I can see where the 2 actual extend partitions exist in WinHex.

My guess looks like
0 = exFat
1 = extended
2 = exFAT
3 = FAT32
but have to grab those sectors to be sure which I will do in the morning
Will play with extended tomorrow…

As for GPT, if desired you can convert…
https://docs.microsoft.com/en-us/wi...management/change-an-mbr-disk-into-a-gpt-disk

Edit: I now have thumb drive setup for extended:

Code:
msc # Partition Table
	part,boot,bgnCHS[3],type,endCHS[3],start,length
FAT32:	1,0,0x20,0x21,0x0,0xC,0xFA,0xFF,0xFB,2048,16384000
FAT32:	2,0,0xFB,0xC1,0xFB,0xC,0xFE,0xFF,0xFF,16386048,8388608
Extend:	3,0,0xFE,0xFF,0xFF,0xF,0xFE,0xFF,0xFF,24774656,35846144
pt_#0:	4,0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0,0

USB Drive: 2 connected

But Think I am done for today
 
Last edited:
If/when I can ferret out all the needed WIP's I can try some of what I have here ...

Windows 10/11 now always make GPT's! This is a hassle when cloning a HDD for a Win 7 machine as Win 7 doesn't know what to do with it. And Acronis makes the new drive GPT - completes a non-booting clone when returned to the Win 7 machine :( And until that is apparent ... a lot of time can be wasted doing it over (and over) assuming the clone failed :mad:

Wood all transferred! All fit in the rack with the new taller roofed rack - no more head bumping from the one I did 20 years ago :) ... that was snapped in half :( Broke cleanly and the old plywood still usable except a 3'x4' make up piece. So all wind safe if trees don't fall.
 
Back power from Hub is odd/good/bad. Indeed once it latches - it sticks/stays on. The External Hub with inbuilt 2.5" HDD carrier here does that. Nice package and assures HDD has needed power - and 3 USB ports ... but it also backfeeds power for better for worse.

USB Host on Teensy has a controlled switch, or is that just a current limiter?

Sunday and Monday strong wind for sure as I read it. If I get the pickup unloaded I can put a tarp away before the wind takes it.

Live in the great northwest like I do?
 
Live in the great northwest like I do?

Yes, I saw a comment of yours the other day indicating we do live in the same corner of the world.

Phone weather notes: "Gale Warning from SUN 12:00 AM PDT until TUE 5:00 AM PDT" - hazardous seas ... winds of 34 to 47 knots (54 MPH) are immanent or occurring
 
Will play with extended tomorrow…

As for GPT, if desired you can convert…
https://docs.microsoft.com/en-us/wi...management/change-an-mbr-disk-into-a-gpt-disk
.....

But Think I am done for today

Morning all
Created a thumb drive version of a GPT disk which is now working.
Code:
USB Drive: 0 connected

msc # Partition Table
	part,boot,bgnCHS[3],type,endCHS[3],start,length
*** GPT Disk not supported ***
GPT guard:	1,0,0x0,0x2,0x0,0xEE,0xFE,0xBF,0xDB,1,4294967295
pt_#0:	2,0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0,0
pt_#0:	3,0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0,0
pt_#0:	4,0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0,0

GPT partition header revision: 10000
LBAs current:1 backup:242075647 first:34 last:242075614
Disk GUID:494E6C4B-0928-4119-6EED-8BD052C8A7BAStart LBA Array: 2 Count: 128 size:128
Part	 Type Guid, Unique Guid, First, last, attr, name
0	EBD0A0A2-B9E5-4433-87C0-68B6B72699C7, 466E5E58-6453-84BB-6CDD-3B813E5167A3, 2048, 242073599, 0, 
>>> Microsoft Basic Data Partition
EB 76 90 45 58 46 41 54 20 20 20 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 :.v.EXFAT   .....................
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 :................................
and does with MTP.

Just one big bit of WARNING: DO NOT USE MTP FORMATTER to reformat the disk. It will wipe the GPT disk clean and just create a MBR exFAT version on the disk since it is only applicable to MBR disks - probably need some sort of guard for GPT disk formatting.
 
Morning all
Created a thumb drive version of a GPT disk which is now working.
Code:
USB Drive: 0 connected

msc # Partition Table
	part,boot,bgnCHS[3],type,endCHS[3],start,length
*** GPT Disk not supported ***
GPT guard:	1,0,0x0,0x2,0x0,0xEE,0xFE,0xBF,0xDB,1,4294967295
pt_#0:	2,0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0,0
pt_#0:	3,0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0,0
pt_#0:	4,0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0,0

GPT partition header revision: 10000
LBAs current:1 backup:242075647 first:34 last:242075614
Disk GUID:494E6C4B-0928-4119-6EED-8BD052C8A7BAStart LBA Array: 2 Count: 128 size:128
Part	 Type Guid, Unique Guid, First, last, attr, name
0	EBD0A0A2-B9E5-4433-87C0-68B6B72699C7, 466E5E58-6453-84BB-6CDD-3B813E5167A3, 2048, 242073599, 0, 
>>> Microsoft Basic Data Partition
EB 76 90 45 58 46 41 54 20 20 20 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 :.v.EXFAT   .....................
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 :................................
and does with MTP.

Just one big bit of WARNING: DO NOT USE MTP FORMATTER to reformat the disk. It will wipe the GPT disk clean and just create a MBR exFAT version on the disk since it is only applicable to MBR disks - probably need some sort of guard for GPT disk formatting.

Sorry, I know I should have mentioned that I have not looked at the format code yet. Note: the changes for this would need to be in our PFSLib format code as well SDFat always format code ignores everything and writes one big volume... As I have been told SDCards were not made for partitions....
 
Sorry, I know I should have mentioned that I have not looked at the format code yet. Note: the changes for this would need to be in our PFSLib format code as well SDFat always format code ignores everything and writes one big volume... As I have been told SDCards were not made for partitions....

You did note format was a collective wipe for MBR's - no surprise the just recognized follow on GPT is similar. Format seemed a dangerous add. Does it perform a low level format - or just clean the disk structures away?
 
You did note format was a collective wipe for MBR's - no surprise the just recognized follow on GPT is similar. Format seemed a dangerous add. Does it perform a low level format - or just clean the disk structures away?

in the end there are three types of partitions: As I know you are extremely knowledgeable in...

The simple MBR ones: And yes these will update the MBR:
Extended MBR Which has an Extended MBR entry, which points off to the rest of the disk:
Under this there are then Logical partitions which each entry has the equivalent of 2 MBR entries, the first for the partition (Logical Partition) that is described and the 2nd one a pointer to the next one of these. You can chain a lot of them this way... With these, the starting/ending LBA number are relative to the start of the Extended partition...

Then there is GPT, which is whole other ball of wax... Most of them are setup to handle a max of 128 partitions. These partitions do not have the same data as the MBR, but instead work with GUIDs, one of these QUIDs is the partition type. There are some known types out there including one Microsoft Basic Data, which then the first block is the Boot Parameter Block, which is what we use to deduce what thpe partition...

So now cleaning up some of the format.

Note: I did push up some more changes to my SDFat branch which updated the init code to follow similar code like SDFat does, where instead of uint32_t you use an array of 4 unit8_t and then there are inline methods which give you the uint32_t value for both Endian type systems.

Code is still not working properly. I have it avoid updating the MBR record, but something still is.

About to add in some more debug code
 
@Paul, @KurtE, @mjs513 - I think I have finally got the gist of what you guy's are trying to accomplish with FS. Using the latest libraries from @KurtE I put together a sketch to test using LittleFS and MSC. Have not tested SD yet but I assume it will use the same principles of FS. That's next. Ran out of time this weekend:(

Kurt, I spent time studying MTP_Teensy and what you have done with it. This led me create this sketch:
Code:
// FStesting.ino
//
#include "Arduino.h"
#include <USBHost_t36.h>
#include <USBHost_ms.h>
#include <LittleFS.h>

// Add USBHost objectsUsbFs
USBHost myusb;
USBHub hub1(myusb);
USBHub hub2(myusb);
USBHub hub(myusb);

// MSC objects.
msController drive1(myusb);
msController drive2(myusb);
msController drive3(myusb);

msFilesystem msFS1(myusb);
msFilesystem msFS2(myusb);
msFilesystem msFS3(myusb);
msFilesystem msFS4(myusb);
msFilesystem msFS5(myusb);

// Quick and dirty
msFilesystem *pmsFS[] = {&msFS1, &msFS2, &msFS3, &msFS4, &msFS5};
#define CNT_MSC  (sizeof(pmsFS)/sizeof(pmsFS[0]))
uint32_t pmsfs_store_ids[CNT_MSC] = {0xFFFFFFFFUL, 0xFFFFFFFFUL, 0xFFFFFFFFUL, 0xFFFFFFFFUL, 0xFFFFFFFFUL};
char  pmsFS_display_name[CNT_MSC][20];

msController *pdrives[] {&drive1, &drive2, &drive3};
#define CNT_DRIVES  (sizeof(pdrives)/sizeof(pdrives[0]))
bool drive_previous_connected[CNT_DRIVES] = {false, false, false};

LittleFS_QPINAND myfs; // This will become an array of LFS devices.

FS *mscDisk;
File dataFile; // Specifes that dataFile is of File type

void setup() {
  // Open serial communications and wait for port to open:
   while (!Serial) {
    SysCall::yield(); // wait for serial port to connect.
  }
#if defined(ARDUINO_TEENSY41)
  if(CrashReport)
    Serial.print(CrashReport);
#endif  
  // This line will only work with VT100 capable terminal program.
  Serial.printf("%c",12); // Clear screen (VT100).

  myusb.begin();
 
  Serial.printf(F("FS Testing\r\n\r\n"));
  Serial.printf(F("Initializing, please wait...\r\n\r\n"));

// lets initialize a QPINAND drive.
  if (myfs.begin()) {
    Serial.printf("QPINAND Drive initialized\r\n");
  }

  myusb.Task();
//delay(1000);
  USBMSCDevice mscDrive;
  PFsLib pfsLIB;
  for (uint8_t i=0; i < CNT_DRIVES; i++) {
	elapsedMillis em = 0;
	while (em < MEDIA_READY_TIMEOUT) {
		myusb.Task();
	}
    if (*pdrives[i]) {
      if (!drive_previous_connected[i]) {
        if (mscDrive.begin(pdrives[i])) {
          Serial.printf("\nUSB Drive: %u connected\r\n", i);
          pfsLIB.mbrDmp(&mscDrive, (uint32_t)-1, Serial);
          drive_previous_connected[i] = true;
         }
      }
    } else {
      drive_previous_connected[i] = false;
    }
  }
  for (uint8_t i=0; i < CNT_MSC; i++) {
    char volName[20];
    if (pmsFS[i]->mscfs.getVolumeLabel(volName, sizeof(volName)))
      snprintf(pmsFS_display_name[i], sizeof(pmsFS_display_name[i]), "MSC%d-%s", i, volName);
    else
      snprintf(pmsFS_display_name[i], sizeof(pmsFS_display_name[i]), "MSC%d", i);
	  Serial.printf("%s\r\n",pmsFS_display_name[i]);
  }
  Serial.printf("Directory Listing for QPINAND\r\n");
  mscDisk = &myfs;
  listFiles(mscDisk);
  Serial.printf("Directory Listing for %s\r\n", pmsFS_display_name[0]);
  mscDisk = pmsFS[0];
  listFiles(mscDisk);
  Serial.printf("Directory Listing for %s\r\n", pmsFS_display_name[1]);
  mscDisk = pmsFS[1];
  listFiles(mscDisk);
  Serial.printf("Directory Listing for %s\r\n", pmsFS_display_name[2]);
  mscDisk = pmsFS[2];
  listFiles(mscDisk);
  Serial.printf("Directory Listing for %s\r\n", pmsFS_display_name[3]);
  mscDisk = pmsFS[3];
  listFiles(mscDisk);
}

void loop() {
	
}  

void listFiles(FS *fs) {
  Serial.print("Space Used = ");
  Serial.println(fs->usedSize());
  Serial.print("Filesystem Size = ");
  Serial.println(fs->totalSize());

  printDirectory(*fs);
}


void printDirectory(FS &fs) {
  Serial.println("Directory\n---------");
  printDirectory(fs.open("/"), 0);
  Serial.println();
}

void printDirectory(File dir, int numSpaces) {
  while (true) {
    File entry = dir.openNextFile();
    if (!entry) {
      // Serial.println("** no more files **");
      break;
    }
    printSpaces(numSpaces);
    Serial.print(entry.name());
    if (entry.isDirectory()) {
      Serial.println("/");
      printDirectory(entry, numSpaces + 2);
    } else {
      // files have sizes, directories do not
      printSpaces(36 - numSpaces - strlen(entry.name()));
      Serial.print("  ");
      Serial.println(entry.size(), DEC);
    }
    entry.close();
  }
}

void printSpaces(int num) {
  for (int i = 0; i < num; i++) {
    Serial.print(" ");
  }
}

Results:
Code:
FS Testing

Initializing, please wait...

QPINAND Drive initialized

USB Drive: 0 connected

msc # Partition Table
        part,boot,bgnCHS[3],type,endCHS[3],start,length
exFAT:  1,0,0x4,0x1,0x4,0x7,0xFE,0xC2,0xFF,2048,14649344
exFAT:  2,0,0xFE,0xC2,0xFF,0x7,0xFE,0xC2,0xFF,14651392,15566848
pt_#0:  3,0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0,0
pt_#0:  4,0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0,0

USB Drive: 1 connected

msc # Partition Table
        part,boot,bgnCHS[3],type,endCHS[3],start,length
FAT32:  1,0,0x4,0x1,0x4,0xC,0xFE,0xC2,0xFF,2048,125001728
exFAT:  2,0,0xFE,0xC2,0xFF,0x7,0xFE,0xC2,0xFF,125003776,116656128
pt_#0:  3,0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0,0
pt_#0:  4,0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0,0
MSC0-16GEXFATP1
MSC1-16GEXFATP2
MSC2-128GFAT32P1
MSC3-128GEXFATP2
MSC4
Directory Listing for QPINAND
Space Used = 33161216
Filesystem Size = 131596288
Directory
---------
32MEGfile.dat                         32768000
test1.txt                             51

Directory Listing for MSC0-16GEXFATP1
Space Used = 400752640
Filesystem Size = 7499481088
Directory
---------
32MEGfile.dat                         32768000
Music/
  armaged.wav                         56863276
  DeffLepardAnimal.wav                43025728
  StatusQuoWhateverYouWant.wav        60148898
  ThePaperboysMolinos.wav             35639166
  ThePaperboysMolinosI'veJustSeenaFace.wav  36381082
  YoureLazyJimmyBarnesJoeBonamassa.wav  102878882
bench.dat                             32768000
test1.txt                             51

Directory Listing for MSC1-16GEXFATP2
Space Used = 298844160
Filesystem Size = 7969177600
Directory
---------
00-teensy.rules                       2062
ListFiles.ino                         2314
LittleFS_Program_Simple_Datalogger-dates.ino  6205
keywords.txt                          218
library.properties                    277
README.md                             6511
README1.md                            1781

Directory Listing for MSC2-128GFAT32P1
Space Used = 36143104
Filesystem Size = 63985188864
Directory
---------
32MEGfile.dat                         32768000
SdFat-GPT_disks.zip                   2868116
USBHost_t36-FS_Integration_MSC.zip    442002

Directory Listing for MSC3-128GEXFATP2
Space Used = 569901056
Filesystem Size = 59725971456
Directory
---------
32MEGfile.dat                         32768000
TeensyduinoInstall-1.56B2.linux64     91586596

It just keeps getting better:)
Winds and rain are now picking up here. Gotta prepare for possible power outages...
 
Last edited:
Another quick update:

We have been having some fun, getting Fat and ExFat to work on both Extended Partitions as well as drives that are configured to use GPT instead of MBR setups.

Thought these steps may be important over time with USB drives as more of them are coming configured with GPT and many disk partition configuration programs on their own decide to create an extended partition...

These changes so far still have some debug output code and are not yet agreed to...

My later changes are up in my:

USBHost_t36 in the FS_Integration_MSC branch: This includes the changes needed for MTP to handle the formatting of these drives (We are still debugging some of this).. Finally added in the extra code to do a new begin of the volume after a format, such that the FS object is not using the layout data that may have changed.

Also while doing this, added in new method to PFSLib to convert the partition number to data like starting LBA and count of sectors, and where the MBR is and which section of the MBR that should be used.

ALso added a quick dirty partition list function....


SDFat in the GPT_disks branch: Added in the code to both Fatlib and ExFatlib volume code to do the same stuff... right now printing out debug stuff... Also am tempted to do like I did in USBHost and move this code to maybe FSLib and make it such that fixes in one place...

MTP_Teensy (main branch): a few changes like going back to send device reset instead of storage info reset after a format. Update the USB_MTP-logger sketch to use the new stuff like printing out the volume information...

For example with GPT drive the list shows for one SSD I have with 2 partitions:

Code:
Try Partition list
PART	Type	Start	Count	(MBR	Part)	Volume Type
1	O	34	32734	1814064751	121
2	G	32768	65536000	4294967295	255	Fat32:
3	G	65568768	434548736	4294967295	255	exFAT
Found new Volume:0
addFilesystem: 1 MSC0-GPT_FAT32 20008a70
Found new Volume:1
addFilesystem: 2 MSC1-GPT_exFAT 20008f40

With a different SSD with 4 partitions on it MBR two in primary and 2 in Extended, you see some data like:

Code:
Try Partition list
PART	Type	Start	Count	(MBR	Part)	Volume Type
1	M	2048	32768000	0	0	Fat32:
2	M	32770048	65536000	0	1	exFAT
3	E	98308096	65536000	98306048	0	Fat32:
4	E	163846144	70592512	163844096	0	exFAT
Found new Volume:0
addFilesystem: 3 MSC0-PRI_FAT32 20008a70
Found new Volume:1
addFilesystem: 4 MSC1-PRI_ExFat 20008f40
Found new Volume:2
addFilesystem: 5 MSC2-EXT_FAT32 20009410
Found new Volume:3
addFilesystem: 6 MSC3-EXT_EXFAT 200098e0

Lots of other debug output but...

Again having some fun...
 
@KurtE - Nice work. Will update. Can't wait to play with it this weekend. Haven't worked GPT and Extended partitions yet.
 
Cool it is progressing and getting better and more capable!

IS the OP update to p#135 still the path? I just got some fresh SD cards and looking to catch up with WIP - please make sure there is a breadcrumb trail to get me up to speed. Also stocked with disposable HDD's and other FLASH media. Trying to built LittleFS QSPI Nand/Nor with Beta2 of 1.56 gave a fail so need updates.

Just pulled two T_MM's and 12 Teensy off my desk - down to one now - so I can start fresh. Have ti find the PJRC SPI memory board and breakout ...
 
Cool it is progressing and getting better and more capable!

IS the OP update to p#135 still the path? I just got some fresh SD cards and looking to catch up with WIP - please make sure there is a breadcrumb trail to get me up to speed. Also stocked with disposable HDD's and other FLASH media. Trying to built LittleFS QSPI Nand/Nor with Beta2 of 1.56 gave a fail so need updates.

Just pulled two T_MM's and 12 Teensy off my desk - down to one now - so I can start fresh. Have ti find the PJRC SPI memory board and breakout ...

Yeah, and more standardized with all of the different file systems:)

What I have now is all of @KurtE's latest updates to TD1.56B2:

https://github.com/KurtE/MTP_Teensy
https://github.com/KurtE/USBHost_t36/tree/FS_Integration_MSC
https://github.com/KurtE/SdFat/tree/GPT_disks
https://github.com/KurtE/SD

Then as of today I followed this link:
https://forum.pjrc.com/threads/68560-LittleFS-SPI-and-QSPI-wrappers post #3 which pointed me to this sketch:
https://github.com/KurtE/MTP_Teensy/blob/main/examples/SD_SPI_QSPI_MTP-logger/SD_SPI_QSPI_MTP-logger.ino

KurtE has moved all needed files from UsbMscFat to USBHost_t36-FS_integration_MSC and updated them so you will not need to use UsbMscFat any more. It's more self contained now for MSC and maybe other options. The SdFat-GPT_disks library is updated to work with GTP and extended partitions which I have not worked with yet.

Hopefully this helps for what you are looking for:)
 
Thanks @wwatson! Will see if I can gather and build.

Doh - found my T_4.1 w/memory board and USB Host cable - had moved it to independent USB port on top of computer last week.
 
Good github list @wwatson!

I can build the QSPI example:
Code:
Multiple libraries were found for "sdios.h"
 Used: C:\T_Drive\tCode\libraries\SdFat
 Not used: C:\T_Drive\Arduino_1.8.16_155\hardware\teensy\avr\libraries\SdFat
Using library LittleFS at version 1.0.0 in folder: C:\T_Drive\Arduino_1.8.16_155\hardware\teensy\avr\libraries\LittleFS 
Using library SPI at version 1.0 in folder: C:\T_Drive\Arduino_1.8.16_155\hardware\teensy\avr\libraries\SPI 
Using library SdFat at version 2.1.0 in folder: C:\T_Drive\tCode\libraries\SdFat


And the linked MTP data logger seems ready to work:
> It logs to LFS_Ram and reports a Time/Date! And opens to notepad.
Code:
Dump Storage list(6)
store:0 storage:10001 name:RAM fs:20006574
store:1 storage:20001 name:QNAND fs:20006478
store:2 storage:30001 name:NAND_3 fs:20005a84
store:3 storage:40001 name:NAND_4 fs:20005d38
store:4 storage:50001 name:Flash_5 fs:20005e3c
store:5 storage:60001 name:Flash_6 fs:200060f0

MTP_FS1.png
 
Went to MSC - expanded mscTesting to msDrive4.

It builds and does okay except calling out one error (not always #1) : msDrive1 not connected: Code: 35

Then then showing four drives:
Code:
MSC TEST

Initializing USB Drive(s)
msDrive1 not connected: Code: 35

msDrive2  connected
msDrive3  connected
msDrive4  connected

Press a key to show USB drive info:

msDrive1  connected/initilized
msDrive1 is NOT Mounted

   connected 1
   initialized 1
   USB Vendor ID: 0080
  USB Product ID: a001
      HUB Number: 1
        HUB Port: 1
  Device Address: 2
Removable Device: NO
        VendorID: TO Exter
       ProductID: nal USB 3.0     
      RevisionID: 0203
         Version: 6
    Sector Count: 234441647
     Sector size: 512
   Disk Capacity: 120034123264 Bytes

msDrive2  connected/initilized
msDrive2 is NOT Mounted

   connected 1
   initialized 1
   USB Vendor ID: 1e1d
  USB Product ID: 1104
      HUB Number: 1
        HUB Port: 2
  Device Address: 3
Removable Device: YES
        VendorID: Kanguru 
       ProductID: FlashBlu 30     
      RevisionID: PMAP
         Version: 6
    Sector Count: 30965759
     Sector size: 512
   Disk Capacity: 15854468608 Bytes

msDrive3  connected/initilized
msDrive3 is NOT Mounted

   connected 1
   initialized 1
   USB Vendor ID: 090c
  USB Product ID: 1000
      HUB Number: 1
        HUB Port: 4
  Device Address: 5
Removable Device: YES
        VendorID: Samsung 
       ProductID: Type-C          
      RevisionID: 1100
         Version: 6
    Sector Count: 501253131
     Sector size: 512
   Disk Capacity: 256641603072 Bytes

msDrive4  connected/initilized
msDrive4 is NOT Mounted

   connected 1
   initialized 1
   USB Vendor ID: 1e1d
  USB Product ID: 1106
      HUB Number: 4
        HUB Port: 4
  Device Address: 6
Removable Device: YES
        VendorID: Kanguru 
       ProductID: FlashBlu        
      RevisionID: PMAP
         Version: 6
    Sector Count: 60555263
     Sector size: 512
   Disk Capacity: 31004294656 Bytes

Then same usb connections unchanged - cleared SerMon and re-Uploaded:
Code:
MSC TEST

Initializing USB Drive(s)
msDrive1 not connected: Code: 35

msDrive2  connected
msDrive3 not connected: Code: 40

msDrive4 not connected: Code: 40


Press a key to show USB drive info:

msDrive1  connected/initilized
msDrive1 is NOT Mounted

   connected 1
   initialized 1
   USB Vendor ID: 0080
  USB Product ID: a001
      HUB Number: 1
        HUB Port: 1
  Device Address: 2
Removable Device: NO
        VendorID: TO Exter
       ProductID: nal USB 3.0     
      RevisionID: 0203
         Version: 6
    Sector Count: 234441647
     Sector size: 512
   Disk Capacity: 120034123264 Bytes

msDrive2  connected/initilized
msDrive2 is NOT Mounted

   connected 1
   initialized 1
   USB Vendor ID: 1e1d
  USB Product ID: 1104
      HUB Number: 1
        HUB Port: 2
  Device Address: 3
Removable Device: YES
        VendorID: Kanguru 
       ProductID: FlashBlu 30     
      RevisionID: PMAP
         Version: 6
    Sector Count: 30965759
     Sector size: 512
   Disk Capacity: 15854468608 Bytes

msDrive3 not connected: Code: 40

msDrive4 not connected: Code: 40
 
This is the DUAL SD USB adapter that works on PC to mount and use both the SD and MicroSD slotted media :: amazon.com/gp/product/B01EFPX9XA

Plugging it in to Above MSC Test - it only finds one of the SD cards.

Turning on _DEBUG shows this in case it helps:
Code:
MSC TEST

Initializing USB Drive(s)
USB2 PLL running
 reset waited 6
USBHS_ASYNCLISTADDR = 0
USBHS_PERIODICLISTBASE = 20003000
periodictable = 20003000
port change: 10001803
    connect
  begin reset
port change: 18001205
  port enabled
  end recovery
new_Device: 480 Mbit/sec
new_Pipe
enumeration:
enumeration:
enumeration:
Device Descriptor:
  12 01 10 02 00 00 00 40 E3 05 49 07 35 15 03 04 05 01 
    VendorID = 05E3, ProductID = 0749, Version = 1535
    Class/Subclass/Protocol = 0 / 0 / 0
    Number of Configurations = 1
enumeration:
enumeration:
Manufacturer: Generic
enumeration:
Product: USB3.0 Card Reader
enumeration:
Serial Number: 000000001536
enumeration:
Config data length = 32
enumeration:
Configuration Descriptor:
  09 02 20 00 01 01 00 80 FA 
    NumInterfaces = 1
    ConfigurationValue = 1
  09 04 00 00 02 08 06 50 00 
    Interface = 0
    Number of endpoints = 2
    Class/Subclass/Protocol = 8(Mass Storage) / 6(SCSI) / 80(Bulk Only)
  07 05 81 02 00 02 00 
    Endpoint = 1 IN
    Type = Bulk
    Max Size = 512
    Polling Interval = 0
  07 05 02 02 00 02 00 
    Endpoint = 2 OUT
    Type = Bulk
    Max Size = 512
    Polling Interval = 0
enumeration:
USBHub memory usage = 960
USBHub claim_device this=200031E0
USBHub memory usage = 960
USBHub claim_device this=200035A0
USBHub memory usage = 960
USBHub claim_device this=20003960
USBHub memory usage = 960
USBHub claim_device this=20003D20
msController claim this=200040E0
msController claim this=200049E0
msController claim this=200052E0
msController claim this=20005BE0
Descriptor 4 = INTERFACE
msController claim this=200040E0
09 04 00 00 02 08 06 50 00 07 05 81 02 00 02 00 07 05 02 02 00 02 00 
numendpoint=2
endpointIn=81
endpointOut=2
packet size in (msController) = 512
packet size out (msController) = 512
polling intervalIn = 0
polling intervalOut = 0
new_Pipe
new_Pipe
Descriptor 5 = ENDPOINT
Descriptor 5 = ENDPOINT
control CallbackIn (msController)
00 00 00 00 00 00 00 00 
control CallbackIn (msController)
01 00 00 00 00 00 00 00 
msController CallbackOut (static)
transfer->qtd.token = 0
msController dataOut (static)31
55 53 42 43 01 00 00 00 00 00 00 00 80 00 06 1B 01 00 00 01 00 00 00 00 00 00 00 00 00 00 00 
msController CallbackIn (static)
transfer->qtd.token = 0
msController dataIn (static): 13
55 53 42 53 01 00 00 00 00 00 00 00 00 
msController CallbackOut (static)
transfer->qtd.token = 0
msController dataOut (static)31
55 53 42 43 02 00 00 00 00 00 00 00 80 00 06 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 
msController CallbackIn (static)
transfer->qtd.token = 0
msController dataIn (static): 13
55 53 42 53 02 00 00 00 00 00 00 00 01 
msController CallbackOut (static)
transfer->qtd.token = 0
msController dataOut (static)31
55 53 42 43 03 00 00 00 00 00 00 00 80 00 06 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 
msController CallbackIn (static)
transfer->qtd.token = 0
msController dataIn (static): 13
55 53 42 53 03 00 00 00 00 00 00 00 00 
msController CallbackOut (static)
transfer->qtd.token = 0
msController dataOut (static)31
55 53 42 43 04 00 00 00 24 00 00 00 80 00 06 12 00 00 00 24 00 00 00 00 00 00 00 00 00 00 00 
msController CallbackIn (static)
transfer->qtd.token = 0
msController dataIn (static): 36
00 80 06 02 3B 00 00 00 47 65 6E 65 72 69 63 20 4D 61 73 73 53 74 6F 72 61 67 65 43 6C 61 73 73 
msController CallbackIn (static)
transfer->qtd.token = 0
msController dataIn (static): 13
55 53 42 53 04 00 00 00 00 00 00 00 00 
msController CallbackOut (static)
transfer->qtd.token = 0
msController dataOut (static)31
55 53 42 43 05 00 00 00 08 00 00 00 80 00 0A 25 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 
msController CallbackIn (static)
transfer->qtd.token = 0
msController dataIn (static): 8
0E E7 BF FF 00 00 02 00 
msController CallbackIn (static)
transfer->qtd.token = 0
msController dataIn (static): 13
55 53 42 53 05 00 00 00 00 00 00 00 00 
msDrive1  connected
msDrive2 not connected: Code: 40

msDrive3 not connected: Code: 40

msDrive4 not connected: Code: 40


Press a key to show USB drive info:
 
Went back to MTP_logger.ino, with added new 128GB Samsung uSD in T_4.1 socket.

Copied Oct phone photos folder : 46 items taking 386 MB - time reported about 2 minutes.

Completed all good.

Opened 2 or 3 larger JPG files that never displayed, left just a BLACK window after the wait circle completed.

Went to smallest of the files - dropped image from another photo of size 729KB that display from PC source disk.

On clicking that image Win 11 reported this as 'corrupted':
MTP_FS_corrupt.png

Other JPG rane from 3,485 KB to 19,582 KB and the sizes all appear to be correct
> 'Created' TD stamps cover 12:31 (one file) to 12.34 (four files)
> 'Modified' TD stamp appears to match source file

Here is SerMon log in case it shows anything:
View attachment MTP_copy1.txt
 
Good Morning all:

@defragster an @wwatson, will take a look at some of your testing stuff soon.

Actually though there may be another level of testing needed... We have been running into several cases of us reformatting disks and copy files to exFat volumes on the disk, and then find the disk corrupted.

Assumed that it is our MTP/PFSVolume/... code that is screwing up.

So tried another test: I have a new (used a few days now) 128gb SDCard (SanDisk). I ran through SDFormatter, which created one exfat partition.

Then ran on T4.1 with the memory chips, sketch mentioned few posts earlier.

I then copied a couple of large files. one 26+mb and the other 158mb+ file, to the SDCard.

I then took it back to Windows and sure enough windows mentioned it being corrupted:
Ran the chkdsk to see if more data available...
Code:
I:\> chkdsk
The type of the file system is exFAT.
Access is denied.

The volume is in use by another process. Chkdsk
might report errors when no corruption is present.
Volume Serial Number is 20A0-4984
Windows is verifying files and folders...
Volume label is SD_EXFAT.
Corruption was found while examining files in directory \ (7).
Corruption was found while examining files in directory \ (8).
Corruption was found while examining files in directory \ (9).
Corruption was found while examining files in directory \ (10).
Corruption was found while examining files in directory \ (11).
Corruption was found while examining files in directory \ (12).
Corruption was found while examining files in directory \ (13).
Corruption was found while examining files in directory \ (14).
An error occurred while examining files and directories.

Windows has checked the file system and found problems.
Run CHKDSK with the /F (fix) option to correct these.

So this is not running any of our PFS stuff. Yes it is receiving data through MTP, but using standard SD (SDFat) calls...

So there appears to be an issue with the underlying exFat code... (maybe).

Would be interesting to test creating large files just using SD/SDFat... and see if same issue...
 
Good Morning all:

@defragster an @wwatson, will take a look at some of your testing stuff soon.

Actually though there may be another level of testing needed... We have been running into several cases of us reformatting disks and copy files to exFat volumes on the disk, and then find the disk corrupted.
...

Not sure the FORMAT is the problem!

Took p#373 NEW uSD card to PC ( I did copy once on PC then delete the same files )

Used for MTP copy as posted. Put uSD onto PC to view same files ...

Windows did some repair successfully - the drive is empty - one zero length file of all 46 that were to be there:
Code:
H:\2021-10>dir
 Volume in drive H has no label.
 Volume Serial Number is 9C33-6BBD

 Directory of H:\2021-10

10/29/2021  12:31 AM    <DIR>          .
10/29/2021  12:31 AM    <DIR>          ..
10/29/2021  12:33 AM                 0 20211014_152218.jpg
               1 File(s)              0 bytes
               2 Dir(s)  127,694,536,704 bytes free
 
Back
Top