Many TLAs: MTP MSC FS SD SDFat LittleFS UsbMSCFat to work with each other 8)

[KurtE]

Not sure if some of this should go on MTP thread(https://forum.pjrc.com/threads/43050...-Contribution(

and/or the MSC thread: https://forum.pjrc.com/threads/55821...l=1#post270839

or ???

But for the fun of it, thought I would see how hard it would be to add one or more MSC devices to our list of MTP devices in the mtp-test sketch.

There are probably many issues and the like, with several different File Systems:

SD for SD stuff

LittleFS for the chips and the like

and currently now UsbMSCFat which is a modified version of SD (or SDFat...) stuff, maybe at some point we can combine?

So far, I have just tried to add the include files and maybe define the start of a few objects:

#define USE_MSC_FAT 1
...
//=============================================================================
// MSC FAT classes
//=============================================================================
#if USE_MSC_FAT == 1
#if defined(__IMXRT1062__) || defined(ARDUINO_TEENSY36)
#include <mscFS.h>
USBHost myusb;
USBHub hub1(myusb);
USBHub hub2(myusb);

// start off with one controller. 
msController msDrive1(myusb);


#else 
// Only those Teensy support USB
#warning "Only Teensy 3.6 and 4.x support MSC"
#endif
#endif

And the build now fails in loop()

    char pathname[MAX_FILENAME_LEN];

where MAX_FILENAME_LEN is not defined...

Turns out just about every library defines some generic name like: MAX_FILENAME_LEN

And in this case, the mscFS.h file not only defines it:
It then un-defines it:

// do not expose these defines in Arduino sketches or other libraries
#undef MSCFAT_FILE
#undef MSCFAT_BASE
#undef MAX_FILENAME_LEN

This one is probably easy to work around, as the only place this define is used in this directory is the actual header, so could change to something else.
Or could change to something in the class itself, like an enum or a static const uint16_t MAX_FILENAME_LEN=256;

But my real question here is should we just continue on the multiple different threads and/or create a new thread for Teensyduino Integration of hopefully getting
all of these capabilities working with each other and hopefully integrated into some version of Teensyduino, and/or some common place to get instructions on
how to get all of this to work?

[mjs513]

@KurtE

My suggestion would be that if it is a specific library related issue it should be on the appropriate thread. For instance if LittleFS is formatting correctly or not recognizing a flash chip it would go on the LittleFS thread. If related to integration of the of all the filesystems probably could go on the integration thread.

If I remember correctly @wwatson did have a mod to the SD library but was working on the FS linkage as well so probably a good place for that discussion would be here. My guess is that we should try and the LittleFS as the FS starting point which links back to SDFat?

EDIT: Was curious so I tried mscFat: see post https://forum.pjrc.com/threads/55821...l=1#post270944

[KurtE]

@mjs513 - I totally agree that in many cases posting to a thread specific to a library for something specific to that library...

But was wondering where for example would be a good place to post things that maybe effect multiple of these libraries.

Example: Like maybe should we add some additional methods to the FS classes:

For example: Should File class have methods to allow you to get and set modify date/time and/or create date and time?

Should FS class have a virtual method for things like:
bool quickFormat();
bool lowLevelFormat(char progressChar=0);
if lowLevelFormat, should there be a callback of some type to allow your own way to show progress>

Should there be a method to get logical (or desired) write sizes (sector, cluster).

Should there be some standard error reporting something? Like if opening file system is because there is no such device, versus device is not formatted, versus we don't support that FS...

...

Note: changed the one define in MSC to MSC_MAX_FILENAME_LEN and now builds... Crashes at startup
Now to start debugging!

[KurtE]

Another issue came up today in the thread: https://forum.pjrc.com/threads/66335...l=1#post270956

in there he is trying to enable the use of UNICODE file names:
Ok, the example in MTP_t4 is working, but only if #define USE_EXFAT_UNICODE_NAMES 0 in ExFatConfig.h

The code fails to build. He mentions in followup message:

WMXZ library in storage.h seems to be char* focused, yes. I guess this would need to be changed.
At the very least it is not compatible with the Unicode mode of SdFat.
In the example, it gets an SDClass from the SD wrapper (in the Teensy libraries), and that also does not seem to support Unicode.
And FS.h in Cores?

[mjs513]

Morning @KurtE

Sorry I didn't get to this yesterday - still recovering a bit and had another distraction.

Should FS class have a virtual method for things like:
bool quickFormat();
bool lowLevelFormat(char progressChar=0);
if lowLevelFormat, should there be a callback of some type to allow your own way to show progress>

Should there be a method to get logical (or desired) write sizes (sector, cluster).

Should there be some standard error reporting something? Like if opening file system is because there is no such device, versus device is not formatted, versus we don't support that FS...

I agree with 100%. For the things that you described those are the things that should/could be discussed on the integration thread. Especially going to come more into play as expanding usage via MSC. This was apparent when I tried the SSD drive. It actually has 2 partitions from when we were testing MSC, it showed up in one of the examples and how to get what we had with MSC into SDFat/FS etc. Same thing with error codes that BG has in the lib.

[mjs513]

Just including this for reference: https://forum.pjrc.com/threads/55821...l=1#post271015 on using partitions.

[wwatson]

@All - Ok, I have discovered a an unneeded delay in MSC that affects the time it takes for initializing a USB drive. In MassStorageDriver.cpp line #200 is:

	msReset(0); // Assume bNumInterfaces = 1 for now.
//	delay(500);
	maxLUN = msGetMaxLun(0); // Assume bNumInterfaces = 1 for now

It is not necessary. Also, I found this comment in hathatch's tinyusb driver:

// MSC interface Reset (not used now)

So I commented that out and found no ill effects when tested with my USB drives.

Using 'volumeName.ino' I tested with and without the delay after 'msReset(0)'.

Results with the delay:

Waiting up to 5 seconds for USB drive 1
^^^^Waiting up to 5 seconds for USB drive 2
^^^^^~Initialize USB drive...^^^^USB drive 1 is present.

Without the delay:

Waiting up to 5 seconds for USB drive 1
^^^^Waiting up to 5 seconds for USB drive 2
^^^^^~Initialize USB drive...USB drive 1 is present.

Commenting out 'msReset(0)' did not change anything else that I could detect so far.

I feel the need, the need for speed

[KurtE]

Sounds good!

Will take in. Right now I am trying to make a version of the fsFat which is partition friendly.
Running into a few compile issues. Will let everyone know if I have any luck.

[KurtE]

@all, again not sure if best to mention here or in the MSC thread...

But I just pushed up a new Branch of the UsbMscFat library... https://github.com/KurtE/UsbMscFat/tree/PFsLib

Where I completely duplicated the SdFat/FsLib sub-library where the only real difference is that in the FsVolume class
The begin method added the two same optional parameters that are in FatLib and ExFatLib:
That is: bool begin(BlockDevice* blockDev);
is now: bool begin(BlockDevice* dev, bool setCwv = true, uint8_t part = 1);

And the implementation, went from:

bool FsVolume::begin(BlockDevice* blockDev) {
  Serial.printf("FsVolume::begin(%x)\n", (uint32_t)blockDev);
  m_blockDev = blockDev;
  m_fVol = nullptr;
  m_xVol = new (m_volMem) ExFatVolume;
  if (m_xVol && m_xVol->begin(m_blockDev, false)) {
    goto done;
  }
  m_xVol = nullptr;
  m_fVol = new (m_volMem) FatVolume;
  if (m_fVol && m_fVol->begin(m_blockDev, false)) {
    goto done;
  }
  m_cwv = nullptr;
  m_fVol = nullptr;
  return false;

 done:
  m_cwv = this;
  return true;
}

To:

bool PFsVolume::begin(BlockDevice* blockDev, bool setCwv, uint8_t part) {
//  Serial.printf("PFsVolume::begin(%x)\n", (uint32_t)blockDev);
  m_blockDev = blockDev;
  m_fVol = nullptr;
  m_xVol = new (m_volMem) ExFatVolume;
  if (m_xVol && m_xVol->begin(m_blockDev, setCwv, part)) {
    goto done;
  }
  m_xVol = nullptr;
  m_fVol = new (m_volMem) FatVolume;
  if (m_fVol && m_fVol->begin(m_blockDev, setCwv, part)) {
    goto done;
  }
  m_cwv = nullptr;
  m_fVol = nullptr;
  return false;

 done:
  m_cwv = this;
  return true;
}

The only other things that changed where the names of files, classes, defines...

I then reworked the VolumeName.ino sketch to use it. I also removed all of the specific get volume label functions for 16 bit 32 bit and ExFat and have it all in my now called getPartitionVolumeLabel, which if I keep the
classes mentioned here. it all uses my new class and I personally think it is a lot cleaner... Note: I also have the SDCard code calling through this as well.
As I now pass in the reference to one of the PfsVolume objects...

bool getPartitionVolumeLabel(PFsVolume &partVol, uint8_t *pszVolName, uint16_t cb) {
  uint8_t buf[512];
  if (!pszVolName || (cb < 12)) return false; // don't want to deal with it

  PFsFile root;
  if (!root.openRoot(&partVol)) return false;
  root.read(buf, 32);
  //print_hexbytes(buf, 32);

  switch (partVol.fatType())
  {
    case FAT_TYPE_FAT12:
    case FAT_TYPE_FAT16:
    case FAT_TYPE_FAT32:
      {
        DirFat_t *dir;
        dir = reinterpret_cast<DirFat_t*>(buf);
        if ((dir->attributes & 0x08) == 0) return false; // not a directory...
        size_t i;
        for (i = 0; i < 11; i++) {
          pszVolName[i]  = dir->name[i];
        }
        while ((i > 0) && (pszVolName[i - 1] == ' ')) i--; // trim off trailing blanks
        pszVolName[i] = 0;
      }
      break;
    case FAT_TYPE_EXFAT:
      {
        DirLabel_t *dir;
        dir = reinterpret_cast<DirLabel_t*>(buf);
        if (dir->type != EXFAT_TYPE_LABEL) return false; // not a label? 
        size_t i;
        for (i = 0; i < dir->labelLength; i++) {
          pszVolName[i] = dir->unicode[2 * i];
        }
        pszVolName[i] = 0;
      }
      break;
    } 
  return true;
}

// Function to handle one MS Drive...
void procesMSDrive(uint8_t drive_number, msController &msDrive, UsbFs &msc)
{
  Serial.printf("Initialize USB drive...");
  cmsReport = 0;
  if (!msc.begin(&msDrive)) {
    Serial.println("");
    msc.errorPrint(&Serial);
    Serial.printf("initialization drive %u failed.\n", drive_number);
  } else {
    Serial.printf("USB drive %u is present.\n", drive_number);
  }
  cmsReport = -1;

  mbrDmp( &msc );
  for (uint8_t i = 1; i < 5; i++) {
    PFsVolume partVol;
    uint8_t volName[32];
    if (!partVol.begin(msc.usbDrive(), true, i)) continue; // not a valid volume.
    partVol.chvol();

    switch (partVol.fatType())
    {
      case FAT_TYPE_FAT12: Serial.print("\n>> Fat12: "); break;
      case FAT_TYPE_FAT16: Serial.print("\n>> Fat16: "); break;
      case FAT_TYPE_FAT32: Serial.print("\n>> Fat32: "); break;
      case FAT_TYPE_EXFAT: Serial.print("\n>> ExFat: "); break;
    }
    if (getPartitionVolumeLabel(partVol, volName, sizeof(volName))) {
      Serial.printf("Volume name:(%s)", volName);
    }
    elapsedMicros em_sizes = 0;
    uint64_t used_size =  (uint64_t)(partVol.clusterCount() - partVol.freeClusterCount())
                      * (uint64_t)partVol.bytesPerCluster();
    uint64_t total_size = (uint64_t)partVol.clusterCount() * (uint64_t)partVol.bytesPerCluster();
    Serial.printf(" Partition Total Size:%llu Used:%llu time us: %u\n", total_size, used_size, (uint32_t)em_sizes);

    partVol.ls();
  }
}

The function processing each SD Drive is pretty simple now as well:

  Serial.printf("\nInitialize SD card...");

  if (!sd.begin(SD_CONFIG)) {
    Serial.println("initialization failed.\n");
  } else {
    Serial.println("SD card is present.\n");
    PFsVolume partVol;
    if (!partVol.begin(sd.card(), true, 1)) Serial.println("SD Did not open partition...");
    switch (partVol.fatType())
    {
      case FAT_TYPE_FAT12: Serial.print("\n>> Fat12: "); break;
      case FAT_TYPE_FAT16: Serial.print("\n>> Fat16: "); break;
      case FAT_TYPE_FAT32: Serial.print("\n>> Fat32: "); break;
      case FAT_TYPE_EXFAT: Serial.print("\n>> ExFat: "); break;
    }
    uint8_t volName[32];
    if (getPartitionVolumeLabel(partVol, volName, sizeof(volName))) {
      Serial.printf("Volume name:(%s)\n", volName);
    }

A test run:

Waiting up to 5 seconds for USB drive 1
^^Waiting up to 5 seconds for USB drive 2
^^^^^^^~~~~:(Initialize USB drive...^^^^^^^FsVolume::begin(20001f70)
USB drive 1 is present.

msc # Partition Table
	part,boot,bgnCHS[3],type,endCHS[3],start,length
FAT32:	1,0,0x4,0x1,0x4,0xB,0xFE,0xC2,0xFF,2048,8192000
FAT16:	2,0,0xE,0x51,0xFE,0xE,0x98,0x98,0x80,8194048,2097152
exFAT:	3,0,0x98,0x99,0x80,0x7,0xEE,0xDC,0xD2,10291200,5435392
pt_#0:	4,0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0,0

>> Fat32: Volume name:(VOLFAT32) Partition Total Size:4186095616 Used:65425408 time us: 6074373
DSC03357.JPG
example.txt
OnFat32.txt
IMG_0277.jpg
IMG_1240.jpg
IMG_1238.jpg
IMG_1236.jpg
IMG_1235.jpg
IMG_1229.jpg
teensy41_BP.jpg
teensy41_2aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa.jpg
T4-Card-Front.jpg
IMG_1176.jpg
IMG_1167.jpg
lynxmotion.png
visualMicro.jpg
IMG_1162.jpg
IMG_1163.jpg
IMG_1164.jpg
IMG_0267.jpg
IMG_0266.jpg
IMG_0264.jpg
IMG_1154.jpg
T4.1-Cardlike.jpg
T4-Cardlike.jpg
IMG_1304.jpg
IMG_1301.jpg
IMG_1303.jpg
IMG_1302.jpg
IMG_1300.jpg
IMG_0287.jpg
IMG_0283.jpg
IMG_1267.jpg
IMG_1260.jpg
IMG_1258.jpg
IMG_1259.jpg
IMG_1255.jpg
IMG_1254.jpg
IMG_0278.jpg
Ausi Pictures/
P3190014.JPG
P3190015.JPG
P3190016.JPG
P3100001.JPG
P3100003.JPG
P3150004.JPG
P3150005.JPG
P3180006.JPG
P3190007.JPG
P3190008.JPG
P3190009.JPG
P3190010.JPG
P3190011.JPG
P3190012.JPG
P3190013.JPG
DSC03521.JPG
DSC03522.JPG
DSC03518.JPG
DSC03519.JPG
DSC03520.JPG

>> Fat16: Volume name:(VOLFAT16) Partition Total Size:1073446912 Used:11714560 time us: 193125
T4-Cardlike.jpg
OnFat16.txt
DSC03357.JPG
Inside-Ps2.jpg
Arduino-close-up.jpg
Birds.jpg
Encoder-plus-serial-overvie.jpg
Oscilloscope.bmp
Encoder-logic-scan.jpg
tri-track-back.jpg
DSC03361.JPG
Serout-and-TASerial-at-1152.jpg
Chr3-rotate-RJoy-LR.jpg
DSC03358.JPG

>> ExFat: Volume name:(VolEXFAT) Partition Total Size:2782461952 Used:232030208 time us: 17002
T4.1-Cardlike.jpg
OnExFat.txt
Hitec-laser-6-receiver.jpg
Arduino-mega-Brat.jpg
Arduino-mega-with-PS2.jpg
Add-pull-up-to-PS2-receiver.jpg
Inside-Ps2.jpg
Arduino-close-up.jpg
Two-Arduino-Pros.jpg
Arduino-SSC32-and-Pro40.jpg
Birds.jpg
Encoder-plus-serial-overvie.jpg
Encoders-plus-serial-to-and.jpg
Encoder-on-USB-Oscilloscope.jpg
Oscilloscope.bmp
Encoder-logic-scan.jpg
nano-serout.jpg
Tri-track-front.jpg
tri-track-back.jpg
DSC03361.JPG
DSC03360.JPG
Minor-Nit-on-startup.jpg
Serout-and-TASerial-at-1152.jpg
Chr3-rotate-RJoy-LR.jpg
Arduino-DIY-remote.jpg
BB-DUINO-Shield-BRD-Bottom.jpg
bb-bap64-xbee.jpg
joysticks.jpg
Botboarduino-top.jpg
Botboarduino-bottom.jpg
DIY-Remote-with-updates.jpg
RC-input-timings.jpg
BB-Duino-Jr-SCH.jpg
Seeeduino-Mega-with-XBee-Sh.jpg
Lynxmotion-Prop-Board-Back.jpg
Lynxmotion-Prop-board-Front.jpg
Lynxmotion-Propeller-Schema.jpg
Lynxmotion-Propeller-Board.jpg
Propeller-Multi-Controller-.jpg
Ps2-with-cable.jpg
Tri-track-with-roboclaw.jpg
webbotlib-serial-closeup.jpg
Webbotlib-serial.jpg
belkin-play-serial-2.jpg
Belkin-play-serial-1.jpg
Seeduino-Mega.jpg
Lantronix-board.jpg
chr3-with-new-body2.jpg
CHR3-with-new-body.jpg
Trace-Nano-I2c-Read.jpg
Trace-Nano-I2C-write.jpg
CHR3-leg-body-up-battery-he.jpg
CHR3-leg-body-on-ground.jpg
Hitec-laser6-logic-scan.jpg
Jake/

Initialize SD card...FsVolume::begin(20003db0)
SD card is present.


>> ExFat: Volume name:(EXFat_Vol)
done...
Press any key to run again

I ran into another interesting thing with an Ubuntu Install (Same type thumb drive):

Waiting up to 5 seconds for USB drive 2
^^^^^^^^^~~~~Initialize USB drive...^^^^^^^^^~FsVolume::begin(20001f70)

Check USB drive format.
initialization drive 1 failed.

msc # Partition Table
	part,boot,bgnCHS[3],type,endCHS[3],start,length
pt_#0:	1,80,0x0,0x1,0x0,0x0,0xAB,0xE0,0xFC,0,5619584
pt_#239:	2,0,0xFE,0xFF,0xFF,0xEF,0xFE,0xFF,0xFF,1700,8000
pt_#131:	3,0,0xCE,0x72,0x5D,0x83,0xF,0xFC,0xD3,5619712,10108928
pt_#0:	4,0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0,0

Initialize SD card...FsVolume::begin(20003db0)
SD card is present.


>> ExFat: Volume name:(EXFat_Vol)
done...
Press any key to run again

Note it does not like any of the partitions... But if plugged into PC the first partiation shows up as a FAT drive:
Note the 80 in the Boot field...

Need to experiment and find RPI SD Card and see what shows up.



Let me know what you think. Maybe I will play with looking to see if the SD card has partitions as well.

I will probably go ahead and issue a Pull request for it.

[wwatson]

@KurtE Was wondering why you were silent for so long This is awesome

Will download and test. That is a lot of work in a short time.

Thanks

[mjs513]

@KurtE
Fantastic job.... think duplicating the sub-library was the best approach and then modifing it. Will download and give it a try.

[mjs513]

@KurtE
Just tried compiling for the T4.1 (VolumeName.ino) in the examples folder and got the following error:

C:\Users\Merli\AppData\Local\Temp\arduino_build_281257\libraries\SdFat\FsLib\FsNew.cpp.o: In function `operator new(unsigned int, unsigned long*)':
F:\arduino-1.8.13-beta6\hardware\teensy\avr\libraries\SdFat\src\FsLib/FsNew.cpp:29: multiple definition of `operator new(unsigned int, unsigned long*)'
C:\Users\Merli\AppData\Local\Temp\arduino_build_281257\libraries\UsbMscFat-PFsLib\PFsLib\PFsNew.cpp.o:D:\Users\Merli\Documents\Arduino\libraries\UsbMscFat-PFsLib\src\PFsLib/PFsNew.cpp:29: first defined here
f:/arduino-1.8.13-beta6/hardware/tools/arm/bin/../lib/gcc/arm-none-eabi/5.4.1/../../../../arm-none-eabi/bin/ld.exe: Disabling relaxation: it will not work with multiple definitions

[wwatson]

@KurtE - Same here as @mjs513.

/home/wwatson/arduino-1.8.13/hardware/teensy/avr/libraries/SdFat/src/FsLib/FsNew.cpp:29: multiple definition of `operator new(unsigned int, unsigned long*)'
/tmp/arduino_build_850402/libraries/UsbMscFat-PFsLib/PFsLib/PFsNew.cpp.o:/home/wwatson/Arduino/libraries/UsbMscFat-PFsLib/src/PFsLib/PFsNew.cpp:29: first defined here
/home/wwatson/arduino-1.8.13/hardware/tools/arm/bin/../lib/gcc/arm-none-eabi/5.4.1/../../../../arm-none-eabi/bin/ld: Disabling relaxation: it will not work with multiple definitions
collect2: error: ld returned 1 exit status

Under Linux 18.04.

[defragster]

Same build issue here on prior code. Must be another moving part?

I like the idea of having one thread for this combination effort for context - so here is fine without jumping threads or polluting the wrong one.

Might even be nice to have a github_Combo with a subfolder of all the LIBS affected to aid in synchonized update ? Sort of like worked with git.LFSIntegrity so WIP updates could go in without banging into an official FORK?

[KurtE]

Try syncing again. I thought I had disabled that new... Turned out I had disabled the one in SDFat library instead

[wwatson]

Same build issue here on prior code. Must be another moving part?

I like the idea of having one thread for this combination effort for context - so here is fine without jumping threads or polluting the wrong one.

Might even be nice to have a github_Combo with a subfolder of all the LIBS affected to aid in synchonized update ? Sort of like worked with git.LFSIntegrity so WIP updates could go in without banging into an official FORK?

Me too, bouncing between threads at my age guaranties I will miss something. We are down one person at work so this week I might not be able play much

Dinner and bed now...

[wwatson]

Try syncing again. I thought I had disabled that new... Turned out I had disabled the one in SDFat library instead

It's working Will do the PR.

[defragster]

@All here : wwatson, mjs513, KurtE ...

If I made the Unified multi Lib on github and gave all write access we wouldn't need PR's (or catching diff edits) - just fixes?

Does that have validity - then this thread would have a single git source for the disparate elements.

[mjs513]

Just re-synched and seems to be working. Put in the SD Card Adatpter with 3 partitions:

Waiting up to 5 seconds for USB drive 1
^Waiting up to 5 seconds for USB drive 2
^^^^^^^^~~~~:(Initialize USB drive...^^^^^^^^USB drive 1 is present.

msc # Partition Table
	part,boot,bgnCHS[3],type,endCHS[3],start,length
FAT32:	1,80,0x0,0x1,0x10,0xC,0x3,0xE0,0xFF,2048,524288
FAT32:	2,0,0xC2,0x23,0x20,0xC,0xFE,0xFF,0xFF,526336,16564224
exFAT:	3,0,0xFE,0xFF,0xFF,0x7,0xFE,0xFF,0xFF,17092608,232642560
pt_#0:	4,0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0,0

>> Fat32: Volume name:(system-boot)###FatPartition::freeClusterCount: FT:32 start:2080 todo:516192
    m_sectorsPerCluster:1
    m_clusterSectorMask:0
    m_sectorsPerClusterShift:0
    m_fatType:32
    m_rootDirEntryCount:0
    m_allocSearchStart:1
    m_sectorsPerFat:4033
    m_dataStartSector:10146
    m_fatStartSector:2080
    m_lastCluster:516191
    m_rootDirStart:2
    m_bytesPerSector:512
 Partition Total Size:264289280 Used:961024 time us: 2116123
bench0.txt
bench1.txt
bench2.txt
bench3.txt
bench4.txt
bench5.txt

>> Fat32: Volume name:(NEW VOLUME1)###FatPartition::freeClusterCount: FT:32 start:526814 todo:2066434
    m_sectorsPerCluster:8
    m_clusterSectorMask:7
    m_sectorsPerClusterShift:3
    m_fatType:32
    m_rootDirEntryCount:0
    m_allocSearchStart:1
    m_sectorsPerFat:16145
    m_dataStartSector:559104
    m_fatStartSector:526814
    m_lastCluster:2066433
    m_rootDirStart:2
    m_bytesPerSector:512
 Partition Total Size:8464105472 Used:983040 time us: 8072748
bench0.txt
bench1.txt
bench2.txt
bench3.txt
bench4.txt
bench5.txt

>> ExFat: Volume name:(New Volume2) Partition Total Size:119107747840 Used:7077888 time us: 111002
CommandStation-EX-Arch-v1-0.pdf
VSLAM_and_Navigation_System_of_Unmanned_Ground_Vehicle_Based_on_RGB-D_Camera.pdf

Initialize SD card...initialization failed.

Did seem like it took a long pause when accessing the 2nd and 3rd partition.

[KurtE]

@All here : wwatson, mjs513, KurtE ...

If I made the Unified multi Lib on github and gave all write access we wouldn't need PR's (or catching diff edits) - just fixes?

Does that have validity - then this thread would have a single git source for the disparate elements.

Maybe... But then there is also MTP with @WMXZ and the like...

Quick fyi - right after the merge,

I updated the sketch again, such that you can dump the MBR for both USB devices as well as SD Cards... I pushed it up in same branch after the merge will do it again in new branch
or new combined project...

In case you wish to edit in (or get from previous branch)

bool mbrDmp(BlockDeviceInterface *blockDev) {
  MbrSector_t mbr;
  // bool valid = true;
  if (!blockDev->readSector(0, (uint8_t*)&mbr)) {
    Serial.print("\nread MBR failed.\n");
    //errorPrint();
    return false;
  }
  Serial.print("\nmsc # Partition Table\n");
  Serial.print("\tpart,boot,bgnCHS[3],type,endCHS[3],start,length\n");
  for (uint8_t ip = 1; ip < 5; ip++) {
    MbrPart_t *pt = &mbr.part[ip - 1];
    //    if ((pt->boot != 0 && pt->boot != 0X80) ||
    //        getLe32(pt->relativeSectors) > sdCardCapacity(&m_csd)) {
    //      valid = false;
    //    }
    switch (pt->type) {
      case 4:
      case 6:
      case 0xe:
        Serial.print("FAT16:\t");
        break;
      case 11:
      case 12:
        Serial.print("FAT32:\t");
        break;
      case 7:
        Serial.print("exFAT:\t");
        break;
      default:
        Serial.print("pt_#");
        Serial.print(pt->type);
        Serial.print(":\t");
        break;
    }
    Serial.print( int(ip)); Serial.print( ',');
    Serial.print(int(pt->boot), HEX); Serial.print( ',');
    for (int i = 0; i < 3; i++ ) {
      Serial.print("0x"); Serial.print(int(pt->beginCHS[i]), HEX); Serial.print( ',');
    }
    Serial.print("0x"); Serial.print(int(pt->type), HEX); Serial.print( ',');
    for (int i = 0; i < 3; i++ ) {
      Serial.print("0x"); Serial.print(int(pt->endCHS[i]), HEX); Serial.print( ',');
    }
    Serial.print(getLe32(pt->relativeSectors), DEC); Serial.print(',');
    Serial.println(getLe32(pt->totalSectors));
  }
  return true;
}

The call for USB Devices looks like: mbrDmp( msc.usbDrive() );

and the one for SDCard looks like: mbrDmp(sd.card() );

Next up actuallly to change the processMSDrive function,
to take in the Blockdevice and then it can handle the whole thing...

[KurtE]

@mjs513 Long Pauses are often the trying to compute the used or free space... As it walk billions and billions of ...

Probably it for today. Probably tomorrow will look into MTP again with these changes.

I am curious about how to get it to work on both PC and Ubuntu with the long long grab of information.

Again thinking of two approaches:

a) Lie at the start: and tell host that some bogus value... Or there 0xffffffff value. then after everything appears to settle, then get the actual values. and send event saying that the properties changed...
b) Allow the Host to talk to MTP while we are still computing stuff, but do like I did when we were in the process of doing format and error out every message they send us with the status we are busy...

Should be fun!

[defragster]

Indeed if it can work include @all

>> @All here : wwatson, mjs513, KurtE. WMXZ ...

Lots of libs in :: MTP-MSC-FS-SD-SDFat-LittleFS-UsbMSCFat

Other: USBHost@PJRC

[KurtE]

Just pushed up a new branch off of main which I synced back up: https://github.com/KurtE/UsbMscFat/tree/VolumeName_SD

It now shows Volume information about SD Card as well:

^^^^^^^^~FsVolume::begin(20001f30)

Check USB drive format.
initialization drive 1 failed.

msc # Partition Table
	part,boot,bgnCHS[3],type,endCHS[3],start,length
pt_#0:	1,80,0x0,0x1,0x0,0x0,0xAB,0xE0,0xFC,0,5619584
pt_#239:	2,0,0xFE,0xFF,0xFF,0xEF,0xFE,0xFF,0xFF,1700,8000
pt_#131:	3,0,0xCE,0x72,0x5D,0x83,0xF,0xFC,0xD3,5619712,10108928
pt_#0:	4,0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0,0

Initialize SD card...FsVolume::begin(20003d70)
SD card is present.


msc # Partition Table
	part,boot,bgnCHS[3],type,endCHS[3],start,length
exFAT:	1,0,0x82,0x3,0x0,0x7,0xFE,0xFF,0xFF,8192,62543872
pt_#0:	2,0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0,0
pt_#0:	3,0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0,0
pt_#0:	4,0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0,0

>> ExFat: Volume name:(EXFat_Vol) Partition Total Size:32017219584 Used:4587520 time us: 7271
T4.1-Cardlike.jpg
T4-Cardlike.jpg
DSC03355.JPG
test1.txt
mtpindex.dat
done...
Press any key to run again
Waiting up to 5 seconds for USB drive 2
^^^^^^^^^~~~~Initialize USB drive...^^^^FsVolume::begin(20001f30)

Check USB drive format.
initialization drive 1 failed.

msc # Partition Table
	part,boot,bgnCHS[3],type,endCHS[3],start,length
pt_#0:	1,80,0x0,0x1,0x0,0x0,0xAB,0xE0,0xFC,0,5619584
pt_#239:	2,0,0xFE,0xFF,0xFF,0xEF,0xFE,0xFF,0xFF,1700,8000
pt_#131:	3,0,0xCE,0x72,0x5D,0x83,0xF,0xFC,0xD3,5619712,10108928
pt_#0:	4,0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0,0

Initialize SD card...FsVolume::begin(20003d70)
SD card is present.


msc # Partition Table
	part,boot,bgnCHS[3],type,endCHS[3],start,length
FAT32:	1,0,0x0,0x1,0x40,0xC,0x3,0xE0,0xFF,8192,524288
pt_#131:	2,0,0x3,0xE0,0xFF,0x83,0x3,0xE0,0xFF,532480,16269312
pt_#0:	3,0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0,0
pt_#0:	4,0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0,0

>> Fat32: Volume name:(boot) Partition Total Size:264289280 Used:47383552 time us: 94778
overlays/
bcm2708-rpi-b-plus.dtb
COPYING.linux
LICENCE.broadcom
issue.txt
bcm2708-rpi-b-rev1.dtb
bcm2708-rpi-b.dtb
bcm2708-rpi-cm.dtb
bcm2708-rpi-zero-w.dtb
bcm2708-rpi-zero.dtb
bcm2709-rpi-2-b.dtb
bcm2710-rpi-2-b.dtb
bcm2710-rpi-3-b-plus.dtb
bcm2710-rpi-3-b.dtb
bcm2710-rpi-cm3.dtb
bcm2711-rpi-4-b.dtb
bcm2711-rpi-400.dtb
bcm2711-rpi-cm4.dtb
bootcode.bin
cmdline.txt
config.txt
fixup.dat
fixup4.dat
fixup4cd.dat
fixup4db.dat
fixup4x.dat
fixup_cd.dat
fixup_db.dat
fixup_x.dat
kernel.img
kernel7.img
kernel7l.img
kernel8.img
start.elf
start4.elf
start4cd.elf
start4db.elf
start4x.elf
start_cd.elf
start_db.elf
start_x.elf
done...
Press any key to run again
Waiting up to 5 seconds for USB drive 1
^^Waiting up to 5 seconds for USB drive 2
^^^^^^^~~~~:(Initialize USB drive...^^^^^^^^^~FsVolume::begin(20001f30)

Check USB drive format.
initialization drive 1 failed.

msc # Partition Table
	part,boot,bgnCHS[3],type,endCHS[3],start,length
pt_#0:	1,80,0x0,0x1,0x0,0x0,0xAB,0xE0,0xFC,0,5619584
pt_#239:	2,0,0xFE,0xFF,0xFF,0xEF,0xFE,0xFF,0xFF,1700,8000
pt_#131:	3,0,0xCE,0x72,0x5D,0x83,0xF,0xFC,0xD3,5619712,10108928
pt_#0:	4,0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0,0

Initialize SD card...FsVolume::begin(20003d70)
SD card is present.


msc # Partition Table
	part,boot,bgnCHS[3],type,endCHS[3],start,length
FAT32:	1,0,0x0,0x1,0x40,0xC,0x3,0xE0,0xFF,8192,524288
pt_#131:	2,0,0x3,0xE0,0xFF,0x83,0x3,0xE0,0xFF,532480,16269312
pt_#0:	3,0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0,0
pt_#0:	4,0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0,0

>> Fat32: Volume name:(boot) Partition Total Size:264289280 Used:47383552 time us: 94772
overlays/
bcm2708-rpi-b-plus.dtb
COPYING.linux
LICENCE.broadcom
issue.txt
bcm2708-rpi-b-rev1.dtb
bcm2708-rpi-b.dtb
bcm2708-rpi-cm.dtb
bcm2708-rpi-zero-w.dtb
bcm2708-rpi-zero.dtb
bcm2709-rpi-2-b.dtb
bcm2710-rpi-2-b.dtb
bcm2710-rpi-3-b-plus.dtb
bcm2710-rpi-3-b.dtb
bcm2710-rpi-cm3.dtb
bcm2711-rpi-4-b.dtb
bcm2711-rpi-400.dtb
bcm2711-rpi-cm4.dtb
bootcode.bin
cmdline.txt
config.txt
fixup.dat
fixup4.dat
fixup4cd.dat
fixup4db.dat
fixup4x.dat
fixup_cd.dat
fixup_db.dat
fixup_x.dat
kernel.img
kernel7.img
kernel7l.img
kernel8.img
start.elf
start4.elf
start4cd.elf
start4db.elf
start4x.elf
start_cd.elf
start_db.elf
start_x.elf
done...
Press any key to run again

[defragster]

IGNORE THIS POST - The external HUB/HDD was unplugged

The SD card on the PC looks like this with 4 Parts:

[defragster]

UPDATING to DELETE PRIOR POST - external drive had POWER OFF - it FELL once - changed Teensy setup on desk and put rubber feet on External HUB/DRIVE.

Got the updated p#23 UsbMsc. Using the VolumeName.ino sketch ( older p#522 sketch doesn't show? )

This shows SD card info - but is wholly missing my USB HDD and FLASH? OPPS - that drive was unplugged!

The SD card on the PC looks like this with 4 Parts:


After HUB internal 120GB and 128GB Flash - The SD CARD output catches 4 partitions - but has wrong TYPE for ExFat 4th Part - and does not do a DIR on it?
-- > It shows Nothing for the 32GB Flash drive, but works for the twin Part 64GB RECOVERY DRIVE - BOTTOM : Both 32GB Parts look okay.
>> IT IS TYPE #15 ??? - adding that to accepted list does not work?
-->> Put the SD card back into PC and :: GOT PC FAULT - Green Screen Fault - as seen during creating each of those 4 SD partitions ???
- Not sure where Dir gets done from - is it Root only?

Waiting up to 5 seconds for USB drive 1
^^^^Waiting up to 5 seconds for USB drive 2
Initialize USB drive...^^^^^^^^USB drive 1 is present.

msc # Partition Table
	part,boot,bgnCHS[3],type,endCHS[3],start,length
exFAT:	1,0,0x20,0x21,0x0,0x7,0xFE,0xFF,0xFF,2048,234436608
pt_#0:	2,0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0,0
pt_#0:	3,0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0,0
pt_#0:	4,0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0,0

>> ExFat: Volume name:(120GB_Ext) Partition Total Size:120026300416 Used:197394432 time us: 84126
A_00001.dat
A_00002.dat
A_00003.dat
A_00004.dat
A_00005.dat
A_00006.dat
A_00007.dat
Initialize USB drive...^^^^^^^
Check USB drive format.
initialization drive 2 failed.

msc # Partition Table
	part,boot,bgnCHS[3],type,endCHS[3],start,length
FAT16:	1,0,0x20,0x21,0x0,0xE,0xE8,0x27,0x1,2048,28672
exFAT:	2,0,0xA,0x9,0x2,0x7,0xFE,0xFF,0xFF,32768,242450176
pt_#0:	3,0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0,0
pt_#0:	4,0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0,0

>> ExFat:  Partition Total Size:124117712896 Used:1179648 time us: 244251
ExFat.txt
ExFatFldr/

Initialize SD card...SD card is present.


msc # Partition Table
	part,boot,bgnCHS[3],type,endCHS[3],start,length
FAT32:	1,0,0x20,0x21,0x0,0xC,0xFE,0xFF,0xFF,2048,16777216
exFAT:	2,0,0xFE,0xFF,0xFF,0x7,0xFE,0xFF,0xFF,16779264,33554432
FAT32:	3,0,0xFE,0xFF,0xFF,0xC,0xFE,0xFF,0xFF,50333696,67108864
exFAT:	4,0,0xFE,0xFF,0xFF,0xF,0xFE,0xFF,0xFF,117442560,126834688  //  THIS PART SHOWED as pt_#15 ???

>> Fat32: Volume name:(SD1_8GB_AD)###FatPartition::freeClusterCount: FT:32 start:10262 todo:1047042
    m_sectorsPerCluster:16
    m_clusterSectorMask:15
    m_sectorsPerClusterShift:4
    m_fatType:32
    m_rootDirEntryCount:0
    m_allocSearchStart:1
    m_sectorsPerFat:8181
    m_dataStartSector:26624
    m_fatStartSector:10262
    m_lastCluster:1047041
    m_rootDirStart:2
    m_bytesPerSector:512
 Partition Total Size:8577351680 Used:40960 time us: 187706
SD1_8GB_AD.txt
SD1_8GB_AD/

>> ExFat: Volume name:(SD2_16GB_AD) Partition Total Size:17175674880 Used:262144 time us: 3727
SD2_16GB_AD/
SD2_16GB_AD.txt

>> Fat32: Volume name:(SD3_32GB_AD)###FatPartition::freeClusterCount: FT:32 start:50341892 todo:1048194
    m_sectorsPerCluster:64
    m_clusterSectorMask:63
    m_sectorsPerClusterShift:6
    m_fatType:32
    m_rootDirEntryCount:0
    m_allocSearchStart:1
    m_sectorsPerFat:8190
    m_dataStartSector:50358272
    m_fatStartSector:50341892
    m_lastCluster:1048193
    m_rootDirStart:2
    m_bytesPerSector:512
 Partition Total Size:34347155456 Used:163840 time us: 187756
SD3_32GB_AD/
SD3_32GB_AD.txt
// No Info or Dir Data for 4th Part that is ExFat 60 GB
done...
Press any key to run again

Here is Restarting with Revovery 64GB drive in the system:

...

Initialize USB drive...^^^^USB drive 2 is present.

msc # Partition Table
	part,boot,bgnCHS[3],type,endCHS[3],start,length
FAT32:	1,80,0x20,0x21,0x0,0xC,0xFE,0xFF,0xFF,2048,67108864
FAT32:	2,0,0xFE,0xFF,0xFF,0xC,0xFE,0xFF,0xFF,67110912,54128640
pt_#0:	3,0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0,0
pt_#0:	4,0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0,0

>> Fat32: Volume name:(B )###FatPartition::freeClusterCount: FT:32 start:10244 todo:1048194
    m_sectorsPerCluster:64
    m_clusterSectorMask:63
    m_sectorsPerClusterShift:6
    m_fatType:32
    m_rootDirEntryCount:0
    m_allocSearchStart:1
    m_sectorsPerFat:8190
    m_dataStartSector:26624
    m_fatStartSector:10244
    m_lastCluster:1048193
    m_rootDirStart:2
    m_bytesPerSector:512
 Partition Total Size:34347155456 Used:28836429824 time us: 3071623
efi/
bootmgr
boot/
bootmgr.efi
sources/
reagent.xml
MBAMJan_2019.PNG
MBAM_TWO.PNG
Mejii_MBAM.txt
HWMonitor_x64.exe
hwmonitorw.ini
MyPhoneExplorer/
arduino_b1.9_B27/
TyComm/
tCode/
UserBack.bat
smonteith/
casa/
irs2017/
TUN-320 Win 10 Driver/

>> Fat32: Volume name:(64G_PT#2)###FatPartition::freeClusterCount: FT:32 start:67117264 todo:1690498
    m_sectorsPerCluster:32
    m_clusterSectorMask:31
    m_sectorsPerClusterShift:5
    m_fatType:32
    m_rootDirEntryCount:0
    m_allocSearchStart:1
    m_sectorsPerFat:13208
    m_dataStartSector:67143680
    m_fatStartSector:67117264
    m_lastCluster:1690497
    m_rootDirStart:2
    m_bytesPerSector:512
 Partition Total Size:27697086464 Used:65536 time us: 4953373
FPartTwo.txt

...