Teensyduino File System Integration, including MTP and MSC

Looks like it working - have 3 drives attached - 1 has 2 partitions and all are accessible and writable:
View attachment 26193

Also by accident I attached a SSD by itself and it worked as well :)

Only one issue I see that I guess at some point we need to talk about that came to mind. Since I couldn't tell which drive was which, is that the drive name should be updated, like before, the volume name or a way to show the partition number for the drive.

But it does work. Had to try format out of curiousity

Ditto with Linux except I tried QPINAND which also works:

Screenshot at 2021-10-16 16-24-47.png

Good progress...
 
Ditto with Linux except I tried QPINAND which also works:

View attachment 26198

Good progress...

Since I am a glutton for punishment I modified the mtp_test sketch to add in @KurtE's code for adding USB Drives. So with the T4.1 setup that would make 12 jumps including the 3 jump drives on a hub:
Capture.PNG

So yeah great progress !!!
 
I just pushed up some fixes/enhancements to the MSC integration stuff.

This includes updating the USB_MTP-logger sketch that the names of the MSC objects displayed on MTP are based off of volume label if the volume returns a label.

Also format was implemented. I added the code from the earlier wrapper class we have to the format method (from FS). So this runs through the PFSLib formatter code.

I tried this on one 32gb Flash drive and it appeared to work :D
 
@KurtE - Spent the day going through all of the changes and additions to MSC. I think I am starting to understand what you are doing with it. Only needed three changes to diskIO and it still works although still using the original PFsVolume way. I see how you are using "myusb.Task();" to claim the msController devices partitions. If I understand it correctly.

Also tested USB_MTP-logger. Worked with the volume names. In Linux I used gMTP to test formatting and that also worked. Great stuff:)
 
@KurtE - Spent the day going through all of the changes and additions to MSC. I think I am starting to understand what you are doing with it. Only needed three changes to diskIO and it still works although still using the original PFsVolume way. I see how you are using "myusb.Task();" to claim the msController devices partitions. If I understand it correctly.

Also tested USB_MTP-logger. Worked with the volume names. In Linux I used gMTP to test formatting and that also worked. Great stuff:)

Warning: As I mentioned: So far an experiment. Will see what Paul thinks and still a WIP. But I think it is coming along. Fixed a few issues running with UBUNTU today, still a few more things I was looking at.

Looks like the Ubuntu library does not provide the Create/Modify dates on the sendObjectInfo data, so not preserving dates and times yet.
I was also seeing that we were not showing dates and times of existing files properly. Turns out we did not fill in the ObjectInfo data with the dates, but worked on windows as windows they did a query for them. Ubuntu did not. So fixed the code to fill in the date/time strings.

Today I also updated code to have the MSC volume object code handle the FS format code.

Other things trying to test include things like:

On Windows (and probably Ubuntu) when you issue a Format request, There appears to be some undescribed timeout time, That is if we do a real short format, the format comlpletes, we return everything is OK and Windows is happy and sends out a request to use for a new session... But if we go longer than the unknown timeout, we properly send back the success, but Windows does not process it. Several seconds later the progress dialog is dismissed ant it brings up an error saying format did not complete...

Copy of files to our storage units, Can timeout and error... Not sure exactly what their formula is for timeout. Part of it is pacing I think. Sometimes the T3.x version maybe is less likely to to timeout, in that for each USB packet of 64 bytes we do a write to the storage... With the T4.x, we stuff into a buffer and try to write out in multiples of 512 bytes.. Need to figure out the sweet spot.

MAC? Not sure how best to get any MTP to work. Is anyone trying out some of the different Add ons to allow a MAC to see an MTP drive?

Yes using the task() one to enumerate the partitions. Could not do it on the claim call as system was not happy for me to do this until it completed the processing of message associated with the claim...

Probably all for tonight
 
Warning: As I mentioned: So far an experiment. Will see what Paul thinks and still a WIP. But I think it is coming along. Fixed a few issues running with UBUNTU today, still a few more things I was looking at.

Looks like the Ubuntu library does not provide the Create/Modify dates on the sendObjectInfo data, so not preserving dates and times yet.
I was also seeing that we were not showing dates and times of existing files properly. Turns out we did not fill in the ObjectInfo data with the dates, but worked on windows as windows they did a query for them. Ubuntu did not. So fixed the code to fill in the date/time strings.

Today I also updated code to have the MSC volume object code handle the FS format code.

Other things trying to test include things like:

On Windows (and probably Ubuntu) when you issue a Format request, There appears to be some undescribed timeout time, That is if we do a real short format, the format comlpletes, we return everything is OK and Windows is happy and sends out a request to use for a new session... But if we go longer than the unknown timeout, we properly send back the success, but Windows does not process it. Several seconds later the progress dialog is dismissed ant it brings up an error saying format did not complete...

Copy of files to our storage units, Can timeout and error... Not sure exactly what their formula is for timeout. Part of it is pacing I think. Sometimes the T3.x version maybe is less likely to to timeout, in that for each USB packet of 64 bytes we do a write to the storage... With the T4.x, we stuff into a buffer and try to write out in multiples of 512 bytes.. Need to figure out the sweet spot.

MAC? Not sure how best to get any MTP to work. Is anyone trying out some of the different Add ons to allow a MAC to see an MTP drive?

Yes using the task() one to enumerate the partitions. Could not do it on the claim call as system was not happy for me to do this until it completed the processing of message associated with the claim...

Probably all for tonight

Me to. Work tomorrow. I have seen the same thing with:
Code:
Looks like the Ubuntu library does not provide the Create/Modify dates on the sendObjectInfo data, so not preserving dates and times yet. 
I was also seeing that we were not showing dates and times of existing files properly.  Turns out we did not fill in the ObjectInfo data with the dates, but worked on windows as windows they did a query for them.  Ubuntu did not.  So fixed the code to fill in the date/time strings.
Linux seems a little feeble with MTP but it is getting closer. I can copy files from a Ubuntu drive to an MTP drive and the reverse but copying a file from between two MTP drives always fails. I think the transfer rates might have something to do with it. Not sure why but I can never get past maybe one 1MBps with MTP. But I know regular transfers can be between 9 to 10 MBps with without MTP and it depends greatly on the two devices used. Usually SDIO cards are so much faster. It's progress:)
 
Me to. Work tomorrow. I have seen the same thing with:
Code:
Looks like the Ubuntu library does not provide the Create/Modify dates on the sendObjectInfo data, so not preserving dates and times yet. 
I was also seeing that we were not showing dates and times of existing files properly.  Turns out we did not fill in the ObjectInfo data with the dates, but worked on windows as windows they did a query for them.  Ubuntu did not.  So fixed the code to fill in the date/time strings.
Linux seems a little feeble with MTP but it is getting closer. I can copy files from a Ubuntu drive to an MTP drive and the reverse but copying a file from between two MTP drives always fails. I think the transfer rates might have something to do with it. Not sure why but I can never get past maybe one 1MBps with MTP. But I know regular transfers can be between 9 to 10 MBps with without MTP and it depends greatly on the two devices used. Usually SDIO cards are so much faster. It's progress:)

Yes the Linux side is getting better, especially after Paul pointed the USB Interface data to the string MTP...

Pushed up another change, to the date time stuff for Ubuntu. Not ideal, but not sure what else to do.
That is when MTP sends us the object. On windows the Created and Modified string areas are filled in with the dates for the file being sent to you... On Ubuntu this value is empty. so added code to detect dates of 0 and not update the file object being created. So it should have the current time. This appeared to work for LittleFS, not sure yet of SD. It worked for Create Directory but not sure for sending a file... Will investigate more.

Right now Debugging and probably coding around how long do we have to respond back to windows when we receive an Format request, before windows gives up and gives us a Format did not complete and leaves the whole interface non-functional.

So did a quick and dirty sketch with real simple FS:
Code:
class FormatFS : public FS
  {
  public:
    FormatFS(uint32_t to) : format_to_(to) {}
    File open(const char *filename, uint8_t mode = FILE_READ) { return f;}
    bool exists(const char *filepath) {return false;}
    bool mkdir(const char *filepath) { return false;}
    bool rename(const char *oldfilepath, const char *newfilepath) { return false;}
    bool remove(const char *filepath) { return false; }
    bool rmdir(const char *filepath) { return false; }
    uint64_t usedSize() { return 0; }
    uint64_t totalSize() { return 1000000l; }
    bool format(int type = 0, char progressChar = 0, Print& pr = Serial) {
      delay(format_to_);
      return true;
    }
  private:
    uint32_t format_to_;
    File f;
  };

So I added Several of these FS's
Code:
FormatFS ffs10(1000);
FormatFS ffs20(2000);
FormatFS ffs25(2500);
FormatFS ffs30(3000);
FormatFS ffs31(3100);
FormatFS ffs35(3500);
FormatFS ffs50(5000);

Appears to work for 3 seconds but times out by 3.1 seconds. On Mine (and @mjs513) Windows 10 machines.
So will put in a code to try to avoid this.

Then need to work on the file transfer. I have had better luck with T3.6 which simply writes each buffer as it receives it (First 54 bytes, followed by full 64 byte writes, then however many left at end...)
The T4.x code tries to buffer all of these into something like an 8KB buffer, and do full writes.... The problem with LittleFS (if it has to erase a cluster) and some others is sometimes these writes can take a long time, And then we get the timeout. I played around earlier with having interval timer try to slowly read ahead until the previous write completes. But it also was finacky.

Paul and others:
Also at times wonder if the FS Object should have some method to return something like: Preferred Read/Write sizes... Not sure if this would be two parts, like sector size, and cluster size?

Also side question to self and others. Wonder if MTP code knows what is on the other side... Am I talking to windows, or ubuntu...
So far the host side is in control, and first message, is an Open Session message, which only thing provided to us is a session id...
 
Quick update: I was running into an issue where the Index list was built, and it started filling in the file index number starting right after the current number storage objects numbers... Then USB device becomes available, and it adds it, which sort of collides with the file that was added at that logical store list location.

Few different solutions possible.

I choose to reserve the first MTPD_MAX_FILESYSTEMS items in storage for filesystems... I first tried to cut down the size of these 20 items, as you don't need 256 characters to hold the file name "/"
Which worked, Where I seeked in the file index took that break in size into account...

Then decided to go one better. The only thing that actually changes at all in the root ones is, have we scanned the root directory and index to first child. So the Read/Write functions now just save that data to storage structure, and then File index starts at the first file object.

Appears to fix the issue I was seeing running the MSC objects on MTP on Ubuntu. Before I could not write any files out, as the generated pathname was garbage.
 
I just want to say thanks to everyone who is working on this! all y'all's work is going to be hugely useful to me in a short while.
 
Quick update: I was running into an issue where the Index list was built, and it started filling in the file index number starting right after the current number storage objects numbers... Then USB device becomes available, and it adds it, which sort of collides with the file that was added at that logical store list location.

Few different solutions possible.

I choose to reserve the first MTPD_MAX_FILESYSTEMS items in storage for filesystems... I first tried to cut down the size of these 20 items, as you don't need 256 characters to hold the file name "/"
Which worked, Where I seeked in the file index took that break in size into account...

Then decided to go one better. The only thing that actually changes at all in the root ones is, have we scanned the root directory and index to first child. So the Read/Write functions now just save that data to storage structure, and then File index starts at the first file object.

Appears to fix the issue I was seeing running the MSC objects on MTP on Ubuntu. Before I could not write any files out, as the generated pathname was garbage.

Won't be able to do much testing with Ubuntu till later this week. We lost two people in the warehouse so I'm doing double duty right now. Did update with your changes. I still still see the RAM drive but cannot get the USB drives to show up until I do a refresh in the file manager then the USB drives show up. This is after power up or doing an upload. I will do more testing when I can. Sorry:(
 
If Beta 3 is soon coming with more rolled in edits? Or not I should give it some runtime ...

WOW... That looks like a back ache to me:) At least you have the right truck for the job...

Bit of ache in one spot that seems to be passing :) - tomorrow will tell. 2 to 3,000 lbs if 'dry' - not all dry with recent rain so upper end. And 5.5' x 3' x 8' - is just a bit (132/128) over a cord measure - not counting the extra small row on the tailgate to keep it honest. Used a 30" saw to cut big pieces yesterday and the rest from smaller only needed the 20" today - Bobby split as I cut and stacked. Not bad for 25 year old 3/4 ton truck - had a few more pieces I left so as to not push my luck :rolleyes: Just need to get a roof back on the rack to hold it - 40' Madrona top piece exploded some 2x4's and ruined it
 
If Beta 3 is soon coming with more rolled in edits? Or not I should give it some runtime ...



Bit of ache in one spot that seems to be passing :) - tomorrow will tell. 2 to 3,000 lbs if 'dry' - not all dry with recent rain so upper end. And 5.5' x 3' x 8' - is just a bit (132/128) over a cord measure - not counting the extra small row on the tailgate to keep it honest. Used a 30" saw to cut big pieces yesterday and the rest from smaller only needed the 20" today - Bobby split as I cut and stacked. Not bad for 25 year old 3/4 ton truck - had a few more pieces I left so as to not push my luck :rolleyes: Just need to get a roof back on the rack to hold it - 40' Madrona top piece exploded some 2x4's and ruined it

Looks like you have had some fun. Have not gathered much yet this year as still have several racks of wood from previous years. Plus don't trust self with large chain saws ;) Plus my body hurts more quicker and take longer to recover than it used to.

As for current setup: For testing the integration. Some of this depends on Feedback from Paul and others.

My current setup, that I am testing with as well as I believe mjs513 and .. Starts with the Beta2 Teensyduino:

Then:

Cores: I think running beta2 so far
SD: https://github.com/PaulStoffregen/SD (not sure if any change since beta2 but I am in sync)
SDFat: https://github.com/PaulStoffregen/SdFat (Changes to speed up Format and free/used cluster count)

LittleFs: I think same as beta2, although I know both Paul and Mike are experimenting


MTP_Teensy: https://github.com/KurtE/MTP_Teensy

What is still probably in flux for next release:

USBHost/MSC - I have been working on integrating the MSC stuff into USBHost_t36 (https://github.com/KurtE/USBHost_t36/tree/FS_Integration_MSC)

To make this easier I setup the library to better conform to standard Arduino library format, and moved sources into src directory. This allowed stuff to be added to sub-directories and build. There are several interesting questions to be decided, that were mentioned earlier, like, this integration currently depends on the PFsLib code, which is a sub-directory. This is more or less same as the SDFat FsLib, but it is changed/enhanced to better know about and work with partitions. That is needed as when currently working with SDCards that plug in, SDFat assumes that it is all one partition, and only hands you the first partition of the MBR. If you do a format, it will blow away the MBR and allocated everything on that device to first partition (and write out new MBR)... With plug in disks, you don't want that, you want it to honer the data in the Master Boot Record...

I also have subdirectory of msc, which has @wwatson classes and the like. But I did create a new USBHost file system object, that ties to make the USB Drives and Volumes work sort of like USB drives with HID devices. More in previous posts.

There are still things to fix in it, like: I have a second header file in the USBHost(USBHost_ms.h) which I split out from USBHost_t36.h as was having difficultities
building some of the code if USBHost_t36.h included FS.h. It gave conflicts to some code that first included SDFat.h and then included USBHost_t3.6 (problems with FILE_READ, FILE_WRITE redefines). Need to readdress, but wanted to make progress.

Think that is it for now...
 
Won't be able to do much testing with Ubuntu till later this week. We lost two people in the warehouse so I'm doing double duty right now. Did update with your changes. I still still see the RAM drive but cannot get the USB drives to show up until I do a refresh in the file manager then the USB drives show up. This is after power up or doing an upload. I will do more testing when I can. Sorry:(

Good luck, sounds like you are busy!

With the startup sequence of things, I want to do some experiments, on the startup code, to see what might work best.

In the USB_MTP-logger sketch, need to experiments some with the ordering of calls like move mtpd.begin() up earlier in code, maybe before myusb.begin()...
Maybe add one or two calls to myusb.Task, before we call off to mtpd.loop() code,

Plus update the code, that detects when new volume has been detected and we add it to storage list. I did not put in the next part yet, of when we detect a drive went away and as such the volumes are no longer valid, to remove them from the list.

I was sort of holding off, until more conversations on maybe adding something like events, such that don't have to manually search for things changing. But will probably add some of that in to experiment more with timings.

also right now when I add new volume, used sledgehammer to tell host (Reset device).

Will like to maybe change this, to maybe be more selective.

Like: If we have not enumerated the list of storage objects to host yet, don't notify.
If simply adding FS, try the send_StoreAddedEvent instead. If store removed, we could try the send_StoreRemovedEvent, but then this would also need to prune out our storage list. Not sure if that would be a win... But interesting to experiment with.

Updates showing up on windows/Ubuntu, without refresh(F5) So far I have always needed to do a refresh. But again Ubuntu is getting better
 
@defragster

Definitely getting to be that time of year. Living in the city don't have the problem with firewood looks backbreaking but at least the truck can take it!!!!

@KurtE pretty much summarized the configuration that we have been using for testing. I have pretty much been using mtp_test and some of the other tests that Kurt put together to try and isolate problems but pretty much MTP_test.
 
Just pushed up changes to the USB_MTP-logger sketch and changes within MTP code for how the code is handling knowing a USB drive comes on line or goes away.
It is now processing both.

It is now sending the storage added or removed messages, which appear to be working better on both Windows and Ubuntu. The drive now shows up or goes away when the even happens without having to do refresh.

There is still things to figure out what best to do in some areas. Like:

Currently configured max 20 stores, today each time new store found, it increments a counter and uses the next slot. So after a few iterations, you may not have any storage slots left. Probably several simple solutions on how to reuse the slots that were used earlier by a device. Simply reuse that same storageID that earlier device had... Might work, or simply change the functions that convert store to storageID and back to have different number for storage.

Today they are simple inline functions:
Code:
  static inline uint32_t Store2Storage(uint32_t store) {
    return ((store + 1) << 16) | 1;
  }
  static inline uint32_t Storage2Store(uint32_t storage) {
    return (storage >> 16) - 1;
  }
Could easily be extended that we add in counter that gets put somewhere in that 32 it number that we ignore coming back.


Another issue: is when I remove a store, I remove it from our list of storage FS.s, I clear some fields, but I don't in this case clear out the index file, so all of the items within the storage list for that older storage unit are orphaned. And as we store the store # in this list not storage, it might be confusing to look at list of items. Could solve, but still send the device reset event in this case, which wipes out this list and starts from scratch.

Could walk the list and mark all of these as removed, and still have the storage file grow with new ones added. Could update the code for getting the item to know about holes and remove large chunks...

Will play with some of this soon.
 
Just pushed up changes to the USB_MTP-logger sketch and changes within MTP code for how the code is handling knowing a USB drive comes on line or goes away.
It is now processing both.

It is now sending the storage added or removed messages, which appear to be working better on both Windows and Ubuntu. The drive now shows up or goes away when the even happens without having to do refresh.

There is still things to figure out what best to do in some areas. Like:

Currently configured max 20 stores, today each time new store found, it increments a counter and uses the next slot. So after a few iterations, you may not have any storage slots left. Probably several simple solutions on how to reuse the slots that were used earlier by a device. Simply reuse that same storageID that earlier device had... Might work, or simply change the functions that convert store to storageID and back to have different number for storage.

Today they are simple inline functions:
Code:
  static inline uint32_t Store2Storage(uint32_t store) {
    return ((store + 1) << 16) | 1;
  }
  static inline uint32_t Storage2Store(uint32_t storage) {
    return (storage >> 16) - 1;
  }
Could easily be extended that we add in counter that gets put somewhere in that 32 it number that we ignore coming back.


Another issue: is when I remove a store, I remove it from our list of storage FS.s, I clear some fields, but I don't in this case clear out the index file, so all of the items within the storage list for that older storage unit are orphaned. And as we store the store # in this list not storage, it might be confusing to look at list of items. Could solve, but still send the device reset event in this case, which wipes out this list and starts from scratch.

Could walk the list and mark all of these as removed, and still have the storage file grow with new ones added. Could update the code for getting the item to know about holes and remove large chunks...

Will play with some of this soon.

I finally had a chance to test your latest updates. They seem to work:) On power up or reset they all show up in the file manager. Also I can disconnect and reconnect a USB device and it will disappear and reappear. Good stuff. Did not have to refresh the file manager. This is with Ubuntu by the way.
Back to work now:(
 
Quick note: I have been experimenting to better understand why some of my external drives don't work... So I have been experimenting...

Will probably remove some of this, but detected that the SSD drive I tried has a GPT setup instead of MBR... Curious how hard it would be to support simple cases of this.

So I updated the USB_MTP-logger test sketch (MTP_Teensy) to detect when the drive is plugged in and then have the drives call the function to print out their MBR information.

So for one drive I have plugged in I see something like:
Code:
USB Drive: 0 connected

msc # Partition Table
	part,boot,bgnCHS[3],type,endCHS[3],start,length
FAT32:	1,80,0x0,0x39,0x0,0xB,0xF,0xFC,0xD3,56,15728584
pt_#0:	2,0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0,0
pt_#0:	3,0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0,0
pt_#0:	4,0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0,0
addFilesystem: 1 MSC0-FAT32 20008a70

The drive I mentioned that does not work, printed out something like this but the first partition had type 0xee which is not Fat16 or 32 ore exFat so we ignore....
Turns out this is the GPT guard MBR...

SO yesterday I hacked on the mbrDmp method, to print out, and then have been playing to see if I can figure out what it is...
So current output looks like:
Code:
USB Drive: 1 connected

msc # Partition Table
	part,boot,bgnCHS[3],type,endCHS[3],start,length
*** GPT Disk not supported ***
GPT guard:	1,0,0x0,0x2,0x0,0xEE,0xFE,0x7F,0x99,1,4294967295
pt_#0:	2,0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0,0
pt_#0:	3,0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0,0
pt_#0:	4,0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0,0

GPT partition header revision: 10000
LBAs current:1 backup:500118191 first:34 last:500118158
Disk GUID:03156F5D-B405-431B-9367-0CF4584D8695Start LBA Array: 2 Count: 128 size:128
Part	 Type Guid, Unique Guid, First, last, attr, name
0	EBD0A0A2-B9E5-4433-87C0-68B6B72699C7, 547BDDC9-1B3A-49A1-858D-BCFBA0FD54BB, 2048, 500117503, 0, Basic data partition
>>> Microsoft Basic Data Partition
EB 76 90 45 58 46 41 54 20 20 20 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 :.v.EXFAT   .....................
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 :................................
00 08 00 00 00 00 00 00 00 28 CF 1D 00 00 00 00 00 08 00 00 00 3C 00 00 00 48 00 00 E0 CE 1D 00 :.........(...........<...H......
05 00 00 00 7A F8 1A 84 00 01 00 00 09 08 01 80 00 00 00 00 00 00 00 00 33 C9 8E D1 8E C1 8E D9 :....z...................3.......
BC D0 7B BD 00 7C 88 16 6F 7C B4 41 BB AA 55 CD 13 72 69 81 FB 55 AA 75 63 F6 C1 01 74 5E FE 06 :..{..|..o|.A..U..ri..U.uc...t^..
02 7C 66 50 B0 65 E8 A6 00 66 58 66 B8 01 00 00 00 8A 0E 6D 7C 66 D3 E0 66 89 46 E8 66 B8 01 00 :.|fP.e...fXf.......m|f..f.F.f...
00 00 8A 0E 6C 7C 66 D3 E0 66 89 46 D8 66 A1 40 7C 66 40 BB 00 7E B9 01 00 66 50 E8 41 00 66 58 :....l|f..f.F.f.@|f@..~...fP.A.fX
66 40 BB 00 80 B9 01 00 E8 34 00 66 50 B0 78 E8 5D 00 66 58 E9 09 01 A0 FC 7D EB 05 A0 FB 7D EB :f@.......4.fP.x.].fX.....}....}.
00 B4 7D 8B F0 AC 98 40 74 0C 48 74 0E B4 0E BB 07 00 CD 10 EB EF A0 FD 7D EB E6 CD 16 CD 19 66 :..}....@t.Ht............}......f
60 66 6A 00 66 50 06 53 66 68 10 00 01 00 B4 42 B2 80 8A 16 6F 7C 8B F4 CD 13 66 58 66 58 66 58 :`fj.fP.Sfh.....B....o|....fXfXfX
66 58 66 61 72 B1 03 5E D8 66 40 49 75 D1 C3 66 60 B4 0E BB 07 00 B9 01 00 CD 10 66 61 C3 42 00 :fXfar..^.f@Iu..f`..........fa.B.
4F 00 4F 00 54 00 4D 00 47 00 52 00 0D 0A 52 65 6D 6F 76 65 20 64 69 73 6B 73 20 6F 72 20 6F 74 :O.O.T.M.G.R...Remove disks or ot
68 65 72 20 6D 65 64 69 61 2E FF 0D 0A 44 69 73 6B 20 65 72 72 6F 72 FF 0D 0A 50 72 65 73 73 20 :her media....Disk error...Press 
61 6E 79 20 6B 65 79 20 74 6F 20 72 65 73 74 61 72 74 0D 0A 00 00 00 00 00 00 00 00 00 00 FF FF :any key to restart..............
FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF :................................
FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF 6C 8B 98 55 AA :...........................l..U.
    EXFAT:
So I properly detected that the first partition was a GPT Guard and read in the sector 1, slightly verified that it was an GPT header...
Printed out a bit of it... Then cycled through the count of partitions (128 with 4 per sector)...

Then only looked at ones that were not all 0s... Looked for GUID=EBD0A0A2-B9E5-4433-87C0-68B6B72699C7
Which stands for: Microsoft Basic Data partition...
I then Read in the first cluster of that partition, which should be a partition boot sector... ... Which I currently dump..
Which so far I detected this case as ExFat (first three bytes as ... and Name: "EXFAT "

Next up maybe playing with seeing how hard to get the ExFat library to like it...

The only place other than format, that I think does anything here is: bool ExFatPartition::init(BlockDevice* dev, uint8_t part) {
And after it verifies MBR it reads in this sector as well, and continues... So may try hacking up version that does some of the steps I just mentioned.

May also be curious at changing this as well, to detect MBR partition types of Extended Partition. If I do, I will probably make an assumption, to only support it in the 4th position and maybe allow partition number > 4...

Let know if anyone wishes to play along and I will push up current changes.
 
Quick note: I have been experimenting to better understand why some of my external drives don't work... So I have been experimenting...

Will probably remove some of this, but detected that the SSD drive I tried has a GPT setup instead of MBR... Curious how hard it would be to support simple cases of this.

So I updated the USB_MTP-logger test sketch (MTP_Teensy) to detect when the drive is plugged in and then have the drives call the function to print out their MBR information.

So for one drive I have plugged in I see something like:
Code:
USB Drive: 0 connected

msc # Partition Table
	part,boot,bgnCHS[3],type,endCHS[3],start,length
FAT32:	1,80,0x0,0x39,0x0,0xB,0xF,0xFC,0xD3,56,15728584
pt_#0:	2,0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0,0
pt_#0:	3,0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0,0
pt_#0:	4,0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0,0
addFilesystem: 1 MSC0-FAT32 20008a70

The drive I mentioned that does not work, printed out something like this but the first partition had type 0xee which is not Fat16 or 32 ore exFat so we ignore....
Turns out this is the GPT guard MBR...

SO yesterday I hacked on the mbrDmp method, to print out, and then have been playing to see if I can figure out what it is...
So current output looks like:
Code:
USB Drive: 1 connected

msc # Partition Table
	part,boot,bgnCHS[3],type,endCHS[3],start,length
*** GPT Disk not supported ***
GPT guard:	1,0,0x0,0x2,0x0,0xEE,0xFE,0x7F,0x99,1,4294967295
pt_#0:	2,0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0,0
pt_#0:	3,0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0,0
pt_#0:	4,0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0,0

GPT partition header revision: 10000
LBAs current:1 backup:500118191 first:34 last:500118158
Disk GUID:03156F5D-B405-431B-9367-0CF4584D8695Start LBA Array: 2 Count: 128 size:128
Part	 Type Guid, Unique Guid, First, last, attr, name
0	EBD0A0A2-B9E5-4433-87C0-68B6B72699C7, 547BDDC9-1B3A-49A1-858D-BCFBA0FD54BB, 2048, 500117503, 0, Basic data partition
>>> Microsoft Basic Data Partition
EB 76 90 45 58 46 41 54 20 20 20 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 :.v.EXFAT   .....................
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 :................................
00 08 00 00 00 00 00 00 00 28 CF 1D 00 00 00 00 00 08 00 00 00 3C 00 00 00 48 00 00 E0 CE 1D 00 :.........(...........<...H......
05 00 00 00 7A F8 1A 84 00 01 00 00 09 08 01 80 00 00 00 00 00 00 00 00 33 C9 8E D1 8E C1 8E D9 :....z...................3.......
BC D0 7B BD 00 7C 88 16 6F 7C B4 41 BB AA 55 CD 13 72 69 81 FB 55 AA 75 63 F6 C1 01 74 5E FE 06 :..{..|..o|.A..U..ri..U.uc...t^..
02 7C 66 50 B0 65 E8 A6 00 66 58 66 B8 01 00 00 00 8A 0E 6D 7C 66 D3 E0 66 89 46 E8 66 B8 01 00 :.|fP.e...fXf.......m|f..f.F.f...
00 00 8A 0E 6C 7C 66 D3 E0 66 89 46 D8 66 A1 40 7C 66 40 BB 00 7E B9 01 00 66 50 E8 41 00 66 58 :....l|f..f.F.f.@|f@..~...fP.A.fX
66 40 BB 00 80 B9 01 00 E8 34 00 66 50 B0 78 E8 5D 00 66 58 E9 09 01 A0 FC 7D EB 05 A0 FB 7D EB :f@.......4.fP.x.].fX.....}....}.
00 B4 7D 8B F0 AC 98 40 74 0C 48 74 0E B4 0E BB 07 00 CD 10 EB EF A0 FD 7D EB E6 CD 16 CD 19 66 :..}....@t.Ht............}......f
60 66 6A 00 66 50 06 53 66 68 10 00 01 00 B4 42 B2 80 8A 16 6F 7C 8B F4 CD 13 66 58 66 58 66 58 :`fj.fP.Sfh.....B....o|....fXfXfX
66 58 66 61 72 B1 03 5E D8 66 40 49 75 D1 C3 66 60 B4 0E BB 07 00 B9 01 00 CD 10 66 61 C3 42 00 :fXfar..^.f@Iu..f`..........fa.B.
4F 00 4F 00 54 00 4D 00 47 00 52 00 0D 0A 52 65 6D 6F 76 65 20 64 69 73 6B 73 20 6F 72 20 6F 74 :O.O.T.M.G.R...Remove disks or ot
68 65 72 20 6D 65 64 69 61 2E FF 0D 0A 44 69 73 6B 20 65 72 72 6F 72 FF 0D 0A 50 72 65 73 73 20 :her media....Disk error...Press 
61 6E 79 20 6B 65 79 20 74 6F 20 72 65 73 74 61 72 74 0D 0A 00 00 00 00 00 00 00 00 00 00 FF FF :any key to restart..............
FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF :................................
FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF 6C 8B 98 55 AA :...........................l..U.
    EXFAT:
So I properly detected that the first partition was a GPT Guard and read in the sector 1, slightly verified that it was an GPT header...
Printed out a bit of it... Then cycled through the count of partitions (128 with 4 per sector)...

Then only looked at ones that were not all 0s... Looked for GUID=EBD0A0A2-B9E5-4433-87C0-68B6B72699C7
Which stands for: Microsoft Basic Data partition...
I then Read in the first cluster of that partition, which should be a partition boot sector... ... Which I currently dump..
Which so far I detected this case as ExFat (first three bytes as ... and Name: "EXFAT "

Next up maybe playing with seeing how hard to get the ExFat library to like it...

The only place other than format, that I think does anything here is: bool ExFatPartition::init(BlockDevice* dev, uint8_t part) {
And after it verifies MBR it reads in this sector as well, and continues... So may try hacking up version that does some of the steps I just mentioned.

May also be curious at changing this as well, to detect MBR partition types of Extended Partition. If I do, I will probably make an assumption, to only support it in the 4th position and maybe allow partition number > 4...

Let know if anyone wishes to play along and I will push up current changes.

Of course I will play along. I have a SSD drive with 1 primary and 1 logical which shows up as extended and is formatted as fat32.

Believe FatPartition has a similar command: bool init(BlockDevice* dev, uint8_t part = 1);
 
Just pushed up changes...

Will hack some on ExFat, yes I believe it will be very similar with Fat32/16...

Would be curious to see your SSD drive MBR print to see where the extended one was added...
That is are the 4 slots like: FAT32 <extended> <unused> <unused> ?

If we support extended partition, will be interesting to decide on partition numbers to pass in: In theory could have:

<Fat32> <extended with: Fat32 ExFat UNUSED UNUSED> <FAT16> <UNUSED>

So If I pass in 3 do I get the Fat16 of MBR, or should it the ExFat of the Extended partition...
 
Quick update: I am playing now with a new Branch of Paul's fork of SDFat (https://github.com/KurtE/SdFat/tree/GPT_disks)

Where I put some of the same debug code into the bool ExFatPartition::init(BlockDevice* dev, uint8_t part) function mentioned earlier, and can now open up an ExFat partition on
an SSD disk with cable adapter... :D

screenshot.jpg

Next up maybe take it back to my PC and add 2nd partition of One ExFat the other Fat32...
 
Good work on extending support @KurtE!

Back to update and a few minor run/edits on the Beta 1 USB Teensy overwhelming PC/Windows, but not gathered WIP parts for this. I did get an amazon UGREEN Dual SD Card Reader to try. Not plugged in yet - but it does both an SD and MicroSD card on one USB plug.

And reroofed woodshed holding off water and one third of the pickup of wood that was transferred so far ...
 
Thanks just pushed up WIP change to also load Fat32 (probably Fat16 as well) off of GPT disk.

I am trying out different pieces of hardware that I have on hand... Did buy some more SD card and some more thumb drives.

Using RSHTECH powered USB hub on it for this. Purchased earlier in case the other powered hub I have on PC fails...

The RSHTECH one is sort of interesting, that if I turn it on before the Teensy is powered up. none of the ports show active, but once the Teensy boots, then the in this case 3 different drives show power and work. But the interesting thing is once it is in this state, it is back powering the Teensy, I have to unplug the power connector from it before the teensy will lose power....

Sounds good for getting your roof back on. Supposed to be some pretty good wind starting tonight!
 
Back
Top