MTP Responder Contribution

On one of the NAND SPI chips LFSIntegrity had run leaving lots of dirs and files - died on delete of folder 24? Teensy still responds to USB CMD 'f' to toggle that.

Running the new LFSIntegrity a recursive delete/WIPE all was created and it is visible in the delete process there are some longer pauses.

Wondering if one of those pauses is what threw off the delete? I said skip the rest and the drive was gone after that.

Here is the end where it failed:
Code:
...
CMD: 1008(GET_OBJECT_INFO)l: 16 T:1e2 : a
RESP:2001(RSP:OK)l: 16 T:1e2 : a
CMD: 9803(GET_OBJECT_PROP_VALUE)l: 20 T:1e3 : a dc01 (STORAGE_ID)
RESP:2001(RSP:OK)l: 20 T:1e3 : a dc01
CMD: 9803(GET_OBJECT_PROP_VALUE)l: 20 T:1e4 : a dc07 (OBJECT NAME)
RESP:2001(RSP:OK)l: 20 T:1e4 : a dc07
CMD: 9803(GET_OBJECT_PROP_VALUE)l: 20 T:1e5 : a dc41 (PERSISTENT_UID)
RESP:2001(RSP:OK)l: 20 T:1e5 : a dc41
CMD: 9803(GET_OBJECT_PROP_VALUE)l: 20 T:1e6 : a dc44 (NAME)
RESP:2001(RSP:OK)l: 20 T:1e6 : a dc44
CMD: 9803(GET_OBJECT_PROP_VALUE)l: 20 T:1e7 : 9 dc02 (FORMAT)
RESP:2001(RSP:OK)l: 20 T:1e7 : 9 dc02
CMD: 9803(GET_OBJECT_PROP_VALUE)l: 20 T:1e8 : 9 dc03 (PROTECTION)
RESP:2001(RSP:OK)l: 20 T:1e8 : 9 dc03
CMD: 9803(GET_OBJECT_PROP_VALUE)l: 20 T:1e9 : 9 dc04 (SIZE)
RESP:2001(RSP:OK)l: 20 T:1e9 : 9 dc04
CMD: 1008(GET_OBJECT_INFO)l: 16 T:1ea : 9
RESP:2001(RSP:OK)l: 16 T:1ea : 9
CMD: 9803(GET_OBJECT_PROP_VALUE)l: 20 T:1eb : 9 dc01 (STORAGE_ID)
RESP:2001(RSP:OK)l: 20 T:1eb : 9 dc01
CMD: 9803(GET_OBJECT_PROP_VALUE)l: 20 T:1ec : 9 dc07 (OBJECT NAME)
RESP:2001(RSP:OK)l: 20 T:1ec : 9 dc07
CMD: 9803(GET_OBJECT_PROP_VALUE)l: 20 T:1ed : 9 dc41 (PERSISTENT_UID)
RESP:2001(RSP:OK)l: 20 T:1ed : 9 dc41
CMD: 9803(GET_OBJECT_PROP_VALUE)l: 20 T:1ee : 9 dc44 (NAME)
RESP:2001(RSP:OK)l: 20 T:1ee : 9 dc44
CMD: 100b(DELETE_OBJECT)l: 20 T:1ef : 24 0
RESP:2001(RSP:OK)l: 20 T:1ef : 24 0
CMD: 100b(DELETE_OBJECT)l: 20 T:1f0 : 25 0
RESP:2001(RSP:OK)l: 20 T:1f0 : 25 0
CMD: 100b(DELETE_OBJECT)l: 20 T:1f1 : 26 0
RESP:2001(RSP:OK)l: 20 T:1f1 : 26 0
CMD: 100b(DELETE_OBJECT)l: 20 T:1f2 : 27 0
RESP:2001(RSP:OK)l: 20 T:1f2 : 27 0
CMD: 100b(DELETE_OBJECT)l: 20 T:1f3 : 28 0
RESP:2001(RSP:OK)l: 20 T:1f3 : 28 0
CMD: 100b(DELETE_OBJECT)l: 20 T:1f4 : 9 0
RESP:2001(RSP:OK)l: 20 T:1f4 : 9 0
CMD: 100b(DELETE_OBJECT)l: 20 T:1f5 : a 0
RESP:2001(RSP:OK)l: 20 T:1f5 : a 0
CMD: 100b(DELETE_OBJECT)l: 20 T:1f6 : b 0
RESP:2001(RSP:OK)l: 20 T:1f6 : b 0
CMD: 100b(DELETE_OBJECT)l: 20 T:1f7 : c 0
RESP:2001(RSP:OK)l: 20 T:1f7 : c 0
CMD: 100b(DELETE_OBJECT)l: 20 T:1f8 : d 0
RESP:2001(RSP:OK)l: 20 T:1f8 : d 0
CMD: 100b(DELETE_OBJECT)l: 20 T:1f9 : e 0
RESP:2001(RSP:OK)l: 20 T:1f9 : e 0
CMD: 100b(DELETE_OBJECT)l: 20 T:1fa : f 0
RESP:2001(RSP:OK)l: 20 T:1fa : f 0
CMD: 100b(DELETE_OBJECT)l: 20 T:1fb : 10 0
RESP:2001(RSP:OK)l: 20 T:1fb : 10 0
CMD: 100b(DELETE_OBJECT)l: 20 T:1fc : 11 0
RESP:2001(RSP:OK)l: 20 T:1fc : 11 0
CMD: 100b(DELETE_OBJECT)l: 20 T:1fd : 12 0
RESP:2001(RSP:OK)l: 20 T:1fd : 12 0
CMD: 100b(DELETE_OBJECT)l: 20 T:1fe : 14 0
RESP:2001(RSP:OK)l: 20 T:1fe : 14 0
CMD: 100b(DELETE_OBJECT)l: 20 T:1ff : 15 0
RESP:2001(RSP:OK)l: 20 T:1ff : 15 0
CMD: 100b(DELETE_OBJECT)l: 20 T:200 : 16 0
RESP:2001(RSP:OK)l: 20 T:200 : 16 0
CMD: 100b(DELETE_OBJECT)l: 20 T:201 : 17 0
RESP:2001(RSP:OK)l: 20 T:201 : 17 0
CMD: 100b(DELETE_OBJECT)l: 20 T:202 : 18 0
RESP:2001(RSP:OK)l: 20 T:202 : 18 0

 *** Command line: f
low level format of LittleFS disks selected

Looked a bit more about adding formatunused as third type - some progress - but unclear ...
 
Hard to say, why it stopped, maybe the command was taking longer to finish, and the PC side decided to complain? Or maybe file size? Or ???

There are times with these FS, I wonder about would it make sense to add some ability to do some background housecleaning?

For example in one of the main loops, have something like: if (drive_needs_cleanup) drive_needs_cleanup= myfs.doFastCleanup();
Where maybe it will do one erase unused write... Maybe top level code will only call this if no MTP messages received for some time...
 
Sounds like you are talking about the old windows defrag idea or propabably format unused?
Probably... ;) Probably won't do anything with that yet as it still is unclear how much will make it into a release... But could be nice, to hopefully in the background cleanup the Chips such that very seldom would users see the MTP drop due to really slow writes. But first still playing with BUILTIN sdcard detection stuff :D
 
Quick update, I updated the code for handling the BUILTIN SDCard, if it is not plugged in. Instead of adding it the first time we detect it on DATA3, I go ahead and add it at startup time. I do however defer adding it until all of the other potential drives are added as to avoid issue where index file would try to write to our non existent drive...
Now when you plug it in, I send a StorageChanged event instead of storageAdded


On Windows, when you open up the Teensy folder it shows up in RED.... If still viewing that folder after you mount the SDCard, you may have to do a folder refresh. It did show up properly when I was not looking at the top level window, and went back to it.

Next up try it on the Ubunutu machine.
 
Evening @KurtE
Just downloaded you latest and greatest and gave it a try. Worked like a charm just as you described it in your post! Of course it was on a Windows 10x64 machine :)
 
Tried running same code on Ubuntu 20.04 (MTP+Serial) so I have debug...

Interesting the window that pops up is Shotwell and it shows that I have 4 storage items all named Teensy MTP Disk/Serial
No where it shows the friendly name. If you hover it does show a full path with storage number as you can see in the image:
Screenshot from 2021-02-18 15-14-19.jpg

If I don't have SD in it only shows 3 storage items, when I plug it in. Debug code shows that there are 4, but it does not show up anywhere. Will try next remove/add storage.
 
Observation

I wanted to share an observation T4.1 with Win10
I typically have 6 SD cards attached to my T4.1 (with sdio)

observation:
MTP may fail to work (shows only sdio, with no files in it), when I populate only a part of the SD slots (e.g. the first 2 slots with SDcards the last 4 slots empty)
MTP works as expected when I populate at least the last of the SD slots (last in terms of call of sdfs.begin(..)).
IMO, this means that somewhere is a global variable that is set (or not set) when sdfs.begin(..) fails, compromising proper functioning of MTP/SdFat combo.
Workaround: populate always the last SD slot in your list of SD storages.
 
If LFS tracked or informed on abandoning a block we could use that info. When they want a new block they ask for 'Format' before use, when they are done with a block they just move on AFAIK.

But after reading and finding the Traverse() showing the USED blocks ( sometimes with great delay ) that is used to KNOW what blocks NOT to format, thus formatUnused. Sort of a PostPreDefrag empty the BitBucket.

Any block used by LFS is header tagged - so if not KNOWN as in use - it would be dirty in the first dozen or so bytes and get formatted to clean with unusedFormat. If ever done LFSIntegrity would catch that with tracking of the number of files as it does and doing a verify on the file data on updates and delete.


I'll glance again at the Format request mechanism. Knowing when LFS asks for Format would say disk data changed ( though it could change internal to a block when a rewrite can do the job without moving to new ). But that 'new block request' from LFS would be enough to either note any pending bitmap is obsolete needing a new traverse - or perhaps mark that block as used in an existing bitmap.
 
@WMXZ - re p#883 - have you tried with KurtE's latest p#880?

Just back and not tried ... Does it prepopulate the empty slots on Windows with RED when no disk is installed?
 
@WMXZ - re p#883 - have you tried with KurtE's latest p#880?
No, also as my application calls in general for all slots populated with SD cards.
However, could add the test on Data3 to void sdfs.begin to tell me that it failed.
 
I wanted to share an observation T4.1 with Win10
I typically have 6 SD cards attached to my T4.1 (with sdio)

observation:
MTP may fail to work (shows only sdio, with no files in it), when I populate only a part of the SD slots (e.g. the first 2 slots with SDcards the last 4 slots empty)
MTP works as expected when I populate at least the last of the SD slots (last in terms of call of sdfs.begin(..)).
IMO, this means that somewhere is a global variable that is set (or not set) when sdfs.begin(..) fails, compromising proper functioning of MTP/SdFat combo.
Workaround: populate always the last SD slot in your list of SD storages.

Note: my experiment for the SD is so far only for the BUILTIN_SDCARD that uses SDIO and I have a fixed pin number for DATA3

OOPS, I see issue in the example: Although probably not ever hit:
Code:
#ifndef BUILTIN_S[COLOR="#FF0000"]C[/COLOR]CARD
#define BUILTIN_SDCARD 254
#endif
When you have 6 SD Cards, I am assuming at least 5 of them use SPI? At least I don't know any way to setup uSDHC except on the main onboard SD Slot.

Curious on how you are hooking up the 6? Your own board or a bunch of breakouts? Is there any wiring setup for Card Detect? Maybe should add code to optionally use this pin in similar way I am playing with the BUILTIN...
 
Curious on how you are hooking up the 6? Your own board or a bunch of breakouts? Is there any wiring setup for Card Detect? Maybe should add code to optionally use this pin in similar way I am playing with the BUILTIN...
I have a T4.1 with a custom daughter board (having 6 spi uSD cards) : total 7 disks
Originally I wanted to use only sdio bus, but unfortunately the uSD cards to not implement the sdio addressing system (similar to i2c) so I had to abandon this idea.
My storage boards implements SPI with 6 chip select lines. they are not wired for card select (would need other 6 lines).

I use it with success for some time now. The observation was only noted when I did a quick test with only sdio +1 spi card inserted randomly and MTP crashed during startup. but with sdio + 1 spi card on last postition (#6) MTP started up correctly.
 
@WMXZ - I have a package coming from digikey with some more external sdcard units... For the fun may play with a few, probably won't get up to 6.
And yes you would need to hook up extra pins for each of the card select pins... How many, obviously the easiest would be 6 pins. But could also probably do with SPI digital MUX of some type.

Or Could use a couple of Analog pins and resistors and compute which pin/pins are down by some form of resistor ladder... I did that once when I was down to maybe one pin left and wanted to check the state of a few more buttons...


@all - I just pushed up a version of the mtp-test sketch in my branch that starts to allow me to plug in an MSC drive and see it through MTP.
I set the define to include it to 0
#define USE_MSC_FAT 0 // set to 1 experiment with MTP (USBHost.t36 + mscFS)

change to a 1 to try it out. But it requires the UsbMscFat library.
Also I issued a PR: https://github.com/wwatson4506/UsbMscFat/pull/1
to change the name of one #define to not conflict with other libraries.

Lots of interesting questions to investigate, more later.
 
@KurtE

You just knew I had to download your changes to test MTP with MSC (UsbMscFat).

Anyway hooked up my SSD and low and behold it showed up in MTP and was able to do several things:
1. Copy files from the SSD to PC
2. Play .wav files in windows media
3. Open files on SSD directly
4. Copy @degragsters T4.1 directory - Interesting Note that Long Filename Transferred without an issue to the SSD.

Only issue, which is a known issue, is that the SSD does have 2 partitions but only the first shows up in MTP.

Nice that it works.
 
Now hopefully will get a little time to play a little more and see about defining multiple MSC objects.

Also see about them coming and going... Sort of along the line of SD card being inserted.

Then wondering about partitions....

Then wondering if drives can come and go and some are sort of fixed, should we/can we let the MTP on host know this? Do they allow an Eject like command? Do we need one with things like SD Fat or MSC or is everything flushed when files change or the like? ...
 
My next question to self and others maybe other thread
is to support multiple MSC drives, I was using the global object (MSC) to add the one MSC object as is in the FilesUSB object.

Thought I would try the stuff out of copyFilesUSB.ino sketch which is setup for two MSC objects which it has as msc1 and msc2.

Probably is: MSC global is of the class MSCClass
which is class MSCClass : public FS

So works to add to storage.

But: UsbFs msc1;
and: class UsbFs : public UsbBase<FsVolume> {

Which is not based on FS...

So need to figure out the relationship...
 
@WMXZ - I have a package coming from digikey with some more external sdcard units... For the fun may play with a few, probably won't get up to 6.
And yes you would need to hook up extra pins for each of the card select pins... How many, obviously the easiest would be 6 pins. But could also probably do with SPI digital MUX of some type.

Or Could use a couple of Analog pins and resistors and compute which pin/pins are down by some form of resistor ladder... I did that once when I was down to maybe one pin left and wanted to check the state of a few more buttons...

FWIW, I've attached a PDF schematic from a data logger from about 12 years ago---back when 32GB SD cards were the largest available and cost $50 to $75 dollars each. We needed 128GB of storage, so I designed a multiplexer system using 74AHC24 chips.

View attachment SDC_Array.pdf


I've fallen far behind on the ongoing implementation of MTP events. I'm still occasionally using older MTP implementations without events in some applications. I think I'm waiting for the time when I don't have to modify the cores files to get MTP + Serial to work.
 
FWIW, I've attached a PDF schematic from a data logger from about 12 years ago---back when 32GB SD cards were the largest available and cost $50 to $75 dollars each. We needed 128GB of storage, so I designed a multiplexer system using 74AHC24 chips.
Looks like sort of a pain... But you got to to do what you got to do :D
 
My next question to self and others maybe other thread
is to support multiple MSC drives, I was using the global object (MSC) to add the one MSC object as is in the FilesUSB object.

Thought I would try the stuff out of copyFilesUSB.ino sketch which is setup for two MSC objects which it has as msc1 and msc2.

Probably is: MSC global is of the class MSCClass
which is class MSCClass : public FS

So works to add to storage.

But: UsbFs msc1;
and: class UsbFs : public UsbBase<FsVolume> {

Which is not based on FS...

So need to figure out the relationship...

Was able to substitute the usage of MSCClass for the UsbFs class and that appears to work.

I just updated my branch (warning left the define = 2, so will require the additional libraries.
But with this I have a USB HUB plugged in with two Thumb drives and both are added to MTP...
Note: I was not 100% sure which formatting you need on one of them for 8GB drive. (Fat32? exFat)... Ran the formatter out of your library which appears to get it up and running.

Next up after some other distractions is to handle them being inserted after program starts!
 
In case anyone wishes to play along.

I have added some code to the MTP-Test with the MSC stuff where your define for including it value > 0 is the Max count of USB drives you wish to use...

I also now have code in that detects when the drive is plugged in and adds it to the list, which appears to be working,
but I also added code to try to remove it when the drive goes away. This is still being debugged.

But pressed up in case anyone wishes to play.
 
In case anyone wishes to play along.

...
I also now have code in that detects when the drive is plugged in and adds it to the list, which appears to be working,
but I also added code to try to remove it when the drive goes away. This is still being debugged.
I put in a fix for now... Which is during the reset when the host asks for Storage IDS, I check to see if the name has been removed and if so it skips that slot...
So now when I remove the drive it updates to not show that drive.
 
Back
Top