Many TLAs: MTP MSC FS SD SDFat LittleFS UsbMSCFat to work with each other 8)

Morning all. Downloaded your updated version and ran it on the TMM.

1. No problem recognizing the hub with the SSD and thumb drive attached.
Code:
MTP_test
SDIO Storage 0 254 sdio failed or missing
SD Storage 1 10 sd1 failed or missing
sd_addFilesystem: 0 20009140 PROGM 200032bc 20009140
Program Storage 0 PROGM 983040 8192

Initializing USB MSC drives...
USB Drive Inserted
UsbBase::mscBegin called 20007640 1 1
    After usbDriveBegin

msc # Partition Table
	part,boot,bgnCHS[3],type,endCHS[3],start,length
exFAT:	1,0,0x20,0x21,0x0,0x7,0xFE,0xFF,0xFF,2048,204800000
FAT32:	2,0,0xFE,0xFF,0xFF,0xC,0xFE,0xFF,0xFF,204802048,29634560
pt_#0:	3,0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0,0
pt_#0:	4,0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0,0
  Try Partiton:1 on MSC Index:0
UsbBase::mscBegin called 20007640 0 1
    After usbDriveBegin
    ** SUCCEEDED **
>> USB partition 1 volume ID: New Volume
sd_addFilesystem: 1 20009208 MSC0-New Volume 200032b4 0
  Try Partiton:2 on MSC Index:1
UsbBase::mscBegin called 20007640 0 2
    After usbDriveBegin
    ** SUCCEEDED **
>> USB partition 2 volume ID: NEW VOLUME
sd_addFilesystem: 2 200096c0 MSC0-NEW VOLUME 200032b4 1
  Try Partiton:3 on MSC Index:2
UsbBase::mscBegin called 20007640 0 3
    After usbDriveBegin
  Try Partiton:4 on MSC Index:2
UsbBase::mscBegin called 20007640 0 4
    After usbDriveBegin
USB Drive Inserted
UsbBase::mscBegin called 20007f40 1 1
    After usbDriveBegin

msc # Partition Table
	part,boot,bgnCHS[3],type,endCHS[3],start,length
exFAT:	1,0,0x20,0x21,0x0,0x7,0xFE,0xFF,0xFF,2048,121145343
pt_#0:	2,0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0,0
pt_#0:	3,0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0,0
pt_#0:	4,0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0,0
  Try Partiton:1 on MSC Index:2
UsbBase::mscBegin called 20007f40 0 1
    After usbDriveBegin
    ** SUCCEEDED **
sd_addFilesystem: 3 20009b78 MSC1-1 200032b4 2
  Try Partiton:2 on MSC Index:3
UsbBase::mscBegin called 20007f40 0 2
    After usbDriveBegin
  Try Partiton:3 on MSC Index:3
UsbBase::mscBegin called 20007f40 0 3
    After usbDriveBegin
  Try Partiton:4 on MSC Index:3
UsbBase::mscBegin called 20007f40 0 4
    After usbDriveBegin
sd_addFilesystem: 4 20004758 sdio 200032c8 0

**** dir of sd[0] ****

Setup done

2. Copied the T4_1 directory to the SSD 2nd volume successfully. Output shows that it read the datetime string but modified dates don't show in WinExp for the SSD:
Code:
}}}}}}}}} MSCMTPCB::usedSizeCB called 200045b0 2 1
MSCMTPCB::usedSizeCB called for Fat32
RESP:2001(RSP:OK)l: 24 T:1a : 30001 ffffffff 6
CMD: 9803(GET_OBJECT_PROP_VALUE)l: 20 T:1b : 6 dc02 (FORMAT)
RESP:2001(RSP:OK)l: 20 T:1b : 6 dc02
CMD: 9803(GET_OBJECT_PROP_VALUE)l: 20 T:1c : 6 dc01 (STORAGE_ID)
RESP:2001(RSP:OK)l: 20 T:1c : 6 dc01
CMD: 9803(GET_OBJECT_PROP_VALUE)l: 20 T:1d : 6 dc07 (OBJECT NAME)
RESP:2001(RSP:OK)l: 20 T:1d : 6 dc07
CMD: 9802(GET_OBJECT_PROP_DESC)l: 20 T:1e : dc0b 3001 (PARENT)
RESP:2001(RSP:OK)l: 20 T:1e : dc0b 3001
CMD: 9803(GET_OBJECT_PROP_VALUE)l: 20 T:1f : 6 dc0b (PARENT)
RESP:2001(RSP:OK)l: 20 T:1f : 6 dc0b
CMD: 9803(GET_OBJECT_PROP_VALUE)l: 20 T:20 : 6 dc41 (PERSISTENT_UID)
RESP:2001(RSP:OK)l: 20 T:20 : 6 dc41
CMD: 9803(GET_OBJECT_PROP_VALUE)l: 20 T:21 : 6 dc44 (NAME)
RESP:2001(RSP:OK)l: 20 T:21 : 6 dc44
CMD: 9803(GET_OBJECT_PROP_VALUE)l: 20 T:22 : 6 dc03 (PROTECTION)
RESP:2001(RSP:OK)l: 20 T:22 : 6 dc03
CMD: 9803(GET_OBJECT_PROP_VALUE)l: 20 T:23 : 6 dc04 (SIZE)
RESP:2001(RSP:OK)l: 20 T:23 : 6 dc04
CMD: 1008(GET_OBJECT_INFO)l: 16 T:24 : 6
RESP:2001(RSP:OK)l: 16 T:24 : 6
CMD: 1007(GET_OBJECT_HANDLES)l: 24 T:25 : 30001 0 6
RESP:2001(RSP:OK)l: 24 T:25 : 30001 0 6
CMD: 100c(SEND_OBJECT_INFO)l: 20 T:26 : 30001 6
DATA:100c(SEND_OBJECT_INFO)l: 164 T:26 : 0 3000 e2e75 3000 0
SendObjectInfo: 196609 6 20005400: 0 3000 0 e2e75 3000 0 0 0 0 0 0 0 0 0 0 : AN12253.pdf
Read DateTime string: 20210407T081045.0
>> date: 5287 2021 4 7
>> time: 4156 8 10 45
Created: 5287 4156
Read DateTime string: 20210209T232116.0
>> date: 5249 2021 2 9
>> time: baa8 23 21 16
Modified: 5249 baa8


}}}}}}}}} MSCMTPCB::usedSizeCB called 200045b0 2 1
MSCMTPCB::usedSizeCB called for Fat32
RESP:2001(RSP:OK)l: 24 T:26 : 30001 6 7
CMD: 100d(SEND_OBJECT)l: 12 T:27
MTPD::SendObject: len:929397
 # USB Packets: 1815 total: 11 avg ms: 0 max: 1
 # Write: 114 total:76 avg ms: 0 max: 1
>>>Total Time: 106699
RESP:2001(RSP:OK)l: 12 T:27
CMD: 9802(GET_OBJECT_PROP_DESC)l: 20 T:28 : dc44 3000 (NAME)
RESP:2001(RSP:OK)l: 20 T:28 : dc44 3000
CMD: 9803(GET_OBJECT_PROP_VALUE)l: 20 T:29 : 7 dc02 (FORMAT)
RESP:2001(RSP:OK)l: 20 T:29 : 7 dc02
CMD: 9802(GET_OBJECT_PROP_DESC)l: 20 T:2a : dc01 3000 (STORAGE_ID)
RESP:2001(RSP:OK)l: 20 T:2a : dc01 3000
CMD: 9803(GET_OBJECT_PROP_VALUE)l: 20 T:2b : 7 dc01 (STORAGE_ID)
RESP:2001(RSP:OK)l: 20 T:2b : 7 dc01
CMD: 9802(GET_OBJECT_PROP_DESC)l: 20 T:2c : dc07 3000 (OBJECT NAME)
RESP:2001(RSP:OK)l: 20 T:2c : dc07 3000
CMD: 9803(GET_OBJECT_PROP_VALUE)l: 20 T:2d : 7 dc07 (OBJECT NAME)
RESP:2001(RSP:OK)l: 20 T:2d : 7 dc07
CMD: 9802(GET_OBJECT_PROP_DESC)l: 20 T:2e : dc0b 3000 (PARENT)
RESP:2001(RSP:OK)l: 20 T:2e : dc0b 3000
CMD: 9803(GET_OBJECT_PROP_VALUE)l: 20 T:2f : 7 dc0b (PARENT)
RESP:2001(RSP:OK)l: 20 T:2f : 7 dc0b
CMD: 9802(GET_OBJECT_PROP_DESC)l: 20 T:30 : dc41 3000 (PERSISTENT_UID)
RESP:2001(RSP:OK)l: 20 T:30 : dc41 3000
CMD: 9803(GET_OBJECT_PROP_VALUE)l: 20 T:31 : 7 dc41 (PERSISTENT_UID)
RESP:2001(RSP:OK)l: 20 T:31 : 7 dc41
CMD: 9803(GET_OBJECT_PROP_VALUE)l: 20 T:32 : 7 dc44 (NAME)
RESP:2001(RSP:OK)l: 20 T:32 : 7 dc44
CMD: 9802(GET_OBJECT_PROP_DESC)l: 20 T:33 : dc03 3000 (PROTECTION)
RESP:2001(RSP:OK)l: 20 T:33 : dc03 3000
CMD: 9803(GET_OBJECT_PROP_VALUE)l: 20 T:34 : 7 dc03 (PROTECTION)
RESP:2001(RSP:OK)l: 20 T:34 : 7 dc03
CMD: 9802(GET_OBJECT_PROP_DESC)l: 20 T:35 : dc04 3000 (SIZE)
RESP:2001(RSP:OK)l: 20 T:35 : dc04 3000
CMD: 9803(GET_OBJECT_PROP_VALUE)l: 20 T:36 : 7 dc04 (SIZE)
RESP:2001(RSP:OK)l: 20 T:36 : 7 dc04
CMD: 1008(GET_OBJECT_INFO)l: 16 T:37 : 7
RESP:2001(RSP:OK)l: 16 T:37 : 7
CMD: 100c(SEND_OBJECT_INFO)l: 20 T:38 : 30001 6
DATA:100c(SEND_OBJECT_INFO)l: 172 T:38 : 0 3000 11a470 3000 0
SendObjectInfo: 196609 6 20005400: 0 3000 0 11a470 3000 0 0 0 0 0 0 0 0 0 0 : aoyue-8032a.pdf
Read DateTime string: 20210407T081046.0
>> date: 5287 2021 4 7
>> time: 4157 8 10 46
Created: 5287 4157
Read DateTime string: 20181103T091507.0
>> date: 4d63 2018 11 3
>> time: 49e3 9 15 7
Modified: 4d63 49e3

3. Deleted the directory that was just created - turned TMM off and the back on and the directory was gone.

Unfortunately didn't have the SD Card attached have to do that next.
 
@All - I have been working on an upper level IF to @Kurte's and @mjs513's hard work on UsbMscFat.

I wanted to:
1) Find and initialize all available Mass Storage devices connected to the Teensy.
2) Create an index to those devices using a logical device number and use a static index variable as the current default device which can be changed with a change drive method.
3) Store information about the current device in a struct array to be used with other methods:
Code:
typedef struct {
	char		name[32];        // Volume name as a logical drive name.
	uint8_t		driveNumber = 0; // Physical drive number.
	uint8_t		ldNumber = 0;    // Logical drive number.
	bool		valid = false;   // If true device is connected and mounted.
	uint8_t		driveType;       // USB, SDHC or SDHX
	uint8_t		fatType = 0;     // FAT32 or ExFat
	char            currentPath[256];    // Current default path spec.
} deviceDecriptorEntry_t;
4) Process relative path specs such as '../' to create an absolute path spec.
5) Process wildcards.
6) Process mounting and un-mounting of devices.

So far I have change drive and change directory functions that are working with 1,2,3,4 and 5. I spent a lot of time on the Internet hunting down examples of processing wildcards and converting relative path spec to absolute path specs. There were a couple of them that I am modeling after:

https://stackoverflow.com/questions/22381647/how-do-you-convert-a-relative-path-to-an-absolute-path-in-c
and:
https://github.com/sparkfun/OpenLog
This last link has a function called 'lsPrint()' that calls 'lsPrintNext() that calls a function that processes wildcards.

I guess what I am saying is I don't want to put a whole lot of useless effort into this if it is not something that would be useful at this time:)

Here is a sample output for what I have so far:
Code:
mscFSTesting

Waiting up to 5 seconds for a USB drives

Initialize USB drive 0...
Initialize USB drive 1...
Initialize USB drive 2...
Initialize SDIO SD card...
Initialize SPI SD card...

Logical Drive Information For Attached Drives
Physical Drive #: 0 | Logical Drive #: 0 | Volume Label:  16GEXFATP1 | valid: 1 | Drive Type: USB
Physical Drive #: 0 | Logical Drive #: 1 | Volume Label:  16GEXFATP2 | valid: 1 | Drive Type: USB
Physical Drive #: 1 | Logical Drive #: 2 | Volume Label:   128GFAT32 | valid: 1 | Drive Type: USB
Physical Drive #: 2 | Logical Drive #: 3 | Volume Label:    32GEXFAT | valid: 1 | Drive Type: USB
Physical Drive #:10 | Logical Drive #: 4 | Volume Label:  32GSDEXFAT | valid: 1 | Drive Type: SDHC/SDXC
Physical Drive #:11 | Logical Drive #: 5 | Volume Label:  TEENSY8GSD | valid: 1 | Drive Type: SDHC/SDXC

Changed To Logical Drive: 2 (128GFAT32)
parsePathSpec(const char *pathSpec) returns:
depth = 1
pathSpec = /
pathOut = /

Changed Logial Drive/Directory To: 128GFAT32/

Directory Listing for Logical Drive 2: 128GFAT32/
Arduino/
A_00001.dat                                                                      32768000
parsePathSpec(const char *pathSpec) returns:
depth = 3
pathSpec = /ChRt/src/teensy4/../arm
pathOut = /ChRt/src/arm

Changed Logial Drive/Directory To: 32GSDEXFAT/ChRt/src/arm

Directory Listing for Logical Drive 4: 32GSDEXFAT/ChRt/src/arm/
chcore.h                                                                         6993
chcore_timer.h                                                                   3737
chcore_v6m.c                                                                     5349
chcore_v6m.h                                                                     14491
chcore_v7m.c                                                                     6448
chcore_v7m.h                                                                     22460
chcoreasm_v6m.S                                                                  4908
chcoreasm_v7m.S                                                                  5453
chtypes.h                                                                        3208
cmparams.h                                                                       504
mpu.h                                                                            9587

Directory Listing for Logical Drive 4: 32GSDEXFAT/ChRt/src/arm/*.h
chcore.h                                                                         6993
chcore_timer.h                                                                   3737
chcore_v6m.h                                                                     14491
chcore_v7m.h                                                                     22460
chtypes.h                                                                        3208
cmparams.h                                                                       504
mpu.h                                                                            9587

Directory Listing for Logical Drive 4: 32GSDEXFAT/ChRt/src/arm/*_???.*
chcore_v6m.c                                                                     5349
chcore_v6m.h                                                                     14491
chcore_v7m.c                                                                     6448
chcore_v7m.h                                                                     22460
chcoreasm_v6m.S                                                                  4908
chcoreasm_v7m.S                                                                  5453

done...

It's still a lot of fun playing with this stuff:)
 
@KurtE - A quick question. I found your routine in MTP_T4 to detect when an SDIO SD card is inserted. Did you ever try to detect insertion of an external SPI SD card? Maybe I missed it which is typical:)

Thanks
 
External SD Card insertions. No I have not done it, but should be easy to do: IF you purchase an SD Card reader that has an IO pin for insertions.
Example: https://www.sparkfun.com/products/544
I believe has IO pin CD, which I think is Chip Detect) I would have to look to see if it makes a circuit or breaks a circuit when chip is inserted...
I did purchase some of these, where are buried somewhere in my office ;)
 
External SD Card insertions. No I have not done it, but should be easy to do: IF you purchase an SD Card reader that has an IO pin for insertions.
Example: https://www.sparkfun.com/products/544
I believe has IO pin CD, which I think is Chip Detect) I would have to look to see if it makes a circuit or breaks a circuit when chip is inserted...
I did purchase some of these, where are buried somewhere in my office ;)

Thanks KurtE - Right now I am using the PJRC SD card adapter (the original one) https://www.pjrc.com/teensy/sd_adaptor.html. I have a few of these that that I am testing with. It has been so long since I looked at the schematic that I forgot about the SW pin.

Back to the testing;)
 
@KurtE @mjs513 @defragster and All - DiskIO for testing unified access to SD, SDIO, LittleFS and MSC. Was not sure where I should post this so I will post it here.

This is a library that I have been working on for a few months now. It is just an experimental library to try out unifying access to all of the different types of MassStorage devices and libraries for this purpose available for the Teensy. I wanted to try ways to access the different types of file systems using a common logical device name such as partition volume names as logical drive names '/volume name/' or logical device index number such as 'nn:'.

Some of this was discussed starting here Post #60:
https://forum.pjrc.com/threads/67588-Teensyduino-1-54-Beta-11/page3

Libraries needed:
DiskIO - https://github.com/wwatson4506/DiskIO
UsbMscFast-FS_DATES - https://github.com/wwatson4506/UsbMscFat/tree/UsbMscFat-FS_DATES

Using PFsLib in UsbMscFat made access to both SdFat and UsbFat easy. LittleFS Was more of a challenge but is working. One road block I ran into with LittleFS was file operations such as open(), close(), read(), write() and others. I had to create function wrappers for these and test for which file system type was being used and call the appropriate function. Directory operations such as mkdir(), rmdir() etc... were easy enough. Right now the only LittleFS drive I have tested is QPINAND. I will add the others later for testing.

I have two examples in the library. 'DiskIOTesting.ino' and 'DiskIOMB.ino'. The first one just runs through various disk IO functions using logical drive (partition names) or logical drive numbers. Examples:
Code:
// Change 'device' to the volume name of one of your drives.
// Or you can specify a logical drive number (partition number)
// followed with a colon before the path name. 32 partitions are allowed. 0-23
// used for PFsVolumes. 24-31 for LittlFS devices.
// Use: 'listAvailableDrives(&Serial)' to list attached available volume labels
// and logical drive numbers.

char *device = "0:test1.txt"; // First logical drive on a USB physical drive.
//char *device = "/16GEXFATP2/test1.txt"; // Second logical drive on a USB physical drive.
//char *device = "/128GFAT32/test1.txt"; // Partition label name
//char *device = "24:test1.txt"; // Logical drive number (in this case QPINAND).

The second example is a simple cli that I modified for testing from this MicroBox library:
http://sebastian-duell.de/en/microbox/index.html

It was written for use with Arduino. I changed and added some commands for use with Teensy and DiskIO. It will work with the serial monitor, TyCommander and works best with a VT100 capable terminal program. It requires both CR and LF be enabled.

Here is some examples of usage.

Help command:
Code:
DiskIOMB

The original version of microBox found here:
 http://sebastian-duell.de/en/microbox/index.html

Initializing, please wait...

root@Teensy:/16GEXFATP1/>help

Available Commands:

clear  - Clear Screen (VT100 terminal Only)
ld     - List available logical drives.
ls     - List files and directories.
cd     - Change logical drives and directories.
mkdir  - Make directory.
rmdir  - Remove directory (must be empty).
rm     - Remove file.
rename - Rename file or directory.
cp     - Copy file (src dest).
cat    - List file (Ascii only).

All commands except clear and ld accept an optional drive spec.
The drive spec can be /volume name/ (forward slashes required)
or a logical drive number 0:-32: (colon after number required).
Examples: cp /QPINAND/test.txt 1:test.txt
          cp test.txt test1.txt
Both cp and rename require a space between arguments.
One space is required between command and argument.
Relative path specs and wilcards are supported.
Example: ls 16:a/b/../*??*.cpp.

root@Teensy:/16GEXFATP1/>

List available drives:
Code:
root@Teensy:/16GEXFATP1/>ld

Found 7 logical drives.

Logical Drive Information For Attached Drives
Physical Drive #: 0 | Logical Drive #:  0 | Volume Label:  16GEXFATP1 | valid: 1 | Drive Type: USB
Physical Drive #: 0 | Logical Drive #:  1 | Volume Label:  16GEXFATP2 | valid: 1 | Drive Type: USB
Physical Drive #: 1 | Logical Drive #:  4 | Volume Label: 128GEXFATP1 | valid: 1 | Drive Type: USB
Physical Drive #: 1 | Logical Drive #:  5 | Volume Label: 128GEXFATP2 | valid: 1 | Drive Type: USB
Physical Drive #: 4 | Logical Drive #: 16 | Volume Label:  TEENSY8GSD | valid: 1 | Drive Type: SDHC/SDXC
Physical Drive #: 5 | Logical Drive #: 20 | Volume Label:  32GSDFAT32 | valid: 1 | Drive Type: SDHC/SDXC
Physical Drive #: 6 | Logical Drive #: 24 | Volume Label:     QPINAND | valid: 1 | Drive Type: LFS
Default Logical Drive: /16GEXFATP1/ (0:)
root@Teensy:/16GEXFATP1/>

If the current USB device is unplugged DiskIO should switch to the next available drive after doing a 'ld' command or using a drive spec with any of the commands. This command refreshes the device table. SD cards are hot plug-able yet. You can test this with 'DiskIOTesting.ino' by pressing enter after plugging or unplugging a USB drive.
********** MAKE SURE THE DEFAULT DRIVE IS NOT THE ONE YOU ARE UN-PLUGGING ***********

List files and directories:
Code:
root@Teensy:/32GEXFATP1/>ls DiskIO
Volume Label: 32GEXFATP1
Full Path: DiskIO
.git/                                          <DIR>
examples/                                      <DIR>
src/                                           <DIR>
.gitignore                                       270           2021-08-08 15:31:04
LICENSE                                         1071           2021-08-08 15:31:04
README.md                                       1737           2021-08-08 15:31:04
library.properties                               140           2021-08-08 15:31:04
root@Teensy:/32GEXFATP1/>ls /QPINAND/
Volume Label: QPINAND
Full Path: /QPINAND/
A/                                             <DIR>
ARMAGED.WAV                                 56862764
Test1.Txt                                         51
procedures.c                                   18394
test1.txt                                         51

Free Space: 40632320 bytes

root@Teensy:/32GEXFATP1/>

File Copy (cp):
Code:
root@Teensy:/16GEXFATP1/>cp /128GEXFATP1/32MEGfile.dat 16:32MEGfile.dat
****************************************************************
************************************
Copied 32768000 bytes in 3.849101 seconds. Speed: 8.513157 MB/s
root@Teensy:/16GEXFATP1/>

There is bound to be a lot of bugs still and this library is just experimental. Next up is adding the rest of the LittleFS supported devices. A lot of work:)

I hope people will play and test with this. Again, this is just experimenting and WIP as proof of concept.
I am open to all comments and suggestions. I am sure there are better ways to do this.
 
Great development @wwatson!

Will see about giving it a go in coming days.

<edit> cloned the git:: DiskIO
For some reason with no changes it wants me to do a pull request to main ???
 
Thanks, will check out the GitHub. I have done a lot of heavy commenting in the source code mostly so I don't forget what I was trying to do but should help users try to understand the mess:)
 
@kurtE @defragster @All - I have finished with experimenting with https://github.com/wwatson4506/DiskIO.

It is functional and I think I got most of the bugs worked out with my smoking bug swatter. Added the ability to play wave files as a test. With what I am seeing in several recent threads on the forum, things are going to start developing and changing.
As @Paul put it and I suspected would eventually happen:
Also to be honest, FS.h is going to become a sort of moving target with version 1.56 and maybe 1.57. More APIs will be added in the near future, requiring updates to SD / SdFat, LittleFS, and other other libraries like Frank's new MEMFILE.

Can't wait to see the changes:)

Edit: Curious to see if using partitions will be added to FS.h. This would allow complete abstraction from all but LittleFS (No partitions?).
 
... should have said weeks ... got distrracted with life and the Secure T4 Beta bootloader testing. Might be good to see this on that now.

Hope the smoke settles on the bug swatter without undo damage for future use :)
 
... should have said weeks ... got distrracted with life and the Secure T4 Beta bootloader testing. Might be good to see this on that now.

Hope the smoke settles on the bug swatter without undo damage for future use :)

I understand Tim. Things seem to be getting a little hectic and it's probably going to get more interesting:) That is why I decided to hold off on any more work with diskIO until the dust settles. Just to verify, diskIO is only one approach to unifying all of the file systems we have. I really hope that it eventually evolves through FS.h. In particular being able to use partitions as logical derives. Correct me if I am wrong LittleFS does not support partitioning so that might be the only exception.

I have pushed up the final version of diskIO and hopefully documented enough to use it. Last editions were playing wave file from all connected drives using a modfied version of FrankB's Teensy-WavePlayer. Curious to see how it works for other users:)

Function seems to be there so now on to improving read and write speeds...

Thanks for responding:)
 
Started up DiskIOTesting on a T_4.0 - then a Locked Beta T_4.0: Both on a TallDog Breeakout that gives access to USB_Host and SDIO SD pads from the bottom.

Note: IDE 1.8.15 and TD 1.55 >> I've updated local libs and CORES to @KurtE's WIP FileSystem dates stuff - and it seems to have caused no problems.

Note the time/date on the test file "2021-01-01 00:00:00" - other than being a bit off it tried.
@KurtE - I saw this testing the other new code too - but didn't see new FS thread to post on yet?

Here's what I see First Production T4 - I plugged back in and hit enter before drive reconnected:
Code:
DiskIO Testing

Initializing, please wait...


Found and mounted 2 logical drives.

Logical Drive Information For Attached Drives
Physical Drive #: 0 | Logical Drive #:  0 | Volume Label:        AD64 | valid: 1 | Drive Type: USB
Physical Drive #: 4 | Logical Drive #: 16 | Volume Label:    xyz8LBR | valid: 1 | Drive Type: SDHC/SDXC
Default Logical Drive: /AD64/ (0:)

Volume Label: AD64
Full Path: /AD64/
System Volume Information/                     <DIR>
examples/                                      <DIR>
extras/                                        <DIR>
src/                                           <DIR>
keywords.txt                                     871           2021-08-04 23:26:06
library.properties                               487           2021-08-04 23:26:06
LICENSE                                         1092           2021-08-04 23:26:06
README.md                                       2828           2021-08-04 23:26:06
test.txt                                         126           2021-01-01 00:00:00
test2.txt                                        295           2021-01-01 00:00:00
i7_11800h_7610_bench.pdf                     1596882           2021-08-19 23:55:48
7610_bench_11800h.txt                            246           2021-08-20 00:00:00

Opening this file for write: '0:test1.txt'
What we wrote: 'This is a test line to test diskIO write() function'
51 bytes written to file.
flushing buff
closing file: 0:test1.txt
Opening this file for read: '0:test1.txt'
Seeking to file position 10
Getting current file position
ftell() returned 10
What we read: 'test line to test diskIO write() function'
bytes read = 41
closing file: 0:test1.txt
Press enter to continue...


********************************************************
Unplug a USB drive then press enter.
Plug the same USB drive in then press enter again.
This shows hot plugging at work. USB drives only so far.
Four USB drives are supported (SD's not yet).
********************************************************


Logical Drive Information For Attached Drives
Physical Drive #: 0 | Logical Drive #:  0 | Volume Label:        AD64 | valid: 1 | Drive Type: USB
Physical Drive #: 4 | Logical Drive #: 16 | Volume Label:    xyz8LBR | valid: 1 | Drive Type: SDHC/SDXC
Default Logical Drive: /AD64/ (0:)
Press enter to continue...


********************************************************
Unplug a USB drive then press enter.
Plug the same USB drive in then press enter again.
This shows hot plugging at work. USB drives only so far.
Four USB drives are supported (SD's not yet).
********************************************************


Logical Drive Information For Attached Drives
Physical Drive #: 4 | Logical Drive #: 16 | Volume Label:    xyz8LBR | valid: 1 | Drive Type: SDHC/SDXC
Default Logical Drive: /xyz8LBR/ (16:)
Press enter to continue...


********************************************************
Unplug a USB drive then press enter.
Plug the same USB drive in then press enter again.
This shows hot plugging at work. USB drives only so far.
Four USB drives are supported (SD's not yet).
********************************************************


Logical Drive Information For Attached Drives
Physical Drive #: 4 | Logical Drive #: 16 | Volume Label:    xyz8LBR | valid: 1 | Drive Type: SDHC/SDXC
Default Logical Drive: /xyz8LBR/ (16:)
Press enter to continue...


********************************************************
Unplug a USB drive then press enter.
Plug the same USB drive in then press enter again.
This shows hot plugging at work. USB drives only so far.
Four USB drives are supported (SD's not yet).
********************************************************


Logical Drive Information For Attached Drives
Physical Drive #: 0 | Logical Drive #:  0 | Volume Label:        AD64 | valid: 1 | Drive Type: USB
Physical Drive #: 4 | Logical Drive #: 16 | Volume Label:    xyz8LBR | valid: 1 | Drive Type: SDHC/SDXC
Default Logical Drive: /xyz8LBR/ (16:)
Press enter to continue...

And the Locked Beta T4:
Code:
DiskIO Testing

Initializing, please wait...


Found and mounted 2 logical drives.

Logical Drive Information For Attached Drives
Physical Drive #: 0 | Logical Drive #:  0 | Volume Label: jjj_BROJ61 | valid: 1 | Drive Type: USB
Physical Drive #: 4 | Logical Drive #: 16 | Volume Label:  SD32GB_F32 | valid: 1 | Drive Type: SDHC/SDXC
Default Logical Drive: /jjj_BROJ61/ (0:)

Volume Label: jjj_BROJ61
Full Path: /jjj_BROJ61/
System Volume Information/                     <DIR>

Opening this file for write: '0:test1.txt'
What we wrote: 'This is a test line to test diskIO write() function'
51 bytes written to file.
flushing buff
closing file: 0:test1.txt
Opening this file for read: '0:test1.txt'
Seeking to file position 10
Getting current file position
ftell() returned 10
What we read: 'test line to test diskIO write() function'
bytes read = 41
closing file: 0:test1.txt
Press enter to continue...


********************************************************
Unplug a USB drive then press enter.
Plug the same USB drive in then press enter again.
This shows hot plugging at work. USB drives only so far.
Four USB drives are supported (SD's not yet).
********************************************************


Logical Drive Information For Attached Drives
Physical Drive #: 0 | Logical Drive #:  0 | Volume Label: jjj_BROJ61 | valid: 1 | Drive Type: USB
Physical Drive #: 4 | Logical Drive #: 16 | Volume Label:  SD32GB_F32 | valid: 1 | Drive Type: SDHC/SDXC
Default Logical Drive: /jjj_BROJ61/ (0:)
Press enter to continue...


********************************************************
Unplug a USB drive then press enter.
Plug the same USB drive in then press enter again.
This shows hot plugging at work. USB drives only so far.
Four USB drives are supported (SD's not yet).
********************************************************


Logical Drive Information For Attached Drives
Physical Drive #: 4 | Logical Drive #: 16 | Volume Label:  SD32GB_F32 | valid: 1 | Drive Type: SDHC/SDXC
Default Logical Drive: /SD32GB_F32/ (16:)
Press enter to continue...


********************************************************
Unplug a USB drive then press enter.
Plug the same USB drive in then press enter again.
This shows hot plugging at work. USB drives only so far.
Four USB drives are supported (SD's not yet).
********************************************************


Logical Drive Information For Attached Drives
Physical Drive #: 0 | Logical Drive #:  0 | Volume Label: jjj_BROJ61 | valid: 1 | Drive Type: USB
Physical Drive #: 4 | Logical Drive #: 16 | Volume Label:  SD32GB_F32 | valid: 1 | Drive Type: SDHC/SDXC
Default Logical Drive: /SD32GB_F32/ (16:)
Press enter to continue...


Then - WHILE POWERED - on the first I plugged the Flash drive into a powered USB HDD and HUB unit:
Code:
Logical Drive Information For Attached Drives
Physical Drive #: 0 | Logical Drive #:  0 | Volume Label:        AD64 | valid: 1 | Drive Type: USB
[B]Physical Drive #: 1 | Logical Drive #:  4 | Volume Label:   120GB_Ext | valid: 1 | Drive Type: USB[/B]
Physical Drive #: 4 | Logical Drive #: 16 | Volume Label:    xyz8LBR | valid: 1 | Drive Type: SDHC/SDXC
Default Logical Drive: /xyz8LBR/ (16:)
 
Not so much luck with DiskIOMB, not getting any Help? And it doesn't like the HDD?
Code:
helpfreeDiskIOMB

The original version of microBox found here:
 http://sebastian-duell.de/en/microbox/index.html

Initializing, please wait...

Type 'help' for a list of commands...


mscError: 0X23
initialization drive 0 failed.
root@Teensy:/120GB_Ext/>helphelpDiskIOMB

The original version of microBox found here:
 http://sebastian-duell.de/en/microbox/index.html

Initializing, please wait...

Type 'help' for a list of commands...

root@Teensy:/AD64/>helpfree
 
Maybe just a startup timing issue:
Code:
DiskIO Testing

Initializing, please wait...


mscError: 0X23
initialization drive 0 failed.

Found and mounted 2 logical drives.

Logical Drive Information For Attached Drives
Physical Drive #: 0 | Logical Drive #:  0 | Volume Label:   120GB_Ext | valid: 1 | Drive Type: USB
Physical Drive #: 4 | Logical Drive #: 16 | Volume Label:  SD32GB_F32 | valid: 1 | Drive Type: SDHC/SDXC
Default Logical Drive: /120GB_Ext/ (0:)

Volume Label: 120GB_Ext
Full Path: /120GB_Ext/
System Volume Information/                     <DIR>
A_00001.dat                                 32768000           2020-01-23 17:04:04
A_00002.dat                                 32768000           2020-01-23 17:04:08
A_00003.dat                                 32768000           2020-01-23 17:04:12
A_00004.dat                                 32768000           2020-01-23 17:04:16
A_00005.dat                                 32768000           2020-01-23 17:04:20
A_00006.dat                                 32768000           2020-01-23 17:04:24
A_00007.dat                                        0           1980-00-00 00:00:00
test.txt                                         342           2021-01-01 00:00:00

Opening this file for write: '0:test1.txt'
What we wrote: 'This is a test line to test diskIO write() function'
51 bytes written to file.
flushing buff
closing file: 0:test1.txt
Opening this file for read: '0:test1.txt'
Seeking to file position 10
Getting current file position
ftell() returned 10
What we read: 'test line to test diskIO write() function'
bytes read = 41
closing file: 0:test1.txt
Press enter to continue...
DiskIO Testing

Initializing, please wait...


Found and mounted 2 logical drives.

Logical Drive Information For Attached Drives
Physical Drive #: 0 | Logical Drive #:  0 | Volume Label:   120GB_Ext | valid: 1 | Drive Type: USB
Physical Drive #: 4 | Logical Drive #: 16 | Volume Label:  SD32GB_F32 | valid: 1 | Drive Type: SDHC/SDXC
Default Logical Drive: /120GB_Ext/ (0:)

Volume Label: 120GB_Ext
Full Path: /120GB_Ext/
System Volume Information/                     <DIR>
A_00001.dat                                 32768000           2020-01-23 17:04:04
A_00002.dat                                 32768000           2020-01-23 17:04:08
A_00003.dat                                 32768000           2020-01-23 17:04:12
A_00004.dat                                 32768000           2020-01-23 17:04:16
A_00005.dat                                 32768000           2020-01-23 17:04:20
A_00006.dat                                 32768000           2020-01-23 17:04:24
A_00007.dat                                        0           1980-00-00 00:00:00
test.txt                                         342           2021-01-01 00:00:00
test1.txt                                         51           2019-01-01 00:44:30

Opening this file for write: '0:test1.txt'
What we wrote: 'This is a test line to test diskIO write() function'
51 bytes written to file.
flushing buff
closing file: 0:test1.txt
Opening this file for read: '0:test1.txt'
Seeking to file position 10
Getting current file position
ftell() returned 10
What we read: 'test line to test diskIO write() function'
bytes read = 41
closing file: 0:test1.txt
Press enter to continue...
 
@defragster - Headed to work right now but will check it out when I get home. I haven't actually tried an HDD drive yet. I know those take long time to come online. May have to tweak the timeouts in 'msc.h' in USBHost_t36/utility folder. About line #72. I tested DiskIOMB with tycommander and minicom. I probably did not mention that you have to turn on 'Add CR' in tycommander or any other terminal program you are using. So it would be LF and CR together. Then you should be able to complete the commnds.
 
@defragster - Came home and started playing with Hard Drives and diskIO. No joy when it comes to using mechanical HDD's. They do work but it takes forever to detect when they are ready to be used. During the initialization process there are two steps. (1) is it connected and (2) is it ready to to be used. The first step needs to be waiting a timeout period for connection to be detected and next waiting a timeout period for it to initialize and be usable. Using a USB to HDD adapter also is involved. This is something I have been fighting with for a while now.

Going on vacation next week so will redo the diskIO initialization process. For now if you wait at least 30 to 60 seconds after power up or resetting and then do the 'ld' command in DiskioMB the HDD should appear in the logical drive list. The 'ld' command along with other commands always checks for the device being connected. If it is not then diskIO will find the next available drive and make it the default drive and the disconnected drive should not appear in the drive list when executing the 'ld' command. Pretty much Hot Plugging.

Using a terminal program with DiskIOMB requires setting it to use a 'LFCR' combination. Two commands require a VT100 capable terminal program, that being the 'clear' and 'play' commands. The 'play' command uses the 'end' key to stop playing the wave file. Unfortunately TyCommander and Arduino's serial monitor are not VT100 compatible so the best experience is using a VT100 capable terminal program such as Putty or minicom.

Anyway, I hope you have time to test this stuff:)
 
I wasn't sure if this 128GB Flash drive was good - it failed the : DiskIO Testing. Got it back to Windows and if was okay showing the following:

Dual128GB.png
>> 14 MB FAT16
>> 115 GB ExFAT

It has two partitions - should that work given their types?

Seems to have been built to see something in Feb 2021,

Retyring this is the test with SD card removed from T_4.0 on TallDog breakout:
Code:
DiskIO Testing

Initializing, please wait...


Check USB drive format.
initialization drive 0 failed.
SDIO card initialization failed.

Check USB drive format.
initialization drive 0 failed.

Found and mounted 0 logical drives.

Logical Drive Information For Attached Drives

Check USB drive format.
initialization drive 0 failed.
Default Logical Drive: // (0:)


Check USB drive format.
initialization drive 0 failed.
Volume Label: 
Full Path: /
lsDir() Failed: /, Code: 1
 
@wwatson
Hope you are enjoying the begining of your vacation.

Just tried running your DiskIOTest sketch with all the new stuff but getting some errors on FatTIme:
Code:
D:\Users\Merli\Documents\Arduino\libraries\DiskIO\src\diskIO.cpp: In function 'tm decode_fattime(uint16_t, uint16_t)':
D:\Users\Merli\Documents\Arduino\libraries\DiskIO\src\diskIO.cpp:42:51: error: return type 'struct tm' is incomplete
 struct tm decode_fattime (uint16_t td, uint16_t tt)
                                                   ^
D:\Users\Merli\Documents\Arduino\libraries\DiskIO\src\diskIO.cpp:44:15: error: aggregate 'tm tx' has incomplete type and cannot be defined
     struct tm tx;
               ^
D:\Users\Merli\Documents\Arduino\libraries\DiskIO\src\diskIO.cpp: In member function 'bool diskIO::lsFiles(void*, char*, bool)':
D:\Users\Merli\Documents\Arduino\libraries\DiskIO\src\diskIO.cpp:930:16: error: variable 'tm tx' has initializer but incomplete type
      struct tm tx = decode_fattime (date, time);
                ^
D:\Users\Merli\Documents\Arduino\libraries\DiskIO\src\play_wav.cpp: In member function 'bool AudioPlayWav::readHeader(int)':
D:\Users\Merli\Documents\Arduino\libraries\DiskIO\src\play_wav.cpp:280:5: warning: 'rd' may be used uninitialized in this function [-Wmaybe-uninitialized]
     if (rd < sizeof(fileHeader)) return false;
 
I wasn't sure if this 128GB Flash drive was good - it failed the : DiskIO Testing. Got it back to Windows and if was okay showing the following:

View attachment 25742
>> 14 MB FAT16
>> 115 GB ExFAT

It has two partitions - should that work given their types?

Seems to have been built to see something in Feb 2021,

Retyring this is the test with SD card removed from T_4.0 on TallDog breakout:
Code:
DiskIO Testing

Initializing, please wait...


Check USB drive format.
initialization drive 0 failed.
SDIO card initialization failed.

Check USB drive format.
initialization drive 0 failed.

Found and mounted 0 logical drives.

Logical Drive Information For Attached Drives

Check USB drive format.
initialization drive 0 failed.
Default Logical Drive: // (0:)


Check USB drive format.
initialization drive 0 failed.
Volume Label: 
Full Path: /
lsDir() Failed: /, Code: 1

Sorry, just got home from work. Both FAT32 and EXFat are supported. diskIO supports up to 4 of these partitions per device using USB, SDIO and external SPI SD. I don't have one of the TallDog 4.0 breakout boards but diskIO worked correctly with the PJRC 4.0 breakout board.

Did you have any luck with the DiskIOMB sketch after adding a CR to LF as per #490?
 
@wwatson
Hope you are enjoying the begining of your vacation.

Just tried running your DiskIOTest sketch with all the new stuff but getting some errors on FatTIme:
Code:
D:\Users\Merli\Documents\Arduino\libraries\DiskIO\src\diskIO.cpp: In function 'tm decode_fattime(uint16_t, uint16_t)':
D:\Users\Merli\Documents\Arduino\libraries\DiskIO\src\diskIO.cpp:42:51: error: return type 'struct tm' is incomplete
 struct tm decode_fattime (uint16_t td, uint16_t tt)
                                                   ^
D:\Users\Merli\Documents\Arduino\libraries\DiskIO\src\diskIO.cpp:44:15: error: aggregate 'tm tx' has incomplete type and cannot be defined
     struct tm tx;
               ^
D:\Users\Merli\Documents\Arduino\libraries\DiskIO\src\diskIO.cpp: In member function 'bool diskIO::lsFiles(void*, char*, bool)':
D:\Users\Merli\Documents\Arduino\libraries\DiskIO\src\diskIO.cpp:930:16: error: variable 'tm tx' has initializer but incomplete type
      struct tm tx = decode_fattime (date, time);
                ^
D:\Users\Merli\Documents\Arduino\libraries\DiskIO\src\play_wav.cpp: In member function 'bool AudioPlayWav::readHeader(int)':
D:\Users\Merli\Documents\Arduino\libraries\DiskIO\src\play_wav.cpp:280:5: warning: 'rd' may be used uninitialized in this function [-Wmaybe-uninitialized]
     if (rd < sizeof(fileHeader)) return false;

Thanks. It actually will not start until Friday after work. Can't wait. As far as the error you are seeing I have not had that error show up... WAIT... Just thought of this. I don't think I have updated to TD1.55B1 yet on any of the three PC's. Yep all three are still 1.54 release. I wonder if that's the difference. I'll update one of them and recompile the program to see if I get that error. Are you using the original FS, SD and UsbMscFat-FS_DATES libraries? Those are the ones I have been using to build diskIO.
 
Just thought of this. I don't think I have updated to TD1.55B1 yet on any of the three PC's. Yep all three are still 1.54 release.

1.55-beta2 is (probably) coming tomorrow, when Arduino releases IDE version 1.8.16. 1.55-beta2 will have the new FS.h changes (separate FileImpl class). Please use that as your starting point for the weekend. ;)
 
Unfortunately no. Have all the updated FS files. I am using the fs-integration branch in my fork your repository. Can’t figure out how to push it to you

Edit: yep watch Paul said.
 
Sorry, just got home from work. Both FAT32 and EXFat are supported. diskIO supports up to 4 of these partitions per device using USB, SDIO and external SPI SD. I don't have one of the TallDog 4.0 breakout boards but diskIO worked correctly with the PJRC 4.0 breakout board.

Did you have any luck with the DiskIOMB sketch after adding a CR to LF as per #490?

For some reason the TINY partition is FAT16 - not supported?

Cleaned that for fresh use to 18GB FAT32 and 100GB NTFS.

Can't build now as the WIP FS updates have broken the DiskIO build it seems?
 
Unfortunately no. Have all the updated FS files. I am using the fs-integration branch in my fork your repository. Can’t figure out how to push it to you

Edit: yep watch Paul said.

Alright guy's. Will hang tough:)

Thanks.
 
Back
Top