MTP Responder Contribution

@KurtE
Out of curiosity I set up MTP-test and LFS to print debug messages including Trace messages. So on the 64MB FLASH this is what I am seeing when I try transferring the 8.5MB workshop.pdf right before and after the timeout. Maybe it will help in figuring out what's going on. Have to go to Dr now so can't spend anymore time till I come home:
Code:
CMD: 100d(SEND_OBJECT)l: 12 T:21
MTPD::SendObject: len:8894873
F:\arduino-1.8.13-beta6\hardware\teensy\avr\libraries\LittleFS\src\littlefs\lfs.c:2875:trace: lfs_file_write(200064C8, 20204780, 200044A0, 8192)
F:\arduino-1.8.13-beta6\hardware\teensy\avr\libraries\LittleFS\src\littlefs\lfs.c:2999:trace: lfs_file_write -> 8192
F:\arduino-1.8.13-beta6\hardware\teensy\avr\libraries\LittleFS\src\littlefs\lfs.c:2875:trace: lfs_file_write(200064C8, 20204780, 200044A0, 8192)
F:\arduino-1.8.13-beta6\hardware\teensy\avr\libraries\LittleFS\src\littlefs\lfs.c:2999:trace: lfs_file_write -> 8192
F:\arduino-1.8.13-beta6\hardware\teensy\avr\libraries\LittleFS\src\littlefs\lfs.c:2875:trace: lfs_file_write(200064C8, 20204780, 200044A0, 8192)
F:\arduino-1.8.13-beta6\hardware\teensy\avr\libraries\LittleFS\src\littlefs\lfs.c:2999:trace: lfs_file_write -> 8192
F:\arduino-1.8.13-beta6\hardware\teensy\avr\libraries\LittleFS\src\littlefs\lfs.c:2875:trace: lfs_file_write(200064C8, 20204780, 200044A0, 8192)
F:\arduino-1.8.13-beta6\hardware\teensy\avr\libraries\LittleFS\src\littlefs\lfs.c:2999:trace: lfs_file_write -> 8192
F:\arduino-1.8.13-beta6\hardware\teensy\avr\libraries\LittleFS\src\littlefs\lfs.c:2875:trace: lfs_file_write(200064C8, 20204780, 200044A0, 8192)
F:\arduino-1.8.13-beta6\hardware\teensy\avr\libraries\LittleFS\src\littlefs\lfs.c:2999:trace: lfs_file_write -> 8192
F:\arduino-1.8.13-beta6\hardware\teensy\avr\libraries\LittleFS\src\littlefs\lfs.c:2875:trace: lfs_file_write(200064C8, 20204780, 200044A0, 8192)
F:\arduino-1.8.13-beta6\hardware\teensy\avr\libraries\LittleFS\src\littlefs\lfs.c:2999:trace: lfs_file_write -> 8192
F:\arduino-1.8.13-beta6\hardware\teensy\avr\libraries\LittleFS\src\littlefs\lfs.c:2875:trace: lfs_file_write(200064C8, 20204780, 200044A0, 8192)
F:\arduino-1.8.13-beta6\hardware\teensy\avr\libraries\LittleFS\src\littlefs\lfs.c:2999:trace: lfs_file_write -> 8192
F:\arduino-1.8.13-beta6\hardware\teensy\avr\libraries\LittleFS\src\littlefs\lfs.c:2875:trace: lfs_file_write(200064C8, 20204780, 200044A0, 8192)
F:\arduino-1.8.13-beta6\hardware\teensy\avr\libraries\LittleFS\src\littlefs\lfs.c:2999:trace: lfs_file_write -> 8192
F:\arduino-1.8.13-beta6\hardware\teensy\avr\libraries\LittleFS\src\littlefs\lfs.c:2875:trace: lfs_file_write(200064C8, 20204780, 200044A0, 8192)
F:\arduino-1.8.13-beta6\hardware\teensy\avr\libraries\LittleFS\src\littlefs\lfs.c:2999:trace: lfs_file_write -> 8192
F:\arduino-1.8.13-beta6\hardware\teensy\avr\libraries\LittleFS\src\littlefs\lfs.c:2875:trace: lfs_file_write(200064C8, 20204780, 200044A0, 8192)
F:\arduino-1.8.13-beta6\hardware\teensy\avr\libraries\LittleFS\src\littlefs\lfs.c:2999:trace: lfs_file_write -> 8192
F:\arduino-1.8.13-beta6\hardware\teensy\avr\libraries\LittleFS\src\littlefs\lfs.c:2875:trace: lfs_file_write(200064C8, 20204780, 200044A0, 8192)
F:\arduino-1.8.13-beta6\hardware\teensy\avr\libraries\LittleFS\src\littlefs\lfs.c:2999:trace: lfs_file_write -> 8192
F:\arduino-1.8.13-beta6\hardware\teensy\avr\libraries\LittleFS\src\littlefs\lfs.c:2875:trace: lfs_file_write(200064C8, 20204780, 200044A0, 8192)
F:\arduino-1.8.13-beta6\hardware\teensy\avr\libraries\LittleFS\src\littlefs\lfs.c:2999:trace: lfs_file_write -> 8192
F:\arduino-1.8.13-beta6\hardware\teensy\avr\libraries\LittleFS\src\littlefs\lfs.c:2875:trace: lfs_file_write(200064C8, 20204780, 200044A0, 8192)
F:\arduino-1.8.13-beta6\hardware\teensy\avr\libraries\LittleFS\src\littlefs\lfs.c:2999:trace: lfs_file_write -> 8192
F:\arduino-1.8.13-beta6\hardware\teensy\avr\libraries\LittleFS\src\littlefs\lfs.c:2875:trace: lfs_file_write(200064C8, 20204780, 200044A0, 8192)
F:\arduino-1.8.13-beta6\hardware\teensy\avr\libraries\LittleFS\src\littlefs\lfs.c:2999:trace: lfs_file_write -> 8192
F:\arduino-1.8.13-beta6\hardware\teensy\avr\libraries\LittleFS\src\littlefs\lfs.c:2875:trace: lfs_file_write(200064C8, 20204780, 200044A0, 8192)
F:\arduino-1.8.13-beta6\hardware\teensy\avr\libraries\LittleFS\src\littlefs\lfs.c:2999:trace: lfs_file_write -> 8192
F:\arduino-1.8.13-beta6\hardware\teensy\avr\libraries\LittleFS\src\littlefs\lfs.c:2875:trace: lfs_file_write(200064C8, 20204780, 200044A0, 8192)
F:\arduino-1.8.13-beta6\hardware\teensy\avr\libraries\LittleFS\src\littlefs\lfs.c:2999:trace: lfs_file_write -> 8192
F:\arduino-1.8.13-beta6\hardware\teensy\avr\libraries\LittleFS\src\littlefs\lfs.c:2875:trace: lfs_file_write(200064C8, 20204780, 200044A0, 8192)
F:\arduino-1.8.13-beta6\hardware\teensy\avr\libraries\LittleFS\src\littlefs\lfs.c:2999:trace: lfs_file_write -> 8192
F:\arduino-1.8.13-beta6\hardware\teensy\avr\libraries\LittleFS\src\littlefs\lfs.c:2875:trace: lfs_file_write(200064C8, 20204780, 200044A0, 8192)
F:\arduino-1.8.13-beta6\hardware\teensy\avr\libraries\LittleFS\src\littlefs\lfs.c:2999:trace: lfs_file_write -> 8192
F:\arduino-1.8.13-beta6\hardware\teensy\avr\libraries\LittleFS\src\littlefs\lfs.c:2875:trace: lfs_file_write(200064C8, 20204780, 200044A0, 8192)
F:\arduino-1.8.13-beta6\hardware\teensy\avr\libraries\LittleFS\src\littlefs\lfs.c:2999:trace: lfs_file_write -> 8192
F:\arduino-1.8.13-beta6\hardware\teensy\avr\libraries\LittleFS\src\littlefs\lfs.c:2875:trace: lfs_file_write(200064C8, 20204780, 200044A0, 8192)
F:\arduino-1.8.13-beta6\hardware\teensy\avr\libraries\LittleFS\src\littlefs\lfs.c:2999:trace: lfs_file_write -> 8192
F:\arduino-1.8.13-beta6\hardware\teensy\avr\libraries\LittleFS\src\littlefs\lfs.c:2875:trace: lfs_file_write(200064C8, 20204780, 200044A0, 8192)
F:\arduino-1.8.13-beta6\hardware\teensy\avr\libraries\LittleFS\src\littlefs\lfs.c:2999:trace: lfs_file_write -> 8192
F:\arduino-1.8.13-beta6\hardware\teensy\avr\libraries\LittleFS\src\littlefs\lfs.c:2875:trace: lfs_file_write(200064C8, 20204780, 200044A0, 8192)
F:\arduino-1.8.13-beta6\hardware\teensy\avr\libraries\LittleFS\src\littlefs\lfs.c:2999:trace: lfs_file_write -> 8192
F:\arduino-1.8.13-beta6\hardware\teensy\avr\libraries\LittleFS\src\littlefs\lfs.c:2875:trace: lfs_file_write(200064C8, 20204780, 200044A0, 8192)
F:\arduino-1.8.13-beta6\hardware\teensy\avr\libraries\LittleFS\src\littlefs\lfs.c:2999:trace: lfs_file_write -> 8192
F:\arduino-1.8.13-beta6\hardware\teensy\avr\libraries\LittleFS\src\littlefs\lfs.c:2875:trace: lfs_file_write(200064C8, 20204780, 200044A0, 8192)
F:\arduino-1.8.13-beta6\hardware\teensy\avr\libraries\LittleFS\src\littlefs\lfs.c:2999:trace: lfs_file_write -> 8192
F:\arduino-1.8.13-beta6\hardware\teensy\avr\libraries\LittleFS\src\littlefs\lfs.c:2875:trace: lfs_file_write(200064C8, 20204780, 200044A0, 8192)
F:\arduino-1.8.13-beta6\hardware\teensy\avr\libraries\LittleFS\src\littlefs\lfs.c:2999:trace: lfs_file_write -> 8192
F:\arduino-1.8.13-beta6\hardware\teensy\avr\libraries\LittleFS\src\littlefs\lfs.c:2875:trace: lfs_file_write(200064C8, 20204780, 200044A0, 8192)
F:\arduino-1.8.13-beta6\hardware\teensy\avr\libraries\LittleFS\src\littlefs\lfs.c:2999:trace: lfs_file_write -> 8192
F:\arduino-1.8.13-beta6\hardware\teensy\avr\libraries\LittleFS\src\littlefs\lfs.c:2875:trace: lfs_file_write(200064C8, 20204780, 200044A0, 8192)
F:\arduino-1.8.13-beta6\hardware\teensy\avr\libraries\LittleFS\src\littlefs\lfs.c:2999:trace: lfs_file_write -> 8192
MTPD::SendObject *** USB Read Timeout ***F:\arduino-1.8.13-beta6\hardware\teensy\avr\libraries\LittleFS\src\littlefs\lfs.c:2875:trace: lfs_file_write(200064C8, 20204780, 200044A0, 2548)
F:\arduino-1.8.13-beta6\hardware\teensy\avr\libraries\LittleFS\src\littlefs\lfs.c:2999:trace: lfs_file_write -> 2548
F:\arduino-1.8.13-beta6\hardware\teensy\avr\libraries\LittleFS\src\littlefs\lfs.c:3123:trace: lfs_file_size(200064C8, 20204780)
F:\arduino-1.8.13-beta6\hardware\teensy\avr\libraries\LittleFS\src\littlefs\lfs.c:3128:trace: lfs_file_size -> 223732
F:\arduino-1.8.13-beta6\hardware\teensy\avr\libraries\LittleFS\src\littlefs\lfs.c:2549:trace: lfs_file_close(200064C8, 20204780)
F:\arduino-1.8.13-beta6\hardware\teensy\avr\libraries\LittleFS\src\littlefs\lfs.c:2736:trace: lfs_file_sync(200064C8, 20204780)
F:\arduino-1.8.13-beta6\hardware\teensy\avr\libraries\LittleFS\src\littlefs\lfs.c:2788:trace: lfs_file_sync -> 0
F:\arduino-1.8.13-beta6\hardware\teensy\avr\libraries\LittleFS\src\littlefs\lfs.c:2568:trace: lfs_file_close -> 0
F:\arduino-1.8.13-beta6\hardware\teensy\avr\libraries\LittleFS\src\littlefs\lfs.c:3006:trace: lfs_file_seek(200064C8, 20204498, 816, 0)
F:\arduino-1.8.13-beta6\hardware\teensy\avr\libraries\LittleFS\src\littlefs\lfs.c:3034:trace: lfs_file_seek -> 816
F:\arduino-1.8.13-beta6\hardware\teensy\avr\libraries\LittleFS\src\littlefs\lfs.c:2795:trace: lfs_file_read(200064C8, 20204498, 2005FE08, 272)
F:\arduino-1.8.13-beta6\hardware\teensy\avr\libraries\LittleFS\src\littlefs\lfs.c:2868:trace: lfs_file_read -> 272
F:\arduino-1.8.13-beta6\hardware\teensy\avr\libraries\LittleFS\src\littlefs\lfs.c:3006:trace: lfs_file_seek(200064C8, 20204498, 816, 0)
F:\arduino-1.8.13-beta6\hardware\teensy\avr\libraries\LittleFS\src\littlefs\lfs.c:3034:trace: lfs_file_seek -> 816
F:\arduino-1.8.13-beta6\hardware\teensy\avr\libraries\LittleFS\src\littlefs\lfs.c:2875:trace: lfs_file_write(200064C8, 20204498, 2005FE08, 272)
F:\arduino-1.8.13-beta6\hardware\teensy\avr\libraries\LittleFS\src\littlefs\lfs.c:2999:trace: lfs_file_write -> 272
 # USB Packets: 436 avg ms: 0 max: 1
>>>Total Time: 3661780
RESP:2007(RSP:INCOMPLETE_TRANSFER)l: 12 T:21
 
That p#824 change is the pretty version of what I did.
Just got new ZIP ...

9 devices mounted - removed prop and SD from build:
Code:
MTP_test

sd_addFilesystem: 0 20002fa8 sdio 0
SDIO Storage 0 254 sdio 31963742208 196542464

sd_addFilesystem: 1 20003464 RAM1 0
RAM Storage 0 RAM1 199936 512

sd_addFilesystem: 2 2000352c RAM2 0
RAM Storage 1 RAM2 3999744 4096

sd_addFilesystem: 3 200037e0 PROGM 0
Program Storage 0 PROGM 983040 8192

sd_addFilesystem: 4 200035f4 QSPI 1
QSPI Storage 0 QSPI 16777216 7585792

sd_addFilesystem: 5 20005d20 sflash5 1
SPIFlash Storage 0 5 sflash5 67108864 3035136

sd_addFilesystem: 6 20005df8 sflash6 1
SPIFlash Storage 1 6 sflash6 67108864 258048

sd_addFilesystem: 7 20002dc8 WINBOND1G 1
Storage 0 3 WINBOND1G 131596288 21233664

sd_addFilesystem: 8 20002eb8 WINBOND2G 1
Storage 1 4 WINBOND2G 265289728 32112640

Should we make a better shared test ZIP of files to xfer to test various edge condition handling?
 
@KurtE - @defragster
On a small file write to the 64MB the output looks like this:
Code:
F:\arduino-1.8.13-beta6\hardware\teensy\avr\libraries\LittleFS\src\littlefs\lfs.c:2541:trace: lfs_file_open(200064C8, 20204780, "/PRINTOUTPUTs2.txt", 103)
F:\arduino-1.8.13-beta6\hardware\teensy\avr\libraries\LittleFS\src\littlefs\lfs.c:2390:trace: lfs_file_opencfg(200064C8, 20204780, "/PRINTOUTPUTs2.txt", 103, 20001414 {.buffer=0, .attrs=0, .attr_count=0})
F:\arduino-1.8.13-beta6\hardware\teensy\avr\libraries\LittleFS\src\littlefs\lfs.c:2527:trace: lfs_file_opencfg -> 0
F:\arduino-1.8.13-beta6\hardware\teensy\avr\libraries\LittleFS\src\littlefs\lfs.c:2544:trace: lfs_file_open -> 0
RESP:2001(RSP:OK)l: 24 T:1e : 1 ffffffff 3
CMD: 100d(SEND_OBJECT)l: 12 T:1f
MTPD::SendObject: len:630
F:\arduino-1.8.13-beta6\hardware\teensy\avr\libraries\LittleFS\src\littlefs\lfs.c:2875:trace: lfs_file_write(200064C8, 20204780, 200044A0, 630)
F:\arduino-1.8.13-beta6\hardware\teensy\avr\libraries\LittleFS\src\littlefs\lfs.c:2999:trace: lfs_file_write -> 630
F:\arduino-1.8.13-beta6\hardware\teensy\avr\libraries\LittleFS\src\littlefs\lfs.c:3123:trace: lfs_file_size(200064C8, 20204780)
F:\arduino-1.8.13-beta6\hardware\teensy\avr\libraries\LittleFS\src\littlefs\lfs.c:3128:trace: lfs_file_size -> 630
F:\arduino-1.8.13-beta6\hardware\teensy\avr\libraries\LittleFS\src\littlefs\lfs.c:2549:trace: lfs_file_close(200064C8, 20204780)
F:\arduino-1.8.13-beta6\hardware\teensy\avr\libraries\LittleFS\src\littlefs\lfs.c:2736:trace: lfs_file_sync(200064C8, 20204780)
F:\arduino-1.8.13-beta6\hardware\teensy\avr\libraries\LittleFS\src\littlefs\lfs.c:2788:trace: lfs_file_sync -> 0
F:\arduino-1.8.13-beta6\hardware\teensy\avr\libraries\LittleFS\src\littlefs\lfs.c:2568:trace: lfs_file_close -> 0
F:\arduino-1.8.13-beta6\hardware\teensy\avr\libraries\LittleFS\src\littlefs\lfs.c:3006:trace: lfs_file_seek(200064C8, 20204498, 816, 0)
F:\arduino-1.8.13-beta6\hardware\teensy\avr\libraries\LittleFS\src\littlefs\lfs.c:3034:trace: lfs_file_seek -> 816
F:\arduino-1.8.13-beta6\hardware\teensy\avr\libraries\LittleFS\src\littlefs\lfs.c:2795:trace: lfs_file_read(200064C8, 20204498, 2005FE08, 272)
F:\arduino-1.8.13-beta6\hardware\teensy\avr\libraries\LittleFS\src\littlefs\lfs.c:2868:trace: lfs_file_read -> 272
F:\arduino-1.8.13-beta6\hardware\teensy\avr\libraries\LittleFS\src\littlefs\lfs.c:3006:trace: lfs_file_seek(200064C8, 20204498, 816, 0)
F:\arduino-1.8.13-beta6\hardware\teensy\avr\libraries\LittleFS\src\littlefs\lfs.c:3034:trace: lfs_file_seek -> 816
F:\arduino-1.8.13-beta6\hardware\teensy\avr\libraries\LittleFS\src\littlefs\lfs.c:2875:trace: lfs_file_write(200064C8, 20204498, 2005FE08, 272)
F:\arduino-1.8.13-beta6\hardware\teensy\avr\libraries\LittleFS\src\littlefs\lfs.c:2999:trace: lfs_file_write -> 272
 # USB Packets: 1 avg ms: 1 max: 1
>>>Total Time: 85504
RESP:2001(RSP:OK)l: 12 T:1f
CMD: 9803(GET_OBJECT_PROP_VALUE)l: 20 T:20 : 3 dc02 (FORMAT)
F:\arduino-1.8.13-beta6\hardware\teensy\avr\libraries\LittleFS\src\littlefs\lfs.c:3006:trace: lfs_file_seek(200064C8, 20204498, 816, 0)
F:\arduino-1.8.13-beta6\hardware\teensy\avr\libraries\LittleFS\src\littlefs\lfs.c:3034:trace: lfs_file_seek -> 816
F:\arduino-1.8.13-beta6\hardware\teensy\avr\libraries\LittleFS\src\littlefs\lfs.c:2795:trace: lfs_file_read(200064C8, 20204498, 2005FCF8, 272)
F:\arduino-1.8.13-beta6\hardware\teensy\avr\libraries\LittleFS\src\littlefs\lfs.c:2868:trace: lfs_file_read -> 272
F:\arduino-1.8.13-beta6\hardware\teensy\avr\libraries\LittleFS\src\littlefs\lfs.c:3006:trace: lfs_file_seek(200064C8, 20204498, 816, 0)
F:\arduino-1.8.13-beta6\hardware\teensy\avr\libraries\LittleFS\src\littlefs\lfs.c:3034:trace: lfs_file_seek -> 816
F:\arduino-1.8.13-beta6\hardware\teensy\avr\libraries\LittleFS\src\littlefs\lfs.c:2795:trace: lfs_file_read(200064C8, 20204498, 2005FCF8, 272)
F:\arduino-1.8.13-beta6\hardware\teensy\avr\libraries\LittleFS\src\littlefs\lfs.c:2868:trace: lfs_file_read -> 272
RESP:2001(RSP:OK)l: 20 T:20 : 3 dc02
CMD: 9803(GET_OBJECT_PROP_VALUE)l: 20 T:21 : 3 dc01 (STORAGE_ID)
F:\arduino-1.8.13-beta6\hardware\teensy\avr\libraries\LittleFS\src\littlefs\lfs.c:3006:trace: lfs_file_seek(200064C8, 20204498, 816, 0)
F:\arduino-1.8.13-beta6\hardware\teensy\avr\libraries\LittleFS\src\littlefs\lfs.c:3034:trace: lfs_file_seek -> 816
F:\arduino-1.8.13-beta6\hardware\teensy\avr\libraries\LittleFS\src\littlefs\lfs.c:2795:trace: lfs_file_read(200064C8, 20204498, 2005FCF8, 272)
F:\arduino-1.8.13-beta6\hardware\teensy\avr\libraries\LittleFS\src\littlefs\lfs.c:2868:trace: lfs_file_read -> 272
F:\arduino-1.8.13-beta6\hardware\teensy\avr\libraries\LittleFS\src\littlefs\lfs.c:3006:trace: lfs_file_seek(200064C8, 20204498, 816, 0)
F:\arduino-1.8.13-beta6\hardware\teensy\avr\libraries\LittleFS\src\littlefs\lfs.c:3034:trace: lfs_file_seek -> 816
F:\arduino-1.8.13-beta6\hardware\teensy\avr\libraries\LittleFS\src\littlefs\lfs.c:2795:trace: lfs_file_read(200064C8, 20204498, 2005FCF8, 272)
F:\arduino-1.8.13-beta6\hardware\teensy\avr\libraries\LittleFS\src\littlefs\lfs.c:2868:trace: lfs_file_read -> 272
RESP:2001(RSP:OK)l: 20 T:21 : 3 dc01
CMD: 9803(GET_OBJECT_PROP_VALUE)l: 20 T:22 : 3 dc07 (OBJECT NAME)
F:\arduino-1.8.13-beta6\hardware\teensy\avr\libraries\LittleFS\src\littlefs\lfs.c:3006:trace: lfs_file_seek(200064C8, 20204498, 816, 0)
F:\arduino-1.8.13-beta6\hardware\teensy\avr\libraries\LittleFS\src\littlefs\lfs.c:3034:trace: lfs_file_seek -> 816
F:\arduino-1.8.13-beta6\hardware\teensy\avr\libraries\LittleFS\src\littlefs\lfs.c:2795:trace: lfs_file_read(200064C8, 20204498, 2005FCF8, 272)
F:\arduino-1.8.13-beta6\hardware\teensy\avr\libraries\LittleFS\src\littlefs\lfs.c:2868:trace: lfs_file_read -> 272
F:\arduino-1.8.13-beta6\hardware\teensy\avr\libraries\LittleFS\src\littlefs\lfs.c:3006:trace: lfs_file_seek(200064C8, 20204498, 816, 0)
F:\arduino-1.8.13-beta6\hardware\teensy\avr\libraries\LittleFS\src\littlefs\lfs.c:3034:trace: lfs_file_seek -> 816
F:\arduino-1.8.13-beta6\hardware\teensy\avr\libraries\LittleFS\src\littlefs\lfs.c:2795:trace: lfs_file_read(200064C8, 20204498, 2005FCF8, 272)
F:\arduino-1.8.13-beta6\hardware\teensy\avr\libraries\LittleFS\src\littlefs\lfs.c:2868:trace: lfs_file_read -> 272
RESP:2001(RSP:OK)l: 20 T:22 : 3 dc07
CMD: 9802(GET_OBJECT_PROP_DESC)l: 20 T:23 : dc0b 3000 (PARENT)
RESP:2001(RSP:OK)l: 20 T:23 : dc0b 3000
CMD: 9803(GET_OBJECT_PROP_VALUE)l: 20 T:24 : 3 dc0b (PARENT)
F:\arduino-1.8.13-beta6\hardware\teensy\avr\libraries\LittleFS\src\littlefs\lfs.c:3006:trace: lfs_file_seek(200064C8, 20204498, 816, 0)
F:\arduino-1.8.13-beta6\hardware\teensy\avr\libraries\LittleFS\src\littlefs\lfs.c:3034:trace: lfs_file_seek -> 816
F:\arduino-1.8.13-beta6\hardware\teensy\avr\libraries\LittleFS\src\littlefs\lfs.c:2795:trace: lfs_file_read(200064C8, 20204498, 2005FCF8, 272)
F:\arduino-1.8.13-beta6\hardware\teensy\avr\libraries\LittleFS\src\littlefs\lfs.c:2868:trace: lfs_file_read -> 272
F:\arduino-1.8.13-beta6\hardware\teensy\avr\libraries\LittleFS\src\littlefs\lfs.c:3006:trace: lfs_file_seek(200064C8, 20204498, 816, 0)
F:\arduino-1.8.13-beta6\hardware\teensy\avr\libraries\LittleFS\src\littlefs\lfs.c:3034:trace: lfs_file_seek -> 816
F:\arduino-1.8.13-beta6\hardware\teensy\avr\libraries\LittleFS\src\littlefs\lfs.c:2795:trace: lfs_file_read(200064C8, 20204498, 2005FCF8, 272)
F:\arduino-1.8.13-beta6\hardware\teensy\avr\libraries\LittleFS\src\littlefs\lfs.c:2868:trace: lfs_file_read -> 272
RESP:2001(RSP:OK)l: 20 T:24 : 3 dc0b
CMD: 9803(GET_OBJECT_PROP_VALUE)l: 20 T:25 : 3 dc41 (PERSISTENT_UID)
F:\arduino-1.8.13-beta6\hardware\teensy\avr\libraries\LittleFS\src\littlefs\lfs.c:3006:trace: lfs_file_seek(200064C8, 20204498, 816, 0)
F:\arduino-1.8.13-beta6\hardware\teensy\avr\libraries\LittleFS\src\littlefs\lfs.c:3034:trace: lfs_file_seek -> 816
F:\arduino-1.8.13-beta6\hardware\teensy\avr\libraries\LittleFS\src\littlefs\lfs.c:2795:trace: lfs_file_read(200064C8, 20204498, 2005FCF8, 272)
F:\arduino-1.8.13-beta6\hardware\teensy\avr\libraries\LittleFS\src\littlefs\lfs.c:2868:trace: lfs_file_read -> 272
F:\arduino-1.8.13-beta6\hardware\teensy\avr\libraries\LittleFS\src\littlefs\lfs.c:3006:trace: lfs_file_seek(200064C8, 20204498, 816, 0)
F:\arduino-1.8.13-beta6\hardware\teensy\avr\libraries\LittleFS\src\littlefs\lfs.c:3034:trace: lfs_file_seek -> 816
F:\arduino-1.8.13-beta6\hardware\teensy\avr\libraries\LittleFS\src\littlefs\lfs.c:2795:trace: lfs_file_read(200064C8, 20204498, 2005FCF8, 272)
F:\arduino-1.8.13-beta6\hardware\teensy\avr\libraries\LittleFS\src\littlefs\lfs.c:2868:trace: lfs_file_read -> 272
RESP:2001(RSP:OK)l: 20 T:25 : 3 dc41
CMD: 9803(GET_OBJECT_PROP_VALUE)l: 20 T:26 : 3 dc44 (NAME)
F:\arduino-1.8.13-beta6\hardware\teensy\avr\libraries\LittleFS\src\littlefs\lfs.c:3006:trace: lfs_file_seek(200064C8, 20204498, 816, 0)
F:\arduino-1.8.13-beta6\hardware\teensy\avr\libraries\LittleFS\src\littlefs\lfs.c:3034:trace: lfs_file_seek -> 816
F:\arduino-1.8.13-beta6\hardware\teensy\avr\libraries\LittleFS\src\littlefs\lfs.c:2795:trace: lfs_file_read(200064C8, 20204498, 2005FCF8, 272)
F:\arduino-1.8.13-beta6\hardware\teensy\avr\libraries\LittleFS\src\littlefs\lfs.c:2868:trace: lfs_file_read -> 272
F:\arduino-1.8.13-beta6\hardware\teensy\avr\libraries\LittleFS\src\littlefs\lfs.c:3006:trace: lfs_file_seek(200064C8, 20204498, 816, 0)
F:\arduino-1.8.13-beta6\hardware\teensy\avr\libraries\LittleFS\src\littlefs\lfs.c:3034:trace: lfs_file_seek -> 816
F:\arduino-1.8.13-beta6\hardware\teensy\avr\libraries\LittleFS\src\littlefs\lfs.c:2795:trace: lfs_file_read(200064C8, 20204498, 2005FCF8, 272)
F:\arduino-1.8.13-beta6\hardware\teensy\avr\libraries\LittleFS\src\littlefs\lfs.c:2868:trace: lfs_file_read -> 272
RESP:2001(RSP:OK)l: 20 T:26 : 3 dc44
CMD: 9803(GET_OBJECT_PROP_VALUE)l: 20 T:27 : 3 dc03 (PROTECTION)
F:\arduino-1.8.13-beta6\hardware\teensy\avr\libraries\LittleFS\src\littlefs\lfs.c:3006:trace: lfs_file_seek(200064C8, 20204498, 816, 0)
F:\arduino-1.8.13-beta6\hardware\teensy\avr\libraries\LittleFS\src\littlefs\lfs.c:3034:trace: lfs_file_seek -> 816
F:\arduino-1.8.13-beta6\hardware\teensy\avr\libraries\LittleFS\src\littlefs\lfs.c:2795:trace: lfs_file_read(200064C8, 20204498, 2005FCF8, 272)
F:\arduino-1.8.13-beta6\hardware\teensy\avr\libraries\LittleFS\src\littlefs\lfs.c:2868:trace: lfs_file_read -> 272
F:\arduino-1.8.13-beta6\hardware\teensy\avr\libraries\LittleFS\src\littlefs\lfs.c:3006:trace: lfs_file_seek(200064C8, 20204498, 816, 0)
F:\arduino-1.8.13-beta6\hardware\teensy\avr\libraries\LittleFS\src\littlefs\lfs.c:3034:trace: lfs_file_seek -> 816
F:\arduino-1.8.13-beta6\hardware\teensy\avr\libraries\LittleFS\src\littlefs\lfs.c:2795:trace: lfs_file_read(200064C8, 20204498, 2005FCF8, 272)
F:\arduino-1.8.13-beta6\hardware\teensy\avr\libraries\LittleFS\src\littlefs\lfs.c:2868:trace: lfs_file_read -> 272
RESP:2001(RSP:OK)l: 20 T:27 : 3 dc03
CMD: 9803(GET_OBJECT_PROP_VALUE)l: 20 T:28 : 3 dc04 (SIZE)
F:\arduino-1.8.13-beta6\hardware\teensy\avr\libraries\LittleFS\src\littlefs\lfs.c:3006:trace: lfs_file_seek(200064C8, 20204498, 816, 0)
F:\arduino-1.8.13-beta6\hardware\teensy\avr\libraries\LittleFS\src\littlefs\lfs.c:3034:trace: lfs_file_seek -> 816
F:\arduino-1.8.13-beta6\hardware\teensy\avr\libraries\LittleFS\src\littlefs\lfs.c:2795:trace: lfs_file_read(200064C8, 20204498, 2005FCF8, 272)
F:\arduino-1.8.13-beta6\hardware\teensy\avr\libraries\LittleFS\src\littlefs\lfs.c:2868:trace: lfs_file_read -> 272
F:\arduino-1.8.13-beta6\hardware\teensy\avr\libraries\LittleFS\src\littlefs\lfs.c:3006:trace: lfs_file_seek(200064C8, 20204498, 816, 0)
F:\arduino-1.8.13-beta6\hardware\teensy\avr\libraries\LittleFS\src\littlefs\lfs.c:3034:trace: lfs_file_seek -> 816
F:\arduino-1.8.13-beta6\hardware\teensy\avr\libraries\LittleFS\src\littlefs\lfs.c:2795:trace: lfs_file_read(200064C8, 20204498, 2005FCF8, 272)
F:\arduino-1.8.13-beta6\hardware\teensy\avr\libraries\LittleFS\src\littlefs\lfs.c:2868:trace: lfs_file_read -> 272
RESP:2001(RSP:OK)l: 20 T:28 : 3 dc04
CMD: 1008(GET_OBJECT_INFO)l: 16 T:29 : 3
F:\arduino-1.8.13-beta6\hardware\teensy\avr\libraries\LittleFS\src\littlefs\lfs.c:3006:trace: lfs_file_seek(200064C8, 20204498, 816, 0)
F:\arduino-1.8.13-beta6\hardware\teensy\avr\libraries\LittleFS\src\littlefs\lfs.c:3034:trace: lfs_file_seek -> 816
F:\arduino-1.8.13-beta6\hardware\teensy\avr\libraries\LittleFS\src\littlefs\lfs.c:2795:trace: lfs_file_read(200064C8, 20204498, 2005FCF0, 272)
F:\arduino-1.8.13-beta6\hardware\teensy\avr\libraries\LittleFS\src\littlefs\lfs.c:2868:trace: lfs_file_read -> 272
F:\arduino-1.8.13-beta6\hardware\teensy\avr\libraries\LittleFS\src\littlefs\lfs.c:3006:trace: lfs_file_seek(200064C8, 20204498, 816, 0)
F:\arduino-1.8.13-beta6\hardware\teensy\avr\libraries\LittleFS\src\littlefs\lfs.c:3034:trace: lfs_file_seek -> 816
F:\arduino-1.8.13-beta6\hardware\teensy\avr\libraries\LittleFS\src\littlefs\lfs.c:2795:trace: lfs_file_read(200064C8, 20204498, 2005FCF0, 272)
F:\arduino-1.8.13-beta6\hardware\teensy\avr\libraries\LittleFS\src\littlefs\lfs.c:2868:trace: lfs_file_read -> 272
RESP:2001(RSP:OK)l: 16 T:29 : 3
CMD: 1005(GET_STORAGE_INFO)l: 16 T:2a : 1
F:\arduino-1.8.13-beta6\hardware\teensy\avr\libraries\LittleFS\src\littlefs\lfs.c:4240:trace: lfs_fs_size(200064C8)
F:\arduino-1.8.13-beta6\hardware\teensy\avr\libraries\LittleFS\src\littlefs\lfs.c:4248:trace: lfs_fs_size -> 0
F:\arduino-1.8.13-beta6\hardware\teensy\avr\libraries\LittleFS\src\littlefs\lfs.c:4240:trace: lfs_fs_size(200064C8)
F:\arduino-1.8.13-beta6\hardware\teensy\avr\libraries\LittleFS\src\littlefs\lfs.c:4248:trace: lfs_fs_size -> 0
RESP:2001(RSP:OK)l: 16 T:2a : 1
 
Last edited:
@mjs513 - With the tracing in #826, what may be interesting is more of delta times. I am going to hack up the sendObject to add more
summary info like I did with the outputs like: # USB Packets: 436 avg ms: 0 max: 1

So it sayed we receive 436 usb packets all of them < 2ms in time until the one that timed out at > 500ms...
We can sort of deduct how much time was taken in the file writes by:
>>>Total Time: 3661780
Subtract the timeout: 500000 = 3161780 - maybe 436000 = 2725780
And we did about: 27 write so each write took about .1 seconds....
Which is about my slowest QSPI writes.
 
Indeed 'r' reset does not help restore a missing storage device.

In fact if an explorer refresh is done then NO devices are shown?
 
Ok I update the simple sendObject to also have elapsedMillis around the write calls:
Code:
    bool MTPD::SendObject() 
    { 
      pull_packet(rx_data_buffer);
      read(0,0);
//      printContainer(); 
      uint32_t len = ReadMTPHeader();
      uint32_t index = sizeof(MTPHeader);
      printf("MTPD::SendObject: len:%u\n", len);
      disk_pos=0;
      elapsedMicros em_total = 0;

      uint32_t sum_read_em = 0;
      uint32_t c_read_em = 0;
      uint32_t read_em_max = 0;

      uint32_t sum_write_em = 0;
      uint32_t c_write_em = 0;
      uint32_t write_em_max = 0;

      while((int)len>0)
      { uint32_t bytes = MTP_RX_SIZE - index;                     // how many data in usb-packet
        bytes = min(bytes,len);                                   // loimit at end
        uint32_t to_copy=min(bytes, DISK_BUFFER_SIZE-disk_pos);   // how many data to copy to disk buffer
        memcpy(disk_buffer_+disk_pos, rx_data_buffer + index,to_copy);
        disk_pos += to_copy;
        bytes -= to_copy;
        len -= to_copy;
        //printf("a %d %d %d %d %d\n", len,disk_pos,bytes,index,to_copy);
        //
        if(disk_pos==DISK_BUFFER_SIZE)
        {
          elapsedMillis emWrite = 0;
          if(storage_->write((const char *)disk_buffer_, DISK_BUFFER_SIZE)<DISK_BUFFER_SIZE) return false;
          uint32_t em = emWrite;
          sum_write_em += em;
          c_write_em++;
          if (em > write_em_max) write_em_max = em;
          disk_pos =0;

          if(bytes) // we have still data in transfer buffer, copy to initial disk_buffer_
          {
            memcpy(disk_buffer_,rx_data_buffer+index+to_copy,bytes);
            disk_pos += bytes;
            len -= bytes;
          }
          //printf("b %d %d %d %d %d\n", len,disk_pos,bytes,index,to_copy);
        }
        if(len>0)  // we have still data to be transfered
        { //pull_packet(rx_data_buffer);
          elapsedMillis emRead = 0;
          bool usb_mtp_avail = false; 
          while (!(usb_mtp_avail = usb_mtp_available()) && (emRead < SENDOBJECT_READ_TIMEOUT_MS)) ;
          if (usb_mtp_avail) 
          {
              uint32_t em = emRead;
              sum_read_em += em;
              c_read_em++;
              if (em > read_em_max) read_em_max = em;
              usb_mtp_recv(rx_data_buffer, 60);                  // read directly in.
          }
          else
          { printf("\nMTPD::SendObject *** USB Read Timeout ***\n");
            break;  //
          }
          index=0;
        }
      }
      //printf("len %d\n",disk_pos);
      if(disk_pos)
      {
        elapsedMillis emWrite = 0;
        if(storage_->write((const char *)disk_buffer_, disk_pos)<disk_pos) return false;
        uint32_t em = emWrite;
        sum_write_em += em;
        c_write_em++;
        if (em > write_em_max) write_em_max = em;
        disk_pos =0;
      }
      storage_->close();
      if (c_read_em) printf(" # USB Packets: %u total: %u avg ms: %u max: %u\n", c_read_em, sum_read_em, sum_read_em / c_read_em, read_em_max);
      if (c_write_em) printf(" # Write: %u total:%u avg ms: %u max: %u\n", c_write_em, sum_write_em, sum_write_em / c_write_em, write_em_max);
      printf(">>>Total Time: %u\n", (uint32_t)em_total);

      return (len == 0);
    }
    #endif
And on my propshield I tried sending my 384kb T4.1 card image...
And:
Code:
RESP:2001(RSP:OK)l: 20 T:1e : a dc44
CMD: 100c(SEND_OBJECT_INFO)l: 20 T:1f : 8 ffffffff
DATA:100c(SEND_OBJECT_INFO)l: 176 T:1f : 0 3000 60109 3000 0
SendObjectInfo: 8 4294967295 20003d00: 0 3000 0 60109 3000 0 0 0 568 35c 18 0 0 0 0 : T4.1-Cardlike.jpg
RESP:2001(RSP:OK)l: 24 T:1f : 8 ffffffff b
CMD: 100d(SEND_OBJECT)l: 12 T:20
MTPD::SendObject: len:393481

MTPD::SendObject *** USB Read Timeout ***
 # USB Packets: 436 total: 6 avg ms: 0 max: 1
 # Write: 28 total:3091 avg ms: 110 max: 160
>>>Total Time: 3600747
*MTPD::send_Event(4001)
RESP:2007(RSP:INCOMPLETE_TRANSFER)l: 12 T:20
So yes it is averaging > 100ms per write of 8kb...
 
Ok I update the simple sendObject to also have elapsedMillis around the write calls:
...
So yes it is averaging > 100ms per write of 8kb...

That is slow!

What does LFSIntegrity show on the media? Speedtest with 'g'? Then 'f' and 'g'?

<edit> : My prop mem on CS==6 - not showing up?
 
Last edited:
@KurtE - @defragster
I replaced my sendObject function with what was posted in post #831 but ran it for the 8.5MB workshop.pdf on the 1G NAND and the 64MB flash:
Code:
1G NAND
 # USB Packets: 17372 total: 225 avg ms: 0 max: 1
 # Write: 1086 total:7285 avg ms: 6 max: 37
>>>Total Time: 7593232

64MB FLASH
MTPD::SendObject *** USB Read Timeout ***
 # USB Packets: 436 total: 7 avg ms: 0 max: 1
 # Write: 28 total:3025 avg ms: 108 max: 146
>>>Total Time: 3585300
RESP:2007(RSP:INCOMPLETE_TRANSFER)l: 12 T:47
The 64MB FLASH seems to running at about the same as the 8MB FLASH.

EDIT> Just tried it on the 16MB flash:
Code:
SendObjectInfo: 1 4294967295 20002d60: 0 3000 0 87b999 3000 0 0 0 0 0 0 0 0 0 0 : workshop.pdf
RESP:2001(RSP:OK)l: 24 T:26 : 1 ffffffff 3
CMD: 100d(SEND_OBJECT)l: 12 T:27
MTPD::SendObject: len:8894873
 # USB Packets: 17372 total: 228 avg ms: 0 max: 1
 # Write: 1086 total:65059 avg ms: 59 max: 112
>>>Total Time: 65354412
Just doing this to see where the transfer time would fail, i.e., something like > 60ms.

Question: usb_mtp_recv(rx_data_buffer, 60); // read directly in.
whats the 60 represent - 0nly 60 bytes at a time?
 
Last edited:
Got this 8MB SPI Prop working in LFSIntegrity. Ran to dirty all blocks then repeated after perhaps 'q' then unusedFormat():

Huge varience 5-6X if Formatted or not with 20+ms per block to format (IIRC).

'#' : Iterations:
Code:
:: /3_dir/E_file.txt  SPI_DISK +++ Add [sz 0 add 3072] @KB/sec 31.48 {21.39}  ++ E   Verify /3_dir/E_file.txt 3072B  @KB/sec 287.48 
:: /3_dir/F_file.txt  SPI_DISK +++ Add [sz 0 add 3584] @KB/sec 57.53 {32.86}  ++ F   Verify /3_dir/F_file.txt 3584B  @KB/sec 314.72 
:: /4_dir/G_file.txt  SPI_DISK +++ Add [sz 0 add 4096] @KB/sec 62.08 {37.15}  ++ G   Verify /4_dir/G_file.txt 4096B  @KB/sec 387.99 
:: /4_dir/H_file.txt  SPI_DISK +++ Add [sz 0 add 4608] @KB/sec 38.47 {27.97}  ++ H   Verify /4_dir/H_file.txt 4608B  @KB/sec 393.07 
:: /5_dir/I_file.txt  SPI_DISK +++ Add [sz 0 add 5120] @KB/sec 42.65 {31.03}  ++ I   Verify /5_dir/I_file.txt 5120B  @KB/sec 423.63 

[B]	 formatUnused :: Done Formatting Low Level in 103757609 us.[/B]

:: /2_dir/Y_file.txt  SPI_DISK +++ Add [sz 0 add 13312] @KB/sec 277.34 {158.72}  ++ Y   Verify /2_dir/Y_file.txt 13312B  @KB/sec 574.56 
:: /2_dir/Z_file.txt  SPI_DISK +++ Add [sz 0 add 13824] @KB/sec 278.76 {159.96}  ++ Z   Verify /2_dir/Z_file.txt 13824B  @KB/sec 571.81 
:: /3_dir/A_file.txt  SPI_DISK +++ Add [sz 0 add 6144] @KB/sec 269.45 {102.93}  ++ A   Verify /3_dir/A_file.txt 6144B  @KB/sec 462.58 
:: /3_dir/B_file.txt  SPI_DISK +++ Add [sz 0 add 1536] @KB/sec 228.47 {34.59}  ++ B   Verify /3_dir/B_file.txt 1536B  @KB/sec 210.50

'g' : speedtest:
Code:
>SPI_DISK Disk Stats:Bytes Used: 2060288, Bytes Total:8388608
write speed and latency
speed,max,min,avg
KB/Sec,usec,usec,usec
56.11,56345,4943,36386

[B]	 formatUnused :: Done Formatting Low Level in 51918682 us.[/B]

>SPI_DISK Disk Stats:Bytes Used: 8192, Bytes Total:8388608
write speed and latency
speed,max,min,avg
KB/Sec,usec,usec,usec
348.60,6499,4942,5847

'S' : 2 MB write:
Code:
Big write /1_2MBfile.txt took 30.93 Sec for 2048000 Bytes : file3.size()=2048000
	Big write KBytes per second 66.22 

[B]	 formatUnused :: Done Formatting Low Level in 45243351 us.[/B]

Big write /1_2MBfile.txt took  5.70 Sec for 2048000 Bytes : file3.size()=2048000
	Big write KBytes per second 359.60 

Delete with read verify all #bigfile's
	Verify /0_2MBfile.txt bytes 2048000 : 	Big read&compare KBytes per second 666.86 

	Verify /1_2MBfile.txt bytes 2048000 : 	Big read&compare KBytes per second 667.08
 
@KurtE - @defragster
I replaced my sendObject function with what was posted in post #831 but ran it for the 8.5MB workshop.pdf on the 1G NAND and the 64MB flash:
...

Question: usb_mtp_recv(rx_data_buffer, 60); // read directly in.
whats the 60 represent - 0nly 60 bytes at a time?

I see this in code:
Code:
int usb_mtp_recv(void *buffer, uint32_t [B]timeout[/B])
{
	uint32_t wait_begin_at = [B]systick_millis_count[/B];
	uint32_t tail = rx_tail;
	while (1) {
		if (!usb_configuration) return -1; // usb not enumerated by host
		if (tail != rx_head) break;
		if (systick_millis_count - wait_begin_at > [B]timeout[/B])  {
			return 0;
		}
		yield();
	}

Reports a FAIL if millis longer than 'timeout' - of 60 ms?
 
I see this in code:
Code:
int usb_mtp_recv(void *buffer, uint32_t [B]timeout[/B])
{
	uint32_t wait_begin_at = [B]systick_millis_count[/B];
	uint32_t tail = rx_tail;
	while (1) {
		if (!usb_configuration) return -1; // usb not enumerated by host
		if (tail != rx_head) break;
		if (systick_millis_count - wait_begin_at > [B]timeout[/B])  {
			return 0;
		}
		yield();
	}

Reports a FAIL if millis longer than 'timeout' - of 60 ms?

Darn it - was looking and didn't see it - thanks for posting it. Guess I needed that nap...
 
Yes it fails if > 60ms however this is not called unless we already have a USB buffer. The important part in this code that is failing:
Code:
          elapsedMillis emRead = 0;
          bool usb_mtp_avail = false; 
          while (!(usb_mtp_avail =[COLOR="#FF0000"] usb_mtp_available()) [/COLOR]&& (emRead < SENDOBJECT_READ_TIMEOUT_MS)) ;
          if (usb_mtp_avail) 
          {
              uint32_t em = emRead;
              sum_read_em += em;
              c_read_em++;
              if (em > read_em_max) read_em_max = em;
              usb_mtp_recv(rx_data_buffer, 60);                  // read directly in.
          }
And we only error out with the timeout if the call to usb_mtp_available does not return true within the specified timeout.
 
Yep - was kind of working backwards to see if changing timeouts or buffers made a difference on the 64MB flash.

SENDOBJECT_READ_TIMEOUT_MS = 500: changed to 1000: same thing - timeout still occurs and gets incomplete transfer.

Tried changing the buffer size as well and still, same thing. Need another chip to test with I think.
 
Out of frustration I just tried to send the workshop.pdf file over pdf and it failed there as well.

Code:
SendObjectInfo: 1 4294967295 20002e00: 0 3000 0 87b999 3000 0 0 0 0 0 0 0 0 0 0 : workshop.pdf
RESP:2001(RSP:OK)l: 24 T:16 : 1 ffffffff 2
CMD: 100d(SEND_OBJECT)l: 12 T:17
MTPD::SendObject: len:8894873

MTPD::SendObject *** USB Read Timeout ***
 # USB Packets: 7636 total: 95 avg ms: 0 max: 1
 # Write: 478 total:12107 avg ms: 25 max: 106
>>>Total Time: 12757519
RESP:2007(RSP:INCOMPLETE_TRANSFER)l: 12 T:17
 
I just pushed up a new hack for the format format code (again format of drive only implemented for LittleFS...

typing: f<cr>
will toggle between low level and quick formats.

But currently doing the low level format (example prop shield) will crash MTP as it will time out waiting for me to respond... Will hack it up to return sooner... And then probably send a notify...

But I did a low level format of the Prop Shield and now I can copy large files:
I copied 3 3+MB files to it.

Timing for the last large copy:
Code:
RESP:2001(RSP:OK)l: 24 T:27 : 8 ffffffff c
CMD: 100d(SEND_OBJECT)l: 12 T:28
MTPD::SendObject: len:3355221
 # USB Packets: 6553 total: 104 avg ms: 0 max: 1
 # Write: 410 total:9025 avg ms: 22 max: 59
>>>Total Time: 9141532
RESP:2001(RSP:OK)l: 12 T:28
 
I just pushed up a new hack for the format format code (again format of drive only implemented for LittleFS...

typing: f<cr>
will toggle between low level and quick formats.

But currently doing the low level format (example prop shield) will crash MTP as it will time out waiting for me to respond... Will hack it up to return sooner... And then probably send a notify...

But I did a low level format of the Prop Shield and now I can copy large files:
I copied 3 3+MB files to it.

Timing for the last large copy:
Code:
RESP:2001(RSP:OK)l: 24 T:27 : 8 ffffffff c
CMD: 100d(SEND_OBJECT)l: 12 T:28
MTPD::SendObject: len:3355221
 # USB Packets: 6553 total: 104 avg ms: 0 max: 1
 # Write: 410 total:9025 avg ms: 22 max: 59
>>>Total Time: 9141532
RESP:2001(RSP:OK)l: 12 T:28

WOW. Think @defragster kept telling us to do a lowlevel format. I'm pretty sure I have did that unless something got corrupted.

Just did a low level format on the 64MB flash and that seemed to clear the problem up!
Code:
SendObjectInfo: 1 4294967295 20002e00: 0 3000 0 87b999 3000 0 0 0 0 0 0 0 0 0 0 : workshop.pdf
RESP:2001(RSP:OK)l: 24 T:16 : 1 ffffffff 2
CMD: 100d(SEND_OBJECT)l: 12 T:17
MTPD::SendObject: len:8894873
 # USB Packets: 17372 total: 246 avg ms: 0 max: 1
 # Write: 1086 total:71003 avg ms: 65 max: 99
>>>Total Time: 71285202
 
@defragster said to do 'f' unusedFormat() a time or two :) - that could be added on MOUNT I saw perhaps:
Code:
#if USE_LFS_SPI==1
                            for(int ii=0; ii<nfs_spi;ii++)
                            {
                            if(!spifs[ii].begin(lfs_cs[ii]))
                            { Serial.printf("SPIFlash Storage %d %d %s failed or missing",ii,lfs_cs[ii],lfs_spi_str[ii]); Serial.println();
                            }
                            else
                            {
                              delay(500); // allow USB send
                            if ( Serial.available() ) {
                               // ANY input - or validate a request with read?
                              Serial.printf("SPIFlash Storage %s unusedFormat %ld ms ... please wait ... ",lfs_spi_str[ii], millis());

                              spifs[ii].formatUnused(0,0);
                              Serial.println( millis() );
                              Serial.println( "Restarting ..." );
                              delay(5);
                              SCB_AIRCR = 0x05FA0004;
                            }
                            storage.addFilesystem(spifs[ii], lfs_spi_str[ii], true);
                            uint64_t totalSize = spifs[ii].totalSize();
                            uint64_t usedSize  = spifs[ii].usedSize();
                            Serial.printf("SPIFlash Storage %d %d %s ",ii,lfs_cs[ii],lfs_spi_str[ii]); Serial.print(totalSize); Serial.print(" "); Serial.println(usedSize);
                            }
                            }
#endif
This is ugly with RESET - and added 500ms window to enter a keystroke - not cleared - but it allowed copy of T_4.1.
> It might work without reset if mtpd.loop() were called in a local void yield() ? - but the long format delay at this point breaks connect to PC ... thus the reset.


He also pointed out that 'q'uick and 'F'/LLformat are harmful to the device life as the LFS tracking of wear leveling will get reset and start abusing the same blocks each time.
 
I was just looking at that myself until I got distracted. As usual.

Was thinking about a sketch for Initial format like EraseEverything sketch for SerialFlash. So on initial use you do the formatting (Low Level or formatUnused) and then you can run your MTP-test sketch or your custom sketch. Right now that MTP-test has everything in it except the kitchen sink.

Would be better if @KurtE gets it working in MTP then you can do a format on load.
 
Yes, need to be done before trying to MTP mount the Teensy

Showing troubles with any delay once mount started ...
 
An example of formatUnused() in setup(). Before Serial online if format unneeded or fast it continues startup. Edge case not tested but hopefully safe.

Until the Disk is FULL I can copy all or part of T_4.1 repeating rename prior root folder copy and repeat and some files complete. Then delete the root folders and restart - when the Teensy arrives ( after time of format )

Search code for "bool spifsTrue[nfs_spi];" As coded this only works with : USE_LFS_SPI==1 - tested here on PROP 8MB SPI Flash.

I put an LED on pin #20 to watch it as it runs before Serial online so no output - unless Serial online when done unusedFormat() it then prints and restarts the Teensy.

As noted in code mtpd.send_DeviceResetEvent(); was tried and if format took too long - the PC already given up and this does not reconnect, as seen before.

The user could do this at runtime - as long as no MTP requests came in? - Maybe mtpd.loop() would need to run - so small batch of format : x=0, then repeat x=formatUnused(x,6); until x==0 with needed call in between.

Process is start the LFS on the SPI and check to format and record when found in spifsTrue that is used later to addStorage as normal without doing another begin.

If it takes too long then restart the Teensy - ideally when all formatted this will happen before Serial arrives.

In use this process could work to take the Teensy offline, formatUnused and then restart using a device button? Maybe a better way - if there was a background format that safely formatted non-blocking?

The mtp-test.ino code:
Code:
[B][ATTACH]23676._xfImport[/ATTACH][/B]

<edit> Note this fails on PJRC 4xSpiMemoryboard. The twin 64MB's are found formatted - but it takes too long to do that and it constantly reboots with Serial online ... so needs a better restart mechanism ... or faster start with cahsed LFS_Traverse of used blocks saved on disk.
This shows first time format time - then repeat with no formatting - but a long time confirming that ... each line is followed by a reset - 145.8 secs first formats then 44 seconds to read all 64KB of blocks to confirm not used on mostly empty media.
Code:
Reset @145836
Reset @44051
Reset @44052
Reset @44053
Reset @44052
Reset @44050
Reset @44050
Reset @44051
Reset @44052
Reset @44051
 
Last edited:
@KurtE - @defragster

Just gave @KurtE's formating a try with the 64MB and nothing happened. Then I gave @defragsters a try and it worked on the 64MB chip which was unformatted .

Think the issue is going to be if we have multiple devices say on SPI how we select which nor/flash/fram we want to format or we have a mix of SPI/SD/QSPI? Have to play some more. Maybe a format function with we tell it which store we want to format? Also if i remember correctly if we do a quick or lowlevel format those functions unmount littleFS and the remounts it after formatting. Format Unused doesn't do that. Maybe thats why its crashing.

EDIT:
Maybe something like this. Know the waiting for serial isn't right but it the thought process,
Code:
void deviceFormat(){

    FS *store_fs = store;
	  LittleFS *lfs = nullptr;
	  SDClass *sfs = nullptr;

      // first dump list of storages:
      uint32_t fsCount = storage.getFSCount();
      Serial.printf("\nDump Storage list(%u)\n", fsCount);
      for (uint32_t ii = 0; ii < fsCount; ii++) {
        Serial.printf("store:%u name:%s fs:%x ROYW:%x\n", ii, storage.getStoreName(ii), (uint32_t)storage.getStoreFS(ii), storage.getReadOnYieldWrites(ii));
      }

      // attempt to notify PC on added files (does not work yet)
      Serial.print("Store "); Serial.println(store);
	  
	  int ch1 = -1;
	  
	  if(Serial.available()) {
		ch1 = Serial.parseInt();
		Serial.println(ch);
		if (ch1== 0)
		{	  
			//FS *store_fs = ch1;
			Serial.println("Quick format");
		}
		if (ch1== 1)
		{	  
			//FS *store_fs = ch1;
			Serial.println("lowlevel format");
		}
		if (ch1== 2)
		{	  
			//FS *store_fs = ch1;
			Serial.println("Format Unused");
		}
		if(ch1 == -1) {Serial.println("waiting");
		
		}
	}
}
 
Last edited:
Ok tried this which almost works. Except that it drops through selecting device without waiting for the input. But the second wait for input seems to work:
Code:
void deviceFormat(){
    LittleFS *lfs = nullptr;
    SDClass *sfs = nullptr;

      // first dump list of storages:
      uint32_t fsCount = storage.getFSCount();
      Serial.printf("\nDump Storage list(%u)\n", fsCount);
      for (uint32_t ii = 0; ii < fsCount; ii++) {
        Serial.printf("store:%u name:%s fs:%x ROYW:%x\n", ii, storage.getStoreName(ii), (uint32_t)storage.getStoreFS(ii), storage.getReadOnYieldWrites(ii));
      }
    int ch1 = -1;
    int ch2 = -1;
 
    Serial.println("Select device number.....");
    while(Serial.available() == 0) {}   // Wait for User to Input Data
    ch2 = Serial.parseInt();
    //FS *store_fs = ch2;
    Serial.println(ch2);
    
    Serial.println("Select format level.....");
    while(Serial.available() == 0) {}   // Wait for User to Input Data
    ch1 = Serial.parseInt();
    Serial.println(ch1);
        if (ch1== 0)
        {   
          Serial.println("Quick format");
        }
        if (ch1== 1)
        {   
          //FS *store_fs = ch1;
          Serial.println("lowlevel format");
        }
        if (ch1== 2)
        {   
          //FS *store_fs = ch1;
          Serial.println("Format Unused");
        }
      if(ch1 == -1) Serial.println("waiting");

}
 
Ok this seems to work but just hangs the Teensy:
Code:
void deviceFormat(){
    LittleFS *lfs = nullptr;
    SDClass *sfs = nullptr;

      // first dump list of storages:
      uint32_t fsCount = storage.getFSCount();
      Serial.printf("\nDump Storage list(%u)\n", fsCount);
      for (uint32_t ii = 0; ii < fsCount; ii++) {
        Serial.printf("store:%u name:%s fs:%x ROYW:%x\n", ii, storage.getStoreName(ii), (uint32_t)storage.getStoreFS(ii), storage.getReadOnYieldWrites(ii));
      }
    int ch1 = -1;
    int ch2 = -1;
    int dummy = -1;

    while(Serial.available() == 0) {}   // Wait for User to Input Data
    dummy = Serial.parseInt();
 
    Serial.println("Select device number.....");
    while(Serial.available() == 0) {}   // Wait for User to Input Data
    ch2 = Serial.parseInt();
    FS *store_fs = ch2;
    Serial.println(ch2);

    while(Serial.available() == 0) {}   // Wait for User to Input Data
    ch2 = Serial.parseInt();
    
    Serial.println("Select format level.....");
    while(Serial.available() == 0) {}   // Wait for User to Input Data
    ch1 = Serial.parseInt();
    Serial.println(ch1);
        if (ch1== 0)
        {   
          Serial.println("Quick format");Serial.flush();
          lfs->quickFormat();
        }
        if (ch1== 1)
        {   
          //FS *store_fs = ch1;
          Serial.println("lowlevel format");Serial.flush();
          lfs->lowLevelFormat('.');
        }
        if (ch1== 2)
        {   
          //FS *store_fs = ch1;
          Serial.println("Format Unused");
          Serial.flush();
          lfs->formatUnused(0,0);
        }
      if(ch1 == -1) Serial.println("waiting");

      delay(10);
      SCB_AIRCR = 0x05FA0004; // reset
      Serial.println("Format complete");
      

}
 
unusedFormat() the best option as noted for media life for reasons noted and leaving media mounted.

For that formattedUnused() option it could offer 'delete All' if the user really wants to start fresh - should have added that to LFSIntegrity.

This order would be more telling to user:)
Code:
      Serial.println("Format complete .. restarting");
      delay(10);
      SCB_AIRCR = 0x05FA0004; // reset

Also without taking Teensy offline add something like - that will still fail if this first call takes too long ... but only checks for first unused&unformatted ... I could add a special call for this to assure it returns asap
Code:
Static uint32_t lastUFDone = 0;

// ask option 3 which bypasses the reset and enters loop
        if (ch1== 3)
        {   
          //FS *store_fs = ch1;
          Serial.println("Format Unused");
          Serial.flush();
          lastUFDone = lfs->formatUnused(lastUFDone,1);
// SKIP RESET - go to loop()
        }

loop() {
...
      if ( lastUFDone ) { // this should format piecemeal - until done
    digitalWriteFast(20, HIGH); // debug visualization only
    digitalWriteFast(21, HIGH); // debug visualization only : media not ready for fast use

          lastUFDone = lfs->formatUnused(lastUFDone,5);
    digitalWriteFast(20, LOW); // debug visualization only
      }
      else // system now online and usable
    digitalWriteFast(21, LOW); // uFor() complete :: debug visualization only

...
}

:):):) - Snow Day here! - going back offline

NOTE: I'm sure code for recursive root down 'Delete All' exists? - pointer please?
 
As you mentioned Snow day here in the Pacific NW... I have not fully measured but earlier at least 8 inches or so...


Will take a look at the functions you mentioned.

The recursive delete, I am not sure if there is one that just prunes the items from his storage ID list, there is a recursive one that deletes all files under a directory:
void MTPStorage_SD::removeFile(uint32_t store, char *file)

One could easily orphan out all of the children by simply clearing out the child index of the top storage item... But again not very clean. But than again the ids are not reused within a session. Obviously not hard to walk the list. and simply clear out the data.

Code:
  struct Record 
  { uint32_t parent;
    uint32_t child;  // size stored here for files
    uint32_t sibling;
    uint8_t isdir;
    uint8_t scanned;
    uint16_t store;  // index int physical storage (0 ... num_storages-1)
    char name[MAX_FILENAME_LEN];

    
  };
Again if you look at the method: bool MTPStorage_SD::DeleteObject(uint32_t object)
It marks the entry as deleted by unlinking itself and then setting the name to null..
 
Back
Top