Teensyduino File System Integration, including MTP and MSC

**KurtE** · 05-19-2022, 12:42 AM

The Cache write stuff looks ok: I am doing an output each time we do a write to Cache:
>> cache write: 3 3 2 6144

So wrote cache 3 which was also block three to file

Then: cache write: 2 5 3 10240
So writing out cache block 2 which is physical block 5, and the > block we have output in the past was 3.
So the extend message is letting me know it then should have output block (buffer?) Block 4 first and then it outputs block 5

Hope that makes sense... Maybe I should have put in another debug message for each block that was output to fill in the gaps

Edit: Sorry as I mentioned I left on lots of debug output...
Will probably turn those off soon.

**mjs513** · 05-19-2022, 12:49 AM

Test #2 - moved MTP.begin to after setting up the file systems and had 1 inaccuarate file transfer

Click image for larger version.

Name: Capture.PNG
Views: 3
Size: 49.4 KB
ID: 28419

**defragster** · 05-19-2022, 01:55 AM

Quick NOTE: Repeated w/MakeFiles >> Empty SD card ('W'ipe files), CP_HDD_USB copy to T_4.1_MTP. all files (1_Primes, 't', 'b') copied 2min 9 sec, 'v'erify on T_4.1 GOOD on all 'checked' files. Return SD to PC and compare to HDD source GOOD, ScanDisk Clean (no FOUND.000), CodeCompare HDD to SD GOOD.
> not sure why the prior p#122 "105.txt" got Zero size ...

Originally Posted by KurtE

I am sort of confused here..

You show the old style of the error messages: $$$ Failed to read Index record: 8 bytes Read: 0

The code that shows read failure:

Code:

		uint64_t seek_pos = block_index * sizeof(RecordBlock);
		bool seek_ok = index_.seek(seek_pos, SeekSet);
		if (!seek_ok) {
			MTP_class::PrintStream()->printf(F("$$$ Failed to seek for read Index record: %u addr:%llu Cur: %llu\n"), 
				recordBlocksInfo_[biIndex].block_index, seek_pos, index_.position());
		}
		size_t bytes_read = index_.read((char *)&recordBlocks_[biIndex], sizeof(RecordBlock));
		if (!seek_ok || (bytes_read != sizeof(RecordBlock))) {
			MTP_class::PrintStream()->printf(F("$$$ Failed to read Index record: %u seek:%u bytes Read: %u\n"), block_index, seek_ok, bytes_read);
		}

So your output is not showing the seek: value...

Kurt, those were the only 'unique' lines among the other spew having the $$$ start chars.
Maybe it wasn't current, but if that not an issue - then nothing else showed.

**mjs513** · 05-19-2022, 10:26 AM

Originally Posted by KurtE

The Cache write stuff looks ok: I am doing an output each time we do a write to Cache:
>> cache write: 3 3 2 6144

So wrote cache 3 which was also block three to file

Then: cache write: 2 5 3 10240
So writing out cache block 2 which is physical block 5, and the > block we have output in the past was 3.
So the extend message is letting me know it then should have output block (buffer?) Block 4 first and then it outputs block 5

Hope that makes sense... Maybe I should have put in another debug message for each block that was output to fill in the gaps

Edit: Sorry as I mentioned I left on lots of debug output...
Will probably turn those off soon.

Didn't really think it was an issue, was just documenting what I was seeing

. I did rerun the Makefiles comparison test.

1. Reformatted the SDCard.
2. Ran Makefiles with the 't' and 'v' option - all good
3. Copied files to PC Directory
4. Removed SD Card from T4.1 and put it into the PC
5. Did a code compare between SD Card and Directory on PC:
NO DIFFERENCES FOUND! between sd card and what was copied.

This is using Tim's sketch with just MTP.begin after storage configured.

**KurtE** · 05-19-2022, 12:50 PM

Originally Posted by mjs513

NO DIFFERENCES FOUND! between sd card and what was copied.

This is using Tim's sketch with just MTP.begin after storage configured.

I just pushed up the changes after a little cleanup. That is I did a few things like remove trailing blanks from lines.
Also I set DEBUG back to 0 and then updated the code such that only errors would print out by default

So it won't print the new records... Unless DEBUG....

I did the full copy again of the generated SDCard and it completed with no messages.

**defragster** · 05-19-2022, 04:31 PM

Got update to MTP_Storage.cpp : Not back at it for another couple hours.

reminder: The MakeFiles.ino sketch has the 'v'erify for files it makes for testing on non-removable media.
> should be able to create desired: number of files, and DIR depth, file size of known content
> verify they are created and stored right
> then MTP copied to host for storage
> "W"iped from local T_4.1 storage
> then copied back to T_4.1 storage with MTP
> 'v'erify should show those files as Good or otherwise

Example output on a GOOD and BAD folder:

Code:

        
D0.7 /                                        
2048_390.txt	XXXX	 File name Size 2048 MISMATCH != size of 2017:            removed file data before copy
	  DONE RESULTS BAD HERE  	XXX 'b XX	  MISSING 1 FILE(S)        Removed this file before copy
 2 File Error(s)  
	 with 6 files of Size 12257 Bytes	4.205872 ms
        
D0.8 /                                                                    
DONE RESULTS GOOD HERE 8 Files(s)	 with 8 files of Size 12000 Bytes	5.388092 ms
    
Total 14 files of Size 24257 Bytes	 Verify Errors 2	11.333312 msec
Bytes Used: 2490368, Bytes Total:125062610944

----	Task 'v' complete!	----

Other LFS data can also "C"opy to SD when USE_SDIO_SD commented out, assume this still works. To allow data written to LFS to be pulled off to do COMPARE on PC via SD card.
Custom 'u'ser data set specific to LFS media
Other options:

Code:

Menu Options:
	t - Test Files write 2204
	b - Test Files write BIG
	s - Test Files write small
	u - Test Files write USER edit      change code at: case 'u': // USER
	n - Test ASCII and UTF8 Filenames
	v - Verify Files
	l - List files on media

	R - Restart Teensy
	U - USB reset
	W - Remove ALL media content
	C - Copy LFS media content to SD

**defragster** · 05-20-2022, 08:46 AM

Made edits to github.com/Defragster/T4LockBeta/tree/main/MakeFiles

> added #define && #ifdef options for LFS:

Code:

//#define USE_SDIO_SD
//#define USE_PSRAM
#define TEST_QSPI // Typical NOR FLASH
//#define TEST_QSPI_NAND // NAND Flash

Also tested "C"opy of LFS drive to SD Card to work.
> Filling selected LFS drive will create a new unique Folder on SD card of the DIR/FILE content
{had a too small char string path buffer now fixed}

Put USE of LFS_PROG for mtpindex under #ifdef

**mjs513** · 05-23-2022, 04:48 PM

@wwatson - @KurtE - @defragster
Decided to have a little fun with MTP/MSC/USBHost so I picked up a M.2 SATA 500GB card and used a M.2 SSD adapter to USB:
https://www.amazon.com/gp/product/B0...0?ie=UTF8&th=1 and https://www.amazon.com/gp/product/B0...0?ie=UTF8&th=1

Formatted in PC as GPT and 1exFAT volume. Plugged it into a USB 3 hub and attached that to the T4.1 and guess what it worked:

Code:

Storage Index 3 Name: MSC0-��H8��Gs�$Vą Selected

 Space Used = 932970496
Filesystem Size = 500022902784
Directory
---------
New folder (8)/
  2001/
    calculations.wav                  426300
    completed.wav                     276460
    dangerous_to_remain.wav           372892
    enough_info.wav                   513388
    functional.wav                    237356
    one_moment.wav                    202236
    operational.wav                   772140
    sorry_dave.wav                    791164
    stop.wav                          200844
  Audacity/
    Away_in_a_Manger.mp3              2014737
    Dont_Rain_on_My_Parade.mp3        3944449
    Take_My_Breathe_Away.mp3          5740819
    Welcome_Christmas.mp3             2985790
  Dont Rain on My Parade.mp3          3944449
  FLAC/
    T1_1024.FLA                       9802802
    T1_128.FLA                        11126659
    T1_256.FLA                        10415954
    T1_512.FLA                        10007370
  GIF/
    2001Station.gif                   4155497
    2001_.wav                         8130604
    giphy.gif                         3057429
    Hubble_small.gif                  563816
    ISS_small.gif                     3757080
    jwst_full.gif                     1832311
    jwst_small.gif                    1959082
    martian2.gif                      3168322
    Martian_cart.gif                  2093083
    Odessy.gif                        2025535
    teacup.gif                        57868
  JWST-SideView.jpg                   4810864
  mtpindex.dat                        0
  odd1.mp3                            46888
  odd1.wav                            553004
  PictureViewOptions.ini              57
  propshield_pinout.png               186124
  R.jpg                               30256
  SDTEST1.raw                         8393684
  servonum.jpg                        158936
  Shaya.png                           200265
  testImage.png                       27111
  testImage2.jpg                      54248
  testImage2.png                      23065
  testimg.png                         1089
  Untitled6.jpg                       254222
  zarathustra.mp3                     489461

Transferring that directory did take a while so probably will have to do a few more tests and maybe partition it. But it is possible and it does work!

**KurtE** · 05-23-2022, 04:58 PM

Looks great. I don't have any that size... The biggest one I have tried with are the 256gb.

Thread: Teensyduino File System Integration, including MTP and MSC

Thread Tools

Search Thread

Display

Posting Permissions