Forum Rule: Always post complete source code & details to reproduce any issue!
Page 11 of 33 FirstFirst ... 9 10 11 12 13 21 ... LastLast
Results 251 to 275 of 811

Thread: LittleFS port to Teensy/SPIFlash

  1. #251
    Senior Member+ mjs513's Avatar
    Join Date
    Jul 2014
    Location
    New York
    Posts
    6,125
    Quote Originally Posted by PaulStoffregen View Post
    Oh, yeah, there's probably bugs in openNextFile() with subdirectories. I had to do some kludgy stuff with pathnames. Some not-so-safe string manipulation is also done in there which really needs to be hardened against buffer overflows. Will dig into it later today...
    Just reverted my changes to the ScanDir function to use the version with file_.openNextFile to see what would happen (more as another data point). For RAM and SPI seems to working with a problem to display sub-directories and files in the root directory. However when you use it for QSPI then you run into the issue what we posted.

    Good luck

  2. #252
    Senior Member+ KurtE's Avatar
    Join Date
    Jan 2014
    Posts
    8,217
    Let me know if you need some more testing. If so which versions of libraries to use, like the main one or the WIP branch...

  3. #253
    Senior Member+ mjs513's Avatar
    Join Date
    Jul 2014
    Location
    New York
    Posts
    6,125
    Morning Kurt

    If you want to give it a try I would use the WIP branch. Its now set up so you can have multiple SPIFlash chips and multiple RAMDISKs if you want. Use the mtp_test_combine.ino sketch. The change was to get in line @WMXZ's changes to use multiple SD Cards

    The one thing I was hoping to do was set it up so you could use all 3 storage devices but ran into a problem that I just can't break the code on. Evident when you run the mpt_test_combined1 sketch.

    Thanks for giving it a try.

  4. #254
    Senior Member+ defragster's Avatar
    Join Date
    Feb 2015
    Posts
    13,203
    Quote Originally Posted by PaulStoffregen View Post
    Oh, yeah, there's probably bugs in openNextFile() with subdirectories. I had to do some kludgy stuff with pathnames. Some not-so-safe string manipulation is also done in there which really needs to be hardened against buffer overflows. Will dig into it later today...
    re p#248 p#247 P#246 - not getting good result on Root either - and goes down hill from there adding subdirs

  5. #255
    Senior Member+ defragster's Avatar
    Join Date
    Feb 2015
    Posts
    13,203
    @Paul - In posted sketch just edited to create and cycle over only 4 files on root not 26 @line 32::
    >> #define MAXNUM 4 //26 // ALPHA A-Z is 26, less for fewer files

    dir 'f'ormat for fresh disk.
    did '1' iterations a few times : repro got to #28. "C" just worked to update and "D" failed
    > This indicates that "D" was 31 bytes and another 31 bytes were written, file closed, then VERIFY to read back the 62 bytes that should have been in "D" when "D" failed to open.
    Code:
     QSPI_DISK +++ Add ++++ [was 21 wrote 21] ++ C 	Verify /C_file.txt bytes read 42 and size 42
    	FILE	A_file.txt		101
    	FILE	B_file.txt		22
    	FILE	C_file.txt		42
    	FILE	D_file.txt		31
    
     0 dirs with 4 files of Size 196 Bytes
    
    [  0.91 M] Awaiting input 0123456789rdchkfvpl? loops left 0 >1
    :: /D_file.txt   waited 28390 us
      waited 157 us
     QSPI_DISK +++ Add ++++ [was 31 wrote 31] ++ D 	V	 Fail File open /D_file.txt
    > HIT ENTER and 'BOOM" { these notes from 3rd repro of process }


    I got these SUPER FAST SPEW errors:
    Code:
    ...
    Error: FLEXSPI2_IPRXFSTS=E0C40001
    Error: FLEXSPI2_IPRXFSTS=E0C50001
    Error: FLEXSPI2_IPRXFSTS=E0C60001
    Error: FLEXSPI2_IPRXFSTS=E0C70001
    Error: FLEXSPI2_IPRXFSTS=E0C80001
    Error: FLEXSPI2_IPRXFSTS=E0C90001
    Error: FLEXSPI2_IPRXFSTS=E0CA0001
    Error: FLEXSPI2_IPRXFSTS=E0CB0001
    Error: FLEXSPI2_IPRXFSTS=E0CC0001
    Error: FLEXSPI2_IPRXFSTS=E0CD0001
    Error: FLEXSPI2_IPRXFSTS=E0CE0001
    Error: FLEXSPI2_IPRXFSTS=E0CF0001
    Error: FLEXSPI2_IPRXFSTS=E0D00001
    Error: FLEXSPI2_IPRXFSTS=E0D10001
    Error: FLEXSPI2_IPRXFSTS=E0D20001
    Error: FLEXSPI2_IPRXFSTS=E0D30001
    Error: FLEXSPI2_IPRXFSTS=E0D40001
    Error: FLEXSPI2_IPRXFSTS=E0D50001
    Error: FLEXSPI2_IPRXFSTS=E0D60001
    Error: FLEXSPI2_IPRXFSTS=E0D70001
    Error: FLEXSPI2_IPRXFSTS=E0D80001
    Error: FLEXSPI2_IPRXFSTS=E0D90001
    Error: FLEXSPI2_IPRXFSTS=E0DA0001
    Error: FLEXSPI2_IPRXFSTS=E0DB0001
    Error: FLEXSPI2_IPRXFSTS=E0DC0001
    Error: FLEXSPI2_IPRXFSTS=E0DD0001
    Error: FLEXSPI2_IPRXFSTS=E0DE0001
    Error: FLEXSPI2_IPRXFSTS=E0DF0001
    Error: FLEXSPI2_IPRXFSTS=E0E00001
    Error: FLEXSPI2_IPRXFSTS=E0E10001
    Error: FLEXSPI2_IPRXFSTS=E0E20001
    Error: FLEXSPI2_IPRXFSTS=E0E30001
    Error: FLEXSPI2_IPRXFSTS=E0E40001
    Error: FLEXSPI2_IPRXFSTS=E0E50001
    Error: FLEXSPI2_IPRXFSTS=E0E60001
    Error: FLEXSPI2_IPRXFSTS=E0E70001
    Error: FLEXSPI2_IPRXFSTS=E0E80001
    Error: FLEXSPI2_IPRXFSTS=E0E90001
    Error: FLEXSPI2_IPRXFSTS=E0EA0001
    Error: FLEXSPI2_IPRXFSTS=E0EB0001
    Error: FLEXSPI2_IPRXFSTS=E0EC0001
    Error: FLEXSPI2_IPRXFSTS=E0ED0001
    Error: FLEXSPI2_IPRXFSTS=E0EE0001
    Error: FLEXSPI2_IPRXFSTS=E0EF0001
    Error: FLEXSPI2_IPRXFSTS=E0F00001
    ...
    And similar steps:
    Code:
    ...
    Error: FLEXSPI2_IPRXFSTS=A7240001
    Error: FLEXSPI2_IPRXFSTS=A7250001
    Error: FLEXSPI2_IPRXFSTS=A7260001
    Error: FLEXSPI2_IPRXFSTS=A7270001
    Error: FLEXSPI2_IPRXFSTS=A7280001
    Error: FLEXSPI2_IPRXFSTS=A7290001
    Error: FLEXSPI2_IPRXFSTS=A72A0001
    Error: FLEXSPI2_IPRXFSTS=A72B0001
    Error: FLEXSPI2_IPRXFSTS=A72C0001
    Error: FLEXSPI2_IPRXFSTS=A72D0001
    Error: FLEXSPI2_IPRXFSTS=A72E0001
    Error: FLEXSPI2_IPRXFSTS=A72F0001
    Error: FLEXSPI2_IPRXFSTS=A7300001
    Error: FLEXSPI2_IPRXFSTS=A7310001
    Error: FLEXSPI2_IPRXFSTS=A7320001
    Error: FLEXSPI2_IPRXFSTS=A7330001
    Error: FLEXSPI2_IPRXFSTS=A7340001
    Error: FLEXSPI2_IPRXFSTS=A7350001
    Error: FLEXSPI2_IPRXFSTS=A7360001
    Error: FLEXSPI2_IPRXFSTS=A7370001
    Error: FLEXSPI2_IPRXFSTS=A7380001
    Error: FLEXSPI2_IPRXFSTS=A7390001
    Error: FLEXSPI2_IPRXFSTS=A73A0001
    Error: FLEXSPI2_IPRXFSTS=A73B0001
    Error: FLEXSPI2_IPRXFSTS=A73C0001
    Error: FLEXSPI2_IPRXFSTS=A73D0001
    Error: FLEXSPI2_IPRXFSTS=A73E0001
    Error: FLEXSPI2_IPRXFSTS=A73F0001
    Error: FLEXSPI2_IPRXFSTS=A7400001
    ...
    And from the THIRD repro - so fast the buffer scrolls off:
    Code:
    ...
    Error: FLEXSPI2_IPRXFSTS=971D0001
    Error: FLEXSPI2_IPRXFSTS=971E0001
    Error: FLEXSPI2_IPRXFSTS=971F0001
    Error: FLEXSPI2_IPRXFSTS=97200001
    Error: FLEXSPI2_IPRXFSTS=97210001
    Error: FLEXSPI2_IPRXFSTS=97220001
    Error: FLEXSPI2_IPRXFSTS=97230001
    Error: FLEXSPI2_IPRXFSTS=97240001
    Error: FLEXSPI2_IPRXFSTS=97250001
    Error: FLEXSPI2_IPRXFSTS=97260001
    Error: FLEXSPI2_IPRXFSTS=97270001
    Error: FLEXSPI2_IPRXFSTS=97280001
    Error: FLEXSPI2_IPRXFSTS=97290001
    Error: FLEXSPI2_IPRXFSTS=972A0001
    Error: FLEXSPI2_IPRXFSTS=972B0001
    Error: FLEXSPI2_IPRXFSTS=972C0001
    Error: FLEXSPI2_IPRXFSTS=972D0001
    Error: FLEXSPI2_IPRXFSTS=972E0001
    Error: FLEXSPI2_IPRXFSTS=972F0001
    Error: FLEXSPI2_IPRXFSTS=97300001
    Error: FLEXSPI2_IPRXFSTS=97310001
    Error: FLEXSPI2_IPRXFSTS=97320001
    Error: FLEXSPI2_IPRXFSTS=97330001
    Error: FLEXSPI2_IPRXFSTS=97340001
    Error: FLEXSPI2_IPRXFSTS=97350001
    ...

    Repro above on second T_4.1 QSPI:
    'f'ormat
    '1' iteration 24 times
    Code:
    :: /A_file.txt 	Verify /A_file.txt bytes read 404 and size 404  waited 81 us
     QSPI_DISK ----DEL------ - -- A
    
     0 dirs with 0 files of Size 0 Bytes
    
    :: /B_file.txt   waited 105 us
      waited 96 us
     QSPI_DISK +++ Add ++++ [was 0 wrote 11] ++ B 	V	 Fail File open /B_file.txt
    [  0.63 M] Awaiting input 0123456789rdchkfvpl? loops left 1 >
    >> REPRO not identical - it does get to a 'Fail File open'
    - Prior 'loops left 0' means it was complete the pass - just above it shows 'loops left 1' - meaning if not '0' Zeroed it will continue.
    - this one does not go BOOM
    Last edited by defragster; 11-16-2020 at 06:31 PM. Reason: retry on 2nd Teensy

  6. #256
    Senior Member PaulStoffregen's Avatar
    Join Date
    Nov 2012
    Posts
    23,485
    I've committed a fix for race conditions while reading the FlexSPI IP RX FIFO. Hopefully this will cure the QSPI bugs.

    https://github.com/PaulStoffregen/Li...5f89189dcd3792

    To add another little test program, I noticed the failures would sometimes cause printDirectory to omit printing 1 or more files. So I created this little test which just repeatedly prints the directory, but pauses for 5 seconds (easily visible) if the wrong number of files are found.


    Code:
    // Repeatedly print a list of all files, but pause if wrong number of files
    
    // set this to the number of files previously written to QSPI
    #define NUMFILES  4
    
    #include <LittleFS.h>
    
    LittleFS_QSPIFlash myfs;
    
    void setup() {
      while (!Serial) ;
      Serial.printf("IPRXFCR = %x\n", FLEXSPI2_IPRXFCR);
      if (!myfs.begin()) {
        Serial.println("error startup");
        while (1) ;
      }
    }
    
    unsigned int count = 0;
    unsigned int errcount = 0;
    unsigned int filecount = 0;
    
    void loop() {
      Serial.print("run ");
      Serial.print(++count);
      Serial.print(", errors ");
      Serial.println(errcount);
      
      filecount = 0;
      printDirectory(myfs);
      if (filecount != NUMFILES) {
        errcount++;
        delay(5000);
      }
      delay(1); // run fast, but don't overwhelm serial monitor
    }
    
    
    
    void printDirectory(FS &fs) {
      Serial.println("printDirectory\n--------------");
      printDirectory(fs.open("/"), 0);
      Serial.println();
    }
    
    void printDirectory(File dir, int numTabs) {
      while (true) {
        File entry =  dir.openNextFile();
        if (! entry) {
          // no more files
          break;
        }
        for (uint8_t i = 0; i < numTabs; i++) {
          Serial.print('\t');
        }
        Serial.print(entry.name());
        if (entry.isDirectory()) {
          Serial.println(" / ");
          printDirectory(entry, numTabs + 1);
        } else {
          // files have sizes, directories do not
          Serial.print("\t\t");
          Serial.println(entry.size(), DEC);
          filecount++;
        }
        entry.close();
      }
    }

  7. #257
    Senior Member+ defragster's Avatar
    Join Date
    Feb 2015
    Posts
    13,203
    ... getting new LittleFS code

  8. #258
    Senior Member+ defragster's Avatar
    Join Date
    Feb 2015
    Posts
    13,203
    The '4' root file case just noted is running single passes okay and a couple of 'h'undred too

  9. #259
    Senior Member+ defragster's Avatar
    Join Date
    Feb 2015
    Posts
    13,203
    Went back to 26 Root files and 5 minutes with '1' single and a 3 'h'undred iterations the Directory Integrity on QSPI seems right.

    Code:
      5.09 M] Awaiting input 0123456789rdchkfvpl? loops left 0 >l 
    	 Loop Count: 438 (#fileCycle=4918), Bytes read 1445136, written 455864

  10. #260
    Senior Member+ defragster's Avatar
    Join Date
    Feb 2015
    Posts
    13,203
    Commenting // ROOTONLY

    Fails right away:
    ...

    DISREGARD - After FORMAT the ROOT DIRS are NOT CREATED until RESTARTED ...

  11. #261
    Senior Member+ mjs513's Avatar
    Join Date
    Jul 2014
    Location
    New York
    Posts
    6,125
    Quote Originally Posted by PaulStoffregen View Post
    I've committed a fix for race conditions while reading the FlexSPI IP RX FIFO. Hopefully this will cure the QSPI bugs.

    https://github.com/PaulStoffregen/Li...5f89189dcd3792

    To add another little test program, I noticed the failures would sometimes cause printDirectory to omit printing 1 or more files. So I created this little test which just repeatedly prints the directory, but pauses for 5 seconds (easily visible) if the wrong number of files are found.
    Well not good - let it run 10 times and had 9 errors, here's the dump (using the original T4.1 beta):
    Code:
    IPRXFCR = 0
    QSPI flash begin
    Flash ID: EF 40 18
    Flash size is 16.00 Mbyte
    attempting to mount existing media
    success
    run 1, errors 0
    printDirectory
    --------------
    PRINTOUTPUT1.txt		628
    PRINTOUTPUT2.txt		630
    file1.txt		32
    file10.txt		16
    file2.txt		32
    file20.txt		16
    file3.txt		32
    file30.txt		16
    structuredData / 
    	logger.txt		480
    test1 / 
    	file1.txt		16
    test2 / 
    	file2.txt		16
    test3 / 
    	file3.txt		16
    
    run 2, errors 1
    printDirectory
    --------------
    PRINTOUTPUT1.txt		628
    PRINTOUTPUT2.txt		630
    file1.txt		32
    file10.txt		16
    file2.txt		32
    file20.txt		16
    file3.txt		32
    file30.txt		16
    structuredData / 
    	logger.txt		480
    test1 / 
    	file1.txt		16
    test2 / 
    	file2.txt		16
    test3 / 
    	file3.txt		16
    
    run 3, errors 2
    printDirectory
    --------------
    PRINTOUTPUT1.txt		628
    PRINTOUTPUT2.txt		630
    file1.txt		32
    file10.txt		16
    file2.txt		32
    file20.txt		16
    file3.txt		32
    file30.txt		16
    structuredData / 
    	logger.txt		480
    test1 / 
    	file1.txt		16
    test2 / 
    	file2.txt		16
    test3 / 
    	file3.txt		16
    
    run 4, errors 3
    printDirectory
    --------------
    PRINTOUTPUT1.txt		628
    PRINTOUTPUT2.txt		630
    file1.txt		32
    file10.txt		16
    file2.txt		32
    file20.txt		16
    file3.txt		32
    file30.txt		16
    structuredData / 
    	logger.txt		480
    test1 / 
    	file1.txt		16
    test2 / 
    	file2.txt		16
    test3 / 
    	file3.txt		16
    
    run 5, errors 4
    printDirectory
    --------------
    PRINTOUTPUT1.txt		628
    PRINTOUTPUT2.txt		630
    file1.txt		32
    file10.txt		16
    file2.txt		32
    file20.txt		16
    file3.txt		32
    file30.txt		16
    structuredData / 
    	logger.txt		480
    test1 / 
    	file1.txt		16
    test2 / 
    	file2.txt		16
    test3 / 
    	file3.txt		16
    
    run 6, errors 5
    printDirectory
    --------------
    PRINTOUTPUT1.txt		628
    PRINTOUTPUT2.txt		630
    file1.txt		32
    file10.txt		16
    file2.txt		32
    file20.txt		16
    file3.txt		32
    file30.txt		16
    structuredData / 
    	logger.txt		480
    test1 / 
    	file1.txt		16
    test2 / 
    	file2.txt		16
    test3 / 
    	file3.txt		16
    
    run 7, errors 6
    printDirectory
    --------------
    PRINTOUTPUT1.txt		628
    PRINTOUTPUT2.txt		630
    file1.txt		32
    file10.txt		16
    file2.txt		32
    file20.txt		16
    file3.txt		32
    file30.txt		16
    structuredData / 
    	logger.txt		480
    test1 / 
    	file1.txt		16
    test2 / 
    	file2.txt		16
    test3 / 
    	file3.txt		16
    
    run 8, errors 7
    printDirectory
    --------------
    PRINTOUTPUT1.txt		628
    PRINTOUTPUT2.txt		630
    file1.txt		32
    file10.txt		16
    file2.txt		32
    file20.txt		16
    file3.txt		32
    file30.txt		16
    structuredData / 
    	logger.txt		480
    test1 / 
    	file1.txt		16
    test2 / 
    	file2.txt		16
    test3 / 
    	file3.txt		16
    
    run 9, errors 8
    printDirectory
    --------------
    PRINTOUTPUT1.txt		628
    PRINTOUTPUT2.txt		630
    file1.txt		32
    file10.txt		16
    file2.txt		32
    file20.txt		16
    file3.txt		32
    file30.txt		16
    structuredData / 
    	logger.txt		480
    test1 / 
    	file1.txt		16
    test2 / 
    	file2.txt		16
    test3 / 
    	file3.txt		16
    
    run 10, errors 9
    printDirectory
    --------------
    PRINTOUTPUT1.txt		628
    PRINTOUTPUT2.txt		630
    file1.txt		32
    file10.txt		16
    file2.txt		32
    file20.txt		16
    file3.txt		32
    file30.txt		16
    structuredData / 
    	logger.txt		480
    test1 / 
    	file1.txt		16
    test2 / 
    	file2.txt		16
    test3 / 
    	file3.txt		16
    Unfortunately don't have another board with flash. Have to solder another up just in case.

    EDIT: not sure, but the directly listings all look correct and are the same. Must be missing something.

  12. #262
    Senior Member PaulStoffregen's Avatar
    Join Date
    Nov 2012
    Posts
    23,485
    Quote Originally Posted by mjs513 View Post
    Well not good - let it run 10 times and had 9 errors
    Check that you edited NUMFILES to 12 (since your flash chip appears to have 12 files stored).

  13. #263
    Senior Member+ mjs513's Avatar
    Join Date
    Jul 2014
    Location
    New York
    Posts
    6,125
    @Paul
    I think its working at least for me it seems. I ran my MTPResponder for QSPI and turned power on and off 5 times to see what the directory structure look liked. All 5 times they were the same even the subdirectory files. which matches the printDirectory from your test sketch and what I see on SPIFlash using the test sketch I have to create files and directories.:

    Click image for larger version. 

Name:	Capture.PNG 
Views:	8 
Size:	28.2 KB 
ID:	22486

  14. #264
    Senior Member+ mjs513's Avatar
    Join Date
    Jul 2014
    Location
    New York
    Posts
    6,125
    Quote Originally Posted by PaulStoffregen View Post
    Check that you edited NUMFILES to 12 (since your flash chip appears to have 12 files stored).
    Recounted the files - there are 13. After the change it works without error with matches what i am seeing in MTP as well

    Code:
    run 12319, errors 0
    printDirectory
    --------------
    PRINTOUTPUT1.txt		628
    PRINTOUTPUT2.txt		630
    file1.txt		32
    file10.txt		16
    file2.txt		32
    file20.txt		16
    file3.txt		32
    file30.txt		16
    mtpindex.dat		0
    structuredData / 
    	logger.txt		480
    test1 / 
    	file1.txt		16
    test2 / 
    	file2.txt		16
    test3 / 
    	file3.txt		16

  15. #265
    Senior Member+ defragster's Avatar
    Join Date
    Feb 2015
    Posts
    13,203
    After writing the following ran the p#256 sketch on resulting disk - fails: >> EDIT >> SEE POST #267 ... after reading code ...
    Code:
    run 9, errors 8
    printDirectory
    
    << BOGUS REMOVED >>
    @mjs513 - I suspect your disk FORMAT is bad? That is - already has underlying disk damage?
    Install posts LFSIntegrity:
    > 'f'ormat
    > '1' and '1'
    > 'd'ir
    > Upload Paul's test code

    I'm running on my #2 and it is streaming by just fine:
    Code:
    ...
    run 371547, errors 0
    printDirectory
    --------------
    A_file.txt		101
    B_file.txt		22
    C_file.txt		42
    D_file.txt		31
    ...
    On the #1 here :: Using new github update I have move to !ROOTONLY and it is showing GREAT ...
    > Iterating over 10 directories and the Root with 26 files Add, extend, delete
    > When that is done in another minute I'll post result notes and then start Paul's post #256 sketch there:

    This shows it ran 26 minutes ROOT and 10 DIRS with 26 files coming and going - read 5MB and wrote 1.62MB
    >> NO PROBLEMS - like RAM or SPI it is not running on QSPI
    Code:
    ... // other 10 dirs prior
     0 dirs with 26 files of Size 4624 Bytes
    FILE	A_file.txt		101
    FILE	B_file.txt		22
    FILE	C_file.txt		42
    FILE	D_file.txt		31
    FILE	E_file.txt		41
    FILE	F_file.txt		102
    FILE	G_file.txt		122
    FILE	H_file.txt		71
    FILE	I_file.txt		81
    FILE	J_file.txt		182
    FILE	K_file.txt		202
    FILE	L_file.txt		222
    FILE	M_file.txt		242
    FILE	N_file.txt		262
    FILE	O_file.txt		282
    FILE	P_file.txt		302
    FILE	Q_file.txt		322
    FILE	R_file.txt		342
    FILE	S_file.txt		362
    FILE	T_file.txt		382
    FILE	U_file.txt		402
    FILE	V_file.txt		422
    FILE	W_file.txt		442
    FILE	X_file.txt		462
    FILE	Y_file.txt		482
    FILE	Z_file.txt		251
    
     10 dirs with 26 files of Size 6176 Bytes
     Total 286 files of Size 55248 Bytes
    Bytes Used: 606208, Bytes Total:16777216
    
    [ 23.22 M] Awaiting input 0123456789rdchkfvpl? loops left 0 >
    
    [ 23.22 M] Awaiting input 0123456789rdchkfvpl? loops left 0 >l 
    	 Loop Count: 130 (#fileCycle=17484), Bytes read 5003936, written 1621272
    
    [ 23.26 M] Awaiting input 0123456789rdchkfvpl? loops left 0 >

  16. #266
    Senior Member PaulStoffregen's Avatar
    Join Date
    Nov 2012
    Posts
    23,485
    I'm a little concerned about the error "spew" Defragster posted in msg #255 (just minutes before I committed this race condition fix).

    Those could be related to the RX FIFO race condition, or it could be a completely different bug. Very difficult to know...

  17. #267
    Senior Member+ defragster's Avatar
    Join Date
    Feb 2015
    Posts
    13,203
    JUST READING p#256 CODE - it has hardcoded :: #define NUMFILES 4

    Anything but that is a FAIL - I just happen to have 4 files on the disk when I ran it on #2 - not the case with #1 with unknown number of files.

    Suggest this changed code:
    Code:
    // Repeatedly print a list of all files, but pause if wrong number of files
    
    // set this to the number of files previously written to QSPI
    uint32_t NUMFILES  = 4;
    
    #include <LittleFS.h>
    
    LittleFS_QSPIFlash myfs;
    
    unsigned int filecount = 0;
    void setup() {
      while (!Serial) ;
      Serial.printf("IPRXFCR = %x\n", FLEXSPI2_IPRXFCR);
      Serial.println("\n" __FILE__ " " __DATE__ " " __TIME__);
      if (!myfs.begin()) {
        Serial.println("error startup");
        while (1) ;
      }
      printDirectory(myfs);
      NUMFILES = filecount;
    }
    
    unsigned int count = 0;
    unsigned int errcount = 0;
    
    void loop() {
      Serial.print("run ");
      Serial.print(++count);
      Serial.print(", errors ");
      Serial.println(errcount);
      
      filecount = 0;
      printDirectory(myfs);
      if (filecount != NUMFILES) {
        errcount++;
        delay(5000);
      }
      delay(1); // run fast, but don't overwhelm serial monitor
    }
    
    
    
    void printDirectory(FS &fs) {
      Serial.println("printDirectory\n--------------");
      printDirectory(fs.open("/"), 0);
      Serial.println();
    }
    
    void printDirectory(File dir, int numTabs) {
      while (true) {
        File entry =  dir.openNextFile();
        if (! entry) {
          // no more files
          break;
        }
        for (uint8_t i = 0; i < numTabs; i++) {
          Serial.print('\t');
        }
        Serial.print(entry.name());
        if (entry.isDirectory()) {
          Serial.println(" / ");
          printDirectory(entry, numTabs + 1);
        } else {
          // files have sizes, directories do not
          Serial.print("\t\t");
          Serial.println(entry.size(), DEC);
          filecount++;
        }
        entry.close();
      }
    }

  18. #268
    Senior Member+ defragster's Avatar
    Join Date
    Feb 2015
    Posts
    13,203
    Quote Originally Posted by PaulStoffregen View Post
    I'm a little concerned about the error "spew" Defragster posted in msg #255 (just minutes before I committed this race condition fix).

    Those could be related to the RX FIFO race condition, or it could be a completely different bug. Very difficult to know...
    Indeed - that was UGLY and shocking to see ...

    It only occurred in the one set of repro's and doesn't repeat as tried with latest LittleFS ....

    Any chance that was a side effect of a corrupted disk directory entry with a bad entry telling it to do something out of bounds on the QSPI?

    Running that LFSIntegrity on root and 10 subdirs with 26 files without verbose DIR printing so it runs a bit faster:
    5 mins did a 100 iteration with no error on #2 T_4.1 w/QSPI:
    Code:
    [  4.42 M] Awaiting input 0123456789rdchkfvplm? loops left 0 >l 
    	 Loop Count: 111 (#fileCycle=15838), Bytes read 4516416, written 1468664
    Then ran about 30 '1' single iterations no problem where about 286 files are involved across root and 10 dirs.

    Then turned it on 'c'ontinuous for a bit after that and with 13 minutes running no errors:
    Code:
     10 dirs with 26 files of Size 5704 Bytes
     Total 286 files of Size 61312 Bytes
    Bytes Used: 704512, Bytes Total:16777216
    
    [ 13.83 M] Awaiting input 0123456789rdchkfvplm? loops left 0 >
    
    [ 13.83 M] Awaiting input 0123456789rdchkfvplm? loops left 0 >l 
    	 Loop Count: 276 (#fileCycle=35182), Bytes read 10257464, written 3264696
    And some time later ... another 15 minutes running:
    Code:
     10 dirs with 24 files of Size 10980 Bytes
     Total 279 files of Size 131181 Bytes
    Bytes Used: 1114112, Bytes Total:16777216
    
    [ 40.01 M] Awaiting input 0123456789rdchkfvplm? loops left 0 >
    
    [ 40.01 M] Awaiting input 0123456789rdchkfvplm? loops left 0 >l 
    	 Loop Count: 860 (#fileCycle=124027), Bytes read 36534907, written 11505408
    I really like the CMD UI I put in that sketch! Extended it to ad 'l' and 'm' today where 'm' makes the needed root dirs after a Format.
    Last edited by defragster; 11-16-2020 at 08:12 PM.

  19. #269
    Senior Member+ defragster's Avatar
    Join Date
    Feb 2015
    Posts
    13,203
    Oh and Teensy #1 running updated p#267 of Paul's Dir count code is doing well on 200+ files now:
    Code:
    Z_file.txt		251
    
    run 4241, errors 0
    printDirectory
    --------------
    0_dir / 
    	A_file.txt		101

  20. #270
    Senior Member+ defragster's Avatar
    Join Date
    Feb 2015
    Posts
    13,203
    @Paul - what is with the 20 to 30 ms waits on QSPI?

    Is that the processor flushing the cache or some such long operation as it waits for the needed data to save/read?

    Code:
     QSPI_DISK +++ Add ++++ [was 0 wrote 111] ++ L 	Verify /L_file.txt bytes read 111 and size 111
    :: /M_file.txt   waited 29376 us
      waited 369 us
      waited 372 us
      waited 372 us
      waited 372 us
      waited 229 us
      waited 27960 us
      waited 369 us
      waited 372 us
      waited 372 us
      waited 372 us
      waited 372 us
      waited 73 us
     QSPI_DISK +++ Add ++++ [was 0 wrote 121] ++ M 	Verify /M_file.txt bytes read 121 and size 121

  21. #271
    Senior Member+ defragster's Avatar
    Join Date
    Feb 2015
    Posts
    13,203
    @Paul latest change looking good from here for QSPI:
    One Teensy:
    Code:
    Z_file.txt		251
    
    run 18787, errors 0
    printDirectory
    --------------
    0_dir / 
    	A_file.txt		101
    The Other with Bigger file changes in LFSIntegrity - running some part of 45 minutes and Bytes read 115,031,770, written 36,276,080:
    Code:
     0 dirs with 26 files of Size 29350 Bytes
    FILE	B_file.txt		1010
    FILE	C_file.txt		1020
    FILE	F_file.txt		1050
    FILE	G_file.txt		1060
    FILE	J_file.txt		1090
    FILE	K_file.txt		1100
    FILE	N_file.txt		1130
    FILE	O_file.txt		1140
    FILE	R_file.txt		1170
    FILE	S_file.txt		1180
    FILE	V_file.txt		1210
    FILE	W_file.txt		1220
    
     10 dirs with 12 files of Size 13380 Bytes
     Total 100 files of Size 112980 Bytes
    Bytes Used: 507904, Bytes Total:16777216
    
    [ 47.59 M] Awaiting input 0123456789rdchkfvplm? loops left 0 >
    
    [ 47.59 M] Awaiting input 0123456789rdchkfvplm? loops left 0 >l 
    	 Loop Count: 568 (#fileCycle=45242), Bytes read 115031770, written 36276080
    That is with "BIGADD 1000" now doing "BIGADD 10000"
    All well after some first loops - files are growing:
    Code:
     0 dirs with 20 files of Size 257820 Bytes
    FILE	A_file.txt		11200
    FILE	B_file.txt		11020
    FILE	C_file.txt		11040
    FILE	D_file.txt		11060
    FILE	E_file.txt		11080
    FILE	F_file.txt		21150
    FILE	G_file.txt		11120
    FILE	H_file.txt		1070
    FILE	I_file.txt		11160
    FILE	J_file.txt		20180
    FILE	K_file.txt		10100
    FILE	L_file.txt		1110
    FILE	O_file.txt		10140
    FILE	P_file.txt		1150
    FILE	U_file.txt		11400
    FILE	V_file.txt		20420
    FILE	W_file.txt		20440
    FILE	X_file.txt		11460
    FILE	Y_file.txt		11480
    FILE	Z_file.txt		11500
    
     10 dirs with 20 files of Size 229280 Bytes
     Total 217 files of Size 3071940 Bytes
    Bytes Used: 3518464, Bytes Total:16777216
    
    [  2.67 M] Awaiting input 0123456789rdchkfvplm? loops left 0 >
    
    [  2.67 M] Awaiting input 0123456789rdchkfvplm? loops left 0 >l 
    	 Loop Count: 25 (#fileCycle=955), Bytes read 14086630, written 6958080
    And 50 more cycles - files growing okay:
    Code:
     10 dirs with 24 files of Size 263230 Bytes
     Total 256 files of Size 2856350 Bytes
    Bytes Used: 3473408, Bytes Total:16777216

  22. #272
    Senior Member+ KurtE's Avatar
    Join Date
    Jan 2014
    Posts
    8,217
    I probably need to setup another T4.1 to try out some of this stuff.

    My latest order from Digikey had:
    5 of the https://www.digikey.com/en/products/...EIG-TR/7393545 (FLASH - NAND Memory IC 1Gb (128M x 8) SPI 104MHz 8-WSON (8x6))

    and
    5 of the
    https://www.digikey.com/en/products/...SIQ-TR/7087212 (FLASH - NOR Memory IC 128Mb (16M x 8) SPI - Quad I/O, QPI, DTR 133MHz 8-SOIC)

    I am guessing for this round the later one is the better choice to wire up another T4.1... Still need to order some more PSRAM, I think I still have one... Need to use for Cameras

  23. #273
    Senior Member PaulStoffregen's Avatar
    Join Date
    Nov 2012
    Posts
    23,485
    Quote Originally Posted by defragster View Post
    @Paul - what is with the 20 to 30 ms waits on QSPI?
    Those are the time the flash chip takes to write a 256 byte page or erase a 4K sector. You should see very similar times for the same type of flash chip on the SPI port (if that code inside LittleFS.cpp is uncommented).

    Here are the specs from Winbond's datasheet. As you can see, these chips are pretty slow for anything but reading.

    Click image for larger version. 

Name:	sc.png 
Views:	9 
Size:	33.3 KB 
ID:	22490

    As the chip ages from heavy use, you may see the wait times gradually increase. Of course, these worst case specs in the datasheet also take extreme temperature and manufacturing tolerance into account, so you're unlikely to see anything take quite that long for ordinary use at room temperature.

  24. #274
    Senior Member+ mjs513's Avatar
    Join Date
    Jul 2014
    Location
    New York
    Posts
    6,125
    Quote Originally Posted by KurtE View Post
    I probably need to setup another T4.1 to try out some of this stuff.

    My latest order from Digikey had:
    5 of the https://www.digikey.com/en/products/...EIG-TR/7393545 (FLASH - NAND Memory IC 1Gb (128M x 8) SPI 104MHz 8-WSON (8x6))

    and
    5 of the
    https://www.digikey.com/en/products/...SIQ-TR/7087212 (FLASH - NOR Memory IC 128Mb (16M x 8) SPI - Quad I/O, QPI, DTR 133MHz 8-SOIC)

    I am guessing for this round the later one is the better choice to wire up another T4.1... Still need to order some more PSRAM, I think I still have one... Need to use for Cameras
    @KurtE
    Yep the later 128MB NOR flash is the one to go with. The NAND is setup for direct memory read and writes. Think Paul mentioned that it is a bit more complicated to implement: https://forum.pjrc.com/threads/58033...l=1#post258762 and he won't be getting to it for awhile.

  25. #275
    Senior Member PaulStoffregen's Avatar
    Join Date
    Nov 2012
    Posts
    23,485
    Quote Originally Posted by KurtE View Post
    I am guessing for this round the later one is the better choice to wire up another T4.1...
    Yes, the smaller & simpler NOR flash is the only type that's likely to work for a while...

    Unless you really *want* to dive into writing NAND flash support code, probably best to leave those in the Digikey bag for a while. Those chips require quite a lot of extra work to check the error correcting code status and manage remapping bad sectors. Winbond has 2 more chips coming soon with 256M and 512M capacity, which are similar in concept but use different ECC and have slightly different commands. I did get the datasheets without signing a NDA (they're not on Winbond's website yet), so email me if you really want the details. My current feeling is to wait until early next year when I can a sample of one of those new chips, rather than writing a lot of code now and having to test adapt it to the new format only months later.

    The NAND chips have larger 2K page size and 128K sector erase size. So even though the chip is much larger, it probably won't be any more useful if you store only small files, because LittleFS allocates sectors to files. Even a file with a few bytes will take up at least 128K of space on those NAND chips. Of course they will be pretty awesome for storing a small number of relatively large files like big images and sounds.

Tags for this Thread

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •