Forum Rule: Always post complete source code & details to reproduce any issue!
Page 13 of 33 FirstFirst ... 3 11 12 13 14 15 23 ... LastLast
Results 301 to 325 of 811

Thread: LittleFS port to Teensy/SPIFlash

  1. #301
    Senior Member
    Join Date
    Jul 2014
    Posts
    3,004
    Quote Originally Posted by mjs513 View Post
    Also had to use FILE_WRITE when opening the file. SDFat used file.open("bench.dat", O_RDWR | O_CREAT | O_TRUNC).
    reason is SD defines mode in open as uint8_t, while SDFAT expects 32 bit.
    This prohibits the use of O_ type modes.

  2. #302
    Senior Member+ mjs513's Avatar
    Join Date
    Jul 2014
    Location
    New York
    Posts
    6,125
    Quote Originally Posted by WMXZ View Post
    reason is SD defines mode in open as uint8_t, while SDFAT expects 32 bit.
    This prohibits the use of O_ type modes.
    Yep - think we touched on this issue earlier in the thread as well. Was just documenting the changes I made from the SDFat benchmark sketch

  3. #303
    Senior Member+ KurtE's Avatar
    Join Date
    Jan 2014
    Posts
    8,217
    Quote Originally Posted by mjs513 View Post
    @KurtE
    Think we are cross-posting now with these edits?

    Check the benchmark sketch is configure for QSPI not Program. The sketch as posted is configured for Program.

    The results you are getting is what I am seeing when testing LittleFS_Program so that may be the issue
    I ran the updated format, and then edited the program to use QSPI both at top and begin...

    And now:
    Code:
    LittleFS Test
    QSPI flash begin
    Flash ID: EF 40 18
    Flash size is 16.00 Mbyte
    attempting to mount existing media
    success
    started
    FILE_SIZE_MB = 5
    BUF_SIZE = 512 bytes
    Starting write test, please wait.
    
    write speed and latency
    speed,max,min,avg
    KB/Sec,usec,usec,usec
    78.77,51849,834,6499
    78.58,55570,836,6515
    
    Starting read test, please wait.
    
    read speed and latency
    speed,max,min,avg
    KB/Sec,usec,usec,usec
    21737.74,129,14,23
    21643.64,130,14,23
    
    Done
    Now to see about if I can see it on the PC... And copy files to it.

  4. #304

  5. #305
    Senior Member+ defragster's Avatar
    Join Date
    Feb 2015
    Posts
    13,204
    Really fun getting here in AM some days with 2-3 folks East Coast and Early risers posting after I logged off

    Fixing no #ifdef class .format() ...

    @KurtE - the 'winbond' font looks funny - maybe normal - suppose it was from digikey and not a fraud? But ones here from digikey match the font if not the cut depth:
    Click image for larger version. 

Name:	20201117_100813.jpg 
Views:	8 
Size:	360.2 KB 
ID:	22503

    @mjs513 - here is a Spew Free Dir Verify from the above LFSintegrity - it returns a filecount to compare to static count:
    Code:
    unsigned int printDirectoryFilecount(File dir) {
    	unsigned int filecnt = 0;
    	while (true) {
    		File entry =  dir.openNextFile();
    		if (! entry) {
    			// no more files
    			break;
    		}
    		if (entry.isDirectory()) {
    			filecnt+=printDirectoryFilecount(entry);
    		} else {
    			filecnt++;
    		}
    		entry.close();
    	}
    	return filecnt;
    }

  6. #306
    Senior Member+ mjs513's Avatar
    Join Date
    Jul 2014
    Location
    New York
    Posts
    6,125
    Thanks Paul - will be nice when this is all in the next beta. Now back to playing.

  7. #307
    Senior Member+ mjs513's Avatar
    Join Date
    Jul 2014
    Location
    New York
    Posts
    6,125
    @defragster - Now you know how I feel when I wake up some days and you all have been working something new as well
    thanks for the updated function - will make it easier to for testing.

    @KurtE - cool that its now working after the format. Think maybe I have to issue the PR to Paul?

  8. #308
    Senior Member+ defragster's Avatar
    Join Date
    Feb 2015
    Posts
    13,204
    Quote Originally Posted by mjs513 View Post
    Thanks Paul - will be nice when this is all in the next beta. Now back to playing.
    Indeed I just pulled three ZIPs to integrate yet again ... but easier that than pushing another beta for each change when it just begets another change

    Quote Originally Posted by mjs513 View Post
    @defragster - Now you know how I feel when I wake up some days and you all have been working something new as well
    thanks for the updated function - will make it easier to for testing.

    @KurtE - cool that its now working after the format. Think maybe I have to issue the PR to Paul?
    I started to add code block for the printDirectoryFilecount() - but decided to drop the whole sketch that had it in.

    Having a Flash Level Format ( QSPI and SPI ) in FLASHMEM could be nice. Though if the right post seen above did a pre-format - it is still slow to write at 58KB/sec? Does the low level code always format space it wants to use?

    Now to integrate updated sources ... and see about LFSintegrity edits ...

  9. #309
    Senior Member+ mjs513's Avatar
    Join Date
    Jul 2014
    Location
    New York
    Posts
    6,125
    Quote Originally Posted by defragster View Post
    Indeed I just pulled three ZIPs to integrate yet again ... but easier that than pushing another beta for each change when it just begets another change



    I started to add code block for the printDirectoryFilecount() - but decided to drop the whole sketch that had it in.

    Having a Flash Level Format ( QSPI and SPI ) in FLASHMEM could be nice. Though if the right post seen above did a pre-format - it is still slow to write at 58KB/sec? Does the low level code always format space it wants to use?

    Now to integrate updated sources ... and see about LFSintegrity edits ...
    Oops forgot to add FLASHMEM to the function. Never mind. Its ok. Haven't done anything with SPI flash. Guess that's next.

    Now I have to go update my copies of everything as well

  10. #310
    Senior Member+ KurtE's Avatar
    Join Date
    Jan 2014
    Posts
    8,217
    Sorry I spoke too soon. I still called format instead of your initialize...

    And still had issues. So converted sketch over at first to call initialize instead of begin... It hung in the call to: flexspi2_ip_command(15, 0x00800000);

    So converted sketch to now call begin followed by initialize...
    Which was the problem all along, in that I just used a raw chip...

    so simple sketch now:
    Code:
    #include <LittleFS.h>
    #include <SPI.h>
    
    LittleFS_QSPIFlash myfs;
    
    void setup() {
      //  pinMode(13, OUTPUT);
      //  digitalWrite(13, HIGH);
      SPI.begin();
      while (!Serial) ; // wait
      Serial.println("LittleFS Test"); delay(5);
    
      if (!myfs.begin()) {
        Serial.println("Begin Failed");
      }
      Serial.println("*** Enter anything to continue to initialize ***");
      while (!Serial.available()) ;
      Serial.println("Configured QSPI");
      myfs.initialize();
      Serial.println("End");
    }
    
    void loop() {}
    Output:
    Code:
    LittleFS Test
    QSPI flash begin
    Flash ID: EF 40 18
    Flash size is 16.00 Mbyte
    attempting to mount existing media
    success
    *** Enter anything to continue to initialize ***
    Configured QSPI
    Erasing... (may take some time)
    ............................................................................
    Chip erased in 38 seconds.
    attempting to mount existing media
    couldn't mount media, attemping to format
    attempting to mount freshly formatted media
    success
    End
    Note: I do get compile warnings with the initialize as you have return false;
    from void function.

    Now to try the real sketch:
    mtp-test_combined... it is still not happy?

    Will have to try some other tests...

  11. #311
    Senior Member PaulStoffregen's Avatar
    Join Date
    Nov 2012
    Posts
    23,485
    Quote Originally Posted by mjs513 View Post
    will be nice when this is all in the next beta.
    Yes, there have been so many changes to multiple repositories! It's pretty amazing you're keeping up with so many commits.

    I am hoping to publish 1.54-beta5 this week.

    But first we probably need to revisit the thorny open modes topic. I've thought much about this over the last several days since we all exchanged so many messages about it. I know this is going to be unpopular, but I've more or less decided not provide a mechanism for filesystem specific flags from the abstract FS.h layer.

  12. #312
    Senior Member+ mjs513's Avatar
    Join Date
    Jul 2014
    Location
    New York
    Posts
    6,125
    Quote Originally Posted by Kurte
    Note: I do get compile warnings with the initialize as you have return false;
    from void function.

    Now to try the real sketch:
    mtp-test_combined... it is still not happy?

    Will have to try some other tests...
    Well just fixed that issue in the PR and on my PC. So hopefully its right now. Yeah, initialize has to be called after begin. All the chip initialization after begin since it has all the LUTS setup there.

    EDIT: forgot - mtp_test_combined still not working. That;s really strange since its working for me. Have no clue why its not working on your end. Just out of curiosity do you still have the issue say if you use SPIFlash?

    Think I am going to have to go and solder up a new T4.1 with PSRAM and Flash as a double check. Speaking of that did you check with the Beta T4.1?

  13. #313
    Senior Member PaulStoffregen's Avatar
    Join Date
    Nov 2012
    Posts
    23,485
    My question for everyone about open modes is what functionality do we *need* that isn't possible, or isn't efficient enough, using the abstract functions?

    For example, with the just-now-added truncate() function, you can achieve the same as SdFat's O_TRUNC and LittleFS's LFS_O_TRUNC by just opening the file and then calling file.truncate(0).

    Likewise, you can call fs.exists(pathname) before opening to get the result as using LittleFS's LFS_O_EXCL flag.

    I am willing to add more abstract open modes. But they need to be abstract to be done at FS.h and used by all programs, so we don't end up with just more of the same situation we have now, where programs have hard-coded dependency on a particular library. We've seen where this ultimately ends up, where people can't use 2 different libraries / examples together, where including one header automatically pulls in a library you didn't want and can't build together with the one you did wish to use. The abstraction layer is meant to solve all those problems long-term, but it can only do so it we resist the urge to take the easy short-term solutions the sacrifice it's abstractness. (is "abstractness" even a real word?)

    To add another open mode, there needs to be reason why something is needed which can't be done with the abstraction layer functions & existing modes, or is too inefficient when done with them.

  14. #314
    Senior Member+ KurtE's Avatar
    Join Date
    Jan 2014
    Posts
    8,217
    Quote Originally Posted by PaulStoffregen View Post
    Yes, there have been so many changes to multiple repositories! It's pretty amazing you're keeping up with so many commits.

    I am hoping to publish 1.54-beta5 this week.

    But first we probably need to revisit the thorny open modes topic. I've thought much about this over the last several days since we all exchanged so many messages about it. I know this is going to be unpopular, but I've more or less decided not provide a mechanism for filesystem specific flags from the abstract FS.h layer.
    Sounds good to have beta - not sure what all needs to sync up at different times.

    As for File open mode, sort of interesting question as it likely break existing programs who use SD that use the additional flags...

    I guess the real question may not really be filesystem specific flags versus a standardized set of options, such as a subset of the fcntl.h options...

    Again since I personally don't use many files with Teensy stuff, I can adapt to most anything, but would be nice to be a reasonable subset and not limited to:
    READ - which is read only and starts at start of file.
    WRITE - which is read and write and is put at end of file...

    But for the files I use with Teensy - usually test runs like this - not a big deal. Although if we actually have a working MTP (hopefully with Serial) where I can extract files when I want to or copy in new stuff... That could change.

    @mjs513 - The LittleFS_FileSize is working fine with that board/chip... Will now try back to mtp to see if that makes difference.

  15. #315
    Senior Member+ mjs513's Avatar
    Join Date
    Jul 2014
    Location
    New York
    Posts
    6,125
    To be honest, out of the one's listed by Kurt earlier in the thread, I've only used the ones I highlighted in red in most of my sketches:
    Code:
    // open flags
        LFS_O_RDONLY = 1,         // Open a file as read only        <===> FILE_READ
        LFS_O_WRONLY = 2,         // Open a file as write only
        LFS_O_RDWR   = 3,         // Open a file as read and write                                 <===> FILE_WRITE
        LFS_O_CREAT  = 0x0100,    // Create a file if it does not exist
        LFS_O_EXCL   = 0x0200,    // Fail if a file already exists                                    <===>  fs.exists(pathname) 
        LFS_O_TRUNC  = 0x0400,    // Truncate the existing file to zero size                   <===> truncate() function
        LFS_O_APPEND = 0x0800,    // Move to end of file on every write
    Listing just for info,
    Last edited by mjs513; 11-17-2020 at 07:17 PM. Reason: UPDATE

  16. #316
    Senior Member PaulStoffregen's Avatar
    Join Date
    Nov 2012
    Posts
    23,485
    Quote Originally Posted by KurtE View Post
    As for File open mode, sort of interesting question as it likely break existing programs who use SD that use the additional flags...
    Officially SD has only ever published 2 open modes.

    Are there significant cases of programs open files in undocumented ways?


    but would be nice to be a reasonable subset and not limited to:
    READ - which is read only and starts at start of file.
    WRITE - which is read and write and is put at end of file...
    I agree in principle. I'm pretty sure we need at least 1 more mode, to write starting at the beginning rather than the end. Even though you could just open with writing at the end and then use seek(0), for very large files seeking to the end is expensive.

    Now is the time to think of what other modes are truly needed (so they get into 1.54-beta5)

  17. #317
    Senior Member+ KurtE's Avatar
    Join Date
    Jan 2014
    Posts
    8,217
    More on my failure CHIP... It does work for smaller files.

    That is I was able to copy source files and the like to it, also a smaller image, create a folder... copy files files there, but not the two larger images.
    Click image for larger version. 

Name:	screenshot.jpg 
Views:	8 
Size:	94.1 KB 
ID:	22504

    Wonder if there is some form of timeout?

    Sort of an interesting thing, is I can drag and drop files from another folder into these folders and do a copy....
    It also looks like I maybe can do a drag and drop out of this folder and drop to another folder.

    But if I try to drag a file out of this directory to sublimetext it shows it as a no drop, where from other directories I can drop the file to open it... Wonder what the difference is?

  18. #318
    Senior Member+ defragster's Avatar
    Join Date
    Feb 2015
    Posts
    13,204
    Quote Originally Posted by KurtE View Post
    More on my failure CHIP... It does work for smaller files.

    That is I was able to copy source files and the like to it, also a smaller image, create a folder... copy files files there, but not the two larger images.
    Click image for larger version. 

Name:	screenshot.jpg 
Views:	8 
Size:	94.1 KB 
ID:	22504

    Wonder if there is some form of timeout?

    Sort of an interesting thing, is I can drag and drop files from another folder into these folders and do a copy....
    It also looks like I maybe can do a drag and drop out of this folder and drop to another folder.

    But if I try to drag a file out of this directory to sublimetext it shows it as a no drop, where from other directories I can drop the file to open it... Wonder what the difference is?
    Problem CHIP? What does LFSintegrity do running on that?
    This writes smaller files a byte at a time - changing either will do bigger with SUBADD having a multiplier if set to 1,000
    Code:
    #define SUBADD 1	// bytes added each pass (*times file number)
    #define BIGADD 10	// bytes added each pass - bigger will quickly consume more space
    SubLime is PICKY about where you drag to drop - perhaps this isn't news - it has to go on the file list bar not above - or the text window I've found even bringing in a source file from disk

  19. #319
    Senior Member
    Join Date
    Jul 2014
    Posts
    3,004
    Quote Originally Posted by KurtE View Post
    As for File open mode, sort of interesting question as it likely break existing programs who use SD that use the additional flags...
    My preference would be to always support a transparent mode, where implementation is restricting option, So I vote for an 'oflag_t mode = O_READ' open

    Edit: As an example: "File file = SD.sdfs.open(fileName, O_CREAT | O_WRITE | O_TRUNC);" is not correct
    and has to be replaced by "File file = File(new SDFile(msd->sdfs.open(fileName, O_CREAT | O_WRITE | O_TRUNC)));"
    Last edited by WMXZ; 11-17-2020 at 08:31 PM.

  20. #320
    Senior Member PaulStoffregen's Avatar
    Join Date
    Nov 2012
    Posts
    23,485
    Quote Originally Posted by WMXZ View Post
    Edit: As an example: "File file = SD.sdfs.open(fileName, O_CREAT | O_WRITE | O_TRUNC);" is not correct
    and has to be replaced by "File file = File(new SDFile(msd->sdfs.open(fileName, O_CREAT | O_WRITE | O_TRUNC)));"
    This is exactly the wrong way to use the File API. Code written in this manner won't be portable to LittleFS or other future filesystems!

  21. #321
    Senior Member
    Join Date
    Jul 2014
    Posts
    3,004
    Quote Originally Posted by PaulStoffregen View Post
    This is exactly the wrong way to use the File API. Code written in this manner won't be portable to LittleFS or other future filesystems!
    Sure, thats why I gave this example, which could easily be avoided having an additional future oriented open method. One always can maintain the backward compatible methods.
    BTW, FILE_WRITE is not at all transparent.

  22. #322
    Senior Member PaulStoffregen's Avatar
    Join Date
    Nov 2012
    Posts
    23,485
    Quote Originally Posted by WMXZ View Post
    FILE_WRITE is not at all transparent.
    The goal is abstraction, not transparency of the things we're trying to abstract!

    That sort of "transparent" is exactly the opposite of what File & FS are meant to accomplish.

  23. #323
    Senior Member+ KurtE's Avatar
    Join Date
    Jan 2014
    Posts
    8,217
    Quote Originally Posted by defragster View Post
    Problem CHIP? What does LFSintegrity do running on that?
    This writes smaller files a byte at a time - changing either will do bigger with SUBADD having a multiplier if set to 1,000
    Code:
    #define SUBADD 1	// bytes added each pass (*times file number)
    #define BIGADD 10	// bytes added each pass - bigger will quickly consume more space
    SubLime is PICKY about where you drag to drop - perhaps this isn't news - it has to go on the file list bar not above - or the text window I've found even bringing in a source file from disk
    I have been running the LFSIntegrety sketch. Not sure of best options... But
    #define SUBADD 5 // bytes added each pass (*times file number)
    #define BIGADD 50 // bytes added each pass - bigger will quickly consume more space

    Ran I think fine. I tried hitting h... got tired so then entered numbers...

    Code:
    [ 25.22 M] Awaiting input 0123456789rdchkfvplmu? loops left 0 > d
    printDirectory QSPI_DISK
    --------------
    DIR	0_dir / 
    
     0 dirs with 0 files of Size 0 Bytes
    DIR	1_dir / 
    
     0 dirs with 0 files of Size 0 Bytes
    DIR	2_dir / 
    
     0 dirs with 0 files of Size 0 Bytes
    DIR	3_dir / 
    	FILE	B_file.txt		55
    
     0 dirs with 1 files of Size 55 Bytes
    DIR	4_dir / 
    	FILE	C_file.txt		60
    	FILE	D_file.txt		65
    
     0 dirs with 2 files of Size 125 Bytes
    DIR	5_dir / 
    	FILE	E_file.txt		70
    	FILE	F_file.txt		75
    
     0 dirs with 2 files of Size 145 Bytes
    DIR	6_dir / 
    	FILE	G_file.txt		80
    	FILE	H_file.txt		85
    
     0 dirs with 2 files of Size 165 Bytes
    DIR	7_dir / 
    	FILE	I_file.txt		90
    	FILE	J_file.txt		95
    
     0 dirs with 2 files of Size 185 Bytes
    DIR	8_dir / 
    	FILE	K_file.txt		100
    	FILE	L_file.txt		105
    
     0 dirs with 2 files of Size 205 Bytes
    DIR	9_dir / 
    	FILE	M_file.txt		110
    	FILE	N_file.txt		115
    
     0 dirs with 2 files of Size 225 Bytes
    DIR	Folder / 
    	FILE	IMG_1260.jpg		121305
    	FILE	LittleFS.cpp		19766
    	FILE	LittleFS.h		13378
    	FILE	T4.1-Cardlike.jpg		0
    
     0 dirs with 4 files of Size 154449 Bytes
    FILE	IMG_1255.jpg		96181
    FILE	O_file.txt		120
    FILE	P_file.txt		125
    FILE	T4.1-Cardlike.jpg		0
    FILE	TEST.txt		66
    FILE	mtpindex.dat		0
    FILE	test.txt		66
    
     11 dirs with 7 files of Size 96558 Bytes
     Total 24 files of Size 252112 Bytes
    Bytes Used: 364544, Bytes Total:16777216
    
    
    	 Loop Count: 39 (#fileCycle=2199), Bytes read 427296, written 159025, #Files=24
    [ 31.38 M] Awaiting input 0123456789rdchkfvplmu? loops left 0 >
    Back to playing.

  24. #324
    Senior Member+ mjs513's Avatar
    Join Date
    Jul 2014
    Location
    New York
    Posts
    6,125
    @KurtE

    LPSIntegrity looks like it working but still now sure why you can not copy a 214Kb file. As a test i tried coping a 4.7Mb pdf file to flash and it failed but a 1.5mb pdf file worked (yes it did open when I clicked on it). Was also able to load a 2.7mb stl file and then double click and it opened.

    Just thought of something. @defragster was having a issue early on and he re-downloaded @WMXZ'z USB2 respository: https://github.com/WMXZ-EU/USB2. Maybe yours needs to be updated?

  25. #325
    Senior Member+ defragster's Avatar
    Join Date
    Feb 2015
    Posts
    13,204
    @KurtE - as noted that looks like a usable CHIP - at least for that sketch usage.

    It does many files with byte writes and with many files it started small with RAMDISK, I did test failure on disk full there - I have not gone so large as to test FLASH DISK full yet.

    ... just now put latest 3 from github on my system and did some code edits and adds about to compile - ... lots of phone, post, github distractions ... and lunch soon ...

Tags for this Thread

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •