reason is SD defines mode in open as uint8_t, while SDFAT expects 32 bit.Originally Posted by mjs513
This prohibits the use of O_ type modes.
reason is SD defines mode in open as uint8_t, while SDFAT expects 32 bit.
This prohibits the use of O_ type modes.
Yep - think we touched on this issue earlier in the thread as well. Was just documenting the changes I made from the SDFat benchmark sketch
I ran the updated format, and then edited the program to use QSPI both at top and begin...
And now:
Now to see about if I can see it on the PC... And copy files to it.Code:LittleFS Test QSPI flash begin Flash ID: EF 40 18 Flash size is 16.00 Mbyte attempting to mount existing media success started FILE_SIZE_MB = 5 BUF_SIZE = 512 bytes Starting write test, please wait. write speed and latency speed,max,min,avg KB/Sec,usec,usec,usec 78.77,51849,834,6499 78.58,55570,836,6515 Starting read test, please wait. read speed and latency speed,max,min,avg KB/Sec,usec,usec,usec 21737.74,129,14,23 21643.64,130,14,23 Done
I've added the missing File truncate(size) function.
https://github.com/PaulStoffregen/co...2c7f28504bda56
https://github.com/PaulStoffregen/Li...56f1184b739bb3
https://github.com/PaulStoffregen/SD...acd0740f854c2d
Really fun getting here in AM some days with 2-3 folks East Coast and Early risers posting after I logged off
Fixing no #ifdef class .format() ...
@KurtE - the 'winbond' font looks funny - maybe normal - suppose it was from digikey and not a fraud? But ones here from digikey match the font if not the cut depth:
@mjs513 - here is a Spew Free Dir Verify from the above LFSintegrity - it returns a filecount to compare to static count:
Code:unsigned int printDirectoryFilecount(File dir) { unsigned int filecnt = 0; while (true) { File entry = dir.openNextFile(); if (! entry) { // no more files break; } if (entry.isDirectory()) { filecnt+=printDirectoryFilecount(entry); } else { filecnt++; } entry.close(); } return filecnt; }
Thanks Paul - will be nice when this is all in the next beta.Now back to playing.
@defragster - Now you know how I feel when I wake up some days and you all have been working something new as well
thanks for the updated function - will make it easier to for testing.
@KurtE - cool that its now working after the format. Think maybe I have to issue the PR to Paul?
Indeed I just pulled three ZIPs to integrate yet again ... but easier that than pushing another beta for each change when it just begets another changeThanks Paul - will be nice when this is all in the next beta.
I started to add code block for the printDirectoryFilecount() - but decided to drop the whole sketch that had it in.@defragster - Now you know how I feel when I wake up some days and you all have been working something new as well
thanks for the updated function - will make it easier to for testing.
@KurtE - cool that its now working after the format. Think maybe I have to issue the PR to Paul?
Having a Flash Level Format ( QSPI and SPI ) in FLASHMEM could be nice. Though if the right post seen above did a pre-format - it is still slow to write at 58KB/sec? Does the low level code always format space it wants to use?
Now to integrate updated sources ... and see about LFSintegrity edits ...
Oops forgot to add FLASHMEM to the function. Never mind. Its ok. Haven't done anything with SPI flash. Guess that's next.Indeed I just pulled three ZIPs to integrate yet again ... but easier that than pushing another beta for each change when it just begets another change
I started to add code block for the printDirectoryFilecount() - but decided to drop the whole sketch that had it in.
Having a Flash Level Format ( QSPI and SPI ) in FLASHMEM could be nice. Though if the right post seen above did a pre-format - it is still slow to write at 58KB/sec? Does the low level code always format space it wants to use?
Now to integrate updated sources ... and see about LFSintegrity edits ...
Now I have to go update my copies of everything as well
Sorry I spoke too soon. I still called format instead of your initialize...
And still had issues. So converted sketch over at first to call initialize instead of begin... It hung in the call to: flexspi2_ip_command(15, 0x00800000);
So converted sketch to now call begin followed by initialize...
Which was the problem all along, in that I just used a raw chip...
so simple sketch now:
Output:Code:#include <LittleFS.h> #include <SPI.h> LittleFS_QSPIFlash myfs; void setup() { // pinMode(13, OUTPUT); // digitalWrite(13, HIGH); SPI.begin(); while (!Serial) ; // wait Serial.println("LittleFS Test"); delay(5); if (!myfs.begin()) { Serial.println("Begin Failed"); } Serial.println("*** Enter anything to continue to initialize ***"); while (!Serial.available()) ; Serial.println("Configured QSPI"); myfs.initialize(); Serial.println("End"); } void loop() {}
Note: I do get compile warnings with the initialize as you have return false;Code:LittleFS Test QSPI flash begin Flash ID: EF 40 18 Flash size is 16.00 Mbyte attempting to mount existing media success *** Enter anything to continue to initialize *** Configured QSPI Erasing... (may take some time) ............................................................................ Chip erased in 38 seconds. attempting to mount existing media couldn't mount media, attemping to format attempting to mount freshly formatted media success End
from void function.
Now to try the real sketch:
mtp-test_combined...it is still not happy?
Will have to try some other tests...
Yes, there have been so many changes to multiple repositories! It's pretty amazing you're keeping up with so many commits.
I am hoping to publish 1.54-beta5 this week.
But first we probably need to revisit the thorny open modes topic. I've thought much about this over the last several days since we all exchanged so many messages about it. I know this is going to be unpopular, but I've more or less decided not provide a mechanism for filesystem specific flags from the abstract FS.h layer.
Well just fixed that issue in the PR and on my PC. So hopefully its right now. Yeah, initialize has to be called after begin. All the chip initialization after begin since it has all the LUTS setup there.
EDIT: forgot - mtp_test_combined still not working. That;s really strange since its working for me. Have no clue why its not working on your end. Just out of curiosity do you still have the issue say if you use SPIFlash?
Think I am going to have to go and solder up a new T4.1 with PSRAM and Flash as a double check. Speaking of that did you check with the Beta T4.1?
My question for everyone about open modes is what functionality do we *need* that isn't possible, or isn't efficient enough, using the abstract functions?
For example, with the just-now-added truncate() function, you can achieve the same as SdFat's O_TRUNC and LittleFS's LFS_O_TRUNC by just opening the file and then calling file.truncate(0).
Likewise, you can call fs.exists(pathname) before opening to get the result as using LittleFS's LFS_O_EXCL flag.
I am willing to add more abstract open modes. But they need to be abstract to be done at FS.h and used by all programs, so we don't end up with just more of the same situation we have now, where programs have hard-coded dependency on a particular library. We've seen where this ultimately ends up, where people can't use 2 different libraries / examples together, where including one header automatically pulls in a library you didn't want and can't build together with the one you did wish to use. The abstraction layer is meant to solve all those problems long-term, but it can only do so it we resist the urge to take the easy short-term solutions the sacrifice it's abstractness. (is "abstractness" even a real word?)
To add another open mode, there needs to be reason why something is needed which can't be done with the abstraction layer functions & existing modes, or is too inefficient when done with them.
Sounds good to have beta - not sure what all needs to sync up at different times.
As for File open mode, sort of interesting question as it likely break existing programs who use SD that use the additional flags...
I guess the real question may not really be filesystem specific flags versus a standardized set of options, such as a subset of the fcntl.h options...
Again since I personally don't use many files with Teensy stuff, I can adapt to most anything, but would be nice to be a reasonable subset and not limited to:
READ - which is read only and starts at start of file.
WRITE - which is read and write and is put at end of file...
But for the files I use with Teensy - usually test runs like this - not a big deal. Although if we actually have a working MTP (hopefully with Serial) where I can extract files when I want to or copy in new stuff... That could change.
@mjs513 - The LittleFS_FileSize is working fine with that board/chip... Will now try back to mtp to see if that makes difference.
To be honest, out of the one's listed by Kurt earlier in the thread, I've only used the ones I highlighted in red in most of my sketches:
Listing just for info,Code:// open flags LFS_O_RDONLY = 1, // Open a file as read only <===> FILE_READ LFS_O_WRONLY = 2, // Open a file as write only LFS_O_RDWR = 3, // Open a file as read and write <===> FILE_WRITE LFS_O_CREAT = 0x0100, // Create a file if it does not exist LFS_O_EXCL = 0x0200, // Fail if a file already exists <===> fs.exists(pathname) LFS_O_TRUNC = 0x0400, // Truncate the existing file to zero size <===> truncate() function LFS_O_APPEND = 0x0800, // Move to end of file on every write
Last edited by mjs513; 11-17-2020 at 07:17 PM. Reason: UPDATE
Officially SD has only ever published 2 open modes.
Are there significant cases of programs open files in undocumented ways?
I agree in principle. I'm pretty sure we need at least 1 more mode, to write starting at the beginning rather than the end. Even though you could just open with writing at the end and then use seek(0), for very large files seeking to the end is expensive.but would be nice to be a reasonable subset and not limited to:
READ - which is read only and starts at start of file.
WRITE - which is read and write and is put at end of file...
Now is the time to think of what other modes are truly needed (so they get into 1.54-beta5)
More on my failure CHIP... It does work for smaller files.
That is I was able to copy source files and the like to it, also a smaller image, create a folder... copy files files there, but not the two larger images.
Wonder if there is some form of timeout?
Sort of an interesting thing, is I can drag and drop files from another folder into these folders and do a copy....
It also looks like I maybe can do a drag and drop out of this folder and drop to another folder.
But if I try to drag a file out of this directory to sublimetext it shows it as a no drop, where from other directories I can drop the file to open it... Wonder what the difference is?
Problem CHIP? What does LFSintegrity do running on that?More on my failure CHIP... It does work for smaller files.
That is I was able to copy source files and the like to it, also a smaller image, create a folder... copy files files there, but not the two larger images.
Wonder if there is some form of timeout?
Sort of an interesting thing, is I can drag and drop files from another folder into these folders and do a copy....
It also looks like I maybe can do a drag and drop out of this folder and drop to another folder.
But if I try to drag a file out of this directory to sublimetext it shows it as a no drop, where from other directories I can drop the file to open it... Wonder what the difference is?
This writes smaller files a byte at a time - changing either will do bigger with SUBADD having a multiplier if set to 1,000
SubLime is PICKY about where you drag to drop - perhaps this isn't news - it has to go on the file list bar not above - or the text window I've found even bringing in a source file from diskCode:#define SUBADD 1 // bytes added each pass (*times file number) #define BIGADD 10 // bytes added each pass - bigger will quickly consume more space
My preference would be to always support a transparent mode, where implementation is restricting option, So I vote for an 'oflag_t mode = O_READ' open
Edit: As an example: "File file = SD.sdfs.open(fileName, O_CREAT | O_WRITE | O_TRUNC);" is not correct
and has to be replaced by "File file = File(new SDFile(msd->sdfs.open(fileName, O_CREAT | O_WRITE | O_TRUNC)));"
Last edited by WMXZ; 11-17-2020 at 08:31 PM.
This is exactly the wrong way to use the File API. Code written in this manner won't be portable to LittleFS or other future filesystems!
Sure, thats why I gave this example, which could easily be avoided having an additional future oriented open method. One always can maintain the backward compatible methods.
BTW, FILE_WRITE is not at all transparent.
The goal is abstraction, not transparency of the things we're trying to abstract!
That sort of "transparent" is exactly the opposite of what File & FS are meant to accomplish.
I have been running the LFSIntegrety sketch. Not sure of best options... But
#define SUBADD 5 // bytes added each pass (*times file number)
#define BIGADD 50 // bytes added each pass - bigger will quickly consume more space
Ran I think fine. I tried hitting h... got tired so then entered numbers...
Back to playing.Code:[ 25.22 M] Awaiting input 0123456789rdchkfvplmu? loops left 0 > d printDirectory QSPI_DISK -------------- DIR 0_dir / 0 dirs with 0 files of Size 0 Bytes DIR 1_dir / 0 dirs with 0 files of Size 0 Bytes DIR 2_dir / 0 dirs with 0 files of Size 0 Bytes DIR 3_dir / FILE B_file.txt 55 0 dirs with 1 files of Size 55 Bytes DIR 4_dir / FILE C_file.txt 60 FILE D_file.txt 65 0 dirs with 2 files of Size 125 Bytes DIR 5_dir / FILE E_file.txt 70 FILE F_file.txt 75 0 dirs with 2 files of Size 145 Bytes DIR 6_dir / FILE G_file.txt 80 FILE H_file.txt 85 0 dirs with 2 files of Size 165 Bytes DIR 7_dir / FILE I_file.txt 90 FILE J_file.txt 95 0 dirs with 2 files of Size 185 Bytes DIR 8_dir / FILE K_file.txt 100 FILE L_file.txt 105 0 dirs with 2 files of Size 205 Bytes DIR 9_dir / FILE M_file.txt 110 FILE N_file.txt 115 0 dirs with 2 files of Size 225 Bytes DIR Folder / FILE IMG_1260.jpg 121305 FILE LittleFS.cpp 19766 FILE LittleFS.h 13378 FILE T4.1-Cardlike.jpg 0 0 dirs with 4 files of Size 154449 Bytes FILE IMG_1255.jpg 96181 FILE O_file.txt 120 FILE P_file.txt 125 FILE T4.1-Cardlike.jpg 0 FILE TEST.txt 66 FILE mtpindex.dat 0 FILE test.txt 66 11 dirs with 7 files of Size 96558 Bytes Total 24 files of Size 252112 Bytes Bytes Used: 364544, Bytes Total:16777216 Loop Count: 39 (#fileCycle=2199), Bytes read 427296, written 159025, #Files=24 [ 31.38 M] Awaiting input 0123456789rdchkfvplmu? loops left 0 >
@KurtE
LPSIntegrity looks like it working but still now sure why you can not copy a 214Kb file. As a test i tried coping a 4.7Mb pdf file to flash and it failed but a 1.5mb pdf file worked (yes it did open when I clicked on it). Was also able to load a 2.7mb stl file and then double click and it opened.
Just thought of something. @defragster was having a issue early on and he re-downloaded @WMXZ'z USB2 respository: https://github.com/WMXZ-EU/USB2. Maybe yours needs to be updated?
@KurtE - as noted that looks like a usable CHIP - at least for that sketch usage.
It does many files with byte writes and with many files it started small with RAMDISK, I did test failure on disk full there - I have not gone so large as to test FLASH DISK full yet.
... just now put latest 3 from github on my system and did some code edits and adds about to compile - ... lots of phone, post, github distractions ... and lunch soon ...
Reply With Quote
