Forum Rule: Always post complete source code & details to reproduce any issue!
Page 37 of 38 FirstFirst ... 27 35 36 37 38 LastLast
Results 901 to 925 of 941

Thread: LittleFS port to Teensy/SPIFlash

  1. #901
    Senior Member
    Join Date
    May 2015
    Location
    USA
    Posts
    1,070
    For what little it's worth, I took the https://github.com/littlefs-project/littlefs code and compiled with "-fsanitize=address". It passed all the "make test" tests on linux.

    Unfortunately, "-fsanitize=address" isn't available for embedded arm.

  2. #902
    Quote Originally Posted by TeensyDigital View Post
    I've got an issue when using LittleFS on a T4.1 with an external flash chip, while running under a separate thread. It hangs the thread randomly, where the code works just fine when compiled without threads. I also see that if I substantially increase the setMicroTimer the condition improves, but does not go away. This wasn't an issue when using the same (similar) code with an SD card.

    Are there any known issues using LittleFS or external Flash with threading?
    I can confirm that LittleFS works well with external Flash memory using threads under 1.54b10. I traced back and isolated my issues yesterday and today. I found two different issues, which is why I was struggling to troubleshoot LittleFS in my "super code base". The first issue was completely unrelated to LFS. In my code I had a char array of size 500 and I was doing a strcpy into it with a string that was 550 characters long. Oddly, under 1.53 this code has worked everyday for six months. The 1.54 betas seem to have tightened up and exposed that oversight on my part. It just so happens to be the first thing I write to LFS, so it seemed like a LFS issue. The second issue was with my thread heap size. Previously, with SD, I could get away with the default heap (1024), but when I replaced SD with LFS, it needed more memory. Both of these were easy fixes. Now I've got LittleFS logging 100 bytes 40 times a second in an isolated thread with even more going on with two other threads in parallel. All is good in the universe again. Thx. Mike

  3. #903
    Senior Member+ defragster's Avatar
    Join Date
    Feb 2015
    Posts
    15,069
    Cleaning up LFSIntegrity to become "Examples/Integrity" - hopefully for directly usable sketches for users to see Hardware/Media working - and get it into TD 1.54.

    I have SEVEN Teensys hanging off my 7 port HUB - Thank you @koromix for TyCommander! - 2 4.0's and 5 4.1's

    See :: Defragster/LittleFS/tree/main/examples/Integrity

    That includes:
    Code:
    ...\GitHub\LittleFS\examples\Integrity
    ALL  // This is the working start to make the others, cleanup and removing #ifdef He11
    MRAMSPI
    PSRAM
    QSPI
    RAM
    SPI
    Wondering about a unique entry for PROG not listed before?
    FLASH on SPI or QSPI can be NAND or NOR and that is diff constructor? Was trying to avoid any user needed #IFDEF?

    After First pass occurs the best thing for user understanding would be to segment and clean up this HELP Command list.
    ReOrder and Group and add group labels:
    Code:
     'a' Auto formatUnused() during iterations - TOGGLE
     'R' Restart Teensy
     'd' Directory of LittleFS
     'D' Walk all Files verify Read Size
     'w' WIPE Directory of LittleFS
     'b' big file delete
     'B' BIG FILE MAKE
     'S' FILE 2MB MAKE
     's' FILE 2MB delete
     'c' Continuous Loop
     'g' run speedBench()
     'h' Hundred loops
     'k' Thousand loops
     'F' LittleFS_ Low Level Format Disk 
     'f' LittleFS::formatUnused( ALL ) : DATA PRESERVED 
     'q' LittleFS_ Quick Format Disk 
     'v' Verbose All Dir Prints - TOGGLE
     'p' Pause after all Dir prints - TOGGLE
     'l' Show count of loop()'s, Bytes Read,Written
     'm' Make ROOT dirs (needed after q/F format !ROOTONLY)
     'u' Update Filecount
     'x' Directory filecount verify - TOGGLE
     'n' No verify on Write- TOGGLE
     '+' more add to delete cycles
     '-' fewer add to delete cycles
     'y' reclaim 1 block :: myfs.formatUnused( 1 )
     'Y' reclaim 15 blocks :: myfs.formatUnused( 15 )
     '?' Help list

  4. #904
    Senior Member+ defragster's Avatar
    Join Date
    Feb 2015
    Posts
    15,069
    First pass complete - All Media accounted for ?

    Six Primary types and in SPI and QSPI is an added IFDEF to choose NAND versus NOR.
    Code:
    T:\arduino-1.8.15\hardware\teensy\avr\libraries\LittleFS\examples\Integrity>dir /b
    ALL
    MRAMSPI
    PROG
    PSRAM
    QSPI
    RAM
    SPI
    The 'NAMED'.INO Sketch holds the ~110 lines of base comments and setup() and loop() and Globals and common code #defines.

    Also in each folder is the extracted "functions.ino" that is common 978 lines of worker code.
    > Ahead is editing that for the HELP text presentation

    Now is the time for any outside test or feedback? ...

    github.com/Defragster/LittleFS/tree/main/examples/Integrity

  5. #905
    Senior Member+ mjs513's Avatar
    Join Date
    Jul 2014
    Location
    New York
    Posts
    7,548
    Quote Originally Posted by defragster View Post
    First pass complete - All Media accounted for ?

    Six Primary types and in SPI and QSPI is an added IFDEF to choose NAND versus NOR.
    Code:
    T:\arduino-1.8.15\hardware\teensy\avr\libraries\LittleFS\examples\Integrity>dir /b
    ALL
    MRAMSPI
    PROG
    PSRAM
    QSPI
    RAM
    SPI
    The 'NAMED'.INO Sketch holds the ~110 lines of base comments and setup() and loop() and Globals and common code #defines.

    Also in each folder is the extracted "functions.ino" that is common 978 lines of worker code.
    > Ahead is editing that for the HELP text presentation

    Now is the time for any outside test or feedback? ...

    github.com/Defragster/LittleFS/tree/main/examples/Integrity
    Just gave SPI a test with the mem board. Unfortunately put everything away so tomorrow may have to reset everything up.

    Would make a suggestion. Change the following lines to default to something smaller:
    Code:
    #define MAXNUM 26	// Number of files : ALPHA A-Z is MAX of 26, less for fewer files
    #define NUMDIRS 32  // Number of Directories to use 0 is Rootonly
    maybe 5 and 5 respectfully. Otherwise if someone is experimenting they are going to hit something that is going to take a while to complete and not remember to us the pause key.

  6. #906
    Senior Member+ defragster's Avatar
    Join Date
    Feb 2015
    Posts
    15,069
    Quote Originally Posted by mjs513 View Post
    Just gave SPI a test with the mem board. Unfortunately put everything away so tomorrow may have to reset everything up.

    Would make a suggestion. Change the following lines to default to something smaller:
    Code:
    #define MAXNUM 26	// Number of files : ALPHA A-Z is MAX of 26, less for fewer files
    #define NUMDIRS 32  // Number of Directories to use 0 is Rootonly
    maybe 5 and 5 respectfully. Otherwise if someone is experimenting they are going to hit something that is going to take a while to complete and not remember to us the pause key.
    Very good 5 and 5 - will do that MAX on all. Was trying to make them media size dependent smaller as needed - but indeed MANY DIRS scrolls too much too fast!

    Just about to call it done for today : here is the look of the HELP RE-ORG ??? :
    Code:
     1-9 '#' passes continue loop before Pause
     'c, or 0': Continuous Loop, or stop Loop in progress
     'h, or k': start Hundred or Thousand loops
     'd' Directory of LittleFS
     'D' Walk all Dir Files verify Read Size
     'l' Show count of loop()'s, Bytes Read,Written
     'B, or b': Make Big file half of free space, or remove all Big files
     'S, or s': Make 2MB file , or remove all 2MB files
     	>> Media Format : 'q' and 'F' remove ALL FS data
     'q' Quick Format Disk 
     'F' Low Level Format Disk 
     'w' WIPE ALL Directories and Files
     'm' Make ROOT dirs (needed after q/F format or 'w')
     'f' LittleFS::formatUnused( ALL ) : DATA PRESERVED, empty blocks preformatted 
     'a' Auto formatUnused() during Loop - TOGGLE
     'y' reclaim 1 block :: myfs.formatUnused( 1 )
     'Y' reclaim 15 blocks :: myfs.formatUnused( 15 )
     	>> Test Features 
     'g' run SpeedBench() Media speed benchmark
     'x' Directory filecount verify - TOGGLE
     'v' Verbose All Dir Prints - TOGGLE
     'p' Pause after all Dir prints - TOGGLE
     'n' No verify on Write- TOGGLE
     'u' Update Filecount
     'R' Restart Teensy - Except 'RAM' - data persists
     '+, or -': more, or less add .vs. delete in Loop
     '?' Help list

  7. #907
    Senior Member+ defragster's Avatar
    Join Date
    Feb 2015
    Posts
    15,069

    Updated LittleFS Media Integrity sketches

    github.com/Defragster/LittleFS/tree/main/examples/Integrity

    Multiple sketches ready to run ( except edit of CS on SPI as needed - or Memory size where used RAM/PROG/PSRAM )

    Some comments added, Primary sketch code cleaned of #multimedia #ifdef, Help reorg, some 'debug' spew tamed.

  8. #908
    Senior Member+ defragster's Avatar
    Join Date
    Feb 2015
    Posts
    15,069
    Will give the LittleFS#readme another look ...

    Then ... I've got 8 Teensy-s sitting plugged in here ... going to run through each sketch some and then do a PR if nothing shows as interesting.

    Any feedback ....

  9. #909
    Senior Member+ defragster's Avatar
    Join Date
    Feb 2015
    Posts
    15,069
    Is :: LittleFS_Program myfs;

    expected to work on a T_3.5?

    Given "program storage space. Maximum is 524288 bytes."

    Asking for 204800 bytes fails?

    RAM works with an appropriate size ...

  10. #910
    Senior Member+ mjs513's Avatar
    Join Date
    Jul 2014
    Location
    New York
    Posts
    7,548
    Quote Originally Posted by defragster View Post
    Is :: LittleFS_Program myfs;

    expected to work on a T_3.5?

    Given "program storage space. Maximum is 524288 bytes."

    Asking for 204800 bytes fails?

    RAM works with an appropriate size ...
    @degragster

    Tried PROGRAM on both a T3.5 and a T3.6. Both fail to start using PROGRAM. Probably because it is not defined for use with Teensy 3.x. From the LittleFS.cpp:

    Code:
    #if defined(__IMXRT1062__)
    
    #if defined(ARDUINO_TEENSY40)
    #define FLASH_SIZE  0x1F0000
    #elif defined(ARDUINO_TEENSY41)
    #define FLASH_SIZE  0x7C0000
    #elif defined(ARDUINO_TEENSY_MICROMOD)
    #define FLASH_SIZE  0xFC0000
    #endif
    only defined to work with T4.x/TMM. Paul designed these into the code so would be his call to add T3.x or we need to put a warning into the readme.

  11. #911
    Senior Member+ defragster's Avatar
    Join Date
    Feb 2015
    Posts
    15,069
    Quote Originally Posted by mjs513 View Post
    @degragster

    Tried PROGRAM on both a T3.5 and a T3.6. Both fail to start using PROGRAM. Probably because it is not defined for use with Teensy 3.x. From the LittleFS.cpp:

    ...
    only defined to work with T4.x/TMM. Paul designed these into the code so would be his call to add T3.x or we need to put a warning into the readme.
    Thnx for looking mjs513 - was past Zzzz-Time ... only did 1062 in all that time - thought I had seen others float T_3.x by ... had to swap one of the eight T_4.x's plugged in for Integrity pass to test that ....

  12. #912
    Senior Member+ mjs513's Avatar
    Join Date
    Jul 2014
    Location
    New York
    Posts
    7,548
    Quote Originally Posted by defragster View Post
    Thnx for looking mjs513 - was past Zzzz-Time ... only did 1062 in all that time - thought I had seen others float T_3.x by ... had to swap one of the eight T_4.x's plugged in for Integrity pass to test that ....
    Don't remember if we ever did PROGMEM though. Think we tested with RAM and the various flash chips .

  13. #913
    Senior Member+ defragster's Avatar
    Join Date
    Feb 2015
    Posts
    15,069
    Quote Originally Posted by mjs513 View Post
    Don't remember if we ever did PROGMEM though. Think we tested with RAM and the various flash chips .
    Not tried here ... before. Suddenly realized ALL my testing has been on 1062's ...

    The FLASH 1062 offchip may be better for LittleFS than smaller onchip T_3.x's - don't remember seeing it noted - but it is now documented 1062 only

  14. #914
    Senior Member+ mjs513's Avatar
    Join Date
    Jul 2014
    Location
    New York
    Posts
    7,548
    Quote Originally Posted by defragster View Post
    Not tried here ... before. Suddenly realized ALL my testing has been on 1062's ...

    The FLASH 1062 offchip may be better for LittleFS than smaller onchip T_3.x's - don't remember seeing it noted - but it is now documented 1062 only
    PROGMEM was one of the first things Paul incorporated into LittleFS so my memory is a bit foggy. Yep the PR was incorporated with the warning It was already there for QSPI but missed the PROGMEM.

  15. #915
    Senior Member+ defragster's Avatar
    Join Date
    Feb 2015
    Posts
    15,069
    Quote Originally Posted by mjs513 View Post
    PROGMEM was one of the first things Paul incorporated into LittleFS so my memory is a bit foggy. Yep the PR was incorporated with the warning It was already there for QSPI but missed the PROGMEM.
    Added similar comments in the Integrity examples

    https://github.com/PaulStoffregen/LittleFS/pull/22

  16. #916
    Senior Member+ defragster's Avatar
    Join Date
    Feb 2015
    Posts
    15,069
    From quoted linked post:

    Quote Originally Posted by defragster View Post
    @Sandro /@all - With released TD 1.54 each "Media Type" is in its own sketch folder:
    Code:
    T:\arduino-1.8.15\hardware\teensy\avr\libraries\LittleFS\examples\Integrity\MRAMSPI
    T:\arduino-1.8.15\hardware\teensy\avr\libraries\LittleFS\examples\Integrity\PROG
    T:\arduino-1.8.15\hardware\teensy\avr\libraries\LittleFS\examples\Integrity\PSRAM
    T:\arduino-1.8.15\hardware\teensy\avr\libraries\LittleFS\examples\Integrity\QSPI
    T:\arduino-1.8.15\hardware\teensy\avr\libraries\LittleFS\examples\Integrity\RAM
    T:\arduino-1.8.15\hardware\teensy\avr\libraries\LittleFS\examples\Integrity\SPI
    I just noted this as a handy sketch extension for if nothing else PROG to save and restore data to SD.
    Hopefully I could get to it - though anyone eager might do it sooner:
    Code:
    LittleFS : TOO and FROM SD 'directory' and LittleFS "Media"
     > All Files and Dirs to SD CARD "DIR" - PROG BACKUP
     > All Files and Dirs From SD CARD "DIR" - PROG RESTORE
     > All Files and Dirs ON SD CARD "DIR" - PROG VERIFY
    Some variation on this code in functions.ino in a separate file that could be included in 'user' sketch, without all the other code.
    Working with a "DIR" on SD because the SD card could hold multiple copies - and lots more data
    Would also be a useful way to retrieve/pre-populate 'LOGGED' data from FLASH or PSRAM, etc:
    Code:
    int DirectoryVerify(File dir) {
    	int errCnt = 0;
    	while (true) {
    		File entry =  dir.openNextFile();
    		if (! entry) {
    			break;
    		}
    		if (entry.isDirectory()) {
    			Serial.print("\tD");
    			Serial.print(entry.name()[0]);
    			Serial.print(" ");
    			errCnt += DirectoryVerify(entry);
    			Serial.print("\n");
    		} else {
    			uint64_t fileSize, sizeCnt = 0;
    			char mm;
    			Serial.print(entry.name()[0]);
    			fileSize = entry.size();
    			while ( entry.available() ) {
    				if (fileSize < sizeCnt ) break;
    				entry.read( &mm , 1 );
    				sizeCnt++;
    			}
    			if (fileSize != sizeCnt ) {
    				Serial.printf("\n File Size Error:: %s found %llu Bytes for Size %llu \n", entry.name(), sizeCnt, fileSize);
    				errCnt++;
    				errsLFS++;
    			}
    		}
    		entry.close();
    	}
    	return errCnt;
    }

  17. #917
    Senior Member+ defragster's Avatar
    Join Date
    Feb 2015
    Posts
    15,069
    Started a quick write of the p#916 code - it copies root files form PSRAM to SD card root - and makes the root dirs on SD.

    Need to create SD dir for the copy and have the recursive code maintain the full path into the subdirs it seems to complete writing those files.

  18. #918
    Senior Member+ defragster's Avatar
    Join Date
    Feb 2015
    Posts
    15,069
    @Paul: <EDIT> : Not made a repro - maybe it was a one off or something else ...

    Will post code shortly - using a T_4.1 a PSRAM disk was made with files including ~4MB "B" bigfile and a 2MB "S" file, and a few subdirs with typical Integrity test files.

    Using the code too be posted the PSRAM files were written to SD card.

    Then - OPPS - I moved to PROG Media and asked for the SD card files to be placed onto PROG FLASH with 6MB reserved and it ran out of ROOM.

    Went to the SD CARD and removed the "B" big file and edited code and tried UPLOAD and it is FAILING?

    This is likely the first time PROG Flash was overfilled? And it seems that may be what is messing with it?

    Next post will be the code used. As long as the SD Card contents can FIT all seems well and files pass Integrity test.

    When a SD file set exceeds the capacity is where this went wrong.

    Code:
    11:17:42.386 (loader):  gauge old value = 118
    11:17:42.386 (loader): flash, block=119, bs=1024, auto=0
    11:17:42.401 (loader):  gauge old value = 119
    11:17:42.401 (loader): flash, block=120, bs=1024, auto=0
    11:17:42.401 (loader):  gauge old value = 120
    11:17:42.424 (loader): end operation, total time = 1.530 seconds
    11:17:42.429 (loader): set background IMG_DOWNLOAD_OK
    11:17:42.434 (loader): redraw timer set, image 12 to show for 1500 ms
    11:17:43.936 (loader): redraw, image 10
    11:17:43.936 (loader): set background IMG_ONLINE
    11:18:38.423 (post_compile 47): Begin, version=1.54, high-res time
    11:18:38.428 (loader): remote connection 2148 opened
    11:18:38.431 (loader): remote cmd from 2148: "comment: Teensyduino 1.54 - WINDOWS (teensy_post_compile)"
    11:18:38.431 (post_compile 47): Sending command: comment: Teensyduino 1.54 - WINDOWS (teensy_post_compile)
    11:18:38.435 (loader): remote cmd from 2148: "status"
    11:18:38.439 (loader): file changed
    11:18:38.463 (loader): File "PROG.ino.hex". 123904 bytes, 2% used
    11:18:38.468 (post_compile 47): Status: 1, 0, 1, 50, 1, 0, t:\temp\arduino_build_PROG.ino\, PROG.ino.hex
    11:18:38.468 (post_compile 47): Sending command: dir:t:\temp\arduino_build_PROG.ino\
    11:18:38.469 (loader): remote cmd from 2148: "dir:t:\temp\arduino_build_PROG.ino\"
    11:18:38.476 (loader): remote cmd from 2148: "file:PROG.ino.hex"
    11:18:38.476 (post_compile 47): Sending command: file:PROG.ino.hex
    11:18:38.499 (loader): File "PROG.ino.hex". 123904 bytes, 2% used
    11:18:38.504 (loader): remote cmd from 2148: "status"
    11:18:38.508 (post_compile 47): Status: 1, 0, 1, 50, 1, 0, t:\temp\arduino_build_PROG.ino\, PROG.ino.hex
    11:18:38.508 (post_compile 47): Disconnect
    11:18:38.534 (loader): remote connection 2148 closed
    Doing a 15s Restore ... and will try upload again with USABLE file size. When that works I can try the OVERFILL case again
    Last edited by defragster; 07-19-2021 at 08:31 PM. Reason: Program hang not repeated - fixed with 15s Restore

  19. #919
    Senior Member+ defragster's Avatar
    Join Date
    Feb 2015
    Posts
    15,069

    SD files copy Save/Restore to LittleFS Media

    A quick edit to the "\libraries\LittleFS\examples\Integrity" example of choice should work with this setup mod.
    Add the BOLD lines in place above the RED lines in the Example. This code uncommented line for destLFS will read ALL SD files onto the LFS Media:
    Code:
    #define destLFS 1
    #define destSD 2
    	//xferSD ( destSD ); // do MediaTransfer with SD
    	xferSD ( destLFS ); // do MediaTransfer with SD
    
    	filecount = printDirectoryFilecount( myfs.open("/") );  // Set base value of filecount for disk
    	printDirectory();
    	parseCmd( '?' );
    
    ALSO REQUIRED is this INO to be placed into the sketch folder: xfersdlfs.ino

    It is a minimal add and "printDirectory()" reliance on the file functions.ino is not required - just for debug status:
    Code:
    #include <SD.h>
    
    void xferSD ( int copyType ) { // do MediaTransfer with SD
    #define destLFS 1
    #define destSD 2
    	static bool initSD = true;
    	Serial.print("Initializing SD card...");
    	// see if the card is present and can be initialized:
    	if ( initSD && !SD.begin(BUILTIN_SDCARD)) {
    		Serial.println("\n\n SD Card failed, or not present - Cannot do Xfer\n");
    	}
    	else {
    		initSD = false;
    		Serial.println("card initialized.\n\n");
    		if ( copyType == destSD ) {
    			char szSDdir[] = "/";
    			Serial.println("\n STARTING :: LittleFS copy to SD card XFER ...\n\n");
    			mediaTransfer( myfs.open("/"), szSDdir, destSD ); // TOO SD
    			Serial.println("\n LittleFS copy to SD card XFER COMPLETE.\n\n");
    		}
    		else {
    			char szLFSdir[] = "/";
    			Serial.println("\n STARTING :: SD card copy to LittleFS XFER ...\n\n");
    			mediaTransfer( SD.open("/"), szLFSdir, destLFS );	// FROM SD
    			Serial.println("\n SD card copy to LittleFS XFER COMPLETE.\n\n");
    		}
    	}
    
    	Serial.printf("\n SD printDirectory \n--------------\n");
    	printDirectory(SD.open("/"), 0);
    	Serial.printf("\n SD printDirectory \n--------------\n");
    }
    
    int mediaTransfer(File dir, char* szDir, int destMedia) {
    	int errCnt = 0;
    	char szNewDir[36];
    	while (true) {
    		File entry =  dir.openNextFile();
    		if (! entry) {
    			break;
    		}
    		if (entry.isDirectory()) {
    			Serial.print("\tinDir >");
    			Serial.print(szDir);
    			Serial.print("\txD >");
    			Serial.print(entry.name());
    			Serial.print(" ");
    			strcpy( szNewDir, szDir);
    			strcat( szNewDir, entry.name());
    			strcat( szNewDir, "/");
    			if ( destMedia == destLFS ) myfs.mkdir( entry.name() );
    			else SD.mkdir( entry.name() );
    			Serial.print("\n");
    			errCnt += mediaTransfer(entry, szNewDir, destMedia);
    		} else {
    			uint64_t fileSize, sizeCnt = 0, xfSize=1;
    			char mm[512];
    			strcpy( szNewDir, szDir);
    			strcat( szNewDir, entry.name() );
    			Serial.print("FILE:\t");
    			Serial.print(szNewDir);
    			File dataFile;
    			if ( destMedia == destLFS ) {
    				dataFile = myfs.open(szNewDir, FILE_WRITE_BEGIN);
    			}
    			else {
    				dataFile = SD.open(szNewDir, FILE_WRITE_BEGIN);
    			}
    			if ( dataFile )
    				Serial.print("\td_FILE: OPEN");
    			else
    				Serial.print("\td_FILE: NOT open\n");
    			fileSize = entry.size();
    			while ( entry.available() ) {
    				if ( fileSize < sizeCnt ) break;
    				if ( fileSize - sizeCnt >= sizeof(mm) ) xfSize = sizeof(mm);
    				else xfSize = fileSize - sizeCnt;
    				entry.read( &mm , xfSize );
    				dataFile.write( &mm , xfSize );
    				sizeCnt+=xfSize;
    			}
    			if (fileSize != sizeCnt ) {
    				Serial.printf("\n File Size Error:: %s found %llu Bytes for Size %llu \n", entry.name(), sizeCnt, fileSize);
    			}
    			dataFile.close();
    		}
    		entry.close();
    	}
    	return errCnt;
    }
    PROBLEM: Lack of SD understanding? It only works on ROOT Files and Files in DIRS off the ROOT.

    2nd level dirs not copied? And starting from SD folder directory { szSDdir[] = "/"; } other than ROOT doesn't seem to work as written.

  20. #920
    Senior Member wwatson's Avatar
    Join Date
    Aug 2017
    Posts
    533
    @ defragster - I was able to recreate the issue. SD being a thin wrapper for SdFat I checked the mkdir() function of SdFat and found that there is an extra flag that can be set to create parent sub directories if they do not exist. By default the flag is set to true when the mkdir() function is called. The SD.h mkdir() function does not supply that flag when calling thee SdFat mkdir() function but the default flag is set to true. So it should always create the parent directories. I was using PSRAM.ino that you were using. and getiing the same results you were getting.

    Code:
     STARTING :: SD card copy to LittleFS XFER ...
    
    
    FILE:   /test1.txt
    Opening File (LFS): /test1.txt
            d_FILE: OPEN    inDir >/        xD >a 
    FILE:   /a/test2.txt
    Opening File (LFS): /a/test2.txt
            d_FILE: OPEN    inDir >/a/      xD >b 
    FILE:   /a/b/test3.txt
    Opening File (LFS): /a/b/test3.txt
            d_FILE: NOT open
    
     SD card copy to LittleFS XFER COMPLETE.
    Added a couple of Serial.print's for the actual full path. I just went 2 deep with the sub directories. I am wondering if the directories are actually being created. I guess the next step would be to check that the directories are actually being created. Maybe a printDirectory() before writing the file.

    Don't have time to pursue it further tonight, but am really curious about what is happening. I have a library that I am working on right now that hopefully will be able to unify all of the access to all of the different Teensy storage devices. Just started adding LFS to it and hit some major roadblocks with file manipulation

    Bedtime now...

  21. #921
    Senior Member+ defragster's Avatar
    Join Date
    Feb 2015
    Posts
    15,069
    Quote Originally Posted by wwatson View Post
    @ defragster - I was able to recreate the issue. SD being a thin wrapper for SdFat I checked the mkdir() function of SdFat and found that there is an extra flag that can be set to create parent sub directories if they do not exist. By default the flag is set to true when the mkdir() function is called.

    ...
    Good feedback - thanks for looking. I was between things and had time to do what I did but not look deeper.

    ... that gives me the clue I needed ... READ my own code

    DOH ... OPPS ... I went to great trouble to CREATE the NEW DIR name - and then USED THE WRONG VAR !!!

    Code:
    			strcpy( szNewDir, szDir);
    			strcat( szNewDir, entry.name());
    			strcat( szNewDir, "/");
    			if ( destMedia == destLFS ) myfs.mkdir( szNewDir );
    			else SD.mkdir( szNewDir );
    
    // IT WAS ::
    //			if ( destMedia == destLFS ) myfs.mkdir( entry.name() );
    //			else SD.mkdir( entry.name() );
    
    That now works to copy a PSRAM LFS image to a folder:
    Code:
    G:\>dir /s /b LFS_CPY
    G:\LFS_CPY\0_2MBfile.txt
    G:\LFS_CPY\0_bigfile.txt
    G:\LFS_CPY\1_dir
    G:\LFS_CPY\2_dir
    G:\LFS_CPY\3_dir
    G:\LFS_CPY\4_dir
    G:\LFS_CPY\5_dir
    G:\LFS_CPY\A_file.txt
    G:\LFS_CPY\C_file.txt
    G:\LFS_CPY\D_file.txt
    G:\LFS_CPY\E_file.txt
    G:\LFS_CPY\1_dir\A_file.txt
    G:\LFS_CPY\1_dir\B_file.txt
    G:\LFS_CPY\1_dir\C_file.txt
    G:\LFS_CPY\1_dir\E_file.txt
    G:\LFS_CPY\2_dir\B_file.txt
    G:\LFS_CPY\2_dir\C_file.txt
    G:\LFS_CPY\2_dir\D_file.txt
    G:\LFS_CPY\2_dir\E_file.txt
    G:\LFS_CPY\3_dir\B_file.txt
    G:\LFS_CPY\3_dir\A_file.txt
    G:\LFS_CPY\3_dir\D_file.txt
    G:\LFS_CPY\3_dir\E_file.txt
    G:\LFS_CPY\4_dir\D_file.txt
    G:\LFS_CPY\4_dir\A_file.txt
    G:\LFS_CPY\4_dir\B_file.txt
    G:\LFS_CPY\4_dir\C_file.txt
    G:\LFS_CPY\5_dir\A_file.txt
    G:\LFS_CPY\5_dir\C_file.txt
    G:\LFS_CPY\5_dir\D_file.txt
    G:\LFS_CPY\5_dir\E_file.txt
    Now to go edit to restore from that ROOT "LFS_CPY" subfolder

  22. #922
    Senior Member+ defragster's Avatar
    Join Date
    Feb 2015
    Posts
    15,069
    {edit NOT} IT WORKS!!!! At least to PSRAM ...
    No it doesn't work on copy back - must have a param wrong ...
    >> Looked at the DIR of SD not the DIR of LFS

    In main sketch use ONE of these:
    Code:
    	// xferSD ( destSD ); // do MediaTransfer with SD
    	xferSD ( destLFS ); // do MediaTransfer with SD
    In PSRAM I did the first to build the SD image in p#921

    Than I ran again with edited code {to follow} and restore the SD folder : char szSDdir[] = "LFS_CPY/";

    To the PSRAM drive - in the middle on PC in "LFS_CPY" I added a unique : FILE LFS_CPY_TEST.txt 95

    That file and the rest of the prior LFS image were restored from that folder : char szSDdir[] = "LFS_CPY/";

    >> EXCEPT same code That fills PSRAM from SD not yet right to fill PROG Flash ... ends up with empty and missing subdirs?

    ...

    Need to check the same code is running in both sketch folders ... and monitor the debug spew ...
    Last edited by defragster; 07-20-2021 at 06:19 AM.

  23. #923
    Senior Member+ defragster's Avatar
    Join Date
    Feb 2015
    Posts
    15,069
    Quote Originally Posted by wwatson View Post
    @ defragster - I was able to recreate the issue. SD being a thin wrapper for SdFat I checked the mkdir() function of SdFat and found that there is an extra flag that can be set to create parent sub directories if they do not exist.
    ...
    Bedtime now...
    Back to this > The DIR does not exist when the recursion process passes through the Dir Walk as written.

    Also what works for "Going TO" SD does NOT work coming "BACK TO" LittleFS media - so perhaps the SD code Will make a path to a new subdir, but LFS does NOT.

    Adding this code to test for DIR exist finds it NOT present and it prints out the <mkdir> did the same creating files with <F mkdir>
    Code:
    			strcpy( szNewDir, szDir);
    			if ( destMedia == destLFS ) {
    				if ( !myfs.exists( szNewDir ) ) {
    					Serial.printf("\t<mkdir>\t%s", szNewDir);
    					myfs.mkdir( szNewDir );
    				}
    ...
    Something odd? It works to write to SD , but not to return to LFS PSRAM.

    Here is a ZIP of the PSRAM sketch - Hacked functions.ino to create files and subdirs. BUT TURNED off default 'makerootdirs()' in setup().

    Code:
    >>>
    PSRAM.zip
    <<<
    To create sample PSRAM to put on SD:
    > start with BLANK PSRAM and SD
    > build with setup() line 72 :: xferSD ( destSD ); // do MediaTransfer LFS to SD
    > enter 'm' to makerootdirs and test subdirs
    > enter '3' to create 3 passes of test files
    > WARM RESTART the Teensy and the PSRAM image goes to the SD card
    > DIR of SD will be shown, compare to 'd' DIR of PSRAM

    To copy that SD image to PSRAM:
    > start with above PSRAM image on SD, but enter "F" to format the PSRAM to BLANK
    > build with setup() line 73 :: xferSD ( destLFS ); // do MediaTransfer SD to LFS
    > Note only 'ROOT' files on PSRAM and no DIRS? Note the debug spew for mkdir for "F" files and DIRS

    Is something coded wrong in sketch - or diff behavior with same code that works on SD?

    Bedtime now...

  24. #924
    Senior Member wwatson's Avatar
    Join Date
    Aug 2017
    Posts
    533
    @defragster - I was looking at SdFat exsits() function and it looks like it only tests for files not sub directories.

    Have to go to doctor and then work now. Will look at this more tonight after work.

  25. #925
    Senior Member+ defragster's Avatar
    Join Date
    Feb 2015
    Posts
    15,069
    Quote Originally Posted by wwatson View Post
    @defragster - I was looking at SdFat exsits() function and it looks like it only tests for files not sub directories.

    Have to go to doctor and then work now. Will look at this more tonight after work.
    Interesting - time was short took exist(dir) at face value - it gave expected results - because they don't exist when done either

    Will see if I get back to it before your day gets there ...

Tags for this Thread

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •