Forum Rule: Always post complete source code & details to reproduce any issue!
Page 2 of 38 FirstFirst 1 2 3 4 12 ... LastLast
Results 26 to 50 of 933

Thread: LittleFS port to Teensy/SPIFlash

  1. #26
    Senior Member PaulStoffregen's Avatar
    Join Date
    Nov 2012
    Posts
    24,452
    Quote Originally Posted by mjs513 View Post
    Note: Good point about the FS.h conflict. Need to change that anyway,

    Wonder where the issue is coming from.
    Inside FS.h, I used a pretty different approach than ESP8266. Looks like I either need to overhaul Teensy's FS.h to be much closer to ESP2866's, or this library (and maybe others) need to have parts rewritten.

    I'm playing with the code right now....

  2. #27
    Senior Member+ mjs513's Avatar
    Join Date
    Jul 2014
    Location
    New York
    Posts
    7,196
    Quote Originally Posted by PaulStoffregen View Post
    Inside FS.h, I used a pretty different approach than ESP8266. Looks like I either need to overhaul Teensy's FS.h to be much closer to ESP2866's, or this library (and maybe others) need to have parts rewritten.

    I'm playing with the code right now....
    To be honest my preference is to re-wicker parts of this lib. Just hooking up a SPI flash chip now but still half asleep so may take a while But playing now with the code to see how it works

    UPDATE: Well got this far:
    Code:
    Deleted
    Ok tested on a T4 with the external SPIFlash,
    Code:
    Chip Diagnostics initiated.
    
    Highspeed mode initiated.
    
    SFDP available
    Chip identified using sfdp. Most of this chip's functions are supported by the library.
    No Chip size defined by user. Checking library support.
    Chip identified. This chip is fully supported by the library.
    __blocksize 4096
    Beginning LittleFS test
    Formatting
    Creating 512KB file, may take a while...
    ==> Time to write 512KB in 512b chunks = 5078 milliseconds
    ==> Created file size = 524288
    Reading 512KB file sequentially in 512b chunks
    ==> Time to read 512KB sequentially in 512b chunks = 205 milliseconds = 2557000 bytes/s
    Reading 512KB file MISALIGNED in flash and RAM sequentially in 512b chunks
    ==> Time to read 512KB sequentially MISALIGNED in flash and RAM in 512b chunks = 206 milliseconds = 2545000 bytes/s
    Reading 512KB file in reverse by 512b chunks
    ==> Time to read 512KB in reverse in 512b chunks = 311 milliseconds = 1685000 bytes/s
    Writing 64K file in 1-byte chunks
    ==> Time to write 64KB in 1b chunks = 669 milliseconds = 97000 bytes/s
    Reading 64K file in 1-byte chunks
    ==> Time to read 64KB in 1b chunks = 42 milliseconds = 1560000 bytes/s
    Done!
    now to have some fun with QSPI unless someone beats me to it.
    Last edited by mjs513; 11-05-2020 at 01:25 PM.

  3. #28
    Senior Member+ KurtE's Avatar
    Join Date
    Jan 2014
    Posts
    9,206
    Sounds like you are having some fun!

    Right now playing with other diversion (Serial...)

  4. #29
    Senior Member+ mjs513's Avatar
    Join Date
    Jul 2014
    Location
    New York
    Posts
    7,196
    Quote Originally Posted by KurtE View Post
    Sounds like you are having some fun!

    Right now playing with other diversion (Serial...)
    I noticed

    Keep playing with the OV7670 - have to post the updates soon then think done

  5. #30
    Senior Member PaulStoffregen's Avatar
    Join Date
    Nov 2012
    Posts
    24,452
    Well, I played with making a Teensy-style FS wrapper for LittleFS. It's not yet working well enough for anything more than glancing at the code to see how different the style is from ESP8266.

    https://github.com/PaulStoffregen/LittleFS

    But here's a little test program which tries to create a file, write some data, then read it, write some more, and read it all back. It gets stuck about half way, because I've obviously not done something quite right.

    Code:
    #include <LittleFS.h>
    
    LittleFS_RAM myfs;
    char buf[200000];
    
    void setup() {
      pinMode(13, OUTPUT);
      digitalWrite(13, HIGH);
      while (!Serial) ; // wait
      Serial.println("LittleFS Test"); delay(5);
      if (!myfs.begin(buf, sizeof(buf))) {
        Serial.println("Error starting ramdisk");
        while (1) ;
      }
      Serial.println("started");
      File f = myfs.open("test.txt", FILE_WRITE);
      Serial.println("opened test.txt");
      f.println("This is a test....");
      Serial.println("wrote to file");
      //f.whoami();
      f.close();
      Serial.println();
    
      File in = myfs.open("test.txt");
      if (in) Serial.println("opened file for read");
      char buf[256];
      int n = in.read(buf, 256);
      Serial.printf("read %d bytes\n", n);
      if (n > 255) n = 255;
      buf[n] = 0;
      Serial.println(buf);
      Serial.println("done printing, now close the file");
      in.close();       //    <--- gets stuck here
      Serial.println();
    
      Serial.println("try writing more");
      f = myfs.open("test.txt", FILE_WRITE);
      if (f) Serial.println("opened test.txt again for writing");
      f.println("another test");
      Serial.println("after write");
      f.whoami();
      f.close();
      Serial.println();
    
      in = myfs.open("test.txt");
      if (in) Serial.println("opened file for read");
      char buf2[256];
      n = in.read(buf2, 256);
      Serial.printf("read %d bytes\n", n);
      if (n > 255) n = 255;
      buf[n] = 0;
      Serial.println(buf2);
      in.close();
    
    }
    
    
    void loop() {
    }

    So far this only supports a ramdisk.

  6. #31
    Senior Member
    Join Date
    Apr 2020
    Location
    DFW area in Texas
    Posts
    265
    Quote Originally Posted by PaulStoffregen View Post
    Well, I played with making a Teensy-style FS wrapper for LittleFS. It's not yet working well enough for anything more than glancing at the code to see how different the style is from ESP8266.

    https://github.com/PaulStoffregen/LittleFS

    But here's a little test program which tries to create a file, write some data, then read it, write some more, and read it all back. It gets stuck about half way, because I've obviously not done something quite right.

    Code:
    #include <LittleFS.h>
    
    LittleFS_RAM myfs;
    char buf[200000];               //  <<<
    
    void setup() {
      pinMode(13, OUTPUT);
      digitalWrite(13, HIGH);
      while (!Serial) ; // wait
      Serial.println("LittleFS Test"); delay(5);
      if (!myfs.begin(buf, sizeof(buf))) {
        Serial.println("Error starting ramdisk");
        while (1) ;
      }
      Serial.println("started");
      File f = myfs.open("test.txt", FILE_WRITE);
      Serial.println("opened test.txt");
      f.println("This is a test....");
      Serial.println("wrote to file");
      //f.whoami();
      f.close();
      Serial.println();
    
      File in = myfs.open("test.txt");
      if (in) Serial.println("opened file for read");
      char buf[256];
      int n = in.read(buf, 256);
      Serial.printf("read %d bytes\n", n);
      if (n > 255) n = 255;
      buf[n] = 0;
      Serial.println(buf);
      Serial.println("done printing, now close the file");
      in.close();       //    <--- gets stuck here
      Serial.println();
    
      Serial.println("try writing more");
      f = myfs.open("test.txt", FILE_WRITE);
      if (f) Serial.println("opened test.txt again for writing");
      f.println("another test");
      Serial.println("after write");
      f.whoami();
      f.close();
      Serial.println();
    
      in = myfs.open("test.txt");
      if (in) Serial.println("opened file for read");
      char buf2[256];
      n = in.read(buf2, 256);
      Serial.printf("read %d bytes\n", n);
      if (n > 255) n = 255;
      buf2[n] = 0;          // <<<================  was buf[n] = 0;
      Serial.println(buf2);
      in.close();
    
    }
    
    
    void loop() {
    }

    So far this only supports a ramdisk.
    Paul:

    I am following this discussion with interest. I see that your test program is getting stuck at the point where it is trying to close the second file handle.

    This is completely unrelated to the results that you are seeing, but to potentially save some future headache, there seems to be a simple typo as annotated in the code listing above (buf[n] = 0; should be buf2[n] = 0;).

    Thanks for working/playing with this & I'm looking forward to the potential of this added capability.

    Mark J Culross
    KD5RXT
    Last edited by defragster; 11-05-2020 at 04:45 PM. Reason: disable SMILES

  7. #32
    Senior Member+ defragster's Avatar
    Join Date
    Feb 2015
    Posts
    14,421
    Switching to github.com/PaulStoffregen/LittleFS sketch in p#30 working ... as far as it gets.
    T_4.1 on IDE 18.13 with TD 1.54b4::
    Code:
    T:\tCode\littleFS\lfsRAM\lfsRAM.ino Nov  5 2020 11:06:23
    LittleFS Test
    started
    opened test.txt
    wrote to file
      close regular file
      end of close
    
    opened file for read
    read 20 bytes
    This is a test....
    
    done printing, now close the file
      close regular file
    TSET IDE Portable build with PJRC loader - lots of CODE [Sketch uses 53296 bytes] - using Dual_Serial to allow TyComm SerMon.
    Uses only PJRC for RAMDISK:: libraries\LittleFS-main
    Code:
    Building PORTABLE: T:\arduino-1.8.13_p54\portable\sketchbook\libraries 
    Building Sketch: ".\lfsRAM.ino"
    Using board 'teensy41' from platform in folder: T:\arduino-1.8.13_p54\hardware\teensy\avr
    Using core 'teensy4' from platform in folder: T:\arduino-1.8.13_p54\hardware\teensy\avr
    Detecting libraries used...
    "T:\\arduino-1.8.13_p54\\hardware\\teensy/../tools/arm/bin/arm-none-eabi-g++" -E -CC -x c++ -w -g -Wall -ffunction-sections -fdata-sections -nostdlib -std=gnu++14 -fno-exceptions -fpermissive -fno-rtti -fno-threadsafe-statics -felide-constructors -Wno-error=narrowing -mthumb -mcpu=cortex-m7 -mfloat-abi=hard -mfpu=fpv5-d16 -D__IMXRT1062__ -DTEENSYDUINO=154 -DARDUINO=10600 -DARDUINO_TEENSY41 -DF_CPU=600000000 -DUSB_DUAL_SERIAL -DLAYOUT_US_ENGLISH "-IT:\\arduino-1.8.13_p54\\hardware\\teensy\\avr\\cores\\teensy4" "T:\\TEMP\\arduino_build_lfsRAM.ino\\sketch\\lfsRAM.ino.cpp" -o nul -DARDUINO_LIB_DISCOVERY_PHASE
    Alternatives for LittleFS.h: [LittleFS-main@1.0.0]
    ResolveLibrary(LittleFS.h)
      -> candidates: [LittleFS-main@1.0.0]
    "T:\\arduino-1.8.13_p54\\hardware\\teensy/../tools/arm/bin/arm-none-eabi-g++" -E -CC -x c++ -w -g -Wall -ffunction-sections -fdata-sections -nostdlib -std=gnu++14 -fno-exceptions -fpermissive -fno-rtti -fno-threadsafe-statics -felide-constructors -Wno-error=narrowing -mthumb -mcpu=cortex-m7 -mfloat-abi=hard -mfpu=fpv5-d16 -D__IMXRT1062__ -DTEENSYDUINO=154 -DARDUINO=10600 -DARDUINO_TEENSY41 -DF_CPU=600000000 -DUSB_DUAL_SERIAL -DLAYOUT_US_ENGLISH "-IT:\\arduino-1.8.13_p54\\hardware\\teensy\\avr\\cores\\teensy4" "-IT:\\arduino-1.8.13_p54\\portable\\sketchbook\\libraries\\LittleFS-main\\src" "T:\\TEMP\\arduino_build_lfsRAM.ino\\sketch\\lfsRAM.ino.cpp" -o nul -DARDUINO_LIB_DISCOVERY_PHASE
    Using cached library dependencies for file: T:\arduino-1.8.13_p54\portable\sketchbook\libraries\LittleFS-main\src\LittleFS.cpp
    Using cached library dependencies for file: T:\arduino-1.8.13_p54\portable\sketchbook\libraries\LittleFS-main\src\littlefs\lfs.c
    Using cached library dependencies for file: T:\arduino-1.8.13_p54\portable\sketchbook\libraries\LittleFS-main\src\littlefs\lfs_util.c
    Generating function prototypes...
    "T:\\arduino-1.8.13_p54\\hardware\\teensy/../tools/arm/bin/arm-none-eabi-g++" -E -CC -x c++ -w -g -Wall -ffunction-sections -fdata-sections -nostdlib -std=gnu++14 -fno-exceptions -fpermissive -fno-rtti -fno-threadsafe-statics -felide-constructors -Wno-error=narrowing -mthumb -mcpu=cortex-m7 -mfloat-abi=hard -mfpu=fpv5-d16 -D__IMXRT1062__ -DTEENSYDUINO=154 -DARDUINO=10600 -DARDUINO_TEENSY41 -DF_CPU=600000000 -DUSB_DUAL_SERIAL -DLAYOUT_US_ENGLISH "-IT:\\arduino-1.8.13_p54\\hardware\\teensy\\avr\\cores\\teensy4" "-IT:\\arduino-1.8.13_p54\\portable\\sketchbook\\libraries\\LittleFS-main\\src" "T:\\TEMP\\arduino_build_lfsRAM.ino\\sketch\\lfsRAM.ino.cpp" -o "T:\\TEMP\\arduino_build_lfsRAM.ino\\preproc\\ctags_target_for_gcc_minus_e.cpp" -DARDUINO_LIB_DISCOVERY_PHASE
    "T:\\arduino-1.8.13_p54\\tools-builder\\ctags\\5.8-arduino11/ctags" -u --language-force=c++ -f - --c++-kinds=svpf --fields=KSTtzns --line-directives "T:\\TEMP\\arduino_build_lfsRAM.ino\\preproc\\ctags_target_for_gcc_minus_e.cpp"
    Compiling sketch...
    "T:\\arduino-1.8.13_p54\\hardware\\teensy/../tools/precompile_helper" "T:\\arduino-1.8.13_p54\\hardware\\teensy\\avr/cores/teensy4" "T:\\TEMP\\arduino_build_lfsRAM.ino" "T:\\arduino-1.8.13_p54\\hardware\\teensy/../tools/arm/bin/arm-none-eabi-g++" -x c++-header -O2 -g -Wall -ffunction-sections -fdata-sections -nostdlib -MMD -std=gnu++14 -fno-exceptions -fpermissive -fno-rtti -fno-threadsafe-statics -felide-constructors -Wno-error=narrowing -mthumb -mcpu=cortex-m7 -mfloat-abi=hard -mfpu=fpv5-d16 -D__IMXRT1062__ -DTEENSYDUINO=154 -DARDUINO=10600 -DARDUINO_TEENSY41 -DF_CPU=600000000 -DUSB_DUAL_SERIAL -DLAYOUT_US_ENGLISH "-IT:\\arduino-1.8.13_p54\\hardware\\teensy\\avr/cores/teensy4" "T:\\TEMP\\arduino_build_lfsRAM.ino/pch/Arduino.h" -o "T:\\TEMP\\arduino_build_lfsRAM.ino/pch/Arduino.h.gch"
    Using previously compiled file: T:\TEMP\arduino_build_lfsRAM.ino\pch\Arduino.h.gch
    "T:\\arduino-1.8.13_p54\\hardware\\teensy/../tools/arm/bin/arm-none-eabi-g++" -c -O2 -g -Wall -ffunction-sections -fdata-sections -nostdlib -MMD -std=gnu++14 -fno-exceptions -fpermissive -fno-rtti -fno-threadsafe-statics -felide-constructors -Wno-error=narrowing -mthumb -mcpu=cortex-m7 -mfloat-abi=hard -mfpu=fpv5-d16 -D__IMXRT1062__ -DTEENSYDUINO=154 -DARDUINO=10600 -DARDUINO_TEENSY41 -DF_CPU=600000000 -DUSB_DUAL_SERIAL -DLAYOUT_US_ENGLISH "-IT:\\TEMP\\arduino_build_lfsRAM.ino/pch" "-IT:\\arduino-1.8.13_p54\\hardware\\teensy\\avr\\cores\\teensy4" "-IT:\\arduino-1.8.13_p54\\portable\\sketchbook\\libraries\\LittleFS-main\\src" "T:\\TEMP\\arduino_build_lfsRAM.ino\\sketch\\lfsRAM.ino.cpp" -o "T:\\TEMP\\arduino_build_lfsRAM.ino\\sketch\\lfsRAM.ino.cpp.o"
    Compiling libraries...
    Compiling library "LittleFS-main"
    Using previously compiled file: T:\TEMP\arduino_build_lfsRAM.ino\libraries\LittleFS-main\LittleFS.cpp.o
    Using previously compiled file: T:\TEMP\arduino_build_lfsRAM.ino\libraries\LittleFS-main\littlefs\lfs_util.c.o
    Using previously compiled file: T:\TEMP\arduino_build_lfsRAM.ino\libraries\LittleFS-main\littlefs\lfs.c.o
    Compiling core...
    Using precompiled core: T:\TEMP\arduino_cache_lfsRAM.ino\core\core_770d4e3ac5403584e529082c30c0f08e.a
    Linking everything together...
    "T:\\arduino-1.8.13_p54\\hardware\\teensy/../tools/arm/bin/arm-none-eabi-gcc" -O2 -Wl,--gc-sections,--relax "-TT:\\arduino-1.8.13_p54\\hardware\\teensy\\avr\\cores\\teensy4/imxrt1062_t41.ld" -mthumb -mcpu=cortex-m7 -mfloat-abi=hard -mfpu=fpv5-d16 -o "T:\\TEMP\\arduino_build_lfsRAM.ino/lfsRAM.ino.elf" "T:\\TEMP\\arduino_build_lfsRAM.ino\\sketch\\lfsRAM.ino.cpp.o" "T:\\TEMP\\arduino_build_lfsRAM.ino\\libraries\\LittleFS-main\\LittleFS.cpp.o" "T:\\TEMP\\arduino_build_lfsRAM.ino\\libraries\\LittleFS-main\\littlefs\\lfs.c.o" "T:\\TEMP\\arduino_build_lfsRAM.ino\\libraries\\LittleFS-main\\littlefs\\lfs_util.c.o" "T:\\TEMP\\arduino_build_lfsRAM.ino/..\\arduino_cache_lfsRAM.ino\\core\\core_770d4e3ac5403584e529082c30c0f08e.a" "-LT:\\TEMP\\arduino_build_lfsRAM.ino" -larm_cortexM7lfsp_math -lm -lstdc++
    "T:\\arduino-1.8.13_p54\\hardware\\teensy/../tools/arm/bin/arm-none-eabi-objcopy" -O ihex -j .eeprom --set-section-flags=.eeprom=alloc,load --no-change-warnings --change-section-lma .eeprom=0 "T:\\TEMP\\arduino_build_lfsRAM.ino/lfsRAM.ino.elf" "T:\\TEMP\\arduino_build_lfsRAM.ino/lfsRAM.ino.eep"
    "T:\\arduino-1.8.13_p54\\hardware\\teensy/../tools/arm/bin/arm-none-eabi-objcopy" -O ihex -R .eeprom "T:\\TEMP\\arduino_build_lfsRAM.ino/lfsRAM.ino.elf" "T:\\TEMP\\arduino_build_lfsRAM.ino/lfsRAM.ino.hex"
    "T:\\arduino-1.8.13_p54\\hardware\\teensy/../tools/stdout_redirect" "T:\\TEMP\\arduino_build_lfsRAM.ino/lfsRAM.ino.lst" "T:\\arduino-1.8.13_p54\\hardware\\teensy/../tools/arm/bin/arm-none-eabi-objdump" -d -S -C "T:\\TEMP\\arduino_build_lfsRAM.ino/lfsRAM.ino.elf"
    "T:\\arduino-1.8.13_p54\\hardware\\teensy/../tools/stdout_redirect" "T:\\TEMP\\arduino_build_lfsRAM.ino/lfsRAM.ino.sym" "T:\\arduino-1.8.13_p54\\hardware\\teensy/../tools/arm/bin/arm-none-eabi-objdump" -t -C "T:\\TEMP\\arduino_build_lfsRAM.ino/lfsRAM.ino.elf"
    "T:\\arduino-1.8.13_p54\\hardware\\teensy/../tools/teensy_post_compile" -file=lfsRAM.ino "-path=T:\\TEMP\\arduino_build_lfsRAM.ino" "-tools=T:\\arduino-1.8.13_p54\\hardware\\teensy/../tools/" -board=TEENSY41
    Opening Teensy Loader...
    Using library LittleFS-main at version 1.0.0 in folder: T:\arduino-1.8.13_p54\portable\sketchbook\libraries\LittleFS-main 
    "T:\\arduino-1.8.13_p54\\hardware\\teensy/../tools/arm/bin/arm-none-eabi-size" -A "T:\\TEMP\\arduino_build_lfsRAM.ino/lfsRAM.ino.elf"
    Sketch uses 53296 bytes (0%) of program storage space. Maximum is 8126464 bytes.
    Global variables use 279348 bytes (53%) of dynamic memory, leaving 244940 bytes for local variables. Maximum is 524288 bytes.
           upload@7684140-Teensy  Uploading to board '7684140-Teensy' (Teensy 4.1)
           upload@7684140-Teensy  Waiting for Teensy Loader
           upload@7684140-Teensy  Failed to reset board '7684140-Teensy'
    estack:20070000 ebss:20034340
    
    FlexRAM section ITCM+DTCM = 512 KB
        Config : aaaaaaaf (DDDDDDDDDDDDDDII)
        ITCM :  41192 B	(62.85% of   64 KB)
        DTCM : 213824 B	(46.61% of  448 KB)
        Available for Stack: 244928
    OCRAM: 512KB
        DMAMEM:  24736 B	( 4.72% of  512 KB)
        Available for Heap: 499552 B	(95.28% of  512 KB)
    Flash:  56376 B	( 0.69% of 7936 KB)
    [Finished in 13.1s]

  8. #33
    Senior Member+ mjs513's Avatar
    Join Date
    Jul 2014
    Location
    New York
    Posts
    7,196
    Quote Originally Posted by PaulStoffregen View Post
    Well, I played with making a Teensy-style FS wrapper for LittleFS. It's not yet working well enough for anything more than glancing at the code to see how different the style is from ESP8266.

    https://github.com/PaulStoffregen/LittleFS

    But here's a little test program which tries to create a file, write some data, then read it, write some more, and read it all back. It gets stuck about half way, because I've obviously not done something quite right.

    Code:
    #include <LittleFS.h>
    
    LittleFS_RAM myfs;
    char buf[200000];
    
    void setup() {
      pinMode(13, OUTPUT);
      digitalWrite(13, HIGH);
      while (!Serial) ; // wait
      Serial.println("LittleFS Test"); delay(5);
      if (!myfs.begin(buf, sizeof(buf))) {
        Serial.println("Error starting ramdisk");
        while (1) ;
      }
      Serial.println("started");
      File f = myfs.open("test.txt", FILE_WRITE);
      Serial.println("opened test.txt");
      f.println("This is a test....");
      Serial.println("wrote to file");
      //f.whoami();
      f.close();
      Serial.println();
    
      File in = myfs.open("test.txt");
      if (in) Serial.println("opened file for read");
      char buf[256];
      int n = in.read(buf, 256);
      Serial.printf("read %d bytes\n", n);
      if (n > 255) n = 255;
      buf[n] = 0;
      Serial.println(buf);
      Serial.println("done printing, now close the file");
      in.close();       //    <--- gets stuck here
      Serial.println();
    
      Serial.println("try writing more");
      f = myfs.open("test.txt", FILE_WRITE);
      if (f) Serial.println("opened test.txt again for writing");
      f.println("another test");
      Serial.println("after write");
      f.whoami();
      f.close();
      Serial.println();
    
      in = myfs.open("test.txt");
      if (in) Serial.println("opened file for read");
      char buf2[256];
      n = in.read(buf2, 256);
      Serial.printf("read %d bytes\n", n);
      if (n > 255) n = 255;
      buf[n] = 0;
      Serial.println(buf2);
      in.close();
    
    }
    
    
    void loop() {
    }

    So far this only supports a ramdisk.
    Been looking at it but sure where the bug is unless its some strange interaction when using extmem_malloc with littleFS.

    Tried just writing, then close then write some, close and then read (skipped the first read). Seems to hang at the SECOND CLOSE?

    Code:
    #include <LittleFS.h>
    
    LittleFS_RAM myfs;
    char buf[200000];
    
    void setup() {
      pinMode(13, OUTPUT);
      digitalWrite(13, HIGH);
      while (!Serial) ; // wait
      Serial.println("LittleFS Test"); delay(5);
      if (!myfs.begin(buf, sizeof(buf))) {
        Serial.println("Error starting ramdisk");
        while (1) ;
      }
      Serial.println("started");
      File f = myfs.open("test.txt", FILE_WRITE);
      Serial.println("opened test.txt");
      f.println("This is a test....");
      Serial.println("wrote to file");
      //f.whoami();
      f.close();
      Serial.println();
    
      File in = myfs.open("test.txt");
      if (in) Serial.println("opened file for read");
      char buf[256];
      int n = in.read(buf, 256);
      Serial.printf("read %d bytes\n", n);
      if (n > 255) n = 255;
      buf[n] = 0;
      Serial.println(buf);
      Serial.println("done printing, now close the file");
      in.close();       //    <--- gets stuck here
      Serial.println();
    
      Serial.println("try writing more");
      f = myfs.open("test.txt", FILE_WRITE);
      if (f) Serial.println("opened test.txt again for writing");
      f.println("another test");
      Serial.println("after write");
      f.whoami();
      f.close();
      Serial.println();
    
      in = myfs.open("test.txt");
      if (in) Serial.println("opened file for read");
      char buf2[256];
      n = in.read(buf2, 256);
      Serial.printf("read %d bytes\n", n);
      if (n > 255) n = 255;
      buf[n] = 0;
      Serial.println(buf2);
      in.close();
    
    }
    
    void loop() {
    }
    Could be that the file is never really closed after the first close?

    Should open be in FS class or vice versa close in FILE class? This is where i get lost with classes like this/

  9. #34
    Senior Member PaulStoffregen's Avatar
    Join Date
    Nov 2012
    Posts
    24,452
    Fixed the crash on 2nd close problem.

    https://github.com/PaulStoffregen/Li...8e752e316e37fc

  10. #35
    Senior Member+ mjs513's Avatar
    Join Date
    Jul 2014
    Location
    New York
    Posts
    7,196
    Quote Originally Posted by PaulStoffregen View Post
    Fixed the crash on 2nd close problem.

    https://github.com/PaulStoffregen/Li...8e752e316e37fc
    Changing const and is_file was the fix --- I need to go take a nap

    UPDATE:
    Had to try it:
    Code:
    LittleFS Test
    started
    
    opened test.txt
    wrote to file
    
    opened file for read
    read 20 bytes
    This is a test....
    
    done printing, now close the file
    
    try writing more
    
    opened test.txt again for writing
    after write
      File    this=2006fdbc, f=20203230
      LittleFSFile this=20203230, refcount=1
    
    opened file for read
    read 34 bytes
    This is a test....
    another test
    Out of curiosity how do you want to hand QSPI Flash and SPIFlash? ESP8266 handles it one way but not sure you want to use the same approach?

  11. #36
    Senior Member+ defragster's Avatar
    Join Date
    Feb 2015
    Posts
    14,421
    Quote Originally Posted by mjs513 View Post
    Changing const and is_file was the fix --- I need to go take a nap

    UPDATE:
    Had to try it:
    ...
    Same results here on T_4.1.

  12. #37
    Senior Member PaulStoffregen's Avatar
    Join Date
    Nov 2012
    Posts
    24,452
    Quote Originally Posted by mjs513 View Post
    Changing const and is_file was the fix
    Nope, that wasn't the problem, just a bunch of code that needed to change to accommodate the actual fix.

    The root problem my use of memcpy() for the lfs_file_t data in LittleFSFile constructor. Once a file is opened, it's lfs_file_t stuct must not be copied. This isn't really documented anywhere, but looking at the lfs_file_t typedef in lfs.h, I noticed there's a "next" pointer with type struct lfs_file *. Looks like lfs_file_open() and lfs_file_close() are maintaining a linked list of all the structs for open files.

    Inside LittleFS.open(), I had created a lfs_file_t struct as a local variable and passed its address to lfs_file_open(). If lfs_file_open() was successful, then I created the LittleFSFile instance and passed its address, where the LittleFSFile would use malloc to make a copy. Then much later, after the 2nd close, the linked list pointers in the copied data would be pointing back to some place on the stack where that temporary copy had been long ago.

    The crash manifests on the 2nd close because a linked list with a single item is usually handled as a case where the link pointers are null or some other special case to know there are no other items on the list. So it was (probably) only accessing the prior stack memory after 2 files were open.

    So the solution is to call malloc() to allocate the lfs_file_t struct on the heap, where it will remain until the file is closed. If the open isn't successful, then the memory is freed. Originally I had wanted to avoid a malloc & free for an unsuccessful open, but that just doesn't seem practical given the way lfs_file_open() and lfs_file_close() actually work.

    As a result of the memory being already allocated before calling the LittleFSFile constructor, I removed const from the pointer because we're now just storing the pointer rather than a copy of the data it references, so we do need the pointer to be writable. I had used const because the constructor wasn't writing to the original, as it was (illegally) making a copy.

    The 2 booleans I had created to track whether we had an open file or an open directory became redundant now that we're storing pointers rather than copies. So I deleted them and changed all the places using those booleans to just check whether the pointers are nullptr.

    Ultimately the real problem is you can't copy a lfs_file_t struct while the file is open. But until about an hour ago, I didn't know that.
    Last edited by PaulStoffregen; 11-05-2020 at 08:42 PM. Reason: added a bit more info

  13. #38
    Senior Member+ KurtE's Avatar
    Join Date
    Jan 2014
    Posts
    9,206
    Sounds great, and downloaded the updates and it works... (I finished my previous diversion )

  14. #39
    Senior Member+ mjs513's Avatar
    Join Date
    Jul 2014
    Location
    New York
    Posts
    7,196
    @PaulStoffregen

    Thanks for the explanation - it really helps me understanding what was going with LittleFS but also with c/c++. PS - also I didn't scroll down enough in commit to see the big change. Told you needed that nap.

  15. #40
    Senior Member PaulStoffregen's Avatar
    Join Date
    Nov 2012
    Posts
    24,452
    Glad you got some rest.

    Truth is, I worked on this all night and got stuck on the 2nd close crash. Then I got 2 hours sleep and woke up obsessing about memory allocation.

    Now I'm filling in the directory features....

  16. #41
    Senior Member+ mjs513's Avatar
    Join Date
    Jul 2014
    Location
    New York
    Posts
    7,196
    Quote Originally Posted by PaulStoffregen View Post
    Glad you got some rest.

    Truth is, I worked on this all night and got stuck on the 2nd close crash. Then I got 2 hours sleep and woke up obsessing about memory allocation.

    Now I'm filling in the directory features....
    That usually happens with me too. Go to sleep thinking about the problem and then waking up and figuring out how to fix it. Amazing what sleep does for you.

    I started looking at extending your LittleFS_RAM class to a LittFS_SPIFlash class using either the SerialFlash class or the SPImemory classes as a second base class. Not 100% sure which way to go on this one.

  17. #42
    Senior Member PaulStoffregen's Avatar
    Join Date
    Nov 2012
    Posts
    24,452
    I've added support for reading file names, so now the normal directory printing code should work.

    https://github.com/PaulStoffregen/Li...bbf1934eb35536

    Here's a growing little test program.

    Code:
    #include <LittleFS.h>
    
    LittleFS_RAM myfs;
    char buf[200000];
    
    void setup() {
      pinMode(13, OUTPUT);
      digitalWrite(13, HIGH);
      while (!Serial) ; // wait
      Serial.println("LittleFS Test"); delay(5);
      if (!myfs.begin(buf, sizeof(buf))) {
        Serial.println("Error starting ramdisk");
        while (1) ;
      }
      Serial.println("started");
      File f = myfs.open("test.txt", FILE_WRITE);
      Serial.println("opened test.txt");
      f.println("This is a test....");
      Serial.println("wrote to file");
      //f.whoami();
      f.close();
      Serial.println();
    
      File in = myfs.open("test.txt");
      if (in) Serial.println("opened file for read");
      char buf[256];
      int n = in.read(buf, 256);
      Serial.printf("read %d bytes\n", n);
      if (n > 255) n = 255;
      buf[n] = 0;
      Serial.println(buf);
      Serial.println("done printing, now close the file");
      in.close();
      Serial.println();
    
      Serial.println("try writing more");
      f = myfs.open("test.txt", FILE_WRITE);
      if (f) Serial.println("opened test.txt again for writing");
      f.println("another test");
      Serial.println("after write");
      f.whoami();
      f.close();
      Serial.println();
    
      in = myfs.open("test.txt");
      if (in) Serial.println("opened file for read");
      char buf2[256];
      n = in.read(buf2, 256);
      Serial.printf("read %d bytes\n", n);
      if (n > 255) n = 255;
      buf2[n] = 0;
      Serial.println(buf2);
      in.close();
    
      delay(10);
      printDirectory();
    
      Serial.println("create folder");
      if (myfs.mkdir("myfolder")) {
        Serial.println("  success");
      } else {
        Serial.println("  failed");
      }
      Serial.println();
    
      printDirectory();
    
      Serial.println("write to file in folder");
      f = myfs.open("/myfolder/morestuff.txt", FILE_WRITE);
      if (f) Serial.println("opened myfolder/morestuff.txt for write");
      f.println("some more text to write into 2nd file");
      f.close();
      Serial.println();
    
      printDirectory();
      
    }
    
    
    void printDirectory() {
      Serial.println("printDirectory\n--------------");
      printDirectory(myfs.open("/"), 0);
      Serial.println();
    }
    
    void printDirectory(File dir, int numTabs) {
      //dir.whoami();
      while (true) {
    
        static int count=0;
        if (++count > 20) while (1) ;
        
        File entry =  dir.openNextFile();
        if (! entry) {
          // no more files
          //Serial.println("**nomorefiles**");
          break;
        }
        for (uint8_t i = 0; i < numTabs; i++) {
          Serial.print('\t');
        }
        Serial.print(entry.name());
        if (entry.isDirectory()) {
          Serial.println(" / ");
          printDirectory(entry, numTabs + 1);
        } else {
          // files have sizes, directories do not
          Serial.print("\t\t");
          Serial.println(entry.size(), DEC);
        }
        entry.close();
      }
    }
    
    
    void loop() {
    }
    If anyone wants to *really* help, expanding this test to create more files and use/test the other functions would really be useful. I'm sure several have bugs. Would be really nice to get those things fixed while using the ramdisk.

    Another minor issue to consider is maximum file names and path name lengths. So far I've set those at 39 and 127 chars.

  18. #43
    Senior Member+ mjs513's Avatar
    Join Date
    Jul 2014
    Location
    New York
    Posts
    7,196
    Quote Originally Posted by PaulStoffregen View Post
    I've added support for reading file names, so now the normal directory printing code should work.

    https://github.com/PaulStoffregen/Li...bbf1934eb35536

    Here's a growing little test program.

    ........

    If anyone wants to *really* help, expanding this test to create more files and use/test the other functions would really be useful. I'm sure several have bugs. Would be really nice to get those things fixed while using the ramdisk.

    Another minor issue to consider is maximum file names and path name lengths. So far I've set those at 39 and 127 chars.
    As for file names the max to support LFN is 255 but don't think that is necessary (sure someone would disagree. Personally 39 and 127 is reasonable.

    Will start playing with the new functions to see what I can break

  19. #44
    Senior Member+ mjs513's Avatar
    Join Date
    Jul 2014
    Location
    New York
    Posts
    7,196
    Started playing with the different functions in the wrapper. One thing I noticed is that a EOF would be a nice thing to have. Say nice because the workaround would be at an end of of a loop to read x records you would need to test:
    Code:
          if((file3.position()+1) > file3.size()) break;
    which in the case I am going to post would be eof test to break the loop.

    Converting one of SPIFFS test cases over LittleFS which tested a few functions like if print, println and printfs would work and also if writing structured data would, seems to be working but will be expanding. Sample output:
    Code:
    D:\Users\Merli\Documents\Arduino\littlefs_teensy_test3\littlefs_teensy_test3.ino Nov  5 2020 21:17:38
    LittleFS Test
    printDirectory
    --------------
    
    Using println and printf to printoutput file
    opened PRINTOUTPUT1.txt
    opened PRINTOUTPUT2.txt
    printDirectory
    --------------
    PRINTOUTPUT1.txt		628
    PRINTOUTPUT2.txt		630
    
    -------------------------------
    File1 contents:
    -------------------------------
    File1 size: 628
    abcdefghijklmnopqrstuvwxyz
    Rec: 0, Float: 26.400000, Int: 98
    abcdefghijklmnopqrstuvwxyz
    Rec: 1, Float: 27.400000, Int: 99
    abcdefghijklmnopqrstuvwxyz
    Rec: 2, Float: 28.400000, Int: 100
    abcdefghijklmnopqrstuvwxyz
    Rec: 3, Float: 29.400000, Int: 101
    abcde
    ghijklmnopqrstuvwxyz
    Rec: 4, Float: 30.400000, Int: 102
    abcdefghijklmnopqrstuvwxyz
    Rec: 5, Float: 31.400000, Int: 103
    abcdefghijklmnopqrstuvwxyz
    Rec: 6, Float: 32.400000, Int: 104
    abcdefghijklmnopqrstuvwxyz
    Rec: 7, Float: 33.400000, Int: 105
    abcdefghi
    klmnopqrstuvwxyz
    Rec: 8, Float: 34.400000, Int: 106
    abcdefghijklmnopqrstuvwxyz
    Rec: 9, Float: 35.400000, Int: 107
    
    -------------------------------
    File3 byte conversion test:
    -------------------------------
    
    Data_0: true
    Data_1: 1.3574999571
    Data_2: 314159
    Data_3: 142
    The Quick Brown Fox
    Init Done - array loaded
    ...... ...... ......
    
    2nd Directory contents:
    printDirectory
    --------------
    PRINTOUTPUT1.txt		628
    PRINTOUTPUT2.txt		630
    logger.txt		480
    
    
    true, 1.3574999571, 314159, 142, The Quick Brown Fox
    {NOTE THIS LINE IS REPEATED 9 more times in the SERMON]
    Hmmmm interesting on the copy paste doesn't copy all from the serial monitor. Should be 9 more lines of the last line - shows in sermon but not when I pasted here.

    Here is the sketch to duplicate it.
    Code:
    #include <LittleFS.h>
    
    LittleFS_RAM myfs;
    char buf[200000];
    
    File file1, file2, file3;
    
    char fname[32] = "my_file1";
    int szLen = strlen( buf );
    elapsedMicros my_us;
    
    //define a struct of various data types
     struct MYDATA_t {
      bool data_0;
      float data_1; 
      long data_2; 
      int data_3;
      char data_4[32];
    };
    
    //define a struct joining MYDATA_t to an array of bytes to be stored
     union MYDATA4RAM_t {
     MYDATA_t datastruct;
     char Packet[sizeof(MYDATA_t)];
    };
    
    MYDATA4RAM_t mydata; //data to be written in memory
    MYDATA4RAM_t readdata; //data read from memory
    
    
    void setup() {
      pinMode(13, OUTPUT);
      digitalWrite(13, HIGH);
      while (!Serial) ; // wait
        Serial.println("\n" __FILE__ " " __DATE__ " " __TIME__);
      Serial.println("LittleFS Test"); delay(5);
      
      if (!myfs.begin(buf, sizeof(buf))) {
        Serial.println("Error starting ramdisk");
        while (1) ;
      }
    
      printDirectory();
    
      //test of print
      Serial.println("Using println and printf to printoutput file");
      file1 = myfs.open("PRINTOUTPUT1.txt", FILE_WRITE);
      Serial.println("opened PRINTOUTPUT1.txt");
      file2 = myfs.open("PRINTOUTPUT2.txt", FILE_WRITE);
      Serial.println("opened PRINTOUTPUT2.txt");
    
      for(uint8_t i = 0; i < 10; i++) {
        file1.println("abcdefghijklmnopqrstuvwxyz");
        file1.printf("Rec: %d, Float: %f, Int: %d\n",i,i+26.4, i+98);
        file2.println("ABCDEFGHIJKLMNOPQRSTUVWXYZ");
        //eRAM.println("THIS IS A TEST");
        file2.printf("Rec: %d, Float: %f, Int: %d\n",i,i+56.4, i+198);
      }
    
      file1.close();
      file2.close();
    
      printDirectory();
      
    
      Serial.println("-------------------------------");
      Serial.println("File1 contents:");
      Serial.println("-------------------------------");
    
      file1 = myfs.open("PRINTOUTPUT1.txt");
      Serial.printf("File1 size: %d\n", file1.size());
      int filesize = file1.size();
      for(uint8_t i = 0; i <30; i++){
          char buf[256];
          int n = file1.read(buf, 256);
          //Serial.printf("read %d bytes\n", n);
          if (n > 255) n = 255;
          buf[n] = 0;
          Serial.println(buf);
          //Used this in place of a file1.eof() function
          if((file1.position()+1) > filesize) break;
      }
      file1.close();
    
      structuredWrite();
    
    }
      
    void structuredWrite(){
      Serial.println("-------------------------------");
      Serial.println("File3 byte conversion test:");
      Serial.println("-------------------------------");
      Serial.println();
      
      uint32_t arraySize = sizeof(MYDATA_t);
      
      //---------init data - load array
      mydata.datastruct.data_0 = true;
      Serial.print("Data_0: ");
      if (mydata.datastruct.data_0) Serial.println("true");
      if (!mydata.datastruct.data_0) Serial.println("false");
      mydata.datastruct.data_1 = 1.3575;
      Serial.print("Data_1: ");
      Serial.println(mydata.datastruct.data_1, DEC);
      mydata.datastruct.data_2 = 314159L;
      Serial.print("Data_2: ");
      Serial.println(mydata.datastruct.data_2, DEC);
      mydata.datastruct.data_3 = 142;
      Serial.print("Data_3: ");
      Serial.println(mydata.datastruct.data_3, DEC);
      //string test
      String string_test = "The Quick Brown Fox";
      string_test.toCharArray(mydata.datastruct.data_4,string_test.length()+1);
      Serial.println(string_test);
        
      Serial.println("Init Done - array loaded");
      Serial.println("...... ...... ......");
    
      //lets try something more interesting and complicated
      file3 = myfs.open("logger.txt", FILE_WRITE);
       for(int32_t i = 0; i < 10; i++) {
        file3.write(mydata.Packet, arraySize);
      }
      file3.close();
    
      Serial.println();
      Serial.println("2nd Directory contents:");
      printDirectory();
      Serial.println();
      
      file3 = myfs.open("logger.txt");
      for(uint8_t i = 0; i <30; i++){
        //if(eRAM.f_eof(file3)) //break loop on EOF
        //    break;
           int n = file3.read(readdata.Packet, arraySize);
          //Used this in place of a file1.eof() function
       
          //---------Send data to serial
          if (readdata.datastruct.data_0) Serial.print("true");
          if (!readdata.datastruct.data_0) Serial.print("false");
          Serial.print(", ");
          Serial.print(readdata.datastruct.data_1, DEC);
          Serial.print(", ");
          Serial.print(readdata.datastruct.data_2, DEC);
          Serial.print(", ");
          Serial.print(readdata.datastruct.data_3, DEC);  
          Serial.print(", ");
            for (uint8_t j = 0; j < 19 + 1; j++) {
              Serial.print(readdata.datastruct.data_4[j]);
            }
          Serial.println();
          if((file3.position()+1) > file3.size()) break;
      }
      
      file3.close();
    }
    
    
    void printDirectory() {
      Serial.println("printDirectory\n--------------");
      printDirectory(myfs.open("/"), 0);
      Serial.println();
    }
    
    void printDirectory(File dir, int numTabs) {
      //dir.whoami();
      while (true) {
    
        static int count=0;
        if (++count > 20) while (1) ;
        
        File entry =  dir.openNextFile();
        if (! entry) {
          // no more files
          //Serial.println("**nomorefiles**");
          break;
        }
        for (uint8_t i = 0; i < numTabs; i++) {
          Serial.print('\t');
        }
        Serial.print(entry.name());
        if (entry.isDirectory()) {
          Serial.println(" / ");
          printDirectory(entry, numTabs + 1);
        } else {
          // files have sizes, directories do not
          Serial.print("\t\t");
          Serial.println(entry.size(), DEC);
        }
        entry.close();
      }
    }
    
    
    void loop() {
    }

  20. #45
    Senior Member+ defragster's Avatar
    Join Date
    Feb 2015
    Posts
    14,421
    re: @mjs513
    Hmmmm interesting on the copy paste doesn't copy all from the serial monitor. Should be 9 more lines of the last line - shows in sermon but not when I pasted here.
    There must be a non-printable type char in the output? Seems I've seen that before some char the SerMon can support that terminates the Copy/Paste xfer.

    Good to see LittleFS coming along - even if glancing after the fact so far ... Indeed doing to RAM must faster and better a start

  21. #46
    Senior Member+ defragster's Avatar
    Join Date
    Feb 2015
    Posts
    14,421
    @mjs513:: Ran the sketch - copy to notepad fails - copy to WORD shows an unknown char that then is translated to work:
    Click image for larger version. 

Name:	lfsFOX.png 
Views:	16 
Size:	66.0 KB 
ID:	22326

    <FINALLY got the image saved to disk ... all else looks good like posted.

    Didn't look what 'char val' might be printing???

    <edit> : It is printing the NULL char literally with this code as 19+1:
    Code:
            for (uint8_t j = 0; j < 19 + 1; j++) {
              Serial.print(readdata.datastruct.data_4[j]);
            }
              Serial.print(byte(readdata.datastruct.data_4[19])); // << This prints '0'
    Last edited by defragster; 11-06-2020 at 04:36 AM.

  22. #47
    Senior Member PaulStoffregen's Avatar
    Join Date
    Nov 2012
    Posts
    24,452
    Quote Originally Posted by mjs513 View Post
    One thing I noticed is that a EOF would be a nice thing to have.
    File available() returning zero gives you EOF. Or at least it should. As you can see in that program, I haven't tested available() yet.


    Hmmmm interesting on the copy paste doesn't copy all from the serial monitor. Should be 9 more lines of the last line - shows in sermon but not when I pasted here.
    Let's discuss all issues about the Arduino Serial Monitor on the beta test threads.

    https://forum.pjrc.com/threads/64303...no-1-54-Beta-4

    Right now, I'm focused on this filesystem stuff. I will again look at the serial monitor, probably in a week or two. Posting serial monitor bugs here is only going to get lost in this LittleFS conversation. I do read through the most recent beta test thread and sometimes the older ones before deciding whether to package up another installer or wait and fix more stuff.

  23. #48
    Senior Member+ defragster's Avatar
    Join Date
    Feb 2015
    Posts
    14,421
    Thought of an easy test for .available() and got this that has '\n' in the wrong place but runs replacing loop() in post #44 code:
    Code:
    uint32_t lCnt = 0;
    char szLoop[] = "#_file.txt";
    void loop() {
    	lCnt++;
    	int ii;
    	byte nNum = lCnt % 10;
    	szLoop[0] = '0' + nNum;
    	if ( myfs.exists(szLoop) ) {
    		file3 = myfs.open(szLoop);
    		ii = 0;
    		char mm;
    		while ( file3.available() ) {
    			file3.read( &mm , 1 );
    			ii++;
    			if ( '@' != mm ) Serial.print( "Bad Byte!\n");
    		}
    		Serial.printf( " file %s had %u bytes\n", szLoop, ii );
    		file3.close();
    		myfs.remove(szLoop);
    	}
    	else {
    		file3 = myfs.open(szLoop, FILE_WRITE);
    		if ( nNum == 0 ) {
    			nNum = 10;
    		}
    		char mm = '@';
    		for ( ii = 0; ii < nNum * 100; ii++ ) {
    			file3.write( &mm , 1 );
    		}
    		file3.close();
    		Serial.print('.');
    		delay(100);
    	}
    }
    Looks like this and repeats fine:
    Code:
    true, 1.3574999571, 314159, 142, The Quick Brown Fox0
    .......... file 1_file.txt had 100 bytes
     file 2_file.txt had 200 bytes
     file 3_file.txt had 300 bytes
     file 4_file.txt had 400 bytes
     file 5_file.txt had 500 bytes
     file 6_file.txt had 600 bytes
     file 7_file.txt had 700 bytes
     file 8_file.txt had 800 bytes
     file 9_file.txt had 900 bytes
     file 0_file.txt had 1000 bytes
    .......... file 1_file.txt had 100 bytes
     file 2_file.txt had 200 bytes
    // ...
    So moving the '\n' to the other half {tested removed above the '\n' as output shows - it is not a factor - just adding to below} of the if to make it pretty something broke and the T_4.1 hangs and needs button push?
    I don't see it in the sketch code below compared to the above?:
    Code:
    uint32_t lCnt = 0;
    char szLoop[] = "#_file.txt";
    void loop() {
    	lCnt++;
    	int ii;
    	byte nNum = lCnt % 10;
    	szLoop[0] = '0' + nNum;
    	if ( myfs.exists(szLoop) ) {
    		if ( nNum == 1 ) {
    			Serial.print('\n');
    		}
    		file3 = myfs.open(szLoop);
    		ii = 0;
    		char mm;
    		while ( file3.available() ) {
    			file3.read( &mm , 1 );
    			ii++;
    			if ( '@' != mm ) Serial.print( "Bad Byte!\n");
    		}
    		Serial.printf( " file %s had %u bytes\n", szLoop, ii );
    		file3.close();
    		myfs.remove(szLoop);
    	}
    	else {
    		file3 = myfs.open(szLoop, FILE_WRITE);
    		if ( nNum == 0 ) {
    			nNum = 10;
    		}
    		char mm = '@';
    		for ( ii = 0; ii < nNum * 100; ii++ ) {
    			file3.write( &mm , 1 );
    		}
    		file3.close();
    		Serial.print('.');
    		delay(100);
    	}
    }
    From the second loop() code I see this where it writes the 10 files with '.' for each - then reading them back before remove, it dies after the file file - where snippet one above runs and runs:
    Code:
    true, 1.3574999571, 314159, 142, The Quick Brown Fox0
    ..........
     file 1_file.txt had 100 bytes
    Time for my nap.

    And yes, now that the problem is SerMon Copy/Paste caused by sketch and not a problem in the FILE I/O - given the invalid char is a known ZERO - it should be noted elsewhere.
    Last edited by defragster; 11-06-2020 at 09:44 AM.

  24. #49
    Senior Member+ mjs513's Avatar
    Join Date
    Jul 2014
    Location
    New York
    Posts
    7,196
    @PaulStoffregen and @defragster

    Coffee in had first I gave file.available() == 0 in place of my file end test and it worked as advertised. So that question is resolved.

    As for the issue with unprintable character, tracked the issue down this morning after this morning to the to how I am converting the string to a char array. Looking at the docs on string.toCharArray() turns out that the actual string length for printing or other uses is at Array.Length - 1. So fixing that in my sketch now works when doing a copy and paste to the serial monitor :
    Code:
    D:\Users\Merli\Documents\Arduino\littlefs_teensy_test3\littlefs_teensy_test3.ino Nov  6 2020 07:30:50
    LittleFS Test
    printDirectory
    --------------
    
    Using println and printf to printoutput file
    opened PRINTOUTPUT1.txt
    opened PRINTOUTPUT2.txt
    printDirectory
    --------------
    FILE	PRINTOUTPUT1.txt		628
    FILE	PRINTOUTPUT2.txt		630
    
    -------------------------------
    File1 contents:
    -------------------------------
    File1 size: 628
    abcdefghijklmnopqrstuvwxyz
    Rec: 0, Float: 26.400000, Int: 98
    abcdefghijklmnopqrstuvwxyz
    Rec: 1, Float: 27.400000, Int: 99
    abcdefghijklmnopqrstuvwxyz
    Rec: 2, Float: 28.400000, Int: 100
    abcdefghijklmnopqrstuvwxyz
    Rec: 3, Float: 29.400000, Int: 101
    abcde
    ghijklmnopqrstuvwxyz
    Rec: 4, Float: 30.400000, Int: 102
    abcdefghijklmnopqrstuvwxyz
    Rec: 5, Float: 31.400000, Int: 103
    abcdefghijklmnopqrstuvwxyz
    Rec: 6, Float: 32.400000, Int: 104
    abcdefghijklmnopqrstuvwxyz
    Rec: 7, Float: 33.400000, Int: 105
    abcdefghi
    klmnopqrstuvwxyz
    Rec: 8, Float: 34.400000, Int: 106
    abcdefghijklmnopqrstuvwxyz
    Rec: 9, Float: 35.400000, Int: 107
    
    -------------------------------
    File3 byte conversion test:
    -------------------------------
    
    Data_0: true
    Data_1: 1.3574999571
    Data_2: 314159
    Data_3: 142
    19
    The Quick Brown Fox
    Init Done - array loaded
    ...... ...... ......
    create folder
      success
    
    
    2nd Directory contents:
    printDirectory
    --------------
    FILE	PRINTOUTPUT1.txt		628
    FILE	PRINTOUTPUT2.txt		630
    DIR	structuredData / 
    	FILE	logger.txt		480
    
    
    true, 1.3574999571, 314159, 142, The Quick Brown Fox
    true, 1.3574999571, 314159, 142, The Quick Brown Fox
    true, 1.3574999571, 314159, 142, The Quick Brown Fox
    true, 1.3574999571, 314159, 142, The Quick Brown Fox
    true, 1.3574999571, 314159, 142, The Quick Brown Fox
    true, 1.3574999571, 314159, 142, The Quick Brown Fox
    true, 1.3574999571, 314159, 142, The Quick Brown Fox
    true, 1.3574999571, 314159, 142, The Quick Brown Fox
    true, 1.3574999571, 314159, 142, The Quick Brown Fox
    true, 1.3574999571, 314159, 142, The Quick Brown Fox
    so the fixed sketch is:
    Code:
    #include <LittleFS.h>
    
    LittleFS_RAM myfs;
    char buf[200000];
    
    File file1, file2, file3;
    
    char fname[32] = "my_file1";
    int szLen = strlen( buf );
    elapsedMicros my_us;
    
    //define a struct of various data types
     struct MYDATA_t {
      bool data_0;
      float data_1; 
      long data_2; 
      int data_3;
      char data_4[32];
    };
    
    //define a struct joining MYDATA_t to an array of bytes to be stored
     union MYDATA4RAM_t {
     MYDATA_t datastruct;
     char Packet[sizeof(MYDATA_t)];
    };
    
    MYDATA4RAM_t mydata; //data to be written in memory
    MYDATA4RAM_t readdata; //data read from memory
    
    
    void setup() {
      pinMode(13, OUTPUT);
      digitalWrite(13, HIGH);
      while (!Serial) ; // wait
        Serial.println("\n" __FILE__ " " __DATE__ " " __TIME__);
      Serial.println("LittleFS Test"); delay(5);
      
      if (!myfs.begin(buf, sizeof(buf))) {
        Serial.println("Error starting ramdisk");
        while (1) ;
      }
    
      printDirectory();
    
      //test of print
      Serial.println("Using println and printf to printoutput file");
      file1 = myfs.open("PRINTOUTPUT1.txt", FILE_WRITE);
      Serial.println("opened PRINTOUTPUT1.txt");
      file2 = myfs.open("PRINTOUTPUT2.txt", FILE_WRITE);
      Serial.println("opened PRINTOUTPUT2.txt");
    
      for(uint8_t i = 0; i < 10; i++) {
        file1.println("abcdefghijklmnopqrstuvwxyz");
        file1.printf("Rec: %d, Float: %f, Int: %d\n",i,i+26.4, i+98);
        file2.println("ABCDEFGHIJKLMNOPQRSTUVWXYZ");
        //eRAM.println("THIS IS A TEST");
        file2.printf("Rec: %d, Float: %f, Int: %d\n",i,i+56.4, i+198);
      }
    
      file1.close();
      file2.close();
    
      printDirectory();
      
      Serial.println("-------------------------------");
      Serial.println("File1 contents:");
      Serial.println("-------------------------------");
    
      file1 = myfs.open("PRINTOUTPUT1.txt");
      Serial.printf("File1 size: %d\n", file1.size());
      int filesize = file1.size();
      for(uint8_t i = 0; i <30; i++){
          char buf[256];
          int n = file1.read(buf, 256);
          //Serial.printf("read %d bytes\n", n);
          if (n > 255) n = 255;
          buf[n] = 0;
          Serial.println(buf);
          //Used this in place of a file1.eof() function
          //if((file1.position()+1) > filesize) break;
          if(file1.available() == 0) break;
      }
      file1.close();
    
      structuredWrite();
    
    }
      
    void structuredWrite(){
      Serial.println("-------------------------------");
      Serial.println("File3 byte conversion test:");
      Serial.println("-------------------------------");
      Serial.println();
      
      uint32_t arraySize = sizeof(MYDATA_t);
      
      //---------init data - load array
      mydata.datastruct.data_0 = true;
      Serial.print("Data_0: ");
      if (mydata.datastruct.data_0) Serial.println("true");
      if (!mydata.datastruct.data_0) Serial.println("false");
      mydata.datastruct.data_1 = 1.3575;
      Serial.print("Data_1: ");
      Serial.println(mydata.datastruct.data_1, DEC);
      mydata.datastruct.data_2 = 314159L;
      Serial.print("Data_2: ");
      Serial.println(mydata.datastruct.data_2, DEC);
      mydata.datastruct.data_3 = 142;
      Serial.print("Data_3: ");
      Serial.println(mydata.datastruct.data_3, DEC);
      //string test
      String string_test = "The Quick Brown Fox";
      int stringLen = string_test.length();
      Serial.println(stringLen);
      string_test.toCharArray(mydata.datastruct.data_4,stringLen+1);
      Serial.println(string_test);
        
      Serial.println("Init Done - array loaded");
      Serial.println("...... ...... ......");
    
      Serial.println("create folder");
      if (myfs.mkdir("structuredData")) {
        Serial.println("  success");
      } else {
        Serial.println("  failed");
      }
      Serial.println();
    
      //lets try something more interesting and complicated
      file3 = myfs.open("/structuredData/logger.txt", FILE_WRITE);
       for(int32_t i = 0; i < 10; i++) {
        file3.write(mydata.Packet, arraySize);
      }
      file3.close();
    
      Serial.println();
      Serial.println("2nd Directory contents:");
      printDirectory();
      Serial.println();
      
      file3 = myfs.open("/structuredData/logger.txt");
      for(uint8_t i = 0; i <30; i++){
        //if(eRAM.f_eof(file3)) //break loop on EOF
        //    break;
           int n = file3.read(readdata.Packet, arraySize);
          //Used this in place of a file1.eof() function
       
          //---------Send data to serial
          if (readdata.datastruct.data_0) Serial.print("true");
          if (!readdata.datastruct.data_0) Serial.print("false");
          Serial.print(", ");
          Serial.print(readdata.datastruct.data_1, DEC);
          Serial.print(", ");
          Serial.print(readdata.datastruct.data_2, DEC);
          Serial.print(", ");
          Serial.print(readdata.datastruct.data_3, DEC);  
          Serial.print(", ");
            for (uint8_t j = 0; j < stringLen; j++) {
              Serial.print(readdata.datastruct.data_4[j]);
            }
          Serial.println();
          if((file3.position()+1) > file3.size()) break;
          //if(file3.available() == 0) break;
      }
      
      file3.close();
    }
    
    
    void printDirectory() {
      Serial.println("printDirectory\n--------------");
      printDirectory(myfs.open("/"), 0);
      Serial.println();
    }
    
    void printDirectory(File dir, int numTabs) {
      //dir.whoami();
      while (true) {
    
        static int count=0;
        if (++count > 20) while (1) ;
        
        File entry =  dir.openNextFile();
        if (! entry) {
          // no more files
          //Serial.println("**nomorefiles**");
          break;
        }
        for (uint8_t i = 0; i < numTabs; i++) {
          Serial.print('\t');
        }
    
        if(entry.isDirectory()) {
          Serial.print("DIR\t");
        } else {
          Serial.print("FILE\t");
        }
        Serial.print(entry.name());
        if (entry.isDirectory()) {
          Serial.println(" / ");
          printDirectory(entry, numTabs + 1);
        } else {
          // files have sizes, directories do not
          Serial.print("\t\t");
          Serial.println(entry.size(), DEC);
        }
        entry.close();
      }
    }
    
    
    void loop() {
    }
    I didn't add @defragsters code in loop yes as it seems to hang the T4.1 and have to press the button to load the sketch again. Guess that will be next.

  25. #50
    Senior Member+ KurtE's Avatar
    Join Date
    Jan 2014
    Posts
    9,206
    Sounds like you are making good progress.

    Wondering about other stressy things. I was wondering about access to multiple files at same time, which looks like you have.

    Like wondering if it is at all sort of thread safe?
    What about reading and writing to multiple files using IntervalTimers? And main sketch. Or TimerTool with different types of interrupts maybe at different priorities?

Tags for this Thread

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •