FS.h - File class goes boom - when virtual functions not overwritten

Status
Not open for further replies.
@Frank and @luni
First are we all using the same sketch that I posted because as @luni said to looks like it writing but not seeing any data from dumping the file or from the directory.

This is the same test sketch that @KurtE and I have been using for quite a while and definitely worked before changes. Going to play a bit more.....
 
No, I was using the known to work examples "DataLogger" and "DumpFile".
I did also test my own example for MemFile. This works now, too. Crashed before. Reason was a recursion on writes in FS.h

Code:
/*
  MEMFile file dump
  This example code is in the public domain.
*/

#include <MemFile.h>

char testData[31];


MemFS  myfs;

void setup()
{
  Serial.print(CrashReport);
  CrashReport.clear();
  // open the file.
  File dataFile = myfs.open((char*) &testData, sizeof(testData), FILE_WRITE);
  for (unsigned i=0; i<sizeof(testData); ++i) testData[i] = 'a'+i;
  if (dataFile)
  {
    while (dataFile.available()) {
      Serial.write(dataFile.read());
    }
    dataFile.seek(1);
    dataFile.print('!');
    dataFile.seek(0);
    while (dataFile.available()) {
      Serial.write(dataFile.read());
    }
    dataFile.close();
  }
}

void loop()
{
}
(ps: Might be worth an investgation later - it even works with EXTMEM - WITHOUT a PSRAM attached - the cache is magic - but this issue has to be fixed. It should NOT work. Really.
I' decided to pause my PRs to the core, so someone else might want to have a look at this magic issue. The MPU can do this, but I already have a very old PR re: MPU EDIT: HA! This should already fix it)

Was not aware of Kurt's sketch - but as the official examples work, the test sketch - or FS.h must have an other problem.

I'm away now for a short while.
 
Last edited:
Here is my FS.h


i'll do other things now.
Thanks Frank. I do believe you but I just copy and pasted your FS.h to teensy cores and I still can not see any data in my datalog.txt file. It all works on using Datalogger and Dumpfile on the SD Card but not for me with LittleFS. Don't know what I am missing.
 
Something is wrong with the file pointer maintained in the littlefs file class. Looks like it gets set correctly on opening a file. But when you try to write the file the pointer is null so it never writes. I'll try to find out what this is but I see this code the first time...
 
IMHO, this datalogger example is doing too much for debugging. I therefore did a very simple test sketch which works in principle:

Code:
#include "Arduino.h"
#include <LittleFS.h>

void printDirectory(File dir, int numSpaces);


LittleFS_RAM ramFS;

void setup()
{
    while (!Serial) {}
    if (CrashReport) Serial.println(CrashReport);

    Serial.println("start");
    ramFS.begin(1024);

    File f1 = ramFS.open("test1.txt", FILE_WRITE);
    f1.write("1234567890\0",11);
    f1.close();

    File f2 = ramFS.open("test2.txt", FILE_WRITE);
    const char thirty[] = "123456789012345678901234567890";
    f2.write(thirty, sizeof(thirty));
    f2.close();

    Serial.println("Print directory:");
    printDirectory(ramFS.open("/"), 2);

    char buf[1000];

    Serial.println();

    File f3 = ramFS.open("test1.txt", FILE_READ);
    f3.read(buf, 100);
    f3.close();
    Serial.printf("test1.txt read: %s\n", buf);

    File f4 = ramFS.open("test2.txt", FILE_READ);
    f4.read(buf, 100);
    f4.close();
    Serial.printf("test2.txt read: %s\n", buf);
}

void loop()
{
}

void printSpaces(int num)
{
    for (int i = 0; i < num; i++)
    {
        Serial.print(" ");
    }
}

void printDirectory(File dir, int numSpaces = 2)
{
    while (true)
    {
        File entry = dir.openNextFile();
        if (!entry)
        {
            //Serial.println("** no more files **");
            break;
        }
        printSpaces(numSpaces);
        Serial.print(entry.name());
        if (entry.isDirectory())
        {
            Serial.println("/");
            printDirectory(entry, numSpaces + 2);
        }
        else
        {
            // files have sizes, directories do not
            printSpaces(36 - numSpaces - strlen(entry.name()));
            Serial.print("  ");
            Serial.println(entry.size(), DEC);
        }
        entry.close();
    }
}

As expected, it prints out:
Code:
start
Print directory:
  test1.txt                           11
  test2.txt                           31

test1.txt read: 1234567890
test2.txt read: 123456789012345678901234567890

Following observations:

1) The write(const char*) function strips off trailing zeros. I don't know if this is intended, at least it is unexpected. To store the string with a trailing zero one needs to use the write version which takes the length of the string +1.
Adding a zero manually by e.g.
Code:
File f1 = ramFS.open("test1.txt", FILE_WRITE);
f1.write("1234567890");
f1.write('\0');
f1.close();
Generates an ambiguity error in the File class.

2) The File objects can't be reused. I would expect that this should work:

Code:
File f1 = ramFS.open("test1.txt", FILE_WRITE);
f1.write("1234567890\0",11);
f1.close();

f1 = ramFS.open("test2.txt", FILE_WRITE);
const char thirty[] = "123456789012345678901234567890";
f1.write(thirty, sizeof(thirty));
f1.close();

however this prints:
Code:
start
Print directory:
  test1.txt                           11
  test2.txt                           0
...

maybe something related to the reference counter? Does this work with SD?
 
Yes, that's what I said - (or I edited it away. hm, shouldn't have done this) strlen is one too less. it needs to be strlen + 1
But then you'll get a zero.. of course. But I think the trailing zero is not wanted. Imaging a simple print to a file. Normally, there will be no trailing zero in the file.


What happens with a Stream?
 
Last edited:
I'm incorporating the write fixes and looking more closely at the boolean test. Probably going to change the FileImpl class slightly to remove its bool operator in favor of a more explicit function call.
 
@luni
Just got back In answer to your questions the datalogger example for LittleFS is essentially the Datalogger/DumpFile SD example used for the SD Card. The SDLogger does not have trailing 0's so I guess its an intended action.

In answer to your second question once you do a File f1 and close the file you can reuse. I did it in the past many times without an issue.

If this happens with SD card can't say for sure but I did run the Datalogger example and the Dumpfile sketch without an issue. It wrote to the file and read from it but it was in 2 separate sketches.

Will have to combine the 2 to say for sure.
 
@Frank: I'd say if you have a function intended to write c-strings it should terminate them with a \0. But I'm more concerned with the non working write('\0') and the problem reusing the File objects. But it is getting late here...
 
Not sure what happens on a PC.. I use C on a PC not often enough. I think a print does not append a zero, there... hm.
 
Sorry for my blah blah (again) I'll shut off.

Edit: Paul, attention if you use my file "FS.h" I had added the "nwrite" - you may want to remove that.
Good night.
 
@luni - @Frank - @Paul
While the SD example sketches work they don't work if you try and combine them into one sketch along the lines of the RamFS logger sketch. And I think the problem is along the lines that @luni mentioned. The File pointer is not retained if you put it at the global scope. For instance in the sketch I have declared dataFile as a File:
Code:
/*
  SD card datalogger
 
 This example shows how to log data from three analog sensors
 to an SD card using the SD library.
   
 The circuit:
 * analog sensors on analog ins 0, 1, and 2
 * SD card attached to SPI bus as follows:
 ** MOSI - pin 11, pin 7 on Teensy with audio board
 ** MISO - pin 12
 ** CLK - pin 13, pin 14 on Teensy with audio board
 ** CS - pin 4,  pin 10 on Teensy with audio board
 
 created  24 Nov 2010
 modified 9 Apr 2012
 by Tom Igoe
 
 This example code is in the public domain.
 */

#include <SD.h>
#include <SPI.h>

const int chipSelect = 8;

[B]File dataFile;  // Specifes that dataFile is of File type
[/B]
int record_count = 0;
bool write_data = false;

void setup()
{
  // Open serial communications and wait for port to open:
  Serial.begin(115200);
  while (!Serial) {
    // wait for serial port to connect.
  }
  Serial.print(CrashReport);
  Serial.println("\n" __FILE__ " " __DATE__ " " __TIME__);

  Serial.print("Initializing SD card...");

  // see if the card is present and can be initialized:
  if (!SD.begin(chipSelect)) {
    Serial.println("Card failed, or not present");
    while (1) {
      // No SD card, so don't do anything more - stay stuck here
    }
  }
  Serial.println("card initialized.");
  
  menu();
  
}

void loop()
{ 
  if ( Serial.available() ) {
    char rr;
    rr = Serial.read();
    switch (rr) {
      case 'l': listFiles(); break;
      case 'e': eraseFiles(); break;
      case 's':
        {
          Serial.println("\nLogging Data!!!");
          write_data = true;   // sets flag to continue to write data until new command is received
          // opens a file or creates a file if not present,  FILE_WRITE will append data to
          // to the file created.
          dataFile = SD.open("datalog.txt", FILE_WRITE);
          logData();
        }
        break;
      case 'x': stopLogging(); break;
      case 'd': dumpLog(); break;
      case '\r':
      case '\n':
      case 'h': menu(); break;
    }
    while (Serial.read() != -1) ; // remove rest of characters.
  } 

  if(write_data) logData();
}

void logData()
{
    // make a string for assembling the data to log:
    String dataString = "";
  
    // read three sensors and append to the string:
    for (int analogPin = 0; analogPin < 3; analogPin++) {
      int sensor = analogRead(analogPin);
      dataString += String(sensor);
      if (analogPin < 2) {
        dataString += ",";
      }
    }
  
    // if the file is available, write to it:
    if (dataFile) {
      dataFile.println(dataString);
      // print to the serial port too:
      Serial.println(dataString);
      record_count += 1;
    } else {
      // if the file isn't open, pop up an error:
      Serial.println("error opening datalog.txt");
    }
    delay(100); // run at a reasonable not-too-fast speed for testing
}

void stopLogging()
{
  Serial.println("\nStopped Logging Data!!!");
  write_data = false;
  // Closes the data file.
  dataFile.close();
  Serial.printf("Records written = %d\n", record_count);
}


void dumpLog()
{
  Serial.println("\nDumping Log!!!");
  // open the file.
  dataFile = SD.open("datalog.txt");

  // if the file is available, write to it:
  if (dataFile) {
    while (dataFile.available()) {
      Serial.write(dataFile.read());
    }
    dataFile.close();
  }  
  // if the file isn't open, pop up an error:
  else {
    Serial.println("error opening datalog.txt");
  } 
}

void menu()
{
  Serial.println();
  Serial.println("Menu Options:");
  Serial.println("\tl - List files on disk");
  Serial.println("\te - Erase files on disk");
  Serial.println("\ts - Start Logging data (Restarting logger will append records to existing log)");
  Serial.println("\tx - Stop Logging data");
  Serial.println("\td - Dump Log");
  Serial.println("\th - Menu");
  Serial.println();
}

void listFiles()
{
  File root = SD.open("/");
  
  printDirectory(root, 0);
}

void eraseFiles()
{
  Serial.println("\nNot supported !");
}

void printDirectory(File dir, int numSpaces) {
   while(true) {
     File entry = dir.openNextFile();
     if (! entry) {
       //Serial.println("** no more files **");
       break;
     }
     printSpaces(numSpaces);
     Serial.print(entry.name());
     if (entry.isDirectory()) {
       Serial.println("/");
       printDirectory(entry, numSpaces+2);
     } else {
       // files have sizes, directories do not
       printSpaces(36 - numSpaces - strlen(entry.name()));
       Serial.print("  ");
       Serial.println(entry.size(), DEC);
     }
     entry.close();
   }
}

void printSpaces(int num) {
  for (int i=0; i < num; i++) {
    Serial.print(" ");
  }
}
If I try and start the data logging process it causes the T4.1 to reboot with (seems to crash on dataFile.open:
Code:
CrashReport:
  A problem occurred at (system time) 17:27:32
  Code was executing from address 0x20203060
  CFSR: 1
	(IACCVIOL) Instruction Access Violation
to prove the point I happen to have a datalog.txt file on the sd card already and if you do a change to the dumpLog function:
Code:
void dumpLog()
{
  Serial.println("\nDumping Log!!!");
  // open the file.
[B]  File dataFile = SD.open("datalog.txt");
[/B]
  // if the file is available, write to it:
  if (dataFile) {
    while (dataFile.available()) {
      Serial.write(dataFile.read());
    }
    dataFile.close();
  }  
  // if the file isn't open, pop up an error:
  else {
    Serial.println("error opening datalog.txt");
  } 
}
dumping the text file works.

As a test I also tried File dataFile.open in startLog case and it did open the log file but fails to write - get the error writting to log file.

So as @luni said the pointer to File dataFile isn't being retained as far as I can see.
 
Yup, saw that.

Today is not the time to expand the public File API.

Darn, and here I was wanting to sneak in a few changes ;)
Code:
#ifdef FS_FILE_SUPPORT_DATES
  		virtual bool getAccessDateTime(uint16_t* pdate, uint16_t* ptime);
  		virtual bool getCreateDateTime(uint16_t* pdate, uint16_t* ptime);
	  	virtual bool getModifyDateTime(uint16_t* pdate, uint16_t* ptime);
	  	virtual bool timestamp(uint8_t flags, uint16_t year, uint8_t month, uint8_t day,
                 uint8_t hour, uint8_t minute, uint8_t second);
#endif
Waiting for updates.

I have also been testing with @mjs513 with MTP stuff and running into faults and the like. I tried earlier today merging in Frank's changes and then compated with mjs513...

Running a version of the SD_MTP logging stuff, and for example starting the writes to log faults. Asking from PC back to format SD faults, although I have not tried that much with current code..
 
Incorporated the changes for T4 only to start and threw in Kurt's date changes for our testing with dates and so far like you said working for SD and LittleFS. Tried the SD Logging sketch that I posted earlier that starts and stops and dumps the files and gives the directory listing as well as a few of the LittleFS logger sketches to start: SPI, Program and QSPI. All were able to start and stop logging multiple times, format the device, dump the file and give me the directory listings.
Capture.PNG
 
Just made similar changes to mscFS.h that was made to LittleFS with a slight difference so was able to use MTP in conjunction with LittleFS (tested with SPI) and SD (used @KurtE SD-MTP_logger test case we have been working on ). Files show in Windows Explorer - able add/delete, etc and format from windows.
Capture.PNG
Output from serial mon on formating External Card Reader:
Code:
CMD: 1007(GET_OBJECT_HANDLES)l: 24 T:7 : 20001 0 ffffffff
RESP:2001(RSP:OK)l: 24 T:7 : 20001 0 ffffffff
CMD: 9803(GET_OBJECT_PROP_VALUE)l: 20 T:8 : 2 dc02 (FORMAT)
RESP:2001(RSP:OK)l: 20 T:8 : 2 dc02
CMD: 9802(GET_OBJECT_PROP_DESC)l: 20 T:9 : dc03 3000 (PROTECTION)
RESP:2001(RSP:OK)l: 20 T:9 : dc03 3000
...............
Write cluster two, bitmap
Writing upcase table
Writing root
Format done
free clusters after begin on partVol: 974318
*** end Interval Timer ***

Attached are the changes made to mscFS.h to make it work. Haven't tried USB drives yet getting late here so maybe I will leave others to do that.
 

Attachments

  • mscFS.zip
    2.6 KB · Views: 42
Note: Will also need to update MSC Library UsbMscFat\src\mscFS.h
In my version line 44: class MSCFile : public File

Here's a first quick attempt. Haven't had time to test, but this at least solves the compile errors. Edits are pretty much the same as with SD.h and LittleFS.h

Edit: looks like I wasn't fast enough....
 

Attachments

  • mscFS.h
    7 KB · Views: 47
Here's a first quick attempt. Haven't had time to test, but this at least solves the compile errors. Edits are pretty much the same as with SD.h and LittleFS.h

Edit: looks like I wasn't fast enough....

:)

Had all the changes in except the isOpen so was pretty quick to make the change and update.

Just ran a quick mtp-test with USB Stick + QSPI + PRogram:
Capture.PNG

Will test more in the morning.
 
Status
Not open for further replies.
Back
Top