Teensyduino File System Integration, including MTP and MSC

[PaulStoffregen]

Looks like the interface swap broken serial emulation on Linux, but only when using USB Type MTP. Windows and MacOS still work. I just fixed teensy_serialmon here.

 

[KurtE]

Looks like the interface swap broken serial emulation on Linux, but only when using USB Type MTP. Windows and MacOS still work. I just fixed teensy_serialmon here.

Wonder why? Would make difference on Ubuntu?

Also wonder why only MTP. The change to:

  #define MTP_INTERFACE		1	// MTP Disk
  #define SEREMU_INTERFACE      0	// Serial emulation

Sort of followed along what USB_AUDIO has:

  #define SEREMU_INTERFACE      0	// Serial emulation
  #define AUDIO_INTERFACE	1	// Audio (uses 3 consecutive interfaces)

Fix for teensy_serlmon - Was that specific for Ubuntu/Linux?

Note: In my experiments I found MTP on Ubuntu was problematic...
More on that when ready.

 

[PaulStoffregen]

Looking at the timestamp stuff today....

setSyncProvider - Would be nice if programs that use File system Dates and Times just worked without having to know about the time/date glue in the Time library.

That is the dates and times will be screwed up if something does not call setSyncProvider()...

Agreed. I would like to bring more of this into the core library without any Time library dependency.

 

[mjs513]

Looking at the timestamp stuff today....
....
Agreed. I would like to bring more of this into the core library without any Time library dependency.

If you just want to try it with LittleFS I modified the LFS_Usage test sketch to print file/directory timestamps.

/*
  LittleFS usuage from the LittleFS library
  
  Starting with Teensyduino 1.54, support for LittleFS has been added.

  LittleFS is a wrapper for the LittleFS File System for the Teensy family of microprocessors and provides support for RAM Disks, NOR and NAND Flash chips, and FRAM chips. For the NOR, NAND Flash support is provided for SPI and QSPI in the case of the Teensy 4.1. For FRAM only SPI is supported. It is also linked to SDFat so many of the same commands can be used as for an SD Card.
  
  This example shows the use of some of the commands provided in LittleFS using a SPI Flash chip such as the W25Q128. 
  
  See the readme for the LittleFS library for more information: https://github.com/PaulStoffregen/LittleFS
  
*/

#include <LittleFS.h>
#include <SD.h>

// Some variables for later use
uint64_t fTot, totSize1;

// To use SPI flash we need to create a instance of the library telling it to use SPI flash.
/* Other options include:
  LittleFS_QSPIFlash myfs;
  LittleFS_Program myfs;
  LittleFS_SPIFlash myfs;
  LittleFS_SPIFram myfs;
  LittleFS_SPINAND myfs;
  LittleFS_QPINAND myfs;
  LittleFS_RAM myfs;
*/
LittleFS_SPIFlash myfs;
time_t getTeensy3Time()
{
  return Teensy3Clock.get();
}

// Since we are using SPI we need to tell the library what the chip select pin
#define chipSelect 6  // use for access flash on audio or prop shield

// Specifies that the file, file1 and file3 are File types, same as you would do for creating files
// on a SD Card
File file, file1, file2;


void setup()
{
  // Open serial communications and wait for port to open:
  Serial.begin(115200);
  while (!Serial) {
    // wait for serial port to connect.
  }
  Serial.println("\n" __FILE__ " " __DATE__ " " __TIME__);

  Serial.print("Initializing LittleFS ...");
  setSyncProvider(getTeensy3Time);

  // see if the Flash is present and can be initialized:
  // Note:  SPI is default so if you are using SPI and not SPI for instance
  //        you can just specify myfs.begin(chipSelect). 
  if (!myfs.begin(chipSelect, SPI)) {
    Serial.printf("Error starting %s\n", "SPI FLASH");
    while (1) {
      // Error, so don't do anything more - stay stuck here
    }
  }
  myfs.quickFormat();
  Serial.println("LittleFS initialized.");
  
  
  // To get the current space used and Filesystem size
  Serial.println("\n---------------");
  Serial.printf("Bytes Used: %llu, Bytes Total:%llu\n", myfs.usedSize(), myfs.totalSize());
  waitforInput();

  // Now lets create a file and write some data.  Note: basically the same usage for 
  // creating and writing to a file using SD library.
  Serial.println("\n---------------");
  Serial.println("Now lets create a file with some data in it");
  Serial.println("---------------");
  char someData[128];
  memset( someData, 'z', 128 );
  file = myfs.open("bigfile.txt", FILE_WRITE);
  file.write(someData, sizeof(someData));

  for (uint16_t j = 0; j < 100; j++)
    file.write(someData, sizeof(someData));
  file.close();
  
  // We can also get the size of the file just created.  Note we have to open and 
  // thes close the file unless we do file size before we close it in the previous step
  file = myfs.open("bigfile.txt", FILE_WRITE);
  Serial.printf("File Size of bigfile.txt (bytes): %u\n", file.size());
  file.close();

  // Now that we initialized the FS and created a file lets print the directory.
  // Note:  Since we are going to be doing print directory and getting disk usuage
  // lets make it a function which can be copied and used in your own sketches.
  listFiles();
  waitforInput();
  
  // Now lets rename the file
  Serial.println("\n---------------");
  Serial.println("Rename bigfile to file10");
  myfs.rename("bigfile.txt", "file10.txt");
  listFiles();
  waitforInput();
  
  // To delete the file
  Serial.println("\n---------------");
  Serial.println("Delete file10.txt");
  //myfs.remove("file10.txt");
  listFiles();
  waitforInput();

  Serial.println("\n---------------");
  Serial.println("Create a directory and a subfile");
  myfs.mkdir("structureData1");

  file = myfs.open("structureData1/temp_test.txt", FILE_WRITE);
  file.println("SOME DATA TO TEST");
  file.close();
  listFiles();
  waitforInput();

  Serial.println("\n---------------");
  Serial.println("Rename directory");
  myfs.rename("structureData1", "structuredData");
  listFiles();
  waitforInput();

  Serial.println("\n---------------");
  Serial.println("Lets remove them now...");
  //Note have to remove directories files first
  //myfs.remove("structuredData/temp_test.txt");
  //myfs.rmdir("structuredData");
  listFiles();
  waitforInput();

  Serial.println("\n---------------");
  Serial.println("Now lets create a file and read the data back...");
  
  // LittleFS also supports truncate function similar to SDFat. As shown in this
  // example, you can truncate files.
  //
  Serial.println();
  Serial.println("Writing to datalog.bin using LittleFS functions");
  file1 = myfs.open("datalog.bin", FILE_WRITE);
  unsigned int len = file1.size();
  Serial.print("datalog.bin started with ");
  Serial.print(len);
  Serial.println(" bytes");
  if (len > 0) {
    // reduce the file to zero if it already had data
    file1.truncate();
  }
  file1.print("Just some test data written to the file (by SdFat functions)");
  file1.write((uint8_t) 0);
  file1.close();

  // You can also use regular SD type functions, even to access the same file.  Just
  // remember to close the file before opening as a regular SD File.
  //
  Serial.println();
  Serial.println("Reading to datalog.bin using LittleFS functions");
  file2 = myfs.open("datalog.bin");
  if (file2) {
    char mybuffer[100];
    int index = 0;
    while (file2.available()) {
      char c = file2.read();
      mybuffer[index] = c;
      if (c == 0) break;  // end of string
      index = index + 1;
      if (index == 99) break; // buffer full
    }
    mybuffer[index] = 0;
    Serial.print("  Read from file: ");
    Serial.println(mybuffer);
  } else {
    Serial.println("unable to open datalog.bin :(");
  }
  file2.close();


  Serial.println("\nBasic Usage Example Finished");
}

void loop() {}

void listFiles()
{
  Serial.println("---------------");
  printDirectory(myfs);
  Serial.printf("Bytes Used: %llu, Bytes Total:%llu\n", myfs.usedSize(), myfs.totalSize());
}

void printDirectory(FS &fs) {
  Serial.println("Directory\n---------");
  printDirectory(fs.open("/"), 0);
  Serial.println();
}

void printDirectory(File dir, int numSpaces) {
   while(true) {
     File entry = dir.openNextFile();
     if (! entry) {
       //Serial.println("** no more files **");
       break;
     }
     printSpaces(numSpaces);
     Serial.print(entry.name());
     if (entry.isDirectory()) {
       Serial.println("/");
       printDirectory(entry, numSpaces+2);
     } else {
       // files have sizes, directories do not
       printSpaces(36 - numSpaces - strlen(entry.name()));
       Serial.print("  ");
      Serial.print(entry.size(), DEC);
      uint16_t c_date; uint16_t c_time;
      entry.getCreateDateTime(&c_date, &c_time);
      Serial.printf("  %d-%02d-%02d %02d:%02d:%02d", FS_YEAR(c_date), FS_MONTH(c_date), FS_DAY(c_date), FS_HOUR(c_time), FS_MINUTE(c_time), FS_SECOND(c_time));
      entry.getModifyDateTime(&c_date, &c_time);
      Serial.printf("    %d-%02d-%02d %02d:%02d:%02d\n", FS_YEAR(c_date), FS_MONTH(c_date), FS_DAY(c_date), FS_HOUR(c_time), FS_MINUTE(c_time), FS_SECOND(c_time));

     }
     entry.close();
   }
}

void printSpaces(int num) {
  for (int i=0; i < num; i++) {
    Serial.print(" ");
  }
}

void waitforInput()
{
  Serial.println("Press anykey to continue");
  while (Serial.read() == -1) ;
  while (Serial.read() != -1) ;
}

Prints alot of extra stuff but really was meant for us to do some debugging.

I just commented out the remove file/directory lines. You are going to notice one issue that we just found. When you have a file in sub-directory you don't see its time stamp. Traced it to the name() function. Seems to only return the file name and not the fullpath to the file. If I comment out the if(p)... line (yeah I know should do that but was curious on what it would return. It does return the fullpath. Maybe we need to add another function to just return the full path???? Still playing with it.

Sample out put with the if(p) line commented out:

---------------
Directory
---------
FO Read::  directory
ONF::  next name = "file10.txt"
ONF:: pathname --- /file10.txt
/file10.txt                           12928  2021-09-24 09:59:40    2021-09-24 09:59:40
ONF::  next name = "structuredData"
ONF:: pathname --- /structuredData
/structuredData/
ONF::  next name = "temp_test.txt"
ONF:: pathname --- /structuredData/temp_test.txt
  /structuredData/temp_test.txt       19  2021-09-24 09:59:42    2021-09-24 09:59:42

Bytes Used: 32768, Bytes Total:67108864
Press anykey to continue

and if I leave it in:

---------------
Directory
---------
FO Read::  directory
ONF::  next name = "file10.txt"
ONF:: pathname --- /file10.txt
file10.txt                            12928  2021-09-24 10:13:36    2021-09-24 10:13:36
ONF::  next name = "structureData1"
ONF:: pathname --- /structureData1
structureData1/
ONF::  next name = "temp_test.txt"
ONF:: pathname --- /structureData1/temp_test.txt
  temp_test.txt                       19  1980-00-00 00:00:00GMT: Error value clearedtemp_test.txt
    1980-00-00 00:00:00

Note: I am using a Locked T4 and the memory board for testing.

As for the TimeLibrary - yes it would be a lot easier.

EDIT: ONF = open next file, GMT - getModifiedTime, FO - fileopen functions.

 

[PaulStoffregen]

Looks like you're storing 2 attributes for the 16 bit numbers of FAT filesystems?

 

[mjs513]

Looks like you're storing 2 attributes for the 16 bit numbers of FAT filesystems?

Well kind of. Using lfs_setattr we are storing the modified timestamp and the created timestamp using the TimeLibraries timeelement structure.

				time_t _now = now();
				rcode = lfs_getattr(&lfs, filepath, 'c', (void *)&filetime, sizeof(filetime));
				if(rcode != sizeof(filetime)) {
					rcode = lfs_setattr(&lfs, filepath, 'c', (const void *) &_now, sizeof(_now));
					if(rcode < 0)
						Serial.println("FO:: set attribute creation failed");
				}
				rcode = lfs_setattr(&lfs, filepath, 'm', (const void *) &_now, sizeof(_now));
				if(rcode < 0)
					Serial.println("FO:: set attribute modified failed");

'c' is for created time and 'm' is for modified time. This is an extract from the open in littleFS.

EDIT: Oh finally woke up enough to realize and just use fullpath in the getmodified and getCreation functions Doh!

 

[mjs513]

Just pushed an update to the FS_Integration branch (https://github.com/mjs513/LittleFS/tree/FS_Integration) that fixes the issue with timestamps on file dates in subdirectories.

 

[KurtE]

Sounds great: I also fixed an issue today in mtp code when you create a directory on the PC side, it was not getting the dates and times from the file... Also found we were getting it when we called it for a normal file, but we did not actually update the mtp record with the data. Probably never saw it as most of the time files created on PC, are by copy of a file, and with this MTP tells us to then update the dates and times of the file...

Yes it would be great to have the core properly setup to allow simple dates/times to work by default... Obviously someone may want to override the default to maybe allow some ethernet call to time server...

 

[KurtE]

@Paul - if you are looking at the Time stamp stuff,

With our current stuff, our current File/FileImpl code follows the same methods for getting/setting the file dates/time as the SDFat code defined:

#ifdef FS_FILE_SUPPORT_DATES
	// These will all return false as only some FS support it.
  	virtual bool getAccessDateTime(uint16_t* pdate, uint16_t* ptime) {
  		return (f) ? f->getAccessDateTime(pdate, ptime) : false;
  	}
  	virtual bool getCreateDateTime(uint16_t* pdate, uint16_t* ptime) {
  		return (f) ? f->getCreateDateTime(pdate, ptime) : false;
  	}
  	virtual bool getModifyDateTime(uint16_t* pdate, uint16_t* ptime) {
  		return (f) ? f->getModifyDateTime(pdate, ptime) : false;
  	}
  	virtual bool timestamp(uint8_t flags, uint16_t year, uint8_t month, uint8_t day,
                 uint8_t hour, uint8_t minute, uint8_t second) {
  		return (f) ? f->timestamp(flags, year, month, day, hour, minute, second) : false;
  	}
#endif

When we added it in I kept it with the timestamp method, as that is in SDFat, but I did not define the flags here.

I was waiting to get your thoughts on it:
Assuming we do integrate dates and times into the system I think we should do:

Either:
a) Keep this way but define an enum within the File class like:
enum {T_ACCESS = 1, T_CREATE = 2, T_WRITE = 4 };
So callers of it can call it like myfile.timestamp(File::T_CREATE, ...

or

b) get rid of timestamp and add

  	virtual bool setAccessDateTime(uint16_t date, uint16_t time) {
  		return (f) ? f->setAccessDateTime(date, time) : false;
  	}
  	virtual bool setCreateDateTime(uint16_t date, uint16_t time) {
  		return (f) ? f->setCreateDateTime(date, time) : false;
  	}
  	virtual bool setModifyDateTime(uint16_t date, uint16_t time) {
  		return (f) ? f->setModifyDateTime(date, time) : false;
  	}

Personally I prefer b) as it is obvious what it should do, and you don't need to pass in, the year, month, day...

Thoughts? if you also prefer b) I am pretty sure we can make the changes pretty quickly.

Kurt

 

[PaulStoffregen]

Ok, so much to discuss. First, let's talk about the File base class.

Looks like everything so far has been built on top of SdFat's approach of a pair of pointers to 16 bit integers for FAT format date & time. For a public API, I really want to use Time's TimeElements or C library struct tm, rather than FAT specific integers. I know that's going to be painful to rewrite some code, but in the long run I believe we'll be much better off with the public API using one of these generic structs rather than FAT's integers.

The TimeElements struct is a lot smaller. It also means we can treat the 32 bit time as unsigned, which goes until 2106 rather than only 2038 if we go with C library stuff and don't use 64 bit time_t. Then again, the C library stuff is "standard"... difficult decisions.... but the main thing I want to see is a single pointer to a more generic struct rather than 2 pointers to 16 bit FAT-specific integers.


For implementation on SD, I'm thinking SdClass will provide a SdFat callback which is automatically assigned in SD.begin(). This is the code I've been experimenting with today.

void SDClass::dateTime(uint16_t *date, uint16_t *time)
{
        time_t t = Teensy3Clock.get();
        if (t < 315532800) {
                // before 1980
                *date = 0;
                *time = 0;
                return;
        }
        struct tm datetime;
        gmtime_r(&t, &datetime);
        // 15-9:  year (0 = 1980)
        // 8-5:   month (1-12)
        // 4-0:   day (1-21)
        *date = ((datetime.tm_year - 80) << 9) | ((datetime.tm_mon + 1) << 5) | datetime.tm_mday;
        // 15-11: hours (0-23)
        // 10-5:  minutes (0-29)
        // 4-0:   seconds/2 (0-29)
        *time = (datetime.tm_hour << 11) | (datetime.tm_min << 5) | (datetime.tm_sec >> 1);
}

Inside LittleFS, you can just call Teensy3Clock.get() rather than TimeLib now().

 

[KurtE]

Hi Paul,

Personally sounds reasonable to me.

I did not really care what exactly the API was, but more figuring out the pieces and wanted the ability to get/set this data.

We have most of pieces compiled with the SDFat version in branch FS_Integration, which once you give word on it will convert over to new stuff

It will great to get this stuff all officially integrated.

Edit: Forgot to say, lets us know when you have a Fork/Branch FS.h and the like to start playing with

 

[mjs513]

Yep I agree.

Just replaced Teensy3Clock.get() with now() in LittleFS and its working. Did find I forget to change the modified when you rename a file so added that in. going to push those changes up now.

 

[mjs513]

Here is a slightly updated test sketch for LittleFS.

/*
  LittleFS usuage from the LittleFS library
  
  Starting with Teensyduino 1.54, support for LittleFS has been added.

  LittleFS is a wrapper for the LittleFS File System for the Teensy family of microprocessors and provides support for RAM Disks, NOR and NAND Flash chips, and FRAM chips. For the NOR, NAND Flash support is provided for SPI and QSPI in the case of the Teensy 4.1. For FRAM only SPI is supported. It is also linked to SDFat so many of the same commands can be used as for an SD Card.
  
  This example shows the use of some of the commands provided in LittleFS using a SPI Flash chip such as the W25Q128. 
  
  See the readme for the LittleFS library for more information: https://github.com/PaulStoffregen/LittleFS
  
*/

#include <LittleFS.h>
#include <SD.h>

// Some variables for later use
uint64_t fTot, totSize1;

// To use SPI flash we need to create a instance of the library telling it to use SPI flash.
/* Other options include:
  LittleFS_QSPIFlash myfs;
  LittleFS_Program myfs;
  LittleFS_SPIFlash myfs;
  LittleFS_SPIFram myfs;
  LittleFS_SPINAND myfs;
  LittleFS_QPINAND myfs;
  LittleFS_RAM myfs;
*/
LittleFS_SPIFlash myfs;

// Since we are using SPI we need to tell the library what the chip select pin
#define chipSelect 6  // use for access flash on audio or prop shield

// Specifies that the file, file1 and file3 are File types, same as you would do for creating files
// on a SD Card
File file, file1, file2;


void setup()
{
  // Open serial communications and wait for port to open:
  Serial.begin(115200);
  while (!Serial) {
    // wait for serial port to connect.
  }
  Serial.println("\n" __FILE__ " " __DATE__ " " __TIME__);

  Serial.print("Initializing LittleFS ...");

  // see if the Flash is present and can be initialized:
  // Note:  SPI is default so if you are using SPI and not SPI for instance
  //        you can just specify myfs.begin(chipSelect). 
  if (!myfs.begin(chipSelect, SPI)) {
    Serial.printf("Error starting %s\n", "SPI FLASH");
    while (1) {
      // Error, so don't do anything more - stay stuck here
    }
  }
  myfs.quickFormat();
  Serial.println("LittleFS initialized.");
  
  
  // To get the current space used and Filesystem size
  Serial.println("\n---------------");
  Serial.printf("Bytes Used: %llu, Bytes Total:%llu\n", myfs.usedSize(), myfs.totalSize());
  waitforInput();

  // Now lets create a file and write some data.  Note: basically the same usage for 
  // creating and writing to a file using SD library.
  Serial.println("\n---------------");
  Serial.println("Now lets create a file with some data in it");
  Serial.println("---------------");
  char someData[128];
  memset( someData, 'z', 128 );
  file = myfs.open("bigfile.txt", FILE_WRITE);
  file.write(someData, sizeof(someData));

  for (uint16_t j = 0; j < 100; j++)
    file.write(someData, sizeof(someData));
  file.close();
  
  // We can also get the size of the file just created.  Note we have to open and 
  // thes close the file unless we do file size before we close it in the previous step
  file = myfs.open("bigfile.txt", FILE_WRITE);
  Serial.printf("File Size of bigfile.txt (bytes): %u\n", file.size());
  file.close();

  // Now that we initialized the FS and created a file lets print the directory.
  // Note:  Since we are going to be doing print directory and getting disk usuage
  // lets make it a function which can be copied and used in your own sketches.
  listFiles();
  waitforInput();
  
  // Now lets rename the file
  Serial.println("\n---------------");
  Serial.println("Rename bigfile to file10");
  myfs.rename("bigfile.txt", "file10.txt");
  listFiles();
  waitforInput();

  Serial.println("\n---------------");
  Serial.println("Create a directory and a subfile");
  myfs.mkdir("structureData1");

  file = myfs.open("structureData1/temp_test.txt", FILE_WRITE);
  file.println("SOME DATA TO TEST");
  file.close();
  listFiles();
  waitforInput();

  Serial.println("\n---------------");
  Serial.println("Rename directory");
  myfs.rename("structureData1", "structuredData");
  listFiles();
  waitforInput();


  Serial.println("\n---------------");
  Serial.println("Now lets create a file and read the data back...");
  
  // LittleFS also supports truncate function similar to SDFat. As shown in this
  // example, you can truncate files.
  //
  Serial.println();
  Serial.println("Writing to datalog.bin using LittleFS functions");
  file1 = myfs.open("datalog.bin", FILE_WRITE);
  unsigned int len = file1.size();
  Serial.print("datalog.bin started with ");
  Serial.print(len);
  Serial.println(" bytes");
  if (len > 0) {
    // reduce the file to zero if it already had data
    file1.truncate();
  }
  file1.print("Just some test data written to the file (by SdFat functions)");
  file1.write((uint8_t) 0);
  file1.close();

  // You can also use regular SD type functions, even to access the same file.  Just
  // remember to close the file before opening as a regular SD File.
  //
  Serial.println();
  Serial.println("Reading to datalog.bin using LittleFS functions");
  file2 = myfs.open("datalog.bin");
  if (file2) {
    char mybuffer[100];
    int index = 0;
    while (file2.available()) {
      char c = file2.read();
      mybuffer[index] = c;
      if (c == 0) break;  // end of string
      index = index + 1;
      if (index == 99) break; // buffer full
    }
    mybuffer[index] = 0;
    Serial.print("  Read from file: ");
    Serial.println(mybuffer);
  } else {
    Serial.println("unable to open datalog.bin :(");
  }
  file2.close();


  Serial.println("\nBasic Usage Example Finished");
}

void loop() {}

void listFiles()
{
  Serial.println("---------------");
  printDirectory(myfs);
  Serial.printf("Bytes Used: %llu, Bytes Total:%llu\n", myfs.usedSize(), myfs.totalSize());
}

void printDirectory(FS &fs) {
  Serial.println("Directory\n---------");
  printDirectory(fs.open("/"), 0);
  Serial.println();
}

void printDirectory(File dir, int numSpaces) {
   while(true) {
     File entry = dir.openNextFile();
     if (! entry) {
       //Serial.println("** no more files **");
       break;
     }
     printSpaces(numSpaces);
     Serial.print(entry.name());
     if (entry.isDirectory()) {
       Serial.print("/");
        uint16_t c_date; uint16_t c_time;
        entry.getCreateDateTime(&c_date, &c_time);
        Serial.printf("  %d-%02d-%02d %02d:%02d:%02d", FS_YEAR(c_date), FS_MONTH(c_date), FS_DAY(c_date), FS_HOUR(c_time), FS_MINUTE(c_time), FS_SECOND(c_time));
        entry.getModifyDateTime(&c_date, &c_time);
        Serial.printf("    %d-%02d-%02d %02d:%02d:%02d\n", FS_YEAR(c_date), FS_MONTH(c_date), FS_DAY(c_date), FS_HOUR(c_time), FS_MINUTE(c_time), FS_SECOND(c_time));
       Serial.println("--------------------------------------------------------------------");
       printDirectory(entry, numSpaces+2);
     } else {
       // files have sizes, directories do not
       printSpaces(36 - numSpaces - strlen(entry.name()));
       Serial.print("  ");
      Serial.print(entry.size(), DEC);
      uint16_t c_date; uint16_t c_time;
      entry.getCreateDateTime(&c_date, &c_time);
      Serial.printf("  %d-%02d-%02d %02d:%02d:%02d", FS_YEAR(c_date), FS_MONTH(c_date), FS_DAY(c_date), FS_HOUR(c_time), FS_MINUTE(c_time), FS_SECOND(c_time));
      entry.getModifyDateTime(&c_date, &c_time);
      Serial.printf("    %d-%02d-%02d %02d:%02d:%02d\n", FS_YEAR(c_date), FS_MONTH(c_date), FS_DAY(c_date), FS_HOUR(c_time), FS_MINUTE(c_time), FS_SECOND(c_time));
     }
     entry.close();
   }
}

void printSpaces(int num) {
  for (int i=0; i < num; i++) {
    Serial.print(" ");
  }
}

void waitforInput()
{
  Serial.println("Press anykey to continue");
  while (Serial.read() == -1) ;
  while (Serial.read() != -1) ;
}

deleted unused code and added dates on directories.

 

[PaulStoffregen]

Took a quick look at ESP cores, as they're the only other (that I know of) defining a File base class. Looks like both are using time_t.

ESP8266 has these:

    time_t fileTime();
    time_t fileCreationTime();

ESP32 has only this:

    time_t getLastWrite();

Neither seems to have a way to set it.

While time_t is tempting and feels "clean", I'm leaning towards TimeElements (or struct tm) partly because the most common use case is FAT32 and using the 3 date and 3 time fields where encoding to time_t for the API inside the library means just unpacking it again in the application, and also because 32 bit signed time_t only goes another 17 years.

 

[mjs513]

Actually looked that the ESP8266 (https://github.com/esp8266/Arduino/...addc336806f/libraries/LittleFS/src/LittleFS.h) implementation as well - the ESP32 core was much help. Was looking for an examples of using get and setattr. Pretty much what we do but kind of implemented to suit our needs.

In the esp8266 core they use lfs_attr in their void void close() override { function. Based on what I read you can set it but just doesn;t get written until you do a synch or file close. Chose to setattr's during open type functions.

 

[KurtE]

Again I think you got to go with your gut on this. Having some similarity with ESP is maybe a good thing, so time_t does as you say sound clean.

You could do it like:

   time_t fileTime();
    time_t fileCreationTime();
    void fileTime(time_t t);
    void fileCreationTime(time_t t);

And obviously similar with get/set methods.

But as for time_t versus TimeElements... Again go for which one you feel most comfortable with!

 

[mjs513]

Hi Paul - sorry got a bit distracted today. As @KurtE said just let us know which way you feel comfortable with.

 

[PaulStoffregen]

I've been experimenting this afternoon with bringing the Time library breakTime() and makeTime() functions into the core library. Sadly, it's looking like a thorny mess of conflicts. The typedef conficts if TimeLib.h is included, and putting it into core_pins.h where the RTC stuff is defined means time_t isn't always defined unless I bring in more C library headers. Arduino ecosystem is already a minefield of include issues and the combinations of which headers include which others isn't something I want to risk changing.

Much as I'd like to use something "standard", going with C library time_t puts us on track for 2038 rollover and struct tm means we use a lot more memory (maybe not a big deal on Teensy 4, but still...) So I'm really leaning towards uint32_t to match our hardware and our own non-standard typedef for broken down time.

Will probably include an inline getLastWrite() for compatibility with ESP32.

 

[mjs513]

Too bad about getting some of the Timelib stuff into the core.

Y2038: you all made me look up what the 2038 problem is, made interesting reading. But from what I read the implementation was to convert to 64bit time_t. Refs: https://en.m.wikipedia.org/wiki/Year_2038_problem and glibc: https://sourceware.org/glibc/wiki/Y2038ProofnessDesign.

getLastWrite and getCreationDate are equivalent to getModifiedTime and getCreationTime that we have private for LittleFS. Think we went with the naming convention that went along with Windows explorer. Sometimes I get confused here in whether we are talking about specific changes to LittleFS or the generic Integration case (yeah probably bad wording) that go across file implementations FS/SD/UsbMSCFat/MTP.

Ok going back to sleep now.

 

[PaulStoffregen]

I committed time functions the core library.

https://github.com/PaulStoffregen/cores/commit/a8b96bb41be4d84cdfed80f67bd2db57fb108175

Will work on FS.h soon...

 

[Mcu32]

Is there a way to write a library that uses a file and/or uses open(), i.e. to write a new file? How to handle all the different media?
It looks like, that it is not possible. I could imagine a way which uses typeof or such - but there is no RTTI.

How to implement this rather simple requirement?
In other words: What would a wrapper for open() look like? openanywhere( xzy, filename) xyz = filesystem { ???? }

 

[Mcu32]

There is an abstraction layer missing. Or maybe there is a C++ trick?

 

[PaulStoffregen]

There is an abstraction layer missing.

FS.h is indeed the abstraction layer, which was added in Teensyduino 1.54 and we're expanding for 1.56 to provide filesystem abstracted access to file creation and modification time.

https://github.com/PaulStoffregen/cores/blob/master/teensy4/FS.h

It's most definitely NOT missing!

Is there a way to write a library that uses a file and/or uses open(), i.e. to write a new file? How to handle all the different media?

Yes. Rather than explain with words, here is a sample program with a function that writes a file to both a SD card and a LittleFS flash chip. The myWrite() function is able to write to any type of media. Or at least any library which uses the FS & File base classes, which today is SD, LittleFS, MSC and (maybe) Frank's MemFS.

#include <SD.h>
#include <LittleFS.h>

LittleFS_SPIFlash myfs;

void myWrite(FS &filesystem, const char *filename) {
  File myfile = filesystem.open(filename, FILE_WRITE_BEGIN);
  if (myfile) {
    myfile.println("This is test data written to a file.");
    myfile.close();
    Serial.println("file written");
  } else {
    Serial.println("error opening file for write");
  }
}

void setup() {
  Serial.begin(9600);
  delay(1500);
  if (SD.begin(BUILTIN_SDCARD)) { // SD card on Teensy 4.1 SD socket
    myWrite(SD, "mytest1.txt");
  } else {
    Serial.println("Unable to access SD card");
  }
  if (myfs.begin(6)) { // SPI flash chip with CS on pin 6 (eg, audio shield)
    myWrite(myfs, "mytest2.txt");
  } else {
    Serial.println("Unable to access flash memory chip card");
  }
}

void loop() {
}

It does indeed work quite well, at least for all the functions FS.h defines... which are all the Arduino SD library API plus a couple extensions. Now we're extending it further to allow media independent file creation and modification time.

 

[luni]

Edit: ups crosspost

Is there a way to write a library that uses a file and/or uses open(), i.e. to write a new file? How to handle all the different media?
It looks like, that it is not possible. I could imagine a way which uses typeof or such - but there is no RTTI.

Here an example how to do this:

#include "Arduino.h"
#include "FS.h"
#include "LittleFS.h"

// library code ------------------------------------------------------
class MyClass
{
 public:
    MyClass(FS& fs) // user passes in the file system to use
        : fileSystem(fs)
    {}

    void writeSomething()
    {
        File file = fileSystem.open("test", FILE_WRITE_BEGIN);
        file.println("Bla bla");
        file.write((uint8_t)0);
        file.close();
    }

    String readSomething()
    {
        File file = fileSystem.open("test", FILE_READ);
        return file.readString(); // file destructor automatically closes file
    }

 protected:
    FS& fileSystem;
};

// usage -------------------------------------------

LittleFS_RAM lfs_ram;

MyClass myClass(lfs_ram);     // use the library on e.g. LittleFS_RAM

void setup()
{
    while (!Serial) {}
    lfs_ram.begin(1024);      // setup the filesystem

    myClass.writeSomething(); // use the library class
    Serial.println(myClass.readSomething());
}

void loop()
{
}

 

[PaulStoffregen]

Probably also worth mentioning LittleFS provides several classes for different type of media (Flash, NAND Flash, FRAM, volatile RAM, Program memory) and different connectivity (1 bit SPI vs 4 bit QSPI). The abstraction layer allows any of them to be used the same way as SD and MSC.