Modify the contents of a text file on an SD card

Status
Not open for further replies.

thecomfychair

Active member
As part of an audio project, I'm attempting modify the contents of text-based files on an SD card (M3U playlists in this case). My initial approach is to copy the contents of the file into a std::vector, erase the contents of the file, modify the vector, then write the contents of the vector back into the file.
(This approach may be flawed, any alternative suggestions are welcome too!)

I can successfully read the contents of a file to a vector. My next step is to erase the contents of the file, but here I run into issues:
In the SdFat_Usage example, the suggested method for quickly wiping the contents of a file is to simply truncate it down to 0. However, this seems to have no effect on the file - when the file is read again after truncating it, it has retained its size and contents.

Source code (compiled with Arduino 1.8.16, using TD 1.55. Running on a Teensy 4.1):
Code:
#include <SD.h>
#include <SPI.h>

//const int chipSelect = BUILTIN_SDCARD;

#include <string>
#include <vector>
#include <algorithm>

// some aliases to reduce typing....
using string = std::string;
using strVec = std::vector<string>;

strVec fileItems; //vector to store contents of text-based file
constexpr size_t fileItemsNameLen  = 256; //max no of characters each item in a text file can be

File myFile;

void setup() {
// Open serial communications and wait for port to open:
  Serial.begin(9600);
   while (!Serial) {;} // wait for serial port to connect
  
  Serial.print("Initializing SD card...");
  bool ok;
  ok = SD.sdfs.begin(SdioConfig(FIFO_SDIO));
  if (!ok) {
      Serial.println("initialization failed!");
      return;
    }
    Serial.println("initialization done.");
    Serial.println();


  readFile();

  clearFile();

  readFile();
  
}

void readFile(){
  myFile = SD.open("/MusicBee/Playlists/Test 3.m3u", FILE_READ); //open the playlist file in read mode

  if (myFile) {
    Serial.print("Opening file...");

      Serial.print("File size is: ");
      Serial.println(myFile.size());
      
      fileItems.clear();//clear the contents of the fileItems vector

      Serial.print("Copying file contents...");
      
      // read from the file until there's nothing else in it: NOTE currently seems to read one extra carriage return
      while (myFile.available()>1) { //if there is more than 1 byte left to read (to prevent single carriage returns being copied - playlist entries will always be >2)
        char buf[fileItemsNameLen];
        myFile.readBytesUntil('\n', buf,fileItemsNameLen); //reads chars from file until carriage return, although terminating char isn't included
        fileItems.emplace_back(buf);  //copy the line of chars to vector end of vector, increasing size automatically
        }
      }
  else {  // if the file didn't open, print an error:
    Serial.println("Error opening file");
    }
  myFile.close();
  Serial.print("File closed...");

  Serial.println("File contains: ");
  for(int i = 0; i<fileItems.size(); i++){
    Serial.println(fileItems.at(i).c_str());
  }
}

void clearFile(){
  Serial.print("Clearing contents of file...");
  
   myFile = SD.open("/MusicBee/Playlists/Test 3.m3u", FILE_READ); //open the playlist file in read mode
  
  Serial.print("File size before truncate: ");
  Serial.println(myFile.size());

  myFile.close();

  FsFile myFsFile = SD.sdfs.open("/MusicBee/Playlists/Test 1.m3u", O_WRITE); //open the playlist file in write mode
  
  Serial.print("Get filesize with FsFile function: ");
  Serial.println(myFsFile.fileSize());
  
  //check to see if there's anything in the playlist file
  if(myFsFile.fileSize() > 0){ 
    myFsFile.truncate(0); //if so, delete everything in the file - seems to work when you specify a size to truncate to, otherwise it just trims off excess file size
    Serial.print("Truncating...");
  }

//  Serial.print("File size after truncate: ");
//  Serial.println(myFsFile.fileSize());

  myFsFile.close();

  myFile = SD.open("/MusicBee/Playlists/Test 3.m3u", FILE_READ); //open the playlist file in read mode
  
  Serial.print("File size after truncate: ");
  Serial.println(myFile.size());

  myFile.close();

}

void loop() {
  // put your main code here, to run repeatedly:

}

Result:
Initializing SD card...initialization done.

Opening file...File size is: 292
Copying file contents...File closed...File contains:
/Music/Deadmau5/4x4=12 (Deluxe Version)/1-01 - Some Chords.m4a
/Music/Deadmau5/4x4=12 (Deluxe Version)/1-09 - Raise Your Weapon.m4a
/Music/Various Artists/Oh My God...It's Electro House Vol. 5/1- - Strobe (Club Edit).m4a
/Music/Deadmau5/4x4=12 (Deluxe Version)/1-09 - Raise Your Weapon.m4a
Clearing contents of file...File size before truncate: 292
Get filesize with FsFile function: 0
File size after truncate: 292
Opening file...File size is: 292
Copying file contents...File closed...File contains:
/Music/Deadmau5/4x4=12 (Deluxe Version)/1-01 - Some Chords.m4a
/Music/Deadmau5/4x4=12 (Deluxe Version)/1-09 - Raise Your Weapon.m4a
/Music/Various Artists/Oh My God...It's Electro House Vol. 5/1- - Strobe (Club Edit).m4a
/Music/Deadmau5/4x4=12 (Deluxe Version)/1-09 - Raise Your Weapon.m4a

After some time spent debugging, I'm wondering if none of the FsFile functions are working - note that when fileSize() is called it returns 0, whereas the standard SD size() function returns 292.

So basically what I'm asking is:
  1. Is this the correct approach to modifying the contents of text based files on an SD card?
  2. Is there a better way to clear the contents of a text file?
  3. Am I incorrectly implemeting the SDFat functions, causing them to fail?

Any thoughts greatly appreciated!
 
The FS parts are under edit since TD 1.55 - try with the latest TD 1.56 Beta ( now at #2 ) - if there are problems it can get resolved.

Seems like that should work.

Another way might be to rename the file to a known .bak copy ( that doesn't exist ) - then start fresh with the expected file name?

But if there is a bug with truncate it should get resolved. If you can post a simple sample repro on the TD 1.56 Beta thread that would help.
 
Looks like you are truncating "/MusicBee/Playlists/Test 1.m3u", yet you seem to be displaying the contents of "/MusicBee/Playlists/Test 3.m3u"
 
Looks like you are truncating "/MusicBee/Playlists/Test 1.m3u", yet you seem to be displaying the contents of "/MusicBee/Playlists/Test 3.m3u"

Indeed it does - good reading:
Code:
void clearFile(){
  Serial.print("Clearing contents of file...");
  
   myFile = SD.open("/MusicBee/Playlists/[U]Test 3.m3u[/U]", FILE_READ); //open the playlist file in read mode
  
  Serial.print("File size before truncate: ");
  Serial.println(myFile.size());

  myFile.close();

  FsFile myFsFile = SD.sdfs.open("/MusicBee/Playlists/[U]Test 1.m3u[/U]", O_WRITE); //open the playlist file in write mode
...
 
This project was shifted to the back burner for a couple of weeks, only just a chance to get back to it - apologies for the delayed reply.

Looks like you are truncating "/MusicBee/Playlists/Test 1.m3u", yet you seem to be displaying the contents of "/MusicBee/Playlists/Test 3.m3u"
You are absolutely correct, I've fixed that line and the output is now:

Initializing SD card...initialization done.

Opening file...File size is: 292
Copying file contents...File closed...File contains:
/Music/Deadmau5/4x4=12 (Deluxe Version)/1-01 - Some Chords.m4a
/Music/Deadmau5/4x4=12 (Deluxe Version)/1-09 - Raise Your Weapon.m4a
/Music/Various Artists/Oh My God...It's Electro House Vol. 5/1- - Strobe (Club Edit).m4a
/Music/Deadmau5/4x4=12 (Deluxe Version)/1-09 - Raise Your Weapon.m4a
Clearing contents of file...File size before truncate: 292
Get filesize with FsFile function: 292
Truncating...File size after truncate: 0
Opening file...File size is: 0
Copying file contents...File closed...File contains:
...and the file contains nothing when opened through Notepad on my PC. These are exactly the results I was looking for.

TL;DR: FS is still working in 1.55 and I'm a twit. Apologies for the confusion on my part, and thank you both for prompt replies!
 
Status
Not open for further replies.
Back
Top