Forum Rule: Always post complete source code & details to reproduce any issue!
Results 1 to 5 of 5

Thread: Modify the contents of a text file on an SD card

  1. #1

    Modify the contents of a text file on an SD card

    As part of an audio project, I'm attempting modify the contents of text-based files on an SD card (M3U playlists in this case). My initial approach is to copy the contents of the file into a std::vector, erase the contents of the file, modify the vector, then write the contents of the vector back into the file.
    (This approach may be flawed, any alternative suggestions are welcome too!)

    I can successfully read the contents of a file to a vector. My next step is to erase the contents of the file, but here I run into issues:
    In the SdFat_Usage example, the suggested method for quickly wiping the contents of a file is to simply truncate it down to 0. However, this seems to have no effect on the file - when the file is read again after truncating it, it has retained its size and contents.

    Source code (compiled with Arduino 1.8.16, using TD 1.55. Running on a Teensy 4.1):
    Code:
    #include <SD.h>
    #include <SPI.h>
    
    //const int chipSelect = BUILTIN_SDCARD;
    
    #include <string>
    #include <vector>
    #include <algorithm>
    
    // some aliases to reduce typing....
    using string = std::string;
    using strVec = std::vector<string>;
    
    strVec fileItems; //vector to store contents of text-based file
    constexpr size_t fileItemsNameLen  = 256; //max no of characters each item in a text file can be
    
    File myFile;
    
    void setup() {
    // Open serial communications and wait for port to open:
      Serial.begin(9600);
       while (!Serial) {;} // wait for serial port to connect
      
      Serial.print("Initializing SD card...");
      bool ok;
      ok = SD.sdfs.begin(SdioConfig(FIFO_SDIO));
      if (!ok) {
          Serial.println("initialization failed!");
          return;
        }
        Serial.println("initialization done.");
        Serial.println();
    
    
      readFile();
    
      clearFile();
    
      readFile();
      
    }
    
    void readFile(){
      myFile = SD.open("/MusicBee/Playlists/Test 3.m3u", FILE_READ); //open the playlist file in read mode
    
      if (myFile) {
        Serial.print("Opening file...");
    
          Serial.print("File size is: ");
          Serial.println(myFile.size());
          
          fileItems.clear();//clear the contents of the fileItems vector
    
          Serial.print("Copying file contents...");
          
          // read from the file until there's nothing else in it: NOTE currently seems to read one extra carriage return
          while (myFile.available()>1) { //if there is more than 1 byte left to read (to prevent single carriage returns being copied - playlist entries will always be >2)
            char buf[fileItemsNameLen];
            myFile.readBytesUntil('\n', buf,fileItemsNameLen); //reads chars from file until carriage return, although terminating char isn't included
            fileItems.emplace_back(buf);  //copy the line of chars to vector end of vector, increasing size automatically
            }
          }
      else {  // if the file didn't open, print an error:
        Serial.println("Error opening file");
        }
      myFile.close();
      Serial.print("File closed...");
    
      Serial.println("File contains: ");
      for(int i = 0; i<fileItems.size(); i++){
        Serial.println(fileItems.at(i).c_str());
      }
    }
    
    void clearFile(){
      Serial.print("Clearing contents of file...");
      
       myFile = SD.open("/MusicBee/Playlists/Test 3.m3u", FILE_READ); //open the playlist file in read mode
      
      Serial.print("File size before truncate: ");
      Serial.println(myFile.size());
    
      myFile.close();
    
      FsFile myFsFile = SD.sdfs.open("/MusicBee/Playlists/Test 1.m3u", O_WRITE); //open the playlist file in write mode
      
      Serial.print("Get filesize with FsFile function: ");
      Serial.println(myFsFile.fileSize());
      
      //check to see if there's anything in the playlist file
      if(myFsFile.fileSize() > 0){ 
        myFsFile.truncate(0); //if so, delete everything in the file - seems to work when you specify a size to truncate to, otherwise it just trims off excess file size
        Serial.print("Truncating...");
      }
    
    //  Serial.print("File size after truncate: ");
    //  Serial.println(myFsFile.fileSize());
    
      myFsFile.close();
    
      myFile = SD.open("/MusicBee/Playlists/Test 3.m3u", FILE_READ); //open the playlist file in read mode
      
      Serial.print("File size after truncate: ");
      Serial.println(myFile.size());
    
      myFile.close();
    
    }
    
    void loop() {
      // put your main code here, to run repeatedly:
    
    }
    Result:
    Initializing SD card...initialization done.

    Opening file...File size is: 292
    Copying file contents...File closed...File contains:
    /Music/Deadmau5/4x4=12 (Deluxe Version)/1-01 - Some Chords.m4a
    /Music/Deadmau5/4x4=12 (Deluxe Version)/1-09 - Raise Your Weapon.m4a
    /Music/Various Artists/Oh My God...It's Electro House Vol. 5/1- - Strobe (Club Edit).m4a
    /Music/Deadmau5/4x4=12 (Deluxe Version)/1-09 - Raise Your Weapon.m4a
    Clearing contents of file...File size before truncate: 292
    Get filesize with FsFile function: 0
    File size after truncate: 292
    Opening file...File size is: 292
    Copying file contents...File closed...File contains:
    /Music/Deadmau5/4x4=12 (Deluxe Version)/1-01 - Some Chords.m4a
    /Music/Deadmau5/4x4=12 (Deluxe Version)/1-09 - Raise Your Weapon.m4a
    /Music/Various Artists/Oh My God...It's Electro House Vol. 5/1- - Strobe (Club Edit).m4a
    /Music/Deadmau5/4x4=12 (Deluxe Version)/1-09 - Raise Your Weapon.m4a
    After some time spent debugging, I'm wondering if none of the FsFile functions are working - note that when fileSize() is called it returns 0, whereas the standard SD size() function returns 292.

    So basically what I'm asking is:
    1. Is this the correct approach to modifying the contents of text based files on an SD card?
    2. Is there a better way to clear the contents of a text file?
    3. Am I incorrectly implemeting the SDFat functions, causing them to fail?


    Any thoughts greatly appreciated!

  2. #2
    Senior Member+ defragster's Avatar
    Join Date
    Feb 2015
    Posts
    15,263
    The FS parts are under edit since TD 1.55 - try with the latest TD 1.56 Beta ( now at #2 ) - if there are problems it can get resolved.

    Seems like that should work.

    Another way might be to rename the file to a known .bak copy ( that doesn't exist ) - then start fresh with the expected file name?

    But if there is a bug with truncate it should get resolved. If you can post a simple sample repro on the TD 1.56 Beta thread that would help.

  3. #3
    Member
    Join Date
    Aug 2018
    Location
    Brisbane, Australia
    Posts
    79
    Looks like you are truncating "/MusicBee/Playlists/Test 1.m3u", yet you seem to be displaying the contents of "/MusicBee/Playlists/Test 3.m3u"

  4. #4
    Senior Member+ defragster's Avatar
    Join Date
    Feb 2015
    Posts
    15,263
    Quote Originally Posted by thebigg View Post
    Looks like you are truncating "/MusicBee/Playlists/Test 1.m3u", yet you seem to be displaying the contents of "/MusicBee/Playlists/Test 3.m3u"
    Indeed it does - good reading:
    Code:
    void clearFile(){
      Serial.print("Clearing contents of file...");
      
       myFile = SD.open("/MusicBee/Playlists/Test 3.m3u", FILE_READ); //open the playlist file in read mode
      
      Serial.print("File size before truncate: ");
      Serial.println(myFile.size());
    
      myFile.close();
    
      FsFile myFsFile = SD.sdfs.open("/MusicBee/Playlists/Test 1.m3u", O_WRITE); //open the playlist file in write mode
    ...

  5. #5
    This project was shifted to the back burner for a couple of weeks, only just a chance to get back to it - apologies for the delayed reply.

    Quote Originally Posted by thebigg View Post
    Looks like you are truncating "/MusicBee/Playlists/Test 1.m3u", yet you seem to be displaying the contents of "/MusicBee/Playlists/Test 3.m3u"
    You are absolutely correct, I've fixed that line and the output is now:

    Initializing SD card...initialization done.

    Opening file...File size is: 292
    Copying file contents...File closed...File contains:
    /Music/Deadmau5/4x4=12 (Deluxe Version)/1-01 - Some Chords.m4a
    /Music/Deadmau5/4x4=12 (Deluxe Version)/1-09 - Raise Your Weapon.m4a
    /Music/Various Artists/Oh My God...It's Electro House Vol. 5/1- - Strobe (Club Edit).m4a
    /Music/Deadmau5/4x4=12 (Deluxe Version)/1-09 - Raise Your Weapon.m4a
    Clearing contents of file...File size before truncate: 292
    Get filesize with FsFile function: 292
    Truncating...File size after truncate: 0
    Opening file...File size is: 0
    Copying file contents...File closed...File contains:
    ...and the file contains nothing when opened through Notepad on my PC. These are exactly the results I was looking for.

    TL;DR: FS is still working in 1.55 and I'm a twit. Apologies for the confusion on my part, and thank you both for prompt replies!

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •