Problems with (external) library using SDFat

But File isn't just a wrapper for SdFat...
It's a generic file handling class. Adding a function to it means every underlying filesystem object must support it. That's exactly the sort of thing that libc's formatted I/O functions are designed to workaround.
 
jmarsh said:
But File isn't just a wrapper for SdFat...
It's a generic file handling class. Adding a function to it means every underlying filesystem object must support it. That's exactly the sort of thing that libc's formatted I/O functions are designed to workaround.
Click to expand...
Think you better that a look at FS.h.
 
@mjs513 - I'm back at it. Going to clean up the code a bit and post it on GitHub. This is what I did for fgets() in readConfigLine():
Code: 
            // Read in a new line - Note: removes '\r' but leaves '\n'
            int bufferLength = origFile.read(lineBuffer, sizeof(lineBuffer));
The author set a fixed line buffer size of 40 bytes which meens that each line that is parsed in the config file cannot be any larger than 40 bytes including the '\r' and '\n'. That is where I am at now with the code. I can read in the config file info line by line but parsing it does not work correctly and it looks like it writes it to temporary file while parsing the info. It is a little hard to follow. Kinda like my code:ROFLMAO:

If I remember correctly you can access SdFat functions from SD using this:
Code: 
"SD.sdfs.xxxx()"
Have not tested this yet. I'll post back when I have the code uploaded to GitHub...
 
Looks like everyone is having fun. Would join in but currently distracted from my distractions. 😀

IMG_0985.jpeg
 
Ok got SDFAT fgets working:
Here is the replacement file for SD.h which goes into the SD library
and FS.h which replaces the one in teensy 4 core:

Test sketch:
C++: 
#include <SD.h>
#include <Streaming.h>
#define cout Serial
File wrfile;
File rdfile;
void setup() {
  Serial.begin(115200);
  delay(5000);
  SD.begin(BUILTIN_SDCARD);
  makeTestFile();
  demoFgets();
}
void loop() {
  // put your main code here, to run repeatedly:
}
void makeTestFile() {
  // create or open test file
  if(SD.exists("fgets.txt")) SD.remove("fgets.txt");
 
  wrfile = SD.open("fgets.txt", FILE_WRITE);
  // check for open error
  if (!wrfile) {
    Serial.println("MakeTestFile Error");
  } else {
    cout << "makeTestFile opened!" << endl;
  }
  // write test file
  wrfile.print(F(
                 "Line with CRLF\r\n"
                 "Line with only LF\n"
                 "Long line that will require an extra read\n"
                 "\n"  // empty line
                 "Line at EOF without NL"
               ));
  wrfile.close();
}
void demoFgets() {
  char line[25];
  int n;
  // open test file
  rdfile = SD.open("fgets.txt", FILE_READ);
  // check for open error
  if (!rdfile) {
    Serial.println("demoFgets error");
  } else {
    cout << "demo read opened" << endl;
  }
  cout << endl << F(
         "Lines with '>' end with a '\\n' character\n"
         "Lines with '#' do not end with a '\\n' character\n"
         "\n");
  // read lines from the file
  while ((n = rdfile.fgets(line, sizeof(line))) > 0) {
    if (line[n - 1] == '\n') {
      cout << '>' << line;
    } else {
      cout << '#' << line << endl;
    }
  }
}

OUTPUT
Code: 
makeTestFile opened!
demo read opened

Lines with '>' end with a '\n' character
Lines with '#' do not end with a '\n' character

>Line with CRLF
>Line with only LF
#Long line that will requ
>ire an extra read
>
#Line at EOF without NL
 

Attachments

  • SD.h
    8.4 KB · Views: 14
  • FS.h
    10.1 KB · Views: 11
@mjs513 - Will test and patch that in.
Here is the link to the work I have done so far. The get() function(s) are only working with "int" values and "arduinoStringValue". This may have to do with a variable called "paramFound" in this function:
Code: 
/**
 * Check whether a string matches the current config entry name
 * @param[in]  itemName  The string against which to check the current entry
 * @return     True if the names match, false otherwise
 */
bool SdConfigFile::checkItemName(const char *itemName) {
    // If a matching parameter has already been found, no need to check again
    if (paramFound) return false;

    // Remove spaces from front of character array
    while (currentPos[0] == ' ') currentPos++;
    int stringLength = strlen(currentPos);

    // Remove spaces, tabs and line ending characters from end of character array
    while (stringLength > 0 && discardChar(currentPos[stringLength - 1])) stringLength--;
    currentPos[stringLength] = '\0';

    // Check if both name strings match and that the string isn't empty
    if (strcmp(itemName, currentPos) == 0 && stringLength != 0) {
        paramFound = true;
        currentPos = strtok(NULL, "=");
        if (currentPos) return true;
    }

    return false;
}
This is output from a modified version of the @Nanoprecision sketch:
Code: 
initialization Passed: OK.

lineBuffer = intValue=1234
longValue=12678
floatValue=0.24689
arduinoStringValue=Hello, this is a string with spaces
cStringValue=This is als)
lineBuffer = o a string

boolValue1=True
boolValue2=0
boolValue3=false

ingValue=Hello, this is a string with spaces
cStringValue=This is als)
intValue: 1234
longValue: 0
Notice that the long value returned 0 instead of 12678. Changing the order of the get() calls did not change the results either.
This is the output so far of the modified original sd-config-file library:
Code: 
initialization Passed: OK.

--- Read Method 1 - While Loop ---
lineBuffer = intValue=1234
longValue=12678
floatValue=0.24689
arduinoStringValue=Hello, this is a string with spaces
cStringValue=This is als0)
lineBuffer = o a string

boolValue1=True
boolValue2=0
boolValue3=false

ingValue=Hello, this is a string with spaces
cStringValue=This is als0)
intValue: 1234
longValue: 0
floatValue: 0.00000
boolValue1: 0
boolValue2: 0
arduinoString:
cStringValue:
Lastly the printToFile() function is not working either shown here:
Code: 
/**
 * Print data to temporary file if tempFile and currentPos.
 */
void SdConfigFile::printLineToFile() {
    if ((tempFile) && (currentPos != NULL)) {
        tempFile.print(currentPos);
        if (equalsSplit) {                  // If the line was already split
            currentPos = strtok(NULL, "="); // Get the value on other side of the equals sign
            checkItemName("");              // Strips spaces, etc. from the value
            tempFile.print("=");            // Add in the equals sign again
            tempFile.println(currentPos);   // Print the value to the file
        }
    }
}
A File object named tempFile is opened elsewhere in the code and is named "_temp".
This is as far as I have gotten with testing so far...
 
@mjs513 - Trying out your test sketch for the modified FS.h and SD.h files, I am getting this error:
Code: 
/home/wwatson/arduino-1.8.19-TD1.60B5/arduino-builder -dump-prefs -logger=machine -hardware /home/wwatson/arduino-1.8.19-TD1.60B5/hardware -hardware /home/wwatson/Arduino/hardware -tools /home/wwatson/arduino-1.8.19-TD1.60B5/tools-builder -tools /home/wwatson/arduino-1.8.19-TD1.60B5/hardware/tools/avr -built-in-libraries /home/wwatson/arduino-1.8.19-TD1.60B5/libraries -libraries /home/wwatson/Arduino/libraries -fqbn=teensy:avr:teensy41:usb=serial,speed=600,opt=o2std,keys=en-us -ide-version=10819 -build-path /tmp/arduino_build_422842 -warnings=none -build-cache /tmp/arduino_cache_308324 -verbose /home/wwatson/Arduino/SD_fgets_Test/SD_fgets_Test.ino
/home/wwatson/arduino-1.8.19-TD1.60B5/arduino-builder -compile -logger=machine -hardware /home/wwatson/arduino-1.8.19-TD1.60B5/hardware -hardware /home/wwatson/Arduino/hardware -tools /home/wwatson/arduino-1.8.19-TD1.60B5/tools-builder -tools /home/wwatson/arduino-1.8.19-TD1.60B5/hardware/tools/avr -built-in-libraries /home/wwatson/arduino-1.8.19-TD1.60B5/libraries -libraries /home/wwatson/Arduino/libraries -fqbn=teensy:avr:teensy41:usb=serial,speed=600,opt=o2std,keys=en-us -ide-version=10819 -build-path /tmp/arduino_build_422842 -warnings=none -build-cache /tmp/arduino_cache_308324 -verbose /home/wwatson/Arduino/SD_fgets_Test/SD_fgets_Test.ino
Using board 'teensy41' from platform in folder: /home/wwatson/arduino-1.8.19-TD1.60B5/hardware/teensy/avr
Using core 'teensy4' from platform in folder: /home/wwatson/arduino-1.8.19-TD1.60B5/hardware/teensy/avr
Detecting libraries used...
/home/wwatson/arduino-1.8.19-TD1.60B5/hardware/teensy/../tools/arm/bin/arm-none-eabi-g++ -E -CC -x c++ -w -g -Wall -ffunction-sections -fdata-sections -nostdlib -std=gnu++17 -fno-exceptions -fpermissive -fno-rtti -fno-threadsafe-statics -felide-constructors -Wno-error=narrowing -Wno-psabi -mthumb -mcpu=cortex-m7 -mfloat-abi=hard -mfpu=fpv5-d16 -D__IMXRT1062__ -DTEENSYDUINO=160 -DARDUINO=10819 -DARDUINO_TEENSY41 -DF_CPU=600000000 -DUSB_SERIAL -DLAYOUT_US_ENGLISH -I/home/wwatson/arduino-1.8.19-TD1.60B5/hardware/teensy/avr/cores/teensy4 /tmp/arduino_build_422842/sketch/SD_fgets_Test.ino.cpp -o /dev/null
Alternatives for SD.h: [SD@1.2.4 SD@2.0.0]
ResolveLibrary(SD.h)
  -> candidates: [SD@1.2.4 SD@2.0.0]
/home/wwatson/arduino-1.8.19-TD1.60B5/hardware/teensy/../tools/arm/bin/arm-none-eabi-g++ -E -CC -x c++ -w -g -Wall -ffunction-sections -fdata-sections -nostdlib -std=gnu++17 -fno-exceptions -fpermissive -fno-rtti -fno-threadsafe-statics -felide-constructors -Wno-error=narrowing -Wno-psabi -mthumb -mcpu=cortex-m7 -mfloat-abi=hard -mfpu=fpv5-d16 -D__IMXRT1062__ -DTEENSYDUINO=160 -DARDUINO=10819 -DARDUINO_TEENSY41 -DF_CPU=600000000 -DUSB_SERIAL -DLAYOUT_US_ENGLISH -I/home/wwatson/arduino-1.8.19-TD1.60B5/hardware/teensy/avr/cores/teensy4 -I/home/wwatson/arduino-1.8.19-TD1.60B5/hardware/teensy/avr/libraries/SD/src /tmp/arduino_build_422842/sketch/SD_fgets_Test.ino.cpp -o /dev/null
Alternatives for Teensy_SdFat.h: []
ResolveLibrary(Teensy_SdFat.h)
  -> candidates: []
In file included from /home/wwatson/Arduino/SD_fgets_Test/SD_fgets_Test.ino:1:
Multiple libraries were found for "SD.h"
/home/wwatson/arduino-1.8.19-TD1.60B5/hardware/teensy/avr/libraries/SD/src/SD.h:27:10: fatal error: Teensy_SdFat.h: No such file or directory
 Used: /home/wwatson/arduino-1.8.19-TD1.60B5/hardware/teensy/avr/libraries/SD
   27 | #include <Teensy_SdFat.h>
 Not used: /home/wwatson/arduino-1.8.19-TD1.60B5/libraries/SD
      |          ^~~~~~~~~~~~~~~~
compilation terminated.
Using library SD at version 2.0.0 in folder: /home/wwatson/arduino-1.8.19-TD1.60B5/hardware/teensy/avr/libraries/SD
Error compiling for board Teensy 4.1.
The original SD.h contains:
Code: 
#include <SdFat.h>
and the new version contains:
Code: 
#include <Teensy_SdFat.h>
I am not sure what I am missing...
Edit: Fixed it by changing #include <Teensy_SdFat.h> to #include <SdFat.h>. Hope that is correct as I saw the fix on the Arduino forum :D
 
Last edited:
@mjs513 There is a while(1){;} in the original "ReadWrite_ConfigFile.ino" sketch just before this:
Code: 
    /**
     * Writing method 1 - using a while loop
     */
Edit: And this is the contents of the "test_file.txt" on the SDcard I am using (all comments removed):
Code: 
intValue=1234
longValue=12678
floatValue=0.24689
arduinoStringValue=Hello, this is a string with spaces
cStringValue=This is also a string

boolValue1=True
boolValue2=0
boolValue3=false
EDIT2: Your sketch and changes to FS.h and SD.h work with TD-1.59 as well...
Sorry I tend to loose track of what I am doing...
 
Last edited:
wwatson said:
Looks suspiciously similar in parsing although not the same. @Nanoprecision - Have you seen this?
Click to expand...
yes, I‘ve seen this, but I need to update values and this lib does not support write.

From the readme.md:
“Write support is a feature that has been requested on several occasions but as I am no longer using the IniFile library I will not add this feature“
 
Nanoprecision said:
yes, I‘ve seen this, but I need to update values and this lib does not support write.

From the readme.md:
“Write support is a feature that has been requested on several occasions but as I am no longer using the IniFile library I will not add this feature“
Click to expand...
Got it.
 
Adding fgets() fixes a few of the issues:
Code: 
initialization Passed: OK.

--- Read Method 1 - While Loop ---
lineBuffer = intValue=1234
lineBuffer = longValue=12678
lineBuffer = floatValue=0.24689
lineBuffer = arduinoStringValue=Hello, this is a string with spaces
lineBuffer = cStringValue=This is also a string
lineBuffer =
lineBuffer = boolValue1=True
lineBuffer = boolValue2=0
lineBuffer = boolValue3=false
lineBuffer =
intValue: 1234
longValue: 12678
floatValue: 0.24689
boolValue1: 1
boolValue2: 0
arduinoString:
cStringValue: This is also a stri
Good stuff :D
 
mjs513 said:
Think you better that a look at FS.h.
Click to expand...
I think you need to actually understand what the FS class is for - what happens when someone calls your new fgets method on an FS implementation (e.g. LittleFS) that doesn't support it? Or even a File object that isn't a currently opened file?
You've just created a headache for anyone who has implemented FS to support other filesystems, i.e. the way it was intended to be used.
 
jmarsh said:
your new fgets method
Click to expand...
As I said it is not a new method.
fgets is a method that is already implemented in SdFat!!! It is now new - it is just be exposed as other functions that are currently in FS class.

Will have little impact to other filesystems unless you deceide you want to use the fgets function. For instance, if you look in LittleFS.h you will see it uses it own wrapper to SdFat methods. Since I haven't added this fgets littleFs does not support calling fgets!
 
Last edited:
mjs513 said:
For instance, if you look in LittleFS.h you will see it uses it own wrapper to SdFat methods.
Click to expand...
No it doesn't. You have it backwards; LittleFS and SdFat are both implementations of the FS interface. Some of the functions may have been "inspired" by the SdFat functions originally but that is because they are generic enough to fit any standard filesystem interface. fgets has nothing to do with filesystems, it is purely a convenience function that operates on top of read/peek. If you implement that one then what's the reasoning for not implementing fprintf, fscanf and all the other libc formatted I/O functions?

mjs513 said:
Since I haven't added this fgets littleFs does not support calling fgets!
Click to expand...
It will be usable with LittleFS because you've modified the base FS class. It just won't work because only SdFat has it implemented instead of being a generic method. Did you try the readBytesUntil method I mentioned earlier, which would work with any FS implementation?

You seem to have a basic misunderstanding about how class inheritance works and what the purpose of the FS class is; it is not just a wrapper around SdFat, it is intended to be an abstract interface so an app can access any filesystem without worrying about what methods are/aren't supported.
 
@jmarsh
Not going to keep arguing about this. You have your opinion and I have mine. Truth probsbly somewhere in between. End of story.
 
From the readme.md:
“Write support is a feature that has been requested on several occasions but as I am no longer using the IniFile library I will not add this feature“
Click to expand...
Okay, good answer. How many parameters do you need to store, and how often will they be updated?
 
joepasquariello said:
Okay, good answer. How many parameters do you need to store, and how often will they be updated?
Click to expand...

The Teensy will control a measurement equipment, so I need to be able to store machine data, data of used sensor(s), motor parameters etc. and to be able to allow user to make adaptions for their needs with different settings.
Number of parameters will be somewhere between 50 and 100. Update frequency: Hmmm. After initial setup with some 10 optimization loops maybe 5x a day?
 
You must log in or register to reply here.
Back
Top