my SDfat code - good in 2017 but won't compile now?

ninja2

ninja2

Well-known member
I have a number of sketches I wrote for my T3.5 & T3.6 back in 2017. They all successfully read &/or wrote to the SD card using the SdFat library by grieman (which, based on my sketch comment, was a beta release) using this code snippet:

Code: 
#include <SdFat.h>                    // grieman's SdFat_beta, unmodified
#include <SPI.h>
SdFatSdio SD;
File  inputfile;                                          // input data file
File outputfile;                                         // output results file

but now it won't compile in IDE 2.3.2 ...

Code: 
In file included from D:\...\02 ARDUINO\SKETCHES\x_ARCHIVES\TEENSY\t3MPUsim02\t3MPUsim02.ino:27:
d:\...\02 ARDUINO\SKETCHES\libraries\SdFat\src/SdFat.h:448:2: warning: #warning File not defined because __has_include(FS.h) [-Wcpp]
  448 | #warning File not defined because __has_include(FS.h)
      |  ^~~~~~~
D:\10 CJ base\02 ARDUINO\SKETCHES\x_ARCHIVES\TEENSY\t3MPUsim02\t3MPUsim02.ino:29:1: error: 'SdFatSdio' does not name a type
   29 | SdFatSdio SD;
      | ^~~~~~~~~
D:\10 CJ base\02 ARDUINO\SKETCHES\x_ARCHIVES\TEENSY\t3MPUsim02\t3MPUsim02.ino:30:1: error: 'File' does not name a type
   30 | File  inputfile;                                         // input data file
      | ^~~~
D:\10 CJ base\02 ARDUINO\SKETCHES\x_ARCHIVES\TEENSY\t3MPUsim02\t3MPUsim02.ino:31:1: error: 'File' does not name a type
   31 | File outputfile;                                         // output results file

Suggestions welcomed

What's the meaning of the inital warning about FS.h?

Looking a bit further I see greiman's SdFat-beta in GitHub. This "seems" to be the same as SdFat provided through the IDE library manager, although I'm not sure yet.

BTW it was this 2016 thread that got me onto this library: Try SdFat for Teensy 3.5/3.6

But maybe the library has changed heaps since 2017 ?? (hoping not ... )

TIA
 
Last edited:
SdFat is now integrated into TD and SD is a wrapper to SdFat.
Best to check Bill Greimans SdFat repository for changes to beta version.

You have two options: Use SD wrapper to SdFat
or use SdFat directly, but this needs to be adapted to the SdFat improvements over the lat 7 years.
 
Try this:

Code: 
#include <SdFat.h>                    // grieman's SdFat_beta, unmodified
#include <SPI.h>
SdFs SD;
FsFile  inputfile;                                          // input data file
FsFile outputfile;                                         // output results file

To start access to the SDIO reader, you must use this line in the setup:
Code: 
SD.begin(SdioConfig(FIFO_SDIO));
 
Ok I did those code changes (plus a few more, mainly File->FsFile) and it now compiles, thanks @TFTLCDCyg

What is relationship between SdFat and SdFat-beta libraries? Prior to compiling I manually dowloaded SdFat-beta ZIP from GitHub and put it in the libraries folder (renamed to SdFat), then compiled OK with above fixes. Then I deleted it and used library manager to install SdFat, and it also compiled. So maybe they are the same (although there are different file counts in the SdFat folders vs SdFat-beta) ?

late edit: just found greiman's SdFat on GitHub (not beta!) but this just make my question more relevant !
 
Last edited:
TD = teensyduino

The SDFat library was updated by Greiman a few years ago to make the library more compatible with SDIO, around the time of the release of Teensy 4. He redesigned several functions of the library and since these new changes were quite profound, he decided to name it SdFat beta before uploading the SDFat 2.0 version.

He shared it himself on this forum. When the improvements were made official, the beta version is now SdFat V2, which is what we can download from Github manually or through the IDE's library manager.

Paul decided to route the SD library (which in itself was much better than the native one in the Arduino IDE) to incorporate SdFat V2 functions. The version of the SD library that accompanies the teensyduino in its latest versions, integrates many of the improvements of SdFat V2, among them being able to use memories of more than 32 GB in the SDIO reader. For example, now I have a memory working in the teensy 4.1 256 Gb exFat, and I have seen messages that 1Tb microSD memories can be used!

Canvas Go Plus! 256 Gb on teensy 4.1@528MHz

SDIO bench T41_256Gb.jpg
 
Last edited:
Ahah, knowing that history helps heaps, thanks. I've got some reading to do (e.g. on meaning/purpose of SDIO)

I'd still like to know the reason behind the first warning message about about FS.h? (in my original post) It seems to happen with all my teensy sketches.

Also I see you're using IDE 1.8.19. Is that common practise for teensy users, and if so, why?
 
The warning is due to the use of FS.h by TD which conflicts with the native SdFat implementation.

It always shows up if you use Bill's nativ SdFat with TD.
The correct way is to only use the SdFat provided by TD and not other installations (e.g. in local library and Library manger).
PJRC has added functionality to SdFat that was requested on this forum.
 
The short and simple answer is you should uninstall or delete SdFat you've installed, so Arduino IDE will use the copy of SdFat which comes with Teensyduino. When you install a library, usually Arduino IDE gives your installed copy priority. When you have any compile error, it should also show a message about duplicate copies of the library with the full pathname of the copy it used and any others it ignored. If you want things to just work, use the SdFat that Teensyduino provides. Easy, right?

Your code which includes SdFat.h can no longer use "File". Change every "File" to "FsFile". Annoying, but also easy, right?

Now for the longer, complicated answer...

To understand FS.h, first think about how you use SdFat or SD.h. You call an open() function which returns a "File" object. Then you use that File object to actually access your data.

Code: 
  File myfile;
  myfile = SD.open("mydata.txt");

If you only use a SD card for data this works well. But consider LittleFS library, which stores files on a variety of flash memory chips. Or USB host which can access USB thumb drives or hard drives. Or hypothetically, imagine a future library that accesses "windows share" network filesystems or a NAS device. If these libraries were to define "File" as their own object, then none of them could be used together because you'd have conflicting definition of File compile errors.

Now imagine you write a library which accesses data from files. JPEGDEC could be a prime example. In the traditional Arduino way, where "File" is definied in both SD.h and SdFat.h, Larry (author of JPEGDEC) would need to pick which to include inside his library. If he includes SD.h, then you can't use his library with SdFat. Likewise if he uses SdFat.h, then people using SD.h can't give a File object to the JPEG library. Either way, anyone using LittleFS, USB host accessing USB disk drives, and other file-providing libraries is locked out.

FS.h is meant to provide a generic File base class. The idea is libraries like SD.h and LittleFS.h can provide a File object and libraries like JPEGDEC can accept a File without needing to know which actual library is implementing the access to media.

Bill doesn't really embrace the concept of FS.h. His code in SdFat doesn't make use of it. But he does at least detect if FS.h exists and skip defining "File" as a SdFat-specific class, which results in a compile error if FS.h is present. Knowing this will break all his example (perhaps older) programs and program anyone has written with "File", his code prints a warning that doesn't really explain this situation. In a more ideal world (of infinite programming hours) we would substantially modify SdFat to actually use the File base class defined by FS.h. But that is unlikely to ever happen, as we have a SD.h wrapper which uses SdFat underneath, so there's really not a compelling reason to put so much work into updating SdFat to actually use FS.h properly.

Hopefully that at least explains what FS.h is about.

Now if you have a program you wrote in 2017 using SdFat.h which has variables of type "File", you're going to need to change every "File" to FsFile or File32 or ExFile, depending on how your copy of SdFat is configured. Teensy's copy defaults to FsFile.

As I mentioned earlier, you should delete or uninstall any other copy of SdFat. Then when you open the SdFat examples, you'll get the examples from Teensy's copy of SdFat. Those examples have been updated. You'll find stuff like this:

Code: 
#if SD_FAT_TYPE == 0
SdFat sd;
File file;
#elif SD_FAT_TYPE == 1
SdFat32 sd;
File32 file;
#elif SD_FAT_TYPE == 2
SdExFat sd;
ExFile file;
#elif SD_FAT_TYPE == 3
SdFs sd;
FsFile file;

I could go on about a long history of short-sighted API decisions, especially on Arduino's part, but the situation we have today on Teensy is File gets defined by FS.h. If you want to keep using SdFat with its own APIs, you must use SdFat's native types (FsFile, File32, or ExFile) because File is now considered part of the system and must not be redefined by any specific library. That's inconvenient for you, and everyone else who used SdFat rather than SD.h, but hopefully this lengthy message can help answer the question of why it is this way now.
 
Last edited:
Excellent explanation, thank you Paul. I deleted my SdFat lib as you suggested and the Fs.h warning dissappeared. (I'd previously replaced File with FsFile, based on TFTLCDCyg's earlier post ... easy indeed).

I looked at some of the TD SdFat examples. If I need to edit SdFatConfig.h, where exactly would I find that?

Can I assume the SdFat in TD has "basically" the same functionality as Bill's SdFat (other than what you've explained above)? I still need to descend to my other non-TD Arduino hardware & sketches many of which use SdFat. (e.g. I note code for my Mega2560 won't compile now that Bill's SdFat is removed, although it's easy to reinstate SdFat when needed).

I see TD includes SD as well as SdFat. Can I assume using SdFat means I won't need to consider the SD library at all?

On a minor note, TD has warnings enabled by default, which is fine, but it seems it can't toggled via usual IDE preferences tick ? (sometimes I need to focus on a few errors first, amongst heaps of warnings)

PS: it seems there's no dark theme for this forum?

PPS: Paul it's high time you updated your avatar :)
 
Last edited:
ninja2 said:
Excellent explanation, thank you Paul. I deleted my SdFat lib as you suggested and the Fs.h warning dissappeared. (I'd previously replaced File with FsFile, based on TFTLCDCyg's earlier post ... easy indeed).
Click to expand...
The one main complication you might run into, is what if you wish to build some sketches for other Arduinos that use SdFat. They will not find the libraries that are installed within the Teensy Board installs.

If you install the normal official SdFat library, then it breaks the Teensy builds. Most of the time now to handle the other boards, I install the Adafruit version of SdFat:
1711325474345.png


The directory name is different, so when building on Teensy, it will use the directory in the Teensy install (I believe because of the directory name sort of is a tie breaker, I think). For other platforms it will use the Adafruit one.

However if I trying to use some of Bill's latest stuff, then I install it and then remove it when doing Teensy stuff, which is sort of a pain.

Sometimes wish we would have renamed the directory and include files that SD includes for SdFat... But...
 
So many questions. Here's a couple quick answers. (sorry, don't have time for another long message)

ninja2 said:
Can I assume the SdFat in TD has "basically" the same functionality as Bill's SdFat
Click to expand...

Yes, basically the same. You can see what's changed on github.

github.com

GitHub - PaulStoffregen/SdFat: Arduino FAT16/FAT32 Library

Arduino FAT16/FAT32 Library. Contribute to PaulStoffregen/SdFat development by creating an account on GitHub.
github.com github.com

The readme has a high-level summary. Of course you can use github compare or click "52 commits ahead of" to see the finer details of every change.


ninja2 said:
I see TD includes SD as well as SdFat. Can I assume using SdFat means I won't need to consider the SD library at all?
Click to expand...

Yes, you can use only SdFat if you like.

As WMXZ mentioned in msg #2, and you said you didn't understand in msg #5, SD.h (on Teensy) is now merely a wrapper for SdFat.

That means it doesn't have any actual SD code, it just uses SdFat to do everything. Again, I don't have time for another long message, so if you want to learn more I'd recommend you read this example:

github.com

SD/examples/SdFat_Usage/SdFat_Usage.ino at Juse_Use_SdFat · PaulStoffregen/SD

Contribute to PaulStoffregen/SD development by creating an account on GitHub.
github.com github.com

You can also read the SD source code. It's just 2 small files, because it doesn't have any "real" code.

github.com

SD/src/SD.h at Juse_Use_SdFat · PaulStoffregen/SD

Contribute to PaulStoffregen/SD development by creating an account on GitHub.
github.com github.com

github.com

SD/src/SD.cpp at Juse_Use_SdFat · PaulStoffregen/SD

Contribute to PaulStoffregen/SD development by creating an account on GitHub.
github.com github.com
 
ninja2 said:
Also I see you're using IDE 1.8.19. Is that common practise for teensy users, and if so, why?
Click to expand...
In my case, I use IDE 1.8.19 because it is the version that does not give me errors with sketches that have many tabs. The IDE 2.x has given me a lot of headaches. In case of tracking down an error in the code it is easier, the excesses of help in version 2.x end up not helping

PS: I made the dark background adjustments and other extra colors in the IDE by manually editing the theme.txt file
(C:\arduino-1.8.19\lib\theme)
 
The FS.h warning is in the SdFat.h file, on line 448, I think it can be omitted by adding a line like this:

Code: 
#if __has_include(<FS.h>)
#define HAS_INCLUDE_FS_H
//   #warning File not defined because __has_include(FS.h)
#endif  // __has_include(<FS.h>)

So far I have not seen any errors in the performance of the SdFat library and we removed a couple of pretty long warnings hehehe

Before...

warning.jpg


After...
warning out.jpg
 
Last edited:
TFTLCDCyg said:
So far I have not seen any errors in the performance of the SdFat library and we removed a couple of pretty long warnings hehehe
Click to expand...

It should be fine. So far almost all the changes we've made in SdFat are for compatibility with other libraries you're not using. The only changes (I can recall) for actual functionality involve support for very old or buggy SD cards. Those cards are rare, but the issues have come up several times on this forum when people tried to use them.
 
You are right Paul, just as very fast and high-capacity microSD memories have emerged, it is also a fact that these small memories have practically disappeared and the ones we have left can no longer be used as before. My shield gameduino 3 that accompanied me so much, I had to put it in the drawer because it could only work with memories of maximum 4 or 8 Gb, some of 16 Gb and with the 32 Gb it never responded to me. When Greiman did the first tests with a teensy 4 and decided to make SDFat beta, I must confess that "my eyes shone", since then I wanted to have a teensy 4.1 with an SDIO reader, a BT817 screen and a 256 Gb microSD memory!!
 
You must log in or register to reply here.
Back
Top