File abstraction and SdFat integration

Doing more testing I have discovered that using cardInfo.ino without my mods to SD.h is hit and miss when trying to use external SPI. It seems like it is a timing problem. Performing multiple programmings will work sometimes but not all the time. Performing multiple programming button presses will work sometimes. Using TyCommander reset function will work sometimes as well. The kicker is that it completely depends on the SD card that I am using. Using some Samsung cards work and some don't. Some PNY SD cards work and some don't. Using sdInfo.ino seems to work with everything consistently.

Just curious as to why:)

Edit: The Modifications I made do not change anything.
 
Last edited:
wwatson said:
Doing more testing I have discovered that using cardInfo.ino without my mods to SD.h is hit and miss when trying to use external SPI. It seems like it is a timing problem. Performing multiple programmings will work sometimes but not all the time. Performing multiple programming button presses will work sometimes. Using TyCommander reset function will work sometimes as well. The kicker is that it completely depends on the SD card that I am using. Using some Samsung cards work and some don't. Some PNY SD cards work and some don't. Using sdInfo.ino seems to work with everything consistently.

Just curious as to why:)

Edit: The Modifications I made do not change anything.
Click to expand...

Actually just got it working - relatively simple fix but didn't try with an external card reader. Change the SD2CARD Init function to:
Code: 
	bool init(uint8_t speed, uint8_t csPin) {
#ifdef BUILTIN_SDCARD
		if (csPin == BUILTIN_SDCARD) {
			return SD.sdfs.begin(SdioConfig(FIFO_SDIO));
			//return sdfs.begin(SdioConfig(DMA_SDIO));
		}
#endif
		return SD.sdfs.begin(csPin, SD_SCK_MHZ(speed));

	}
then it starts working fine.
 
Fixed.
https://github.com/PaulStoffregen/SD/commit/03071e943ec5c631a4bb3419d5ae7077fa520eae

Also added support for Teensy 2.0, but only for up to 32G cards. The SdFs class uses too much flash memory for those old 8 bit boards.

I'm running lots of tests right now with these boards and 2 SD cards.

sdlib.jpg

I am seeing some issues on Teensy 3.x running the JPEG slideshow with a 128G card accessed over SPI. Sometimes works, but often can't initialize the card. 128G works fine on SDIO, and 32G works great on both SPI & SDIO.
 
Looks like you have been busy with those :D I was setup yesterday to test some, but got distracted.

Is there other parts you need tested?

P.S. - It will be great to have all of this in builds!
 
Just tried the fix on a T4.1 and worked fine:
Code: 
Initializing SD card...Wiring is correct and a card is present.


Card type: SDHC


Volume type is FAT32



Volume size (Kbytes): 31154688

Volume size (Mbytes): 30424
 
KurtE said:
Is there other parts you need tested?
Click to expand...

Yes. The one place I'm seeing problems so far is with my 1 and only 128GB card accessed over SPI. It always works great over SDIO (at least in the ways I've tested so far). And it always seems to work over SPI for simple stuff. But running the JPEG slideshow, it sometimes works over SPI, sometimes won't start up.

Once the card is initialized, it seems to run great over SPI. I'm running a long test right now, so far working find. The issue seems to be only at startup to detect the card.

If you have any 64GB or larger cards, can you try using them over SPI?

Robin just put in an order for several more cards from 64GB to 512GB. Amazon says a couple will arrive later today! But until then, this one 128GB card is the only one I have for testing. Hard to know if it's a problem with just this 1 card.

It might also be some sort of subtle issue with sharing the SPI between the SD card and the ILI9341 display?
 
Just gave it a try on a T4.1 (first did a resize of demo jpgs) - example for the ILI9341 works like a charm Paul.

Very cool - now have to find a JPEG encoder to same images as JPEGS :)
 
PaulStoffregen said:
Yes. The one place I'm seeing problems so far is with my 1 and only 128GB card accessed over SPI. It always works great over SDIO (at least in the ways I've tested so far). And it always seems to work over SPI for simple stuff. But running the JPEG slideshow, it sometimes works over SPI, sometimes won't start up.

Once the card is initialized, it seems to run great over SPI. I'm running a long test right now, so far working find. The issue seems to be only at startup to detect the card.

If you have any 64GB or larger cards, can you try using them over SPI?

Robin just put in an order for several more cards from 64GB to 512GB. Amazon says a couple will arrive later today! But until then, this one 128GB card is the only one I have for testing. Hard to know if it's a problem with just this 1 card.

It might also be some sort of subtle issue with sharing the SPI between the SD card and the ILI9341 display?
Click to expand...

Bill had another option in the SD Fat lib for when SPI is shared:
Code: 
 SdSpiConfig(SD_CS_PIN, SHARED_SPI, SD_SCK_MHZ(16))
Don't have any >= 64GB cards to test this with, just ordered a couple but wont be here until tomorrow.
 
I also don't think I have any cards >32GB cards, most of mine are <= 16gb. Although I will look around.
 
KurtE said:
I also don't think I have any cards >32GB cards, most of mine are <= 16gb. Although I will look around.
Click to expand...

Same here. I have plenty of 64GB and 128GB full size SD cards for my cameras, but the micro-SD cards I have, I specifically have kept to 32GB or less to avoid having to deal with things that don't support exFat file-systems and SDXC cards.
 
Ok found a SansDisk Ultra 64gb and hooked up a external card reader so SPI and pin4 for CS.

So far recycled and reloaded a few times with no problems on card initialization.

Not sure how you want to test to make sure
 
As I mentioned so far have not found a 64 or > but have tried 32GB on T4.1 both Internal SD as well as external to ST7789 and so far showing parrot is working fine.
 
Hi Paul,

Perfect timing and great news from my perspective! I have spent most of this weekend so far with the Teensy 4.1 and SdFat.

I am in the process of porting a MIDI controller and sequencer solution that I built on 3 x Teensy 3.2's several years ago. Happens to use both SD and ethernet (to MQTT broker on a RPI) on the 3.2, so the Teensy 4.1. integration is a great fit - in addition to performance upgrades.

The MD_MIDIFile sequencer library I have been using relies on SdFat. It was dormant for a while, but recently has been updated to work with the SdFat V "2.0.0" beta version. I managed to get the MIDI sequencer running and playing MIDI files on the Teensy 4.1 yesterday after going to bed pretty miserable early Saturday morning.

Great to see 'official' support for SdFat from you! I will be switching to your updated release today.

Thank you!

Anton
 
My actual application uses T4.1 + SdFat-beta with SDIO + SIX (6) SPI disks with "SHARED_SPI" (So it can be done)
To save my pocket money, I only tested with 32 GB disks (ExFAT formatted). No problems so far. but I only write SPI sequentially (fill one disk after the other)
Will next try to see if I can use new File abstraction
 
Last edited:
Last edited:
PaulStoffregen said:
Oh, opps, I missed that earlier. Committed a change to use it.

https://github.com/PaulStoffregen/SD/commit/52c44d76abb45d9547a524c808eec665719a72be

Sadly my 128GB card still isn't always starting up properly when accessed by SPI with the ILI9341 connected using the JPEG slideshow program. I'm hoping to get a couple 256GB cards later today.
Click to expand...
Ordered a 128 and 512GB sansdisk which should be tomorrow. Couldn’t resist.

Have you tried to change the clock to 16mhz for your 128gb as card?
 
I have a unopened 64GB aData uSD here I can try on a T_3.2 PJRC Audio Breadboard kit with ILI9341.

I have another 64 or 128 open somewhere - but all I find are ~10 empty Adapter cards ... uSD is too small.

Will need to collect the CORE and SD updates again and know what sketch and files are needed to repro. @mjs513 - can you post test parts on your github maybe as you have them?

Offline again for an hour or two - but will give it a try asap ...
 
mjs513 said:
Have you tried to change the clock to 16mhz for your 128gb as card?
Click to expand...

Not yet, but will try slower clocks tomorrow.

Even though there may be an issue with large cards, I believe things are good enough for a beta test. That should make it easier than having to grab files from github and we can be more certain everyone has the same setup.
 
PaulStoffregen said:
Not yet, but will try slower clocks tomorrow.

Even though there may be an issue with large cards, I believe things are good enough for a beta test. That should make it easier than having to grab files from github and we can be more certain everyone has the same setup.
Click to expand...
Great and I agree for what’s it worth. Think you got a great baseline for FS.
 
For my logging application I use the following different SdFatConfig.h definitions
Code: 
#define SD_CHIP_SELECT_MODE 1
to allow
Code: 
void sdCsWrite(SdCsPin_t pin, bool level) {digitalWrite(pin, level);}
be replaced with
Code: 
void sdCsWrite(SdCsPin_t pin, bool level) {digitalWriteFast(pin, level);}

and further I change in SdFatConfig.h
Code: 
#define SPI_DRIVER_SELECT 3
to use a modified version of "SPI driver" that allows me to control transaction activation.
For pure logging, I only write to a single uSD card at a time and consequently I can keep the card always activated.
For this I simply have the following class member functions
Code: 
        // Activate SPI hardware with correct speed and mode.
        void activate() { if(doTransactions) SPI.beginTransaction(m_spiSettings); }
        // Deactivate SPI hardware.
        void deactivate() { if(doTransactions) SPI.endTransaction(); }
and a public variable "doTransactions" that I switch on before logging and off after logging
 
Not sure if I see it correctly
in SD.h
Code: 
File open(const char *filepath, uint8_t mode = FILE_READ)
but
in FsVolume.h
Code: 
FsFile FsVolume::open(const char *path, oflag_t oflag)
with
Code: 
typedef int oflag_t;
defined in FsApiConstants.h for "#define USE_FCNTL_H 1" in SdFatConfig.h

IMO, open mode should be declared as "oflag_t" so one cane use "file=sd.open(fname, O_CREAT|O_WRITE|O_TRUNC);"
 
I didn't see any way to create meta data using SDfat libs, but does anyone have some code that will do that? My datalogger also captures information such as test number and I'm currently writing a large header with background infor. I'd love to be able to create metadata and store there.

If no one has any code, I'll keep searching.
 
KrisKasprzak said:
I didn't see any way to create meta data using SDfat libs, but does anyone have some code that will do that? My datalogger also captures information such as test number and I'm currently writing a large header with background infor. I'd love to be able to create metadata and store there.

If no one has any code, I'll keep searching.
Click to expand...

Is there any other file system that has build-in meta data?
I guess you wanted to keep meta data together with file. So, you have to either add a header or a trailer to your data. Not sure if there is any other way.
I assume, you are not looking simply into a formatting question.
 
Yes, I'm looking at a powerpoint file and it has several attributes in the properties dialog (Right click on the file to access). I realize this is a complex file and PPT probably wrote some extension for the OS to suck out the data, but that's my goal. Also in file explorer (Windows) i can display columns that are based on the file properties (metadata). My goals are to add meta and populate things like the following

1. test_number
2. battery_number
3. sensor_type

The in file explorer I could list all the properties, sort and find data based on that.

I have a feeling this data is stored in the file and an extension was added to the OS to extract and display it.
 
You must log in or register to reply here.
Back
Top