"async write" SD card access, filesystem library (Teensy 3.6)

Status
Not open for further replies.

ig-or

Active member
Hello everybody!! I have a question about existing SD card access libraries for the Teensy. I tried the "standard" library, and also "Sdfat". Looks like SDFat is faster, but sometimes when I'm writing the info I'm getting delays up to 20 ms. The data rate is not big - 10 kb/sec maximum. I'm trying to log some info onto SD card, but sometimes (not very often) the "write" function take some significant amount of time. Do you thing there is any possibility to make the "write" function work in an async way - maybe like UART, in order to not wait for this function to complete? Maybe such possibility was already implemented somewhere?

Best regards, Igor
 
Take a look at the low latency logger example in the sdfat library. This should easily meet your needs. However if you don't post your code, its hard for anyone to see if there might be an issue with your code. However sd writing is known for large latency at times. If your data is simple, you might get away with having 2 arrays with your data, where you fill one, while the other writes. Also, you say you are writing only 10kb/sec, this means 20 512 byte packets should be writing per second, so even with a delay of 20ms, you should not have any issues. However I am going to guess you are not writing a full 512, which is also not optimal.
So take a look at the low latency example, plus search low latency in the forum, you will find more examples. And post your code.
 
Thank you for the comment!
This is the code:
INIT function:
Code:
void fInit() {
	//
	
	fsWorking = false;
#ifdef USING_SD_FAT_LIB
	if (!SD.begin()) {
#endif
	
#ifdef USING_STANDARD_SD_LIB
	if (!SD.begin(BUILTIN_SDCARD)) {
#endif

		Serial.println(F("SD card init failed"));
	} else {
#ifdef USING_SD_FAT_LIB
		// make sdEx the current volume.
		SD.chvol();
#endif
		printSDcardInfo();
		fsWorking = true;
	}
}


File open:
Code:
void sdLogStart() {
	logFile = SD.open(logName,  O_WRITE | O_CREAT | O_TRUNC);
	sdLog = true;
}

Write function:
Code:
int XQSendInfo(unsigned char* data, unsigned short int size) {
	logFile.write(&data, size);
}

This "write" in the function above can take quite a time sometimes; I was wondering it it can do all the job somewhere in a background? I have to make it faster than 1 ms somehow.

Best regards, Igor
 
So you want to write 512 bytes every 1ms? Like I said, there are a few great examples in the forum that use low latency logging for analog collection. All of these examples use the pre-allocated file size formate, which set a file size based on number of 512 blocks, which can be as big as you want (up to SD card size). You then save your data and when done the file is truncated down to the number of blocks you actually filled. I am using a similar system right now and see very low latency. As for the job happening in the background, not sure what you mean.
What is your data format? I don't see it above in your code snippets, as per forum rules, you need to post enough code that will allow others to run it. You will find the teeny community is very helpful, but need all the info to help.
 
Dear turtle9er, sorry I didn't explain it clearly. Below is my software:
Code:
#define USING_SD_FAT_LIB
#define USING_SdFatSdioEX

#include "WProgram.h"
#include "SdFat.h"


extern "C" int main(void) {
    char usefulInfo[32];
    initEverything();
    while (1) {
        //     the function below should run (at least) every 10 ms; and it's not very quick, it runs approximately 2 - 9 ms, 
        //     so we have only 1 ms left for the logging function
        do_very_important_thing();

        //   this function below logs some info to the SD card. But the info size is not big- about 32 bytes.  But we have only 1 ms for this function. 
        //   so looks like we have about 10 ms to write those 32 bytes, but we have to do this somewhere in a background
        write_info_to_the_log_file_on_SD_card(usefulInfo, 32);


    }

}
 
Dear turtle9er, sorry I didn't explain it clearly. Below is my software:
Code:
#define USING_SD_FAT_LIB
#define USING_SdFatSdioEX

#include "WProgram.h"
#include "SdFat.h"


extern "C" int main(void) {
    char usefulInfo[32];
    initEverything();
    while (1) {
        //     the function below should run (at least) every 10 ms; and it's not very quick, it runs approximately 2 - 9 ms, 
        //     so we have only 1 ms left for the logging function
        do_very_important_thing();

        //   this function below logs some info to the SD card. But the info size is not big- about 32 bytes.  But we have only 1 ms for this function. 
        //   so looks like we have about 10 ms to write those 32 bytes, but we have to do this somewhere in a background
        write_info_to_the_log_file_on_SD_card(usefulInfo, 32);


    }

}
All high speed logging, say up to 1 MByte/s for T3.5/6, including the examples discussed in this forum, follow the following principle:
All data acquisition and processing is done at elevated priority level. The only activity that is done in main() (or loop()) is to access the uSD (open file, writing data, closing file).

In order to succeed, a write buffer must be provided to cover maximal uSD delay (typically 0.15 ms, or 150 kB for 1MB/s data rate).
So, the ISR level processing is filling up a large buffer continuously and the main/loop level processing is ONLY draining the buffer and writing to uSD as fast as possible.

How you do the buffering, is a question of taste. Check the different examples in the TD distribution and presented in the forum.
 
WMXZ,
thanks for the info. Now I'm starting to understand.

turtle9er,
thanks for the good and useful example!
 
Status
Not open for further replies.
Back
Top