Teensy 4.1/Micromod Teensy and SDIO

Status
Not open for further replies.

bvernham

Well-known member
So I am running into what I think is a band width limit of streaming raw CAN data to an SD card.

I am running TD 1.54 release and IDE 1.8.15. SDFat 2.1.0 (latest release)

I am using micro SD card on the teensy 4.1. For micromod teensy I am using the Adafruit SDIO micro SD with a "shield" I designed for carry the 3 can TX's and the SDIO micro socket and a 5V PS for the field.

I did some of the bench marks and the SD card I showing consistent 20ish kbyte/sec write times from both set ups which seems to be be "good"?

The issue is occasionally the write time seems to get long and stalls everything so is there a simple way to monitor the buffer to see of it is taking too long to write?

My thought process instead of stalling everything I can just skip some captured frames until the SD catches up and going back to unthrottled logging until we get back up again.

The CAN is coming in using and interrupt but I am trying to handle everything thing else in the loop.

Any thoughts?

Thanks

Bruce
 
TD 1.54 and after ... where 1.55 is current, include the SdFat library in the install - so that should not be a library from sketchbook/libraries on building ... but from the teensyduino install?

Odd the SDIO write speed should be over 5MB/sec (IIRC) with a good card.
 
I am basing by "good" comment based on:
Code:
https://forum.pjrc.com/threads/57669-SdFat-SDIO-for-Teensy-4-0

I ran the same sketch and got similar #'s to what was reported in the forum post above.

I updated SdFat within the TD 1.54 released libraries under the Teensy hardware folder to SDFat 2.1.0 release.

I ran both the bench mark and the Teensy SDIO Demo with SDIO FIFO and got consistent results with both the Teensy 4.1 and the Teensy Micromod.

Would I expect some big improvement with TD 1.55?

Thanks

Bruce
 
...

I updated SdFat within the TD 1.54 released libraries under the Teensy hardware folder to SDFat 2.1.0 release.

...

The version of SdFat installed with TeensyDuino is the one to use. It may get some 'shaping' to fit the wrapping with SD.h and as tested with Teensy.

i.e. - what is installed is the one to use. Moving to TD_1.55 may include a newer version - but either as installed should exceed 5 if not 10 or 20 MB/sec writes, depending on card if properly hooked to SDIO pins.
 
I ran both the bench and TeensySDIO Demo with both boards.

You have to excuse my "duh" moment but I quote # off by 1000...20K KB/set.....is not 20 kb/sec.

Anyways here are the numbers:

Micromod with SDIO bench.ino:
write speed and latency
speed,max,min,avg
KB/Sec,usec,usec,usec
15723.27,81581,22,32
21739.13,617,22,23

Starting read test, please wait.

read speed and latency
speed,max,min,avg
KB/Sec,usec,usec,usec
22624.43,275,22,22
22624.43,275,22,22

Micomod SDIO TeensySdioDemo:
FIFO SDIO mode.

size,write,read
bytes,KB/sec,KB/sec
512,19889.48,22731.75
1024,19855.16,22710.52
2048,19870.59,22810.50
4096,19897.83,22787.14
8192,19866.17,22853.69
16384,19918.34,22812.18
32768,19651.62,22864.78

totalMicros 5534208
yieldMicros 199528
yieldCalls 183
yieldMaxUsec 3046

And the Teensy 4.1 bench.ino:
FILE_SIZE_MB = 5
BUF_SIZE = 512 bytes
Starting write test, please wait.

write speed and latency
speed,max,min,avg
KB/Sec,usec,usec,usec
21276.60,3666,22,23
21367.52,4632,22,23

Starting read test, please wait.

read speed and latency
speed,max,min,avg
KB/Sec,usec,usec,usec
22624.43,283,22,22
22624.43,283,22,22

Teensy4.1 TeensySdioDemo:

FIFO SDIO mode.

size,write,read
bytes,KB/sec,KB/sec
512,19955.11,22684.67
1024,20044.03,22758.64
2048,19901.33,22762.29
4096,19897.03,22839.82
8192,19916.02,22805.29
16384,19881.89,22861.48
32768,19939.69,22815.59

totalMicros 5522445
yieldMicros 201484
yieldCalls 183
yieldMaxUsec 3065

For both demo's I get the following warning which I do not understand why:
SdFat\src/SdFat.h:438:2: warning: #warning File not defined because __has__include(FS.h) [-Wcpp]
#warning File not defined because __has__include(FS.h)

So the write performance seems "good" but it still does not explain why randomly I am getting some really long write times and how to know that this is happening?

If I run the same code without streaming the raw data to the SD card I do not run into this issue so it seems to point to the random long write times.

Any thought on how to work around this and or being able to detect these long writes in advance.

Thank for any help in advance.

Bruce
 
I ran both the bench and TeensySDIO Demo with both boards.

You have to excuse my "duh" moment but I quote # off by 1000...20K KB/set.....is not 20 kb/sec.

Anyways here are the numbers:

Micromod with SDIO bench.ino:
Code:
write speed and [B]latency[/B]
speed,[B]max[/B],min,avg
KB/Sec,[B]usec[/B],usec,usec
15723.27,[B]81581[/B],22,32
21739.13,[B]617[/B],22,23

Starting read test, please wait.

read speed and latency
speed,max,min,avg
KB/Sec,usec,usec,usec
22624.43,275,22,22
22624.43,275,22,22

Micomod SDIO TeensySdioDemo:
Code:
...
yieldMaxUsec 3046

And the Teensy 4.1 bench.ino:
Code:
write speed and [B]latency[/B]
speed,[B]max[/B],min,avg
KB/Sec,usec,[B]usec[/B],usec
21276.60,[B]3666[/B],22,23
21367.52,[B]4632[/B],22,23

Starting read test, please wait.

read speed and [B]latency[/B]
speed,[B]max[/B],min,avg
KB/Sec,[B]usec[/B],usec,usec
22624.43,[B]283[/B],22,22
22624.43,[B]283[/B],22,22

Teensy4.1 TeensySdioDemo:
...
yieldMaxUsec 3065

For both demo's I get the following warning which I do not understand why:


So the write performance seems "good" but it still does not explain why randomly I am getting some really long write times and how to know that this is happening?

If I run the same code without streaming the raw data to the SD card I do not run into this issue so it seems to point to the random long write times.

Any thought on how to work around this and or being able to detect these long writes in advance.

Thank for any help in advance.

Bruce

That's like a Thousand times better :)

Copied the speed info back into the reply above with highlight edits to show latency and 'time spent in yield'.

Those are the measured times the SD card takes to respond when it stops to do some housekeeping. They are known just AFTER they happen :) They show up intermittently as the SD processor has to do that housekeeping. Some of that can be minimized when the file can be pre-allocated IIRC.
 
Any thought on how to work around this and or being able to detect these long writes in advance.

Unfortunately these infrequent but long card latency times are the reality of modern SD cards. They're unpredictable. The only thing you can do is allocate a large enough buffer in RAM to store the incoming data while the card is busy.


For both demo's I get the following warning which I do not understand why:

Starting with Teensyduino 1.54, we're defining File in the core library. This is done so multiple libraries like SD and LittleFS can provide File and you can use File interchangeably with libraries like JPEGDEC and Audio which read or write files.

In the traditional way, where SdFat or SD or Bridge or LittleFS defines "File", when libraries are written to actually use File can only work with files from the 1 library which defines what File is, and they normally would have to be hard-coded to include that 1 library. Historically this resulted in most file-using libraries working only with File from SD, or some only with SdFat - and if you wanted to use them together in a project, it quickly turned into a huge mess of library conflicts.

This warning is SdFat wanting to make you painfully aware that it is no longer giving you File in the traditional way. A more helpful message might advise you that you need to use FsFile... or the other 2 File classes SdFat defines when configured to only handle certain filesystem types.
 
FS.h doesn't let you preallocate.
Preallocating a file isn't part of the abstraction layer. It's also not in Arduino's SD library API,

I don't know why there are several other methods that are not part of the Arduino SD library.
For users it looks random what is there and what not.
 
Last edited:
There is an example in : {local install}\hardware\teensy\avr\libraries\SD\examples\SdFat_Usage\SdFat_Usage.ino

Code:
  // You can access files using SdFat which uses "FsFile" for open files
  // FsFile offers more capability than regular SD "File".  As shown in this
  // example, you can truncate tiles.  You can also pre-allocate a file on
  // the SD card (if it does not yet have any data, the reason we truncate
  // first).  Pre-allocation impoves the speed of writes within the already
  // allocated space while data logging or performing other small writes.
  //
  Serial.println();
  Serial.println("Writing to datalog.bin using SdFat functions");
  FsFile myfile = SD.sdfs.open("datalog.bin", O_WRITE | O_CREAT);
  unsigned int len = myfile.fileSize();
  Serial.print("datalog.bin started with ");
  Serial.print(len);
  Serial.println(" bytes");
  if (len > 0) {
    // reduce the file to zero if it already had data
    myfile.truncate();
  }
  if (myfile.preAllocate(40*1024*1024)) {
    Serial.print("  Allocate 40 megabytes for datalog.bin");
  } else {
    Serial.print("  unable to preallocate this file");
  }
 
Indeed Defragster is correct. Use SD.sdfs to gain access to any underlying SdFat. I created that SdFat_Usage example to document how.


I don't know why ....

The message you quoted tries to explain the rationale. Maybe I didn't write clearly?

FS.h is still relatively new for Teensy. It may get expanded with more features, like pre-allocation. SD, LittleFS, MSC are important for FS.h, so any addition needs to be considered for how it would be implemented on those filesystems. That simply hasn't happened yet for pre-allocation on LittleFS.

FS.h is not yet a widely used convention across many boards. As far as I know, ESP8266 and ESP32 are the only others currently offering FS.h abstraction for Arduino File API. Their's don't have pre-allocation either. They both do offer their own extensions beyond Arduino's APIs, some of which differ even between ESP8266 and ESP32.

The reality of pretty much all software development is imperfect time-constrained effort. If you expect abstraction layers with complete coverage of all possible features even in their early releases, well, you're going to be disappointed.
 
Might also help if we could see photos or a clear diagram of how you've connected the SD card. Bill's test results showing 22 Mbyte/sec write speeds are with the SD card in the socket on Teensy 4.1.

SD capture.JPG

This is what the board lay out in case anyone is interested.

I set up a circular buffer which seems to handle this better.

Probably will move the 12v line in a new version.

Presently there is nothing on the 12V because I am just bench testing off of the USB-C cable.

Anyways thanks for the help.

Bruce
 

Attachments

  • SD capture.JPG
    SD capture.JPG
    71.2 KB · Views: 46
...
FS.h is still relatively new for Teensy. It may get expanded with more features, like pre-allocation. SD, LittleFS, MSC are important for FS.h, so any addition needs to be considered for how it would be implemented on those filesystems. That simply hasn't happened yet for pre-allocation on LittleFS.
...

LittleFS preAlloc could make use of the .formatUnused() - but that doesn't know where the LFS 'write head' will be (without lfs/src changes). It starts at block '0' {or last block checked IIRC} and runs as long as requested for ALL or given 'x' blocks to the end. If only a portion of blocks were given - .formatUnused() starting at the beginning may leave the end unchecked - and writes there would require pre-format. And that is only part of course as the growing file needs not only fresh data space - but 'FS/dir' blocks updated as it grows. Having pre-formatted blocks for FS details would help - but not stop it needing to munge around.

And even if blocks were pre assigned to save that - AFAIK it tags each block allocated - which means doing the write would find the block dirty when the data arrived. So maybe offering .formatUnused() is the best, but it could take some time if it found lots of blocks to format and was doing all the media surface.
 
OK so now I seem to be having another problem.

Same scenario as above. Same SD card. Same code between the

For the Teensy 4.1 I am always getting data stored.

For the Micromod with the SDIO interface sometimes yes, sometimes no.

The file is indicated that is open when checked but noting gets written to it. The file name shows up but not date and time. Seems random. I have run the SDFat bench marks and get identical results. Two boards, same SD card, different results.

Has anyone experienced the same thing? Is it possible to detect this is happening so something can be reset? File closed and reopened?

Any thoughts would be appreciated.

Thanks

Bruce
 
Might help to know what setup you are using.... And software.
That is if you re using for example SDFat directly and before current beta Teensyduino... Dates/times you need extra stuff in your code.

One current sketch I did help debug some current stuff. I found you needed to do things like: include <TimeLib.h>

You needed to setup a sync provider... SO my test sketch has:
Code:
  // set the Time library to use Teensy 3.0's RTC to keep time
  setSyncProvider(getTeensy3Time);

For SDFat you then needed to setup a callback like:
Code:
  // Set callback
  FsDateTime::setCallback(dateTime);

And then have the code for the two callbacks:
Code:
//------------------------------------------------------------------------------
// Call back for file timestamps.  Only called for file create and sync().
void dateTime(uint16_t* date, uint16_t* time, uint8_t* ms10) {

  // Return date using FS_DATE macro to format fields.
  *date = FS_DATE(year(), month(), day());

  // Return time using FS_TIME macro to format fields.
  *time = FS_TIME(hour(), minute(), second());

  // Return low time bits in units of 10 ms.
  *ms10 = second() & 1 ? 100 : 0;
}
//------------------------------------------------------------------------------
time_t getTeensy3Time()
{
  return Teensy3Clock.get();
}

The code we are doing now in FS integration stuff hopefully will eliminate the need for most of this...
 
I am using Arduino 1.8.16/TD1.55 release. SDFat from TD1.55 release.

Time stamp is not the. I am using the Teensy3 call back with no issue.

Data streaming with 4.1 works all the time....same code with Micromod teensy works most of the time (99%) but sometimes it hangs when trying to open the file with the Micromod.

Random in nature. I have streamed GB's of CAN data with this Micromod hardware so this is purely an intermittent problem.

If there was a way in determining this was happening I could do something about it. Any ideas how so "see" this happening? To see if data really is getting written/stored to the SD card?

Thanks

Bruce
 
Status
Not open for further replies.
Back
Top