SRAM augmented SD card

Status
Not open for further replies.
W

WMXZ

Well-known member
I'm planning to make Teensy SD adapter with two SRAM chips that can function as temporary buffer to cover the case SD card when is busy with internal activities.

The initial schematic is
Sch.jpg
and the dual layer board layout is
PCB.jpg

In order to be useful for stand alone Teensy, I use spi0 for access to SD-card and SRAM.
In case it is used with TLC external data may be transferred using spi1.

I'm open to comments and suggestions.
 
No, there are no bigger sizes.

Some time ago, i had an idea... it was adding an serial ram to the ili-display, very similar to your idea.
maybe it is possible to - with some logic-gates - to swap them ram's miso and mosi. This way, you can fill the ram as usual, but if you want to write it's contents to the card, swap the ram mis/mois lines and just clk it. the data are "automatically" transfered to the card (which must porperly setup before to write a block). You could save the rad-from-ram cycle this way. It needs some additional code and hardware, but you save a big amount of time...
 
linuxgeek said:
I like the idea. It would be great to have an easy way to power the sdcard without using the teensy power. I suspect the sdcard erratic power draws cause problems for the ADC.
Also, if this could easily be used with the onehorse's appallingly small lipo charger that would be a plus: https://forum.pjrc.com/threads/26462-Appallingly-small-LiPo-charger-add-on-for-Teensy-3-1

Just out of curiousity, why the two chips? Do they not come in bigger sizes?
Click to expand...

I will check own 3.3V generation (maybe similar to T3.2).
I would prefer a 2Mbit chip, but have not found one yet.
 
Have you thought in detail on how the software side of this might work?

Are you going to use SdFat or patch the SD library somehow?
 
PaulStoffregen said:
Have you thought in detail on how the software side of this might work?

Are you going to use SdFat or patch the SD library somehow?
Click to expand...

At the moment I was testing SdFat, but I should see the SD library as well.
whichever SW I'm using, I assume that I have to modify it.

The basic idea is as follows
Assume a dedicated TLC hosting this adapter and T3.1 as 'master'

Ideally, the TLC would replace a real sd-card, interpreting and interfering with the low level spi transfers.
As soon the real sd-card becomes too busy, the TLC would park the incoming data on sram and launch am asynchronous task that transfers parked data as fast as possible.
If sram is empty, data would be transferred directly from TLC input to real sd-card.

this would minimize the code required on TLC and make its use transparent.

I hope this rough description makes sense.
I know, the devil is in the details, but the more I think about, the more I like it.
On the other side I need a reliable high-speed data logger for remote acoustic recorder.
 
WMXZ said:
At the moment I was testing SdFat, but I should see the SD library as well.
whichever SW I'm using, I assume that I have to modify it.

The basic idea is as follows
Assume a dedicated TLC hosting this adapter and T3.1 as 'master'

Ideally, the TLC would replace a real sd-card, interpreting and interfering with the low level spi transfers.
As soon the real sd-card becomes too busy, the TLC would park the incoming data on sram and launch am asynchronous task that transfers parked data as fast as possible.
If sram is empty, data would be transferred directly from TLC input to real sd-card.

this would minimize the code required on TLC and make its use transparent.

I hope this rough description makes sense.
I know, the devil is in the details, but the more I think about, the more I like it.
On the other side I need a reliable high-speed data logger for remote acoustic recorder.
Click to expand...

Does TLC mean Teensy-LC?

So you are expecting to use x2 teensy's to get the job done? That seems less than ideal, but I understand the rationale. Extra power requirements, more potential ground loops maybe?
Maybe DMA on the T3 could handle the sdcard writes? Would be awesome to get something built into Paul's sd library.
 
WMXZ said:
I know, the devil is in the details
Click to expand...

One little detail you might consider is the maximum SPI clock rate.

If you operate either board in SPI slave mode, the maximum SPI clock depends on the slave mode limit. Slave mode doesn't support clocks as fast as master mode.
 
TLC is Teensy-LC.

When I read this MS-DOS SmartDrv came to mind. It used RAM to cache used disk data. If the TLC pre-read the directory tree to RAM (native or SPI) that would be cool. It could make up for any loss from over head in raw read speed by providing non-blocking directory data. Even if in SPI <RAM> it would be able to return it during writes.

If I see this right the LC would sit on SPI and emulate as a go between direct access to the SD card on the TLC second SPI port. In this role it would perform read and write and return the data. When any in progress write blocked a new write it would cache the data to SPI_RAM and be ready for more operations. This device would need good (battery) power and a shutdown/safe_to_remove feature or the SD would end up trashed and it would have to be prepared to ID or correct that.

I just re-read and saw SRAM so in progress 'transactions' could be continued on power loss?
 
Last edited:
If there was something that functioned (as far as the user could see) similiarly to openlog except using SPI rather than serial and the various advantages of Teensy over an Atmega 328 to allow much higher rates of writing data, I'd sure be interested.

https://www.sparkfun.com/products/9530

Seems like there are a lot of "how fast can I write to an SD card?" "well it depends but you'll need a buffer..."

If it could just do something like fastSD.Write("stuff to write") and "just work" at higher rates, I know I for one would find that interesting.

Absolutely no idea how technically feasible that is though.

Also with a T3.1 it might be possible to do that with just a library and not even need any hardware. It's still a bit over my head.
 
PaulStoffregen said:
One little detail you might consider is the maximum SPI clock rate.

If you operate either board in SPI slave mode, the maximum SPI clock depends on the slave mode limit. Slave mode doesn't support clocks as fast as master mode.
Click to expand...

Yes, I'm aware of that, that is why one SPI is dedicated to import/export the data and the other is interacting with buffer or sd card.
It also means that T3.1 to TLC can only go at half speed (12 MHz) and not 24 MHz. So there is a penalty, but I try to overcome the impact on the CPU with doing the 512-byte block transfers with DMA.

@linuxgeek
The card should work also with a single teensy. In fact, I will develop SW on a single T3.1 (it is easier for me to separate added functionality from inter-cpu communication) and then port the low-level stuff to a Teensy-LC (TLC).
I plan to have a modified SPI library that will allow deviating data writes to ram and automatic transfer data from ram to sd card.

@defragster

I'm not sure that one can have the whole directory and FAT in sram as this may be needed as data buffer. But the actual directory and fat buffer could be stored, in fact all read buffers could be stored on SRAM. This would not speed-up data-transfer in a master-slave configuration, but avoid excessive sd-card busy times.

@storbeck
in fact the frequent discussion about sd-card write latency triggered this development
A google search got me only
https://community.particle.io/t/external-ram-shield/3116/6
From the end of that discussion I deduce that such a shield has not been build yet.
Also , I'm using Teensy and I hope that this exercise does not remain purely academic (I enjoy SW but I'm a disaster in HW )
 
WMXZ said:
I'm not sure that one can have the whole directory and FAT in sram as this may be needed as data buffer.
Click to expand...

Tell me about it . . . Indeed xfer buffer would be highest priority. But the whole 32 byte dir entry isn't needed for reference/lookup purposes. If you only need the 8.3 and 2 byte Start Cluster you are at 13 bytes, you could store the file length in clusters <sectors> in 2 bytes. Not good enough for all purposes when files change size. On low file count drives the directory wouldn't need compressed and you could save again not needing to read before write. Depending on what is SD typical other optimizations may be called for given you don't even have 1MB or even 640KB.

As far as the FAT keeping it all in local RAM is big - you can make a few cache blocks and just window around as needed, rewriting when dirtied. Depending on the writer focus 2 entries (file and dir) per growing file - but if you have a few of 512bytes blocks (is that min sector write size?) they can be good for a while, depending on how the directory was 'grown' that will all be localized as the data runs off the end.

Anything you do here 'with a few processor cycles' will pay back toward any loss for lower (but steady) data rates when you don't have to wait to read for data for routine queries or normal operations. If you cache enough FAT and DIR data you may avoid needing to block direct writes as much in the general case. If you are streaming writes hard ideally it is one file and a minimized set of FAT and DIR data is all you need.

Also doing a walk&cache of the FAT and DIR on startup gives you what you need to detect lost clusters and such errors for long term use that otherwise would lead to a failed system after an 'untimely shutdown'. Even on a fragmented SD the read and processing of 128KB wouldn't take long.
 
Last edited:
defragster said:
But the whole 32 byte dir entry isn't needed for reference/lookup purposes. If you only need the 8.3 and 2 byte Start Cluster you are at 13 bytes, you could store the file length in clusters in 2 bytes. Not good enough for all purposes when files change size.
Click to expand...
Yes, that would work. Initially I would constrain myself for 8.3 filename and logging, and not necessarily for random, fragmented writing.
So I expect continuous FAT.
But if the simple logging works, why not extending it to fragmented writing?

As far as the FAT keeping it all in local RAM is big - you can make a few cache blocks and just window around as needed, rewriting when dirtied.
Click to expand...
That is what I'm planning to do.

Anything you do here 'with a few processor cycles' will pay back toward any loss for lower (but steady) data rates when you don't have to wait to read for data for routine queries or normal operations. If you cache enough FAT and DIR data you may avoid needing to block direct writes as much in the general case. If you are streaming writes hard ideally it is one file and a minimized set of FAT and DIR data is all you need.

Also doing a walk&cache of the FAT and DIR on startup gives you what you need to detect lost clusters and such errors for long term use that otherwise would lead to a failed system after an 'untimely shutdown'. Even on a fragmented SD the read and processing of 128KB wouldn't take long.
Click to expand...

As reading can be done sub-block size (i.e <512 byte), I thought to keep the last read always in cache.
 
Storbeck said:
At what rate might this setup be able to record continuously?
Click to expand...

there two cases to be considered
TLC-SRMA-SDCard (TLS as slave and T3.1 as Master) then the max data transfer speed is 12Mbit/s say about 1 Mbyte/s (allowing some overhead)
T3.1-SRAM-SDCard (T3.1 as Master), then max transfer speed is 24 Mbit/s say 2 Mbyte/s

My aim is to achieve about 600 kByte/s in both cases, as this would allow me to archive 4 audio channels with 24 bit at 48 kHz sampling (that is what I ideally wanted).
To achieve these data rate heavy use of DMA may be necessary.
 
Paolo said:
@WMXZ, just curious how this project is coming along?
Click to expand...

I was working on software using different hardware, but may implement this one in near future.

Caveat: the presented schematic and PCB layout has an error in the uSD pinout, that I discovered afterwards.
 
Good Morning WNXZ,
Google searching for help with SRAM cards, I came across your thread. I apologize if I am asking a dumb question but you appear to be very well versed in SRAM card knowledge on how they work.
Trying to solve a mystery that is way over my head and have been tasked with finding a solution for the 20 something systems still in use.
We have a German-made machine that we can no longer get PCMCIA SRAM cards to work. The machine is old DOS-based equipment, we can use a PCMCIA card reader (recently purchased) to see the data on the cards. We can copy the data from the card or at least appear to be coping with the data. Move the data to a newly purchased PCMCIA card but the equipment does not recognize the new cards.
I am trying to find anyone who can help fix my problem? Can you please point me in a direction to find help or can you help with this problem?
Best Regards,
Raymond
 
@Raymond_Decker
Sorry, I have no experience with PCMCIA.

However, if you want to get help from someone here on the forum, I suggest to
create a new thread with better title and be specific about your issue.
 
WMXZ said:
@Raymond_Decker
Sorry, I have no experience with PCMCIA.

However, if you want to get help from someone here on the forum, I suggest to
create a new thread with better title and be specific about your issue.
Click to expand...

Thank you for your information, sir
 
Status
Not open for further replies.
Back
Top