SPI Flash Allocation Table

I would vote for the linked-list, too.

But, sometime ago,(i hope i find it again!) i found a paper,describing a new algorithm (but simple) for wear-levelling which provided more flashing-cyclyes in most cases.
For wear-levelling, a fixed FAT is better than the linked list. Plus, you can delete data in a better way.

@iwanders: you know that the "medium" is soldered and not changeable ?
 
Last edited:
Yes I know that's the case for the SPI memory chip on the audio board, but that's not necessary if the system is hardware independent.

For example; one could use an I2C flash memory chip which is connected with a header. I thike it would be useful if you can detect whether the memory is already formatted or not.
 
the 16 or 32MB SPI flash chips (8 pin) are $1 or less.
If there's an I2C version, it would avoid all the issues with the T3 CPU having but one SPI port.
 
Some ARM vendors have NAND flash wear-leveling code in their open source ARM HAL libraries. Normally used with 8 bit parallel flash chips. Maybe can work with serial flash too.
 
Ok, sounds like there's a pretty good consensus forming around using a linked list approach.

My main concern with the proposal in msg #24 is the lack of ability to locate all of a file's data aligned to 4K sector boundaries. Some types of files, like MP3 and wavetables, will never need to be written. We can pack those tightly. But files intended for recording will want to be aligned onto erasable sectors. A tightly packed linked list format can't do that.

The linked list concept could still be used, if more bytes were allocated to indicate the data location and length, rather than requiring the data to immediately follow the header info, and requiring it's length to be implied by the space between headers.
 
I just spent some time reading about SPIFFS. This is exactly the type of complexity I do NOT want.

For the audio library, we need files to be stored as continuous data. When an audio library object accesses a file on the flash, it will need only 2 numbers: the beginning address and the allocated length. As an object plays samples, or reads a wave table for synthesis, or pulls in more of an MP3 stream, we NEVER want to incur extra overhead of reading index sectors or other metadata, like SPIFFS requires.

When 128 samples need to be read from the flash, the most important goal is keeping our flash access to only those 256 bytes, for consistent and predictable timing. Remember, this will be embedded within some object like a sample player. Users will drag any number of them onto the GUI and click Export to get a sketch that might try to read 12 simultaneous streams. In such a case, 12 instances will end up pulling a total of 3072 bytes from the flash every 2.9 ms, which happens to be about 1/3rd of the total SPI bandwidth, not including slight protocol overhead required by the SPI flash chip.

Continuous, non-fragmented file allocation is essential. Every file's storage must be described by only 2 numbers, beginning address and total data length.

Tools that format the entire flash and decide what space each file gets can have complexity. Most files can be allocated and packed byte-wise, and others meant for recording or writing can be allocated to 4k sector boundaries. The allocation table can provide plenty of metadata about the file, though data type and a text name are probably all we need.

However, being able to quickly locate a file, with a minimum number of SPI transfer bytes, is valuable. An expected use case will be the main program searching for the file based on its text filename, while the library is accessing many other files. One or more SPI transactions will delay the audio update interrupt, so we want to minimize the time we're preventing the audio library from updating.

I fear this conversation has wandered pretty far from focusing on the real-time requirements of the audio library. Indeed there are lots of great general purpose file systems, like SPIFFS, that would be wonderful if the goal was file storage for general purpose computing.

Please remember, the goal here is CD quality audio, possibly a dozen or more polyphonic streams, on a low power microcontroller.
 
Last edited:
Yes, it's the question what the goal is.

Ok, another idea, the other "extreme" :)
I wrote a very simple sketch which copys files to the SPIFlash. Very simple to use. I could extend it a little bit to output a small piece of c-sourcecode (some #defines or const array. If wanted, with filenames..).
Then, all the user has to do is copy&paste this to his sketch, and voilà.

https://forum.pjrc.com/threads/27409-Play-RAW-from-Serial-Flash?p=61750#post61750

Ok, since it will take some time until we have our FAT, i'll do it this way. The sketch can flash every file and is not restricted to audio.
 
Last edited:
Frank B said:
Ok, another idea, the other "extreme" :)
I wrote a very simple sketch which copys files to the SPIFlash. Very simple to use. I could extend it a little bit
Click to expand...

Yes, I looked at your code briefly yesterday. Extending that program sounds like a great path forward.

to output a small piece of c-sourcecode (some #defines or const array. If wanted, with filenames..).
Click to expand...

Yes, something like this. I have a bit of C++ trickery in mind for the defines, to carry both the address and length in a single name and allow the compiler to optimally inline both.

I do also want the sketch to store filenames and allocation data in the flash. It can be a fixed table, a linked list, or any number of other structures. Later I want to build other sketches that have a GUI front end, so the allocation info and filenames need to be stored in the flash to later allow a beautiful front end.
 
Great.
What c++ trickery do you have in mind ?
It's on Github.. erverybody is free to submit patches.
But i can modify it too. Just tell me what do you want... :)
 
Paul, i totally agree with you. I currently do that Kind of adress + length for reading from sdcard in my drum computer prototype.

If we focus on streams, simplicity and performance, index and length is sufficient for me, as i know what kind of data I've written and want i need to read.
 
The ARM HALs (some anyway) have libraries for parallel and SPI NAND flash I/O, wear-leveling and bad-block remapping. Source for that is open, so perhaps that source can be adapted for the Arduino environment. Perhaps Freescale has one too.
The step after that is 8KB for FATfs that mates with the APIs in the above - such as open source
http://elm-chan.org/fsw/ff/00index_p.html
or
http://elm-chan.org/fsw/ff/00index_e.html

BUT the goal here may have been DIY to make it tiny. Wheras the above is less frugal since many current low cost ARM M4's have 512 or 1MB of flash and 256KB or more of RAM.
 
Last edited:
Steve, we're talking specifically about supporting the optional W25W128FV (and equivalent part numbers) NOR Flash chip on the audio board.

This thread has already gotten side-tracked pretty badly!

If you want to talk about NAND Flash chips, or *any* chip other than 8-pin SPI NOR Flash compatible parts, and any usage case other than the specific real-time media requirements of the audio library, please start a new thread.

The audio board is already made. A couple thousand have already shipped, so the hardware is absolutely fixed with that 8-pin location to add optional SPI NOR Flash. Future boards might get designed with all sorts of other chips, but here any now, we're working with hardware that's already made. Other types of chips simply aren't up for discussion right now (except in a new thread, where anything is fair game)

What's needed is software. Frank and Pete have both made a start, writing directly to the chip. It's time to finalize some basic allocation scheme, so as we make several different object that can use this SPI Flash, they can work together nicely.
 
Relampago said:
Paul, i totally agree with you. I currently do that Kind of adress + length for reading from sdcard in my drum computer prototype.

If we focus on streams, simplicity and performance, index and length is sufficient for me, as i know what kind of data I've written and want i need to read.
Click to expand...

wanted to second that. opening by filename should be avoided or at least be made optional.

as an aside -- so if you reading from SDcard by address and length, i take it you pulled uint8_t open(SdFile* dirFile, uint16_t index, uint8_t oflag) into SD/the audio lib? i'd be curious: are there any performance gains to be had from that? or is it mostly to define the start/end points of the sample?
 
For me, it does not really matter how exactly the files are stored. The only thing is, i need to know, how you want it. Then i can do something. Or you do it. Then i need a lib. But without knowing...

It would be much more interesting (for me) to have the possibility to stick the teensy in an usb-port and have 2 or 3 drives..(sd-card, spiflash, maybe teensyflash)...
Unfortunately, i don't have the time and knowlage to do it.

Edit: Or one drive with 3 directories. Or MTP.
 
Last edited:
I'm currently working on the bed-of-nails tester for the upcoming low-cost Teensy board (Cortex M0+), so I can't do much at the moment. I'll get back to this in a few days, with some "final" decisions.
 
Great.
If you decide to work on the USB-side (MTP?) i offer my help. I know nothing about usb, but maybe i could help with the file/memory/flash/spi accesses..
Hey, you don't need a GUI with that :) the operating-systems are the GUI :)
 
Later this year - after the low-cost board is out, after another bootloader upgrade, and after a couple months of more audio work, I'm planning to implement 3 more USB types: audio, MTP and UDP-only network (mainly for E1.31/Artnet lighting and low-latency Open Sound Control).

But first, I need to get the low-cost boards released, and in the process, make some decisions and fiddle with APIs a bit on this SPI flash stuff, so we can all make progress within a simple allocation scheme that will let many different ways to use the SPI flash work together nicely and allow beautiful GUI tools to someday be built, of course in my spare time!
 
Last edited:
PaulStoffregen said:
Later this year - after the low-cost board is out, after another bootloader upgrade, and after a couple months of more audio work, I'm planning to implement 3 more USB types: audio, MTP and UDP-only network (mainly for E1.31/Artnet lighting and low-latency Open Sound Control).

But first, I need to get the low-cost boards released, and in the process, make some decisions and fiddle with APIs a bit on this SPI flash stuff, so we can all make progress within a simple allocation scheme that will let many different ways to use the SPI flash work together nicely and allow beautiful GUI tools to someday be built, of course in my spare time!
Click to expand...


When will the M0 Board be released? Can you give technical info..( which processor? )
I would like to adapt my public libs as soon as possible :) I buy two.. (Is there a way to reduce the shipping cost ( > 10 $) to Germany? ;-)
And, to stay on topic: Is it possible to use the audioboard or is there an adapter for spiflash ?
 
The coder knows where everything is in the FLASH when they write it.

but the flash persists a reboot, the variables storing that information do not.

Therefore some kind of filename will be essential, otherwise you have to erase and rewrite the flash each time you boot, and collect the file offset positions.
 
You must log in or register to reply here.
Back
Top