SD Fat32 Sector layout

[TheMadHouse]

I'm looking at storing a mix of raw binary data and Fat32 files on an SD card (using a Teensy 3.5).

The plan is to format the SD card below it's actual size, leaving sectors un-used by the file system, so I can write to them directly (raw data is only used by the Teensy).

My question is, when formatting an SD card, are sectors allocated sequentially from 0?

Example would be (in round figures, appreciate there is MBR and FAT overhead etc. so this isn't exact)
- 4GB card (8 million sectors of 512 bytes)
- Format for 3GB (using 6m sectors)
- Have 2m sectors for the binary data

Am I safe to assume sector 6m+1 up to 8m are not used by the file system, or is there likely to be some wear-spreading logic going on the controller that changes this?



Jim

 

[KurtE]

Sorry others can answer this better than I can. But several of were playing with Fat16/Fat32 and ExFat to make them work with MTP, and MSC and ... So learned a little bit.

Fat32 - can address up to 127GB volume...
More details on cluster sizes and the like for the different file systems is several places, like: https://events.static.linuxfound.org/images/stories/pdf/lceu11_munegowda_s.pdf
From that:

And If I remember correctly, the SD Group sort of has a standard that if you card is under size X it will be formatted with Fat16, If under Y Fat32 and above ExFat.
Where I believe X is 2GB and Y is 32GB...

There are sort of rules and stuff on how each type of Fat system sets up its storage... Again the systems work using clusters. So 8GB disk would have 4kb clusters on Fat32, so each cluster has 16 sectors...

Also with these disks, with MBR type disk, you can have 4 partitions. The data in the MBR will tell you where the sector count starts and how many and the type of partition.
https://en.wikipedia.org/wiki/Master_boot_record

Not sure why you would want to go outside the bounds of the Filesystem to tore your raw data. But you can query to get the total size of a disk, you can read in the MBR (sector 0), you can look at the 4 partition areas and then see where it ends and what is still free...

For example with our MSC stuff, we have a sketch that allows us to format a drive and print out partition data, create partitions ...
You can see some of this stuff up in the library: https://github.com/wwatson4506/UsbMscFat
Note: several of us are running a different branch of it where we are experimenting with File dates...

Again not sure what you are trying to do or why. In the past, I have seen people do stuff like wanting a large contiguous area, and they did it by creating a very large file, where they I think told the FS to allocate a large contiguous area and then do random access within that area... But not sure if that will suffice.

 

[TheMadHouse]

Very many thanks for the reply, lots of useful information.

I'd found plenty of detail on the FAT32 structure (have played around with FAT-type stuff and other DFS systems a long time ago!), but no in the detail in the document you reference.

Not sure why you would want to go outside the bounds of the Filesystem to tore your raw data

I'm using the SD card for a persistant queue in a data logging system. My primary concern is robustness in case someome turns the power off during i/o operations. If I'm handling the queue at binary level there is little to get corrupted, but if power drops during a file update, there is a risk of the file system becoming corrupted and screwing the whole system.

Reason for wanting some file system space is for more general event logging, for post-use analysis. Not so worried if that gets corrupted.



Jim

 

[WMXZ]

I was there and decided to go with exFAT.
problem with FAT32 is that you have to maintain the file allocation table (i.e. whenever you write to a new cluster you have to modify the FAT). So what you do is to do by program what the File system is doing.
Using exFAT, FAT is somewhat optional al long as you write sequentially to an empty disk.
So, I was able to have a minimal exFAT FS that allowed me to maintain the File descriptors and to write sequentially directly to disk.
However, it is much easier and more robust to use the SdFAT-V2 that is build in in TD 1.54b7, and there is minimal speed penalty.
SdFAT-V2 is robust and if you frequently flush files, or consider your disk queue storage in chunks of small/but not too small files (say 1-10MB), then you will only loos a small amount of data due to power failure.