SD card corruptions

flok

flok

Well-known member
Hi,

I think I'm doing something wrong but I can't figure out what. The situation is that if I write a block of data to the SD-card and then read it back again, then parts of the written data are missing (some parts are then all 0x00) or it is all totally different.
Source code: (zip)

Code: 
Sleep
Go!
Init SD-card backend...
Init SD-card succeeded
2024-02-13 10:09  30064771072 test.dat
Virtual disk size: 28672MB
begin: 1
Write to block 0, 1 blocks
write to offset 0
2d cf 46 29 04 b4 78 d8 68 a7 ff 3f 2b f1 fc d9 7a 96 09 2c a5 57 74 64 c4 af 15 28 a4 e9 57 db 5e 20 fb 38 a8 4e a6 14 93 25 56 24 44 df 59 8d 43 7b be
...
21 cd 0d d9 46 71 b1 9a 72 2a 25 11 8c 81 1a d4 2d 34 aa 68 ac 76 72 81 87 8b 2b 5a 2f 26 be de ec bf 38 ad b0 df a3 0d 29 c8 6f df 78 06 b8 84 0a 72 a2
 2a 4c ae 77 c7
WRITE: 1
Read from block 0, 1 blocks
read from offset 0
32 54 2d 5a 54 32 37 56 37 4b 2d 54 4f 4a 4f 4d 55 35 2d 47 42 47 42 5a 43 4c 2d 53 50 41 36 4d 42 52 2d 4f 4f 4d 44 4d 51 48 26 6e 65 74 77 6f 72 6b 54
...
4c 36 dd 83 d9 10 d5 9c fc 7f 3a f0 7e 0e 82 e3 10 00 99 84 3f 59 db f8 1a 67 2d a6 7e bc 1f 18 be a2 aa 2c ef 33 aa 8d 7d 43 23 9a 48 53 b7 f0 18 bc bf
 9d ad 88 4d f6
READ: 1
equal: 5

"WRITE 1" and "READ 1" say that the sd class says that write/read succeeded.
Yet the contents is different ("equal" should say 0).

Anyone an idea?
 
If you use new or malloc to allocate the sector buffers they'll be in DMAMEM (aka RAM2) which is cached. The SD library probably doesn't handle flushing/invalidating the memory before it writes/reads it.
 
Hi JMarsh!

Something like?

Code: 
                arm_dcache_delete(data, n_bytes_to_read);
                size_t bytes_read = file.read(data, n_bytes_to_read);
                rc = bytes_read == n_bytes_to_read;
                if (rc) {
                        arm_dcache_flush(data, n_bytes_to_read);
                        break;
                }

I also tried the other way around but neither of them resolves the problem.

But a caching problem sounds likely to me because that would explain the partial garbages.
 
Last edited:
Did you add flush before writing too?
Have you checked the contents of the file on a different device, to see what is actually being written?
 
jmarsh said:
Did you add flush before writing too?
Have you checked the contents of the file on a different device, to see what is actually being written?
Click to expand...

Oh indeed. I now do not get an error.
It's not clear to me though which one to use when arm_dcache_delete or arm_dcache_flush.
For write I do now both before the file.write() call.

I verified the contents of the sd card on my pc but the data is not what I expected it to be.

Shouldn't the sdfat library take care of cache invalidating and such?
 
You'd want to flush before writing (push data out of the CPU cache into the memory) and delete before reading (discard whatever is in the cache, to ensure the memory is read).
If it was up to me I would say yes, the SD library should be taking care of this but the cache calls use up a little bit of CPU time which is just wasted when uncacheable memory (DTCM) is used, and that's the majority of use cases.
 
True.

Ok it now works. On the Teensy4.1 and when verified on the pc.
Thanks for your help.
 
I got back to my project to see if I can do any performance tweaks.
In the March version of my project I just did a arm_dcache_flush_delete for both read and writes, but that may be a bit overdoing it? Especially as @jmarsh wrote that (only) flush is for writing and delete for reading.
Now I replaced arm_dcache_flush_delete for arm_dcache_delete before reading and behold: data corruption (did not try writing yet). So did I misunderstood it?
 
flush after write (if you want DMA to see it), delete before read (if DMA has changed RAM). While nothing else is writing the RAM you don't need to use either since you only see the cache contents.

The normal scheme of things is read then write, which in the face of other actors means delete, read, write, flush
 
MarkT said:
flush after write (if you want DMA to see it), delete before read (if DMA has changed RAM). While nothing else is writing the RAM you don't need to use either since you only see the cache contents.

The normal scheme of things is read then write, which in the face of other actors means delete, read, write, flush
Click to expand...

Yeah so that makes:
arm_dcache_flush_delete(buffer, block_size);
ssize_t rc = file.read(buffer, block_size);
to make sure that after the file.read, it will go through the cache and fetch from ram.

And write:
arm_dcache_flush(buffer, block_size);
ssize_t rc2 = file.write(&data_write[i * block_size], block_size);
so that the cache is flushed to ram first.

Yet I see still garbage.
 
flok said:
Yeah so that makes:
arm_dcache_flush_delete(buffer, block_size);
ssize_t rc = file.read(buffer, block_size);
to make sure that after the file.read, it will go through the cache and fetch from ram.

And write:
arm_dcache_flush(buffer, block_size);
ssize_t rc2 = file.write(&data_write[i * block_size], block_size);
so that the cache is flushed to ram first.

Yet I see still garbage.
Click to expand...

Yeah please ignore that copy/paste error for the write. that arm_dcache_flush should've been for data_write etc as well
 
I've reduced my code 100-fold (6k lines to 56 lines) and this shows it failing:

C++: 
#include <SD.h>

FsFile file;

void setup() {
  Serial.begin(115200);

  if (SD.sdfs.begin(SdioConfig(DMA_SDIO)))
          Serial.println(F("Init SD-card succeeded"));
  else
          Serial.println(F("Init SD-card failed!"));

  file = SD.sdfs.open("test.dat", O_RDWR);

  Serial.println(F("Go!"));
}

#define iscsi_block_size 512

void loop() {
  Serial.println(F("Testing"));
  uint8_t buffer1[iscsi_block_size];
  uint8_t buffer2[iscsi_block_size];

  for(int i=123; i<123+99; i++) {
    uint64_t byte_address     = i * iscsi_block_size;

    if (file.seekSet(byte_address) == false) {
      Serial.println(F("Seek failed"));
      return;
    }

    memset(buffer1, i, iscsi_block_size);

    // write
        arm_dcache_flush(buffer1, iscsi_block_size);
        size_t bytes_written = file.write(buffer1, iscsi_block_size);
        if (bytes_written != iscsi_block_size)
      Serial.println(F("Write fail"));

    // read
        size_t bytes_read = file.read(buffer2, iscsi_block_size);
        arm_dcache_delete(buffer2, iscsi_block_size);
        if (bytes_read != iscsi_block_size)
      Serial.println(F("Read fail"));

    for(int k=0; k<iscsi_block_size; k++) {
      if (buffer2[k] != buffer1[k]) {
        Serial.printf("offset %d mismatch, should be: %02x, is: %02x\r\n", k, buffer1[k], buffer2[k]);
      }
    }
    }

  Serial.println(F("sleep 300s"));
  delay(300000l);
}
 
flok said:
I've reduced my code 100-fold (6k lines to 56 lines) and this shows it failing:
Click to expand...

I'm guessing that your code is failing in the comparison of buffer1 with buffer2. Here is what your code does:

- open file
- seek to address X
- memset buffer1
- write buffer1 to file
- read from file into buffer2
- compare buffer1 to buffer2

I haven't run your code, but I think the problem is that you are reading from the address AFTER buffer1. If you want to read back what you wrote you must do this

- seek to X
- write buffer1
- seek to X
- read into buffer2
 
Haha yes indeed I forgot that seek in the test-code. What I saw though without the seek was not the data of the earlier iteration (what was expected) but garbage. *with* the seek, I get all correct data. Puzzled.
 
You must log in or register to reply here.
Back
Top