Possible uSDFS problem.

[Bill Greiman]

I have been running some tests using uSDFS with SDIO on a Tennsy 3.6.

Some files seem to have incorrect data. Here is a minimal test program that demonstrates the problem.

Could be a bug in my test program but I suspect it is a DMA byte alignment problem.

// Run on Teensy 3.6 with SDIO.

#include <wchar.h>
#include "ff.h"

// Works for BUF_DIM < 512.
const size_t BUF_DIM = 512;

uint16_t buf[BUF_DIM];
FRESULT rc;        /* Result code */
FATFS fatfs;      /* File system object */
FIL fil;
UINT wr;

void error(const char* msg) {
  while (1) {
    Serial.println(msg);
    delay(1000);
  }
}

void setup() {
  Serial.begin(9600);
  while (!Serial) {
    delay(100);
  }
  Serial.println("Type any character to begin");  
  while (!Serial.available()) {
    delay(100);
  }  
  TCHAR *device = (TCHAR *)_T("0:/");
  f_mount (&fatfs, device, 0);
  
  f_unlink((TCHAR*)_T("uSDFS.bin"));
  
  rc = f_open(&fil, (TCHAR*)_T("uSDFS.bin"), FA_WRITE | FA_CREATE_ALWAYS);
  if (rc) error("f_open write");
  uint8_t ff = 0XFF;
  rc = f_write(&fil, &ff, 1, &wr);
  if (rc) error("write ff");
  for (size_t i = 0; i < BUF_DIM; i++) {
    buf[i] = i;
  }
  rc = f_write(&fil, buf, sizeof(buf), &wr);
  if (rc) error("write buf");
  rc = f_close(&fil);
  if (rc) error("close after write");
  rc = f_open(&fil, (TCHAR*) _T("uSDFS.bin"), FA_READ);
  if (rc) error("open read");
  rc = f_read(&fil, &ff, 1, &wr);
  if (rc) error("read ff");
  if (ff != 0XFF) error("ff test");
  for (size_t i = 0; i < BUF_DIM; i++) {
    uint16_t n;
    rc = f_read(&fil, &n, 2, &wr);
    if (rc) error("read n ");
    if (i != n) {
      Serial.print(i);
      Serial.print(" != ");
      Serial.println(n);
    }
  }
  rc = f_close(&fil);
  if (rc) error("close after read");
  Serial.println("Done");
}

void loop() {
  // put your main code here, to run repeatedly:
}

If I run the program with BUF_DIM less than 512, it works. Here is the output for BUF_DIM equal to 511.

Type any character to begin
Done

Here is the output for BUF_DIM equal to 512.

Type any character to begin
255 != 65279
256 != 65280
257 != 0
258 != 257
259 != 513
260 != 769
261 != 1025
262 != 1281
263 != 1537
264 != 1793
265 != 2049
266 != 2305
267 != 2561
268 != 2817
269 != 3073
270 != 3329
271 != 3585
272 != 3841
273 != 4097
274 != 4353
275 != 4609
276 != 4865
277 != 5121
278 != 5377
279 != 5633
280 != 5889
281 != 6145
282 != 6401
283 != 6657
284 != 6913
285 != 7169
286 != 7425
287 != 7681
288 != 7937
289 != 8193
290 != 8449
291 != 8705
292 != 8961
293 != 9217
294 != 9473
295 != 9729
296 != 9985
297 != 10241
298 != 10497
299 != 10753
300 != 11009
301 != 11265
302 != 11521
303 != 11777
304 != 12033
305 != 12289
306 != 12545
307 != 12801
308 != 13057
309 != 13313
310 != 13569
311 != 13825
312 != 14081
313 != 14337
314 != 14593
315 != 14849
316 != 15105
317 != 15361
318 != 15617
319 != 15873
320 != 16129
321 != 16385
322 != 16641
323 != 16897
324 != 17153
325 != 17409
326 != 17665
327 != 17921
328 != 18177
329 != 18433
330 != 18689
331 != 18945
332 != 19201
333 != 19457
334 != 19713
335 != 19969
336 != 20225
337 != 20481
338 != 20737
339 != 20993
340 != 21249
341 != 21505
342 != 21761
343 != 22017
344 != 22273
345 != 22529
346 != 22785
347 != 23041
348 != 23297
349 != 23553
350 != 23809
351 != 24065
352 != 24321
353 != 24577
354 != 24833
355 != 25089
356 != 25345
357 != 25601
358 != 25857
359 != 26113
360 != 26369
361 != 26625
362 != 26881
363 != 27137
364 != 27393
365 != 27649
366 != 27905
367 != 28161
368 != 28417
369 != 28673
370 != 28929
371 != 29185
372 != 29441
373 != 29697
374 != 29953
375 != 30209
376 != 30465
377 != 30721
378 != 30977
379 != 31233
380 != 31489
381 != 31745
382 != 32001
383 != 32257
384 != 32513
385 != 32769
386 != 33025
387 != 33281
388 != 33537
389 != 33793
390 != 34049
391 != 34305
392 != 34561
393 != 34817
394 != 35073
395 != 35329
396 != 35585
397 != 35841
398 != 36097
399 != 36353
400 != 36609
401 != 36865
402 != 37121
403 != 37377
404 != 37633
405 != 37889
406 != 38145
407 != 38401
408 != 38657
409 != 38913
410 != 39169
411 != 39425
412 != 39681
413 != 39937
414 != 40193
415 != 40449
416 != 40705
417 != 40961
418 != 41217
419 != 41473
420 != 41729
421 != 41985
422 != 42241
423 != 42497
424 != 42753
425 != 43009
426 != 43265
427 != 43521
428 != 43777
429 != 44033
430 != 44289
431 != 44545
432 != 44801
433 != 45057
434 != 45313
435 != 45569
436 != 45825
437 != 46081
438 != 46337
439 != 46593
440 != 46849
441 != 47105
442 != 47361
443 != 47617
444 != 47873
445 != 48129
446 != 48385
447 != 48641
448 != 48897
449 != 49153
450 != 49409
451 != 49665
452 != 49921
453 != 50177
454 != 50433
455 != 50689
456 != 50945
457 != 51201
458 != 51457
459 != 51713
460 != 51969
461 != 52225
462 != 52481
463 != 52737
464 != 52993
465 != 53249
466 != 53505
467 != 53761
468 != 54017
469 != 54273
470 != 54529
471 != 54785
472 != 55041
473 != 55297
474 != 55553
475 != 55809
476 != 56065
477 != 56321
478 != 56577
479 != 56833
480 != 57089
481 != 57345
482 != 57601
483 != 57857
484 != 58113
485 != 58369
486 != 58625
487 != 58881
488 != 59137
489 != 59393
490 != 59649
491 != 59905
492 != 60161
493 != 60417
494 != 60673
495 != 60929
496 != 61185
497 != 61441
498 != 61697
499 != 61953
500 != 62209
501 != 62465
502 != 62721
503 != 62977
504 != 63233
505 != 63489
506 != 63745
507 != 64001
508 != 64257
509 != 64513
510 != 64769
511 != 257
Done

 

[WMXZ]

I have been running some tests using uSDFS with SDIO on a Tennsy 3.6.

Some files seem to have incorrect data. Here is a minimal test program that demonstrates the problem.

Could be a bug in my test program but I suspect it is a DMA byte alignment problem.

OK, same program, generates same errors, will have a look into it tomorrow.

 

[Bill Greiman]

OK, same program, generates same errors, will have a look into it tomorrow.

I looked at your SDIO driver and the problem is byte alignment.

FatFS, like most small file system libraries, uses direct read/write when possible to avoid a copy to the cache. If a byte is already in the cache, 511 more bytes are copied to the cache and the block is written. The next 512 bytes won't be four-byte aligned.

You can use a temp buffer and copy data for correct alignment or use programed I/O directly from the FIFO to user memory.

Also users expect read/write calls on areas that are not four-byte aligned to work.

 

[WMXZ]

I have been running some tests using uSDFS with SDIO on a Tennsy 3.6.

Some files seem to have incorrect data. Here is a minimal test program that demonstrates the problem.

Could be a bug in my test program but I suspect it is a DMA byte alignment problem.

It was a little bit difficult to understand the problem on the bit and byte level ( without having Debugger available) and to ask the right question.
But, I think, I got it.

Situation:
Bill's test program writes a single byte with one f_write statement followed by 1024 or more bytes in a second f_write statement.
Obviously this test program was constructed to make a point.

The following is happening in f_write:
1) As writes to disk are happening only in multiple of 512 bytes and when closing file, the single byte is copied to cache. So far so good.
2) writing then 1024 bytes with f_write results that, first the cache is first filled with 511 bytes and send to disk,
3) and as there are still 513 bytes available, 512 of the are also send to disk, simply by passing a new buffer address (buffer start + 511 bytes).
4) the last remaining byte is copied to cache, which is sent to disk upon closing the file.

The problem is with step 3 and the use of sdhc-dma
for this the realize we need the addresses that are sent from FATFS to SDHC interface
the cache address was 0x 1ffff 13d0
the intermediate buffer address (step 3) was 0x 1ffff 17de

Now, the dma ignores the last 2 bits of the address (expects integer (4 bytes) aligned addresses, 1fff 17de becomes 1fff17dc)
Consequently, the dma transfers data in the 3rd step that start 3 bytes earlier, resulting in the wrong disk content, as pointed out by Bill.

Bill's problem disappears if the first single word is a 4 or a multiple of 4 bytes.
Having the first word as short, the data are simply off by 1 short.

possible work arounds:
- change diskio.c (interface between FATFS and SDHC) such that data are always proper aligned ( I thing, Bill is doing that). IMO, this results is slow down of large data transfer due to unnecessary copying and needs scares memory resources.
- never write more than 511 bytes with a single f_write statement, so all data get cached, but this results in too slow disk access.
- make sure to write multiple of 4 bytes before writing large data chunks (If one really wanted to write small amount of data AND large amount of data). I prefer this option

- switch SDHC from DMA to SWPOLL, where this problem not exists. This is done be having in sdhc.h

#define SDHC_TRANSFERTYPE                   SDHC_TRANSFERTYPE_SWPOLL

- use Bill's SW

Anyhow, It was good that this side-effect of dma was pointed out.

 

[Bill Greiman]

It is possible to get high speed with small transfers and very little cache. I use two 512 byte blocks of cache.

My fastest SdFat class uses programmed I/O. I maintain the SD in multiple block transfer mode when possible. This is not possible with DMA due to errata in the controller.

This method allows 14 MB/sec read/write with 101 byte non-aligned transfers. Here is the SdFat bench example.

File size 5 MB
Buffer size 101 bytes
Starting write test, please wait.

write speed and latency
speed,max,min,avg
KB/Sec,usec,usec,usec
14285.44,9006,1,6
14367.54,8204,1,6

Starting read test, please wait.

read speed and latency
speed,max,min,avg
KB/Sec,usec,usec,usec
14880.67,986,1,6
14925.09,986,1,6

With overclocking of the SD and 512 byte transfers I have achieved 30 MB/sec read/write speeds and 20MB/sec without overclocking.

At 80 MHz SDIO clock = 240 MHz / 3.

size,write,read
bytes,KB/sec,KB/sec
512,29232.88,30546.46
1024,29664.37,31105.21
2048,29523.73,31342.76
4096,29550.25,31830.49
8192,30222.68,31637.34
16384,30247.86,31734.04
32768,29727.13,31926.68

Don't expect me to support problems but at line 518 of SdioTeensy.cpp this is the mod.

//    kHzSdClk = 50000;
    kHzSdClk = 80000;

Overclocking at this level may not work with many SD cards. I used a card that supports 208 MHz 1.8V UHS I signalling. It is not rated over 50 MHz in 3.3V mode.