Forum Rule: Always post complete source code & details to reproduce any issue!
Page 2 of 2 FirstFirst 1 2
Results 26 to 45 of 45

Thread: Try SdFat forTeensy 3.5/3.6

  1. #26
    Senior Member
    Join Date
    Nov 2012
    Posts
    271
    Quote Originally Posted by WMXZ View Post
    There is no problem with setting uSDFS Baudrate to 45 MHz, simply change the number in diskio.c.
    If there is no switch to high speed mode, the clock must be limited to 25MHz or less. CMD6 is used to switch modes and I don't see that in uSDFS. I only see the ACMD6 switch to 4-bit mode.

    Here is the relevant part of the SD spec.

    4.3.11 High-Speed Mode (25 MB/sec interface speed)

    Although the Rev 1.01 SD memory card supports up to 12.5 MB/sec interface speed, the speed of 25
    MB/sec is necessary to support increasing performance needs of the host and because memory size
    continues to grow.
    To achieve the 25 MB/sec interface speed, the clock rate is increased to 50 MHz and CLK/CMD/DAT
    signal timing and circuit conditions are reconsidered and changed from the Physical Layer Specification
    Version 1.01.
    After power up, the SD memory card is in the default speed mode, and by using Switch Function
    command (CMD6), the Version 1.10 and higher SD memory card can be placed in High-Speed mode.
    The High-Speed function is a function in the access mode group (see Table 4-11). Supporting High-
    Speed mode is optional.
    Because it is not possible to control two cards or more in the case that each of them has a different
    timing mode (Default and High-Speed mode) and in order to satisfy severe timing, the host shall drive
    only one card. CLK/CMD/DAT signal shall be connected in 1-to-1 between the host and the card.
    Some cards may work at higher speeds in standard mode but my experience is that many will be unstable without the mode switch. I have not tried this with the SDHC controller. The signal timing on the bus is very different for the two modes.

    The mode switch is easy to do with CMD6.

    One other thing I just noticed in uSDFS, it appears no check is made for 4-byte buffer alignment with DMA transfers. While the user's buffer may be 4-byte aligned, unless the file position is also 4-byte aligned, a transfer in FatFS or SdFat may not be correctly aligned. For example, if there is single byte in the cache, 511 more bytes will be copied to the cache and written. If multiple blocks remain to be written, a call starting at byte 511 of the user buffer will be used and data will not be copied to the cache.

    The SDHC just silently ignore the low bits of an address for DMA transfers.
    DMA System Address

    Contains the 32-bit system memory address for a DMA transfer. Because the address must be word (4bytes) align, the least 2 bits are reserved, always 0.
    Currently I use single block transfers for the case of improper alignment and Sdfat is extremely slow. I use memcpy and an aligned 512 byte buffer.

    Here is what happens to performance if you do a single byte write before the write test and a single byte read before the read test.

    Code:
    size,write,read
    bytes,KB/sec,KB/sec
    512,596.11,2200.46
    1024,588.22,2201.92
    2048,587.98,2211.84
    4096,591.05,2209.36
    8192,589.02,2206.83
    16384,592.81,2210.23
    32768,589.53,2211.12
    
    totalMicros  126015931
    yieldMicros  123773879
    yieldCalls   229645
    yieldMaxUsec 46178
    kHzSdClk     45000
    I am now working on an improved SDHC driver similar to the SPI driver for SdFatEX.
    Last edited by Bill Greiman; 09-07-2016 at 11:57 AM.

  2. #27
    Senior Member
    Join Date
    Nov 2012
    Posts
    271
    I did a test with and without CMD6 switch to high speed mode. The cards I tested tolerated running at 45 MHz in standard mode but had different performance.

    Here is a Samsung PRO+

    Code:
    Samsung PRO+ no CMD6
    
    size,write,read
    bytes,KB/sec,KB/sec
    512,516.12,2034.51
    1024,967.52,2500.94
    2048,1809.78,4494.90
    4096,3357.10,7499.22
    8192,3698.82,8316.77
    16384,6828.89,11559.49
    32768,12779.27,13391.54
    
    totalMicros  49033135
    yieldMicros  48624415
    yieldCalls   78574
    yieldMaxUsec 42729
    kHzSdClk     45000
    Code:
    Samsung PRO+ with CMD6
    
    size,write,read
    bytes,KB/sec,KB/sec
    512,618.44,2203.42
    1024,1155.51,3189.70
    2048,2180.28,5155.51
    4096,3765.16,8153.64
    8192,4249.29,9398.14
    16384,7515.36,13390.41
    32768,12123.63,15255.79
    
    totalMicros  41843274
    yieldMicros  41435218
    yieldCalls   78586
    yieldMaxUsec 36466
    kHzSdClk     45000
    I ran the test several times and the variations are reproducible. For 32KiB, the card reads faster in High Speed Mode but write is slightly faster in Standard Mode.

    Still, the SD standard and the SDHC reference manual require the CMD6 high speed mode switch. See section 60.7.4 of the K66 reference manual.
    Last edited by Bill Greiman; 09-07-2016 at 01:05 PM.

  3. #28
    Senior Member
    Join Date
    Nov 2012
    Posts
    271
    After striking out twice, I have a first cut of the optimized SDIO driver.

    I had to give up DMA and used polled I/O with the FIFO. Using direct access to the FIFO is amazingly fast.

    I plan to keep the DMA class since it allows CPU time to be recovered with yield().

    For most users the extended multi block driver will be the winner.

    Here are first results. Transfer size almost doesn't matter and it handles byte misalignment well unlike the DMA version.

    Code:
    240 MHz  Samsung 32GB PRO+
    
    size,write,read
    bytes,KB/sec,KB/sec
    512,19144.99,20172.44
    1024,19309.19,20288.65
    2048,19585.41,20483.20
    4096,19568.55,20600.55
    8192,19736.79,20646.09
    16384,19829.82,20642.22
    32768,19734.56,20734.52
    
    totalMicros  5866016
    yieldMicros  0
    yieldCalls   0
    yieldMaxUsec 0
    kHzSdClk     48000
    There is almost no SD card busy time so I have not allowed any yield calls.

    It will be a while before I post the code. It is a mess and not well tested.

  4. #29
    Senior Member
    Join Date
    Jul 2014
    Posts
    2,333
    Quote Originally Posted by Bill Greiman View Post
    After striking out twice, I have a first cut of the optimized SDIO driver.

    I had to give up DMA and used polled I/O with the FIFO. Using direct access to the FIFO is amazingly fast.

    I plan to keep the DMA class since it allows CPU time to be recovered with yield().

    For most users the extended multi block driver will be the winner.

    Here are first results. Transfer size almost doesn't matter and it handles byte misalignment well unlike the DMA version.

    Code:
    240 MHz  Samsung 32GB PRO+
    
    size,write,read
    bytes,KB/sec,KB/sec
    512,19144.99,20172.44
    1024,19309.19,20288.65
    2048,19585.41,20483.20
    4096,19568.55,20600.55
    8192,19736.79,20646.09
    16384,19829.82,20642.22
    32768,19734.56,20734.52
    
    totalMicros  5866016
    yieldMicros  0
    yieldCalls   0
    yieldMaxUsec 0
    kHzSdClk     48000
    There is almost no SD card busy time so I have not allowed any yield calls.

    It will be a while before I post the code. It is a mess and not well tested.
    I look forward to see exFAT running at 19/20 MByte/s
    Unfortunately I need dma to free the CPU for signal processing, so I have to wait a little bit longer.

    As you have constant data rate, are you using a fixed buffer?

    BTW, reading the docs, I could not figure out where the RU is stored in the disk? Any hint?

  5. #30
    Senior Member
    Join Date
    Nov 2012
    Posts
    271
    Quote Originally Posted by WMXZ View Post
    I look forward to see exFAT running at 19/20 MByte/s
    Unfortunately I need dma to free the CPU for signal processing, so I have to wait a little bit longer.

    As you have constant data rate, are you using a fixed buffer?

    BTW, reading the docs, I could not figure out where the RU is stored in the disk? Any hint?
    I only have two 512 byte cache buffers. one for the FAT and one for user data.

    The magic happens in the SD card. I write or read very large sequences of blocks as a single multi block transfer, much larger than the 256 KB of RAM in the K66.

    I could not make DMA work in this scenario. I did a lot of research and others have had the same problem with simple DMA on the K66. I gave up the idea of using Advance DMA since there were still serious problems.

    exFAT won't be faster and won't help the DMA problem. The single FAT cache block means I only need to go to the SD once for every 4MB.

    I will implement exFAT but it will be a while. I want to do a prototype implementation then restructure SdFat for the future. It's now been over seven years since the first version that would just run on a 168 AVR. The official Arduino SD.h library has an early 2010 version of SdFat.

    I am looking at reverse engineering documents for exFAT and several implementations.

    I don't think the RU size is stored on the SD. Section 4.13.1.8 of the SD spec states that the RU for class 10 cards is 512 KB.

    I plan to keep the DMA version of SdFat. If you write or read files using 16KB or 32KB transfers almost no CPU time is required.
    Last edited by Bill Greiman; 09-09-2016 at 11:21 PM.

  6. #31
    Senior Member
    Join Date
    Nov 2012
    Posts
    271
    SdFat-beta on GitHub has the new SdFatSdioEX class. The new SDIO driver is very complex so while I have done a number of tests, it is likely to have bugs.

    Try the TeensySdioDemo example. It compares the new SdFatSdioEX class with the more traditional SdFatSdio class.

    SdFatSdioEX uses maximum length multi-block transfers but does not use DMA. SDHC Errata and SDHC DMA limitations prevented DMA use.

    The SdFatSdio class uses DMA and is similar to traditional SDIO implementations used with FatFs. SdFatSdio uses very little CPU time and calls yield() while the SD card is busy and during DMA transfers. Over 99% of the time is spent in yield() in the TeensySdioDemo example.

    Here are results from the TeensySdioDemo.

    Code:
    SdFatSdioEX,  240 MHz, 32GB Samsung PRO+
     
    size,write,read
    bytes,KB/sec,KB/sec
    512,19330.77,20032.54
    1024,19627.66,20083.81
    2048,19780.53,20275.12
    4096,19691.66,20369.89
    8192,19885.47,20415.55
    16384,19785.71,20358.67
    32768,19868.80,20494.96
    
    totalMicros  5873656
    yieldMicros  243914
    yieldCalls   176
    yieldMaxUsec 5804
    kHzSdClk     48000
    Code:
    SdFatSdio,  240 MHz, 32GB Samsung PRO+
    
    size,write,read
    bytes,KB/sec,KB/sec
    512,623.25,2202.59
    1024,1174.00,3197.87
    2048,2213.93,5564.46
    4096,4047.73,7612.76
    8192,4230.11,9241.63
    16384,7625.73,13764.11
    32768,12841.34,15748.88
    
    totalMicros  41293455
    yieldMicros  40876765
    yieldCalls   78576
    yieldMaxUsec 26244
    kHzSdClk     48000
    Here is the bench example for small 50 byte transfers.
    Code:
    SdFatSdioEX - 32GB Samsung PRO+
    
    File size 5 MB
    Buffer size 50 bytes
    
    write speed and latency
    speed,max,min,avg
    KB/Sec,usec,usec,usec
    12531.33,11111,1,3
    12500.00,8198,1,3
    
    read speed and latency
    speed,max,min,avg
    KB/Sec,usec,usec,usec
    12919.90,958,1,3
    12919.90,957,1,3
    Code:
    SdFatSdio - 32GB Samsung PRO+
    
    File size 5 MB
    Buffer size 50 bytes
    
    write speed and latency
    speed,max,min,avg
    KB/Sec,usec,usec,usec
    624.84,19050,1,79
    613.35,17481,1,81
    
    read speed and latency
    speed,max,min,avg
    KB/Sec,usec,usec,usec
    2225.19,1225,1,22
    2221.24,1228,1,22

  7. #32
    Senior Member
    Join Date
    Jul 2014
    Posts
    2,333
    Quote Originally Posted by Bill Greiman View Post
    SDHC Errata and SDHC DMA limitations prevented DMA use.
    you mind to provide some references?
    (It seems that I'm not using the right keyword for google)
    Last edited by WMXZ; 09-13-2016 at 05:36 PM.

  8. #33
    Senior Member
    Join Date
    Nov 2012
    Posts
    271
    Quote Originally Posted by WMXZ View Post
    you mind to provide some references?
    (It seems that I'm not using the right keyword for google)

    MK66FX1M0xxx18 Errata

  9. #34
    Senior Member
    Join Date
    Jul 2014
    Posts
    2,333
    Quote Originally Posted by Bill Greiman View Post
    thanks, very useful.

  10. #35
    Senior Member+ Frank B's Avatar
    Join Date
    Apr 2014
    Location
    Germany NRW
    Posts
    5,679
    This is very impressive. It reaches (almost) the max. speed with a blocksize of only 512 bytes ?

  11. #36
    Senior Member+ defragster's Avatar
    Join Date
    Feb 2015
    Posts
    9,700
    No Problems: To test the EEPROM write at HSRUN using your yield() calls - I hardcoded your read test to start at 8196 bytes and on yield calls I diverted to run a series of EEPROM updates at 240 MHz.

    I did this on 6 fresh T_3.'6 using the same 16GB flash.

    <edit>: I put it before you intended [ when !sdBusy() ] and after as commented

    <edit2> : putting code in yield was touchy. I could only get OPTION_2 to work on my system - OPTION_1 never worked :: Option11 Now it is working?
    Last edited by defragster; 09-30-2016 at 08:07 AM.

  12. #37
    Senior Member
    Join Date
    Oct 2012
    Location
    Portland OR
    Posts
    676

    Teensy Beta at 180 MHz w/ Amazon 8GB card

    In case of interest- using a Beta T3.6 with an "Amazon Basics" 8 GB class 10 SDHC card, formatted in the Teensy, running at 180 MHz.

    Code:
    SdFatSdioEX
    
    size,write,read
    bytes,KB/sec,KB/sec
    512,6865.54,18488.55
    1024,6988.21,18699.19
    2048,11382.32,18885.99
    4096,6976.06,19019.59
    8192,11714.28,19068.27
    16384,11702.87,19063.72
    32768,6970.21,19069.22
    
    totalMicros  10105510
    yieldMicros  391232
    yieldCalls   186
    yieldMaxUsec 14059
    kHzSdClk     45000
    
    -------------
    SdFatSdio
    
    size,write,read
    bytes,KB/sec,KB/sec
    512,426.85,1498.42
    1024,743.56,2583.71
    2048,1302.65,4472.45
    4096,2127.74,7112.37
    8192,3142.29,10579.55
    16384,6224.64,13955.08
    32768,5560.55,16697.97
    
    totalMicros  60638345
    yieldMicros  60242671
    yieldCalls   73470
    yieldMaxUsec 493057
    kHzSdClk     45000
    Code:
    SdFat version: 20160913
    
    type any character to start
    
    init time: 184 ms
    
    Card type: SDHC
    
    Manufacturer ID: 0X73
    OEM ID: BG
    Product: NCard
    Version: 1.0
    Serial number: 0X5093013
    Manufacturing date: 7/2012
    
    cardSize: 8035.24 MB (MB = 1,000,000 bytes)
    flashEraseSize: 128 blocks
    eraseSingleBlock: true
    OCR: 0XC0FF8000
    
    SD Partition Table
    part,boot,type,start,length
    1,0X0,0XB,8192,15685632
    2,0X0,0X0,0,0
    3,0X0,0X0,0,0
    4,0X0,0X0,0,0
    
    Volume is FAT32
    blocksPerCluster: 64
    clusterCount: 244960
    freeClusters: 244550
    freeSpace: 8013.41 MB (MB = 1,000,000 bytes)
    fatStartBlock: 12556
    fatCount: 2
    blocksPerFat: 1914
    rootDirStart: 2
    dataStartBlock: 16384

  13. #38
    Senior Member
    Join Date
    Oct 2012
    Location
    Portland OR
    Posts
    676

    how to use LowLatencyLogger with SDIO on T3.6 ?

    I would like to try the LowLatencyLogger example with the T3.6 board (SDIO) but it is written to use the SPI interface only. I tried to modify it for SDIO as below, and it seems to open a file, but then halts with an error- any ideas? I'm using Teensyduino 1.31 (Arduino 1.6.12) with the Beta board running at 180 MHz. This SD card worked OK with the "TeensySdioDemo" example as shown in my previous post. Is there some reference online I can consult to learn the meaning of "SD errorCode: 0X65,0X1" ? I don't understand how to read SdFat/src/SdCard/SdInfo.h but does 0x65 match to SD_CARD_ERROR_FUNCTION_NOT_SUPPORTED ? If so, the SDIO flavor of the library does not support all that the "plain" SPI version does?

    EDIT: I guess the problem is that
    bool SdSpiCard::writeStart (uint32_t blockNumber)
    is a low-level SPI function which is not available through the SDIO interface.

    Probably I should instead be using
    bool SdioCard::writeBlock ( uint32_t lba, const uint8_t * src )

    Code:
    FreeStack: 255371
    Records/block: 63
    
    type:
    b - open existing bin file
    c - convert file to csv
    d - dump data to Serial
    e - overrun error details
    l - list files
    r - record data
    t - test without logging
    
    Creating new file
    Erasing all data
    error: writeStart failed
    SD errorCode: 0X65,0X1
    Code:
    /**
     * This program logs data to a binary file.  Functions are included
     * to convert the binary file to a csv text file.
     *
     * Samples are logged at regular intervals.  The maximum logging rate
     * depends on the quality of your SD card and the time required to
     * read sensor data.  This example has been tested at 500 Hz with
     * good SD card on an Uno.  4000 HZ is possible on a Due.
     *
     * If your SD card has a long write latency, it may be necessary to use
     * slower sample rates.  Using a Mega Arduino helps overcome latency
     * problems since 12 512 byte buffers will be used.
     *
     * Data is written to the file using a SD multiple block write command.
     */
    #include <SPI.h>
    #include "SdFat.h"
    #include "FreeStack.h"
    #include "UserTypes.h"
    
    #ifdef __AVR_ATmega328P__
    #include "MinimumSerial.h"
    MinimumSerial MinSerial;
    #define Serial MinSerial
    #endif  // __AVR_ATmega328P__
    //==============================================================================
    // Start of configuration constants.
    //==============================================================================
    // Abort run on an overrun.  Data before the overrun will be saved.
    #define ABORT_ON_OVERRUN 1
    //------------------------------------------------------------------------------
    //Interval between data records in microseconds.
    const uint32_t LOG_INTERVAL_USEC = 20000;
    //------------------------------------------------------------------------------
    // Set USE_SHARED_SPI non-zero for use of an SPI sensor.
    // May not work for some cards.
    #ifndef USE_SHARED_SPI
    #define USE_SHARED_SPI 0
    #endif  // USE_SHARED_SPI
    //------------------------------------------------------------------------------
    // Pin definitions.
    //
    // SD chip select pin.
    //const uint8_t SD_CS_PIN = SS;
    
    //
    // Digital pin to indicate an error, set to -1 if not used.
    // The led blinks for fatal errors. The led goes on solid for
    // overrun errors and logging continues unless ABORT_ON_OVERRUN
    // is non-zero.
    #ifdef ERROR_LED_PIN
    #undef ERROR_LED_PIN
    #endif  // ERROR_LED_PIN
    const int8_t ERROR_LED_PIN = 13;
    //------------------------------------------------------------------------------
    // File definitions.
    //
    // Maximum file size in blocks.
    // The program creates a contiguous file with FILE_BLOCK_COUNT 512 byte blocks.
    // This file is flash erased using special SD commands.  The file will be
    // truncated if logging is stopped early.
    const uint32_t FILE_BLOCK_COUNT = 256000;
    //
    // log file base name if not defined in UserTypes.h
    #ifndef FILE_BASE_NAME
    #define FILE_BASE_NAME "data"
    #endif  // FILE_BASE_NAME
    //------------------------------------------------------------------------------
    // Buffer definitions.
    //
    // The logger will use SdFat's buffer plus BUFFER_BLOCK_COUNT-1 additional
    // buffers.
    //
    #ifndef RAMEND
    // Assume ARM. Use total of ten 512 byte buffers.
    const uint8_t BUFFER_BLOCK_COUNT = 10;
    //
    #elif RAMEND < 0X8FF
    #error Too little SRAM
    //
    #elif RAMEND < 0X10FF
    // Use total of two 512 byte buffers.
    const uint8_t BUFFER_BLOCK_COUNT = 2;
    //
    #elif RAMEND < 0X20FF
    // Use total of four 512 byte buffers.
    const uint8_t BUFFER_BLOCK_COUNT = 4;
    //
    #else  // RAMEND
    // Use total of 12 512 byte buffers.
    const uint8_t BUFFER_BLOCK_COUNT = 12;
    #endif  // RAMEND
    //==============================================================================
    // End of configuration constants.
    //==============================================================================
    // Temporary log file.  Will be deleted if a reset or power failure occurs.
    #define TMP_FILE_NAME FILE_BASE_NAME "##.bin"
    
    // Size of file base name.
    const uint8_t BASE_NAME_SIZE = sizeof(FILE_BASE_NAME) - 1;
    const uint8_t FILE_NAME_DIM  = BASE_NAME_SIZE + 7;
    char binName[FILE_NAME_DIM] = FILE_BASE_NAME "00.bin";
    
    // SdFat sd;
    //SdFatSdioEX sd;
    SdFatSdio sd;
    
    SdBaseFile binFile;
    
    //------------------------------------------------------------------------------
    // store error strings in flash
    #define sdErrorMsg(msg) sd.errorPrint(F(msg));
    //------------------------------------------------------------------------------
    
    // Number of data records in a block.
    const uint16_t DATA_DIM = (512 - 4)/sizeof(data_t);
    
    //Compute fill so block size is 512 bytes.  FILL_DIM may be zero.
    const uint16_t FILL_DIM = 512 - 4 - DATA_DIM*sizeof(data_t);
    
    struct block_t {
      uint16_t count;
      uint16_t overrun;
      data_t data[DATA_DIM];
      uint8_t fill[FILL_DIM];
    };
    //==============================================================================
    // Error messages stored in flash.
    #define error(msg) {sd.errorPrint(&Serial, F(msg));fatalBlink();}
    //------------------------------------------------------------------------------
    //
    void fatalBlink() {
      while (true) {
        SysCall::yield();
        if (ERROR_LED_PIN >= 0) {
          digitalWrite(ERROR_LED_PIN, HIGH);
          delay(200);
          digitalWrite(ERROR_LED_PIN, LOW);
          delay(200);
        }
      }
    }
    //------------------------------------------------------------------------------
    // read data file and check for overruns
    void checkOverrun() {
      bool headerPrinted = false;
      block_t block;
      uint32_t bn = 0;
    
      if (!binFile.isOpen()) {
        Serial.println();
        Serial.println(F("No current binary file"));
        return;
      }
      binFile.rewind();
      Serial.println();
      Serial.print(F("FreeStack: "));
      Serial.println(FreeStack());
      Serial.println(F("Checking overrun errors - type any character to stop"));
      while (binFile.read(&block, 512) == 512) {
        if (block.count == 0) {
          break;
        }
        if (block.overrun) {
          if (!headerPrinted) {
            Serial.println();
            Serial.println(F("Overruns:"));
            Serial.println(F("fileBlockNumber,sdBlockNumber,overrunCount"));
            headerPrinted = true;
          }
          Serial.print(bn);
          Serial.print(',');
          Serial.print(binFile.firstBlock() + bn);
          Serial.print(',');
          Serial.println(block.overrun);
        }
        bn++;
      }
      if (!headerPrinted) {
        Serial.println(F("No errors found"));
      } else {
        Serial.println(F("Done"));
      }
    }
    //-----------------------------------------------------------------------------
    // Convert binary file to csv file.
    void binaryToCsv() {
      uint8_t lastPct = 0;
      block_t block;
      uint32_t t0 = millis();
      uint32_t syncCluster = 0;
      SdFile csvFile;
      char csvName[FILE_NAME_DIM];
    
      if (!binFile.isOpen()) {
        Serial.println();
        Serial.println(F("No current binary file"));
        return;
      }
      Serial.println();
      Serial.print(F("FreeStack: "));
      Serial.println(FreeStack());
      
      // Create a new csvFile.
      strcpy(csvName, binName);
      strcpy(&csvName[BASE_NAME_SIZE + 3], "csv");
    
      if (!csvFile.open(csvName, O_WRITE | O_CREAT | O_TRUNC)) {
        error("open csvFile failed");
      }
      binFile.rewind();
      Serial.print(F("Writing: "));
      Serial.print(csvName);
      Serial.println(F(" - type any character to stop"));
      printHeader(&csvFile);
      uint32_t tPct = millis();
      while (!Serial.available() && binFile.read(&block, 512) == 512) {
        uint16_t i;
        if (block.count == 0 || block.count > DATA_DIM) {
          break;
        }
        if (block.overrun) {
          csvFile.print(F("OVERRUN,"));
          csvFile.println(block.overrun);
        }
        for (i = 0; i < block.count; i++) {
          printData(&csvFile, &block.data[i]);
        }
        if (csvFile.curCluster() != syncCluster) {
          csvFile.sync();
          syncCluster = csvFile.curCluster();
        }
        if ((millis() - tPct) > 1000) {
          uint8_t pct = binFile.curPosition()/(binFile.fileSize()/100);
          if (pct != lastPct) {
            tPct = millis();
            lastPct = pct;
            Serial.print(pct, DEC);
            Serial.println('%');
          }
        }
        if (Serial.available()) {
          break;
        }
      }
      csvFile.close();
      Serial.print(F("Done: "));
      Serial.print(0.001*(millis() - t0));
      Serial.println(F(" Seconds"));
    }
    //-----------------------------------------------------------------------------
    void createBinFile() {
      // max number of blocks to erase per erase call
      const uint32_t ERASE_SIZE = 262144L;
      uint32_t bgnBlock, endBlock;
      
      // Delete old tmp file.
      if (sd.exists(TMP_FILE_NAME)) {
        Serial.println(F("Deleting tmp file " TMP_FILE_NAME));
        if (!sd.remove(TMP_FILE_NAME)) {
          error("Can't remove tmp file");
        }
      }
      // Create new file.
      Serial.println(F("\nCreating new file"));
      binFile.close();
      if (!binFile.createContiguous(TMP_FILE_NAME, 512 * FILE_BLOCK_COUNT)) {
        error("createContiguous failed");
      }
      // Get the address of the file on the SD.
      if (!binFile.contiguousRange(&bgnBlock, &endBlock)) {
        error("contiguousRange failed");
      }
      // Flash erase all data in the file.
      Serial.println(F("Erasing all data"));
      uint32_t bgnErase = bgnBlock;
      uint32_t endErase;
      while (bgnErase < endBlock) {
        endErase = bgnErase + ERASE_SIZE;
        if (endErase > endBlock) {
          endErase = endBlock;
        }
        if (!sd.card()->erase(bgnErase, endErase)) {
          error("erase failed");
        }
        bgnErase = endErase + 1;
      }
    }
    //------------------------------------------------------------------------------
    // dump data file to Serial
    void dumpData() {
      block_t block;
      if (!binFile.isOpen()) {
        Serial.println();
        Serial.println(F("No current binary file"));
        return;
      }
      binFile.rewind();
      Serial.println();
      Serial.println(F("Type any character to stop"));
      delay(1000);
      printHeader(&Serial);
      while (!Serial.available() && binFile.read(&block , 512) == 512) {
        if (block.count == 0) {
          break;
        }
        if (block.overrun) {
          Serial.print(F("OVERRUN,"));
          Serial.println(block.overrun);
        }
        for (uint16_t i = 0; i < block.count; i++) {
          printData(&Serial, &block.data[i]);
        }
      }
      Serial.println(F("Done"));
    }
    //------------------------------------------------------------------------------
    // log data
    void logData() {
      createBinFile();
      recordBinFile();
      renameBinFile();
    }
    //------------------------------------------------------------------------------
    void openBinFile() {
      char name[FILE_NAME_DIM];
      strcpy(name, binName);
      Serial.println(F("\nEnter two digit version"));
      Serial.write(name, BASE_NAME_SIZE);
      for (int i = 0; i < 2; i++) {
        while (!Serial.available()) {
         SysCall::yield();
        }
        char c = Serial.read();
        Serial.write(c);
        if (c < '0' || c > '9') {
          Serial.println(F("\nInvalid digit"));
          return;
        }
        name[BASE_NAME_SIZE + i] = c;
      }
      Serial.println(&name[BASE_NAME_SIZE+2]);
      if (!sd.exists(name)) {
        Serial.println(F("File does not exist"));
        return;
      }
      binFile.close();
      strcpy(binName, name);
      if (!binFile.open(binName, O_READ)) {
        Serial.println(F("open failed"));
        return;
      }
      Serial.println(F("File opened"));
    }
    //------------------------------------------------------------------------------
    void recordBinFile() {
      const uint8_t QUEUE_DIM = BUFFER_BLOCK_COUNT + 1;
      // Index of last queue location.
      const uint8_t QUEUE_LAST = QUEUE_DIM - 1;
      
      // Allocate extra buffer space.
      block_t block[BUFFER_BLOCK_COUNT - 1];
      
      block_t* curBlock = 0;
      
      block_t* emptyStack[BUFFER_BLOCK_COUNT];
      uint8_t emptyTop;
      uint8_t minTop;
    
      block_t* fullQueue[QUEUE_DIM];
      uint8_t fullHead = 0;
      uint8_t fullTail = 0;  
    
      // Use SdFat's internal buffer.
      emptyStack[0] = (block_t*)sd.vol()->cacheClear();
      if (emptyStack[0] == 0) {
        error("cacheClear failed");
      }
      // Put rest of buffers on the empty stack.
      for (int i = 1; i < BUFFER_BLOCK_COUNT; i++) {
        emptyStack[i] = &block[i - 1];
      }
      emptyTop = BUFFER_BLOCK_COUNT;
      minTop = BUFFER_BLOCK_COUNT;
      
      // Start a multiple block write.
      if (!sd.card()->writeStart(binFile.firstBlock())) {
        error("writeStart failed");
      }
      Serial.print(F("FreeStack: "));
      Serial.println(FreeStack());
      Serial.println(F("Logging - type any character to stop"));
      bool closeFile = false;
      uint32_t bn = 0;  
      uint32_t maxLatency = 0;
      uint32_t overrun = 0;
      uint32_t overrunTotal = 0;
      uint32_t logTime = micros();
      while(1) {
         // Time for next data record.
        logTime += LOG_INTERVAL_USEC;
        if (Serial.available()) {
          closeFile = true;
        }  
        if (closeFile) {
          if (curBlock != 0) {
            // Put buffer in full queue.
            fullQueue[fullHead] = curBlock;
            fullHead = fullHead < QUEUE_LAST ? fullHead + 1 : 0;
            curBlock = 0;
          }
        } else {
          if (curBlock == 0 && emptyTop != 0) {
            curBlock = emptyStack[--emptyTop];
            if (emptyTop < minTop) {
              minTop = emptyTop;
            }
            curBlock->count = 0;
            curBlock->overrun = overrun;
            overrun = 0;
          }
          if ((int32_t)(logTime - micros()) < 0) {
            error("Rate too fast");             
          }
          int32_t delta;
          do {
            delta = micros() - logTime;
          } while (delta < 0);
          if (curBlock == 0) {
            overrun++;
            overrunTotal++;
            if (ERROR_LED_PIN >= 0) {
              digitalWrite(ERROR_LED_PIN, HIGH);
            }        
    #if ABORT_ON_OVERRUN
            Serial.println(F("Overrun abort"));
            break;
     #endif  // ABORT_ON_OVERRUN       
          } else {
    #if USE_SHARED_SPI
            sd.card()->spiStop();
    #endif  // USE_SHARED_SPI   
            acquireData(&curBlock->data[curBlock->count++]);
    #if USE_SHARED_SPI
            sd.card()->spiStart();
    #endif  // USE_SHARED_SPI      
            if (curBlock->count == DATA_DIM) {
              fullQueue[fullHead] = curBlock;
              fullHead = fullHead < QUEUE_LAST ? fullHead + 1 : 0;
              curBlock = 0;
            } 
          }
        }
        if (fullHead == fullTail) {
          // Exit loop if done.
          if (closeFile) {
            break;
          }
        } else if (!sd.card()->isBusy()) {
          // Get address of block to write.
          block_t* pBlock = fullQueue[fullTail];
          fullTail = fullTail < QUEUE_LAST ? fullTail + 1 : 0;
          // Write block to SD.
          uint32_t usec = micros();
          if (!sd.card()->writeData((uint8_t*)pBlock)) {
            error("write data failed");
          }
          usec = micros() - usec;
          if (usec > maxLatency) {
            maxLatency = usec;
          }
          // Move block to empty queue.
          emptyStack[emptyTop++] = pBlock;
          bn++;
          if (bn == FILE_BLOCK_COUNT) {
            // File full so stop
            break;
          }
        }
      }
      if (!sd.card()->writeStop()) {
        error("writeStop failed");
      }
      Serial.print(F("Min Free buffers: "));
      Serial.println(minTop);
      Serial.print(F("Max block write usec: "));
      Serial.println(maxLatency);
      Serial.print(F("Overruns: "));
      Serial.println(overrunTotal);
      // Truncate file if recording stopped early.
      if (bn != FILE_BLOCK_COUNT) {
        Serial.println(F("Truncating file"));
        if (!binFile.truncate(512L * bn)) {
          error("Can't truncate file");
        }
      }
    }
    //------------------------------------------------------------------------------
    void recoverTmpFile() {
      uint16_t count;
      if (!binFile.open(TMP_FILE_NAME, O_RDWR)) {
        return;
      }
      if (binFile.read(&count, 2) != 2 || count != DATA_DIM) {
        error("Please delete existing " TMP_FILE_NAME);
      }
      Serial.println(F("\nRecovering data in tmp file " TMP_FILE_NAME));
      uint32_t bgnBlock = 0;
      uint32_t endBlock = binFile.fileSize()/512 - 1;
      // find last used block.
      while (bgnBlock < endBlock) {
        uint32_t midBlock = (bgnBlock + endBlock + 1)/2;
        binFile.seekSet(512*midBlock);
        if (binFile.read(&count, 2) != 2) error("read");
        if (count == 0 || count > DATA_DIM) {
          endBlock = midBlock - 1;
        } else {          
          bgnBlock = midBlock;
        }
      }
      // truncate after last used block.
      if (!binFile.truncate(512*(bgnBlock + 1))) {
        error("Truncate " TMP_FILE_NAME " failed");
      }
      renameBinFile();
    }
    //-----------------------------------------------------------------------------
    void renameBinFile() {
      while (sd.exists(binName)) {
        if (binName[BASE_NAME_SIZE + 1] != '9') {
          binName[BASE_NAME_SIZE + 1]++;
        } else {
          binName[BASE_NAME_SIZE + 1] = '0';
          if (binName[BASE_NAME_SIZE] == '9') {
            error("Can't create file name");
          }
          binName[BASE_NAME_SIZE]++;
        }
      }
      if (!binFile.rename(sd.vwd(), binName)) {
        error("Can't rename file");
        }
      Serial.print(F("File renamed: "));
      Serial.println(binName);
      Serial.print(F("File size: "));
      Serial.print(binFile.fileSize()/512);
      Serial.println(F(" blocks"));
    }
    //------------------------------------------------------------------------------
    void testSensor() {
      const uint32_t interval = 200000;
      int32_t diff;
      data_t data;
      Serial.println(F("\nTesting - type any character to stop\n"));
      // Wait for Serial Idle.
      delay(1000);
      printHeader(&Serial);
      uint32_t m = micros();
      while (!Serial.available()) {
        m += interval;
        do {
          diff = m - micros();
        } while (diff > 0);
        acquireData(&data);
        printData(&Serial, &data);
      }
    }
    //------------------------------------------------------------------------------
    void setup(void) {
      if (ERROR_LED_PIN >= 0) {
        pinMode(ERROR_LED_PIN, OUTPUT);
      }
      Serial.begin(9600);
      
      // Wait for USB Serial 
      while (!Serial) {
        SysCall::yield();
      }
      Serial.print(F("\nFreeStack: "));
      Serial.println(FreeStack());
      Serial.print(F("Records/block: "));
      Serial.println(DATA_DIM);
      if (sizeof(block_t) != 512) {
        error("Invalid block size");
      }
      // Allow userSetup access to SPI bus.
      //pinMode(SD_CS_PIN, OUTPUT);
      //digitalWrite(SD_CS_PIN, HIGH);
      
      // Setup sensors.
      userSetup();
      
      // Initialize at the highest speed supported by the board that is
      // not over 50 MHz. Try a lower speed if SPI errors occur.
      /*
      if (!sd.begin(SD_CS_PIN, SD_SCK_MHZ(50))) {
        sd.initErrorPrint(&Serial);
        fatalBlink();
      } */
    
        if (!sd.begin()) {
          sd.initErrorHalt("SdFatSdio begin() failed");
        }
        // make sd the current volume.
        sd.chvol();  
      
      // recover existing tmp file.
      if (sd.exists(TMP_FILE_NAME)) {
        Serial.println(F("\nType 'Y' to recover existing tmp file " TMP_FILE_NAME));
        while (!Serial.available()) {
          SysCall::yield();
        }
        if (Serial.read() == 'Y') {
          recoverTmpFile();
        } else {
          error("'Y' not typed, please manually delete " TMP_FILE_NAME);
        }
      }
    }
    //------------------------------------------------------------------------------
    void loop(void) {
      // Read any Serial data.
      do {
        delay(10);
      } while (Serial.available() && Serial.read() >= 0);
      Serial.println();
      Serial.println(F("type:"));
      Serial.println(F("b - open existing bin file"));  
      Serial.println(F("c - convert file to csv"));
      Serial.println(F("d - dump data to Serial"));
      Serial.println(F("e - overrun error details"));
      Serial.println(F("l - list files"));  
      Serial.println(F("r - record data"));
      Serial.println(F("t - test without logging"));
      while(!Serial.available()) {
        SysCall::yield();
      }
    #if WDT_YIELD_TIME_MICROS
      Serial.println(F("LowLatencyLogger can not run with watchdog timer"));
      SysCall::halt();
    #endif
      
      char c = tolower(Serial.read());
    
      // Discard extra Serial data.
      do {
        delay(10);
      } while (Serial.available() && Serial.read() >= 0);
    
      if (ERROR_LED_PIN >= 0) {
        digitalWrite(ERROR_LED_PIN, LOW);
      }
      if (c == 'b') {
        openBinFile();
      } else if (c == 'c') {
        binaryToCsv();
      } else if (c == 'd') {
        dumpData();
      } else if (c == 'e') {
        checkOverrun();
      } else if (c == 'l') {
        Serial.println(F("\nls:"));  
        sd.ls(&Serial, LS_SIZE);  
      } else if (c == 'r') {
        logData();
      } else if (c == 't') {
        testSensor();    
      } else {
        Serial.println(F("Invalid entry"));
      }
    }
    Last edited by JBeale; 09-30-2016 at 09:01 PM. Reason: add notes on SDIO writeBlock vs SPI writeStart

  14. #39
    Senior Member
    Join Date
    Aug 2016
    Posts
    149
    I have the version 1.8.4 in arduino and teensyduino 1.38,i test the example "readwrite" of the library sdfat and on the serial screen i get the message
    "Initializing SD card...initialization failed!",the sd card is connected in sd card port.Μy sd card is configured in fat32,i have the teensy3.6.
    Code:
    /*
      SD card read/write
    
     This example shows how to read and write data to and from an SD card file
     The circuit:
     * SD card attached to SPI bus as follows:
     ** MOSI - pin 11
     ** MISO - pin 12
     ** CLK - pin 13
    
     created   Nov 2010
     by David A. Mellis
     modified 9 Apr 2012
     by Tom Igoe
    
     This example code is in the public domain.
    
     */
    
    #include <SPI.h>
    //#include <SD.h>
    #include "SdFat.h"
    SdFat SD;
    
    #define SD_CS_PIN SS
    File myFile;
    
    void setup() {
      // Open serial communications and wait for port to open:
      Serial.begin(9600);
      while (!Serial) {
        ; // wait for serial port to connect. Needed for native USB port only
      }
    
    
      Serial.print("Initializing SD card...");
    
      if (!SD.begin(SD_CS_PIN)) {
        Serial.println("initialization failed!");
        return;
      }
      Serial.println("initialization done.");
    
      // open the file. note that only one file can be open at a time,
      // so you have to close this one before opening another.
      myFile = SD.open("test.txt", FILE_WRITE);
    
      // if the file opened okay, write to it:
      if (myFile) {
        Serial.print("Writing to test.txt...");
        myFile.println("testing 1, 2, 3.");
        // close the file:
        myFile.close();
        Serial.println("done.");
      } else {
        // if the file didn't open, print an error:
        Serial.println("error opening test.txt");
      }
    
      // re-open the file for reading:
      myFile = SD.open("test.txt");
      if (myFile) {
        Serial.println("test.txt:");
    
        // read from the file until there's nothing else in it:
        while (myFile.available()) {
          Serial.write(myFile.read());
        }
        // close the file:
        myFile.close();
      } else {
        // if the file didn't open, print an error:
        Serial.println("error opening test.txt");
      }
    }
    
    void loop() {
      // nothing happens after setup
    }
    Last edited by thanos; 09-03-2017 at 12:18 PM.

  15. #40
    Senior Member
    Join Date
    Nov 2012
    Posts
    271
    Quote Originally Posted by thanos View Post
    I have the version 1.8.4 in arduino and teensyduino 1.38,i test the example "readwrite" of the library sdfat and on the serial screen i get the message
    "Initializing SD card...initialization failed!",the sd card is connected in sd card port.Μy sd card is configured in fat32,i have the teensy3.6.
    The readWrite example does not support the Teensy 3.6 built-in uSD. You must use either the SdFatSdio or SdFatSdioEX class.

    Only the bench.ino, SdFormatter.ino, SdInfo.ino, and TeensySdioDemo.ino examples support the Teensy 3.6 built-in uSD.

    Here is a modifed version of the readWrite example that should tun on Teensy 3.6.
    Code:
    /*
      SD card read/write
    
     This example shows how to read and write data to and from an SD card file
     The circuit:
     * SD card attached to SPI bus as follows:
     ** MOSI - pin 11
     ** MISO - pin 12
     ** CLK - pin 13
    
     created   Nov 2010
     by David A. Mellis
     modified 9 Apr 2012
     by Tom Igoe
    
     This example code is in the public domain.
    
     */
    
    #include <SPI.h>
    //#include <SD.h>
    #include "SdFat.h"
    SdFatSdioEX SD;
    
    //#define SD_CS_PIN SS
    File myFile;
    
    void setup() {
      // Open serial communications and wait for port to open:
      Serial.begin(9600);
      while (!Serial) {
        ; // wait for serial port to connect. Needed for native USB port only
      }
    
    
      Serial.print("Initializing SD card...");
    
      if (!SD.begin()) {
        Serial.println("initialization failed!");
        return;
      }
      Serial.println("initialization done.");
    
      // open the file. note that only one file can be open at a time,
      // so you have to close this one before opening another.
      myFile = SD.open("test.txt", FILE_WRITE);
    
      // if the file opened okay, write to it:
      if (myFile) {
        Serial.print("Writing to test.txt...");
        myFile.println("testing 1, 2, 3.");
        // close the file:
        myFile.close();
        Serial.println("done.");
      } else {
        // if the file didn't open, print an error:
        Serial.println("error opening test.txt");
      }
    
      // re-open the file for reading:
      myFile = SD.open("test.txt");
      if (myFile) {
        Serial.println("test.txt:");
    
        // read from the file until there's nothing else in it:
        while (myFile.available()) {
          Serial.write(myFile.read());
        }
        // close the file:
        myFile.close();
      } else {
        // if the file didn't open, print an error:
        Serial.println("error opening test.txt");
      }
    }
    
    void loop() {
      // nothing happens after setup
    }
    Here is the output:
    Code:
    Initializing SD card...initialization done.
    Writing to test.txt...done.
    test.txt:
    testing 1, 2, 3.

  16. #41
    Senior Member
    Join Date
    Aug 2016
    Posts
    149
    Ok bill thanks.

  17. #42
    Junior Member
    Join Date
    Dec 2018
    Posts
    9
    Bill, I have a quick question.
    Does
    Code:
    SdFatSdioEX sdEx;
    mean the library works with Teensy 3.6 on-board SD card?
    Does
    Code:
    SdFatSdio sd;
    mean the library works with another SD card on another SPI bus?

    I need to play WAV file and record some logs at the same time. Thinking to use this wonderful library to work with two separate SD cards

  18. #43
    [QUOTE=s_arty;194386]Bill, I have a quick question.
    Does
    Code:
    SdFatSdioEX sdEx;
    mean the library works with Teensy 3.6 on-board SD card?

    Yes this statement is true.
    I have been using SdFatSdioEX for a while now to access the internal SD on a 3.6.

    I have not tested your other statement though.
    I do see that the SdFat library has an example called "TwoCards.ino".
    There is a warning in the sketch that it can crash an UNO as it can run it out of RAM.

  19. #44
    Junior Member
    Join Date
    Dec 2018
    Posts
    9
    I've seen the TwoCards.ino example,
    thank you for the hint!

    My understanding is the two cards are on the same SPI bus and the only one difference is Chip Select pin.
    I still need to check whether I can read one card while write function is finalizing on another one. This involves some schematic change, though.

    Thanks

  20. #45
    Sorry I somehow missed that crucial detail.
    In the immortal words of that famous philosopher, Maurice Moss, "egg and my face were in alignment".

    Going go back to adding modules for this GT-521F52 fingerprint scanner...

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •