how to open SD card files by index instead of name to reduce latency?

Status
Not open for further replies.

zachtos

Well-known member
I'm looking to try and make the audio board open wave files on the SD card by index rather than file name. I've done this in the past on waveShield which came with an example shown below. The reason why, is opening by index made a big difference on file open time/latency. It made a difference in my old project, and I would like to try the same thing here. I open files rapidly (say open and close w/in 80ms) and there is a slight squeek sound when I open too rapidly. I'm hoping it would go away if I reduce latency possibly...

Basically I just want to figure out how to cycle through a few wave files on the card, and save the memory position into an array, then playback by the memory position instead of filename. Would only open by filename once at start, using RAM to store memory positions. Any help on this project would be helpful. The standard example wavplayback example modified a bit would be handy starting point.

Note this is also helpful for opening files randomly if needed (specify numbers in array), say play(index[random(1,100)]);

Code:
/*
 * This sketch illustrates opening files by index
 * which can significantly reduce latency.
 *
 * How to prepare a test SD:
 * Start with a clean newly formatted SD.
 * First copy the 400 files in the 'fill' folder to the SD.
 * Next copy the 16 files in the 'DTMF' folder to the SD.
 * There should be 416 files in the SD root directory.
 *
 * You must copy the files in the above order so the 'fill'
 * files occur in the directory before the DTMF files.
 *
 * Run this sketch using the prepared SD.  Notice the
 * difference in latency between play by name and
 * play by index.
 */

#include <WaveHC.h>
#include <WaveUtil.h>

SdReader card;    // This object holds the information for the card
FatVolume vol;    // This holds the information for the partition on the card
FatReader root;   // This holds the information for the volumes root directory
FatReader file;   // This object represent the WAV file 
WaveHC wave;      // This is the only wave (audio) object, since we will only play one at a time

// time to play each tone in milliseconds
#define PLAY_TIME 200

/*
 * Define macro to put error messages in flash memory
 */
#define error(msg) error_P(PSTR(msg))

//////////////////////////////////// SETUP
void setup() {
  Serial.begin(9600);

  if (!card.init()) error("card.init");

  // enable optimized read - some cards may timeout
  card.partialBlockRead(true);

  if (!vol.init(card)) error("vol.init");

  if (!root.openRoot(vol)) error("openRoot");

  PgmPrintln("Index files");
  indexFiles();

  PgmPrintln("Play files by index");
  playByIndex();

  PgmPrintln("Play files by name");
  playByName();
}

//////////////////////////////////// LOOP
void loop() { }

/////////////////////////////////// HELPERS
/*
 * print error message and halt
 */
void error_P(const char *str) {
  PgmPrint("Error: ");
  SerialPrint_P(str);
  sdErrorCheck();
  while(1);
}
/*
 * print error message and halt if SD I/O error, great for debugging!
 */
void sdErrorCheck(void) {
  if (!card.errorCode()) return;
  PgmPrint("\r\nSD I/O error: ");
  Serial.print(card.errorCode(), HEX);
  PgmPrint(", ");
  Serial.println(card.errorData(), HEX);
  while(1);
}

// Number of files.
#define FILE_COUNT 16

// Files are 'touch tone phone' DTMF tones, P = #, S = *
// Most phones don't have A, B, C, and D tones.
// file names are of the form DTMFx.WAV where x is one of
// the letters from fileLetter[]
char fileLetter[] =  {'0', '1', '2', '3', '4', '5', '6', 
      '7', '8', '9', 'A', 'B', 'C', 'D', 'P', 'S'}; 
      
// index of DTMF files in the root directory
uint16_t fileIndex[FILE_COUNT];
/*
 * Find files and save file index.  A file's index is is the
 * index of it's directory entry in it's directory file. 
 */
void indexFiles(void) {
  char name[10];
  
  // copy flash string to RAM
  strcpy_P(name, PSTR("DTMFx.WAV"));
  
  for (uint8_t i = 0; i < FILE_COUNT; i++) {
    
    // Make file name
    name[4] = fileLetter[i];
    
    // Open file by name
    if (!file.open(root, name)) error("open by name");
    
    // Save file's index (byte offset of directory entry divided by entry size)
    // Current position is just after entry so subtract one.
    fileIndex[i] = root.readPosition()/32 - 1;   
  }
  PgmPrintln("Done");
}
/*
 * Play file by index and print latency in ms
 */
void playByIndex(void) {
  for (uint8_t i = 0; i < FILE_COUNT; i++) {
    
    // start time
    uint32_t t = millis();
    
    // open by index
    if (!file.open(root, fileIndex[i])) {
      error("open by index");
    }
    
    // create and play Wave
    if (!wave.create(file)) error("wave.create");
    wave.play();
    
    // print time to open file and start play
    Serial.println(millis() - t);
    
    // stop after PLAY_TIME ms
    while((millis() - t) < PLAY_TIME);
    wave.stop();
    
    // check for play errors
    sdErrorCheck();
  }
  PgmPrintln("Done");
}
/*
 * Play file by name and print latency in ms
 */
void playByName(void) {
  char name[10];
  
  // copy flash string to RAM
  strcpy_P(name, PSTR("DTMFx.WAV"));
  
  for (uint8_t i = 0; i < FILE_COUNT; i++) {
    // start time
    uint32_t t = millis();
    
    // make file name
    name[4] = fileLetter[i];
    
    // open file by name
    if (!file.open(root, name)) error("open by name"); 
    
    // create wave and start play
    if (!wave.create(file))error("wave.create");
    wave.play();
    
    // print time
    Serial.println(millis() - t);
    
    // stop after PLAY_TIME ms
    while((millis() - t) < PLAY_TIME);
    wave.stop();
    
    // check for play errors
    sdErrorCheck();
  }
  PgmPrintln("Done");
}
 
as noted in the audio thread, i was curious about this, too; but (still) didn't get round to it.

what have you done?

i'd imagine it (hopefully) would boil down to:

- in your sketch, come up with the file index array (rather than an array of file names)
- add a "play" function (in sd_play_wav or sd_play_raw) that uses this


i never understood (or tried to understand) what those "400" fill files were about. i hope it could be avoided. quick look at the "ope by index" examples suggests the index thing might be easier with SdFat-beta
 
fwiw, i tried this today and it's fairly straightforward in the end. i didn't test whether it actually improves the latency (the better way at any rate seems to consist in opening the file in advance and not closing it again, ie wherever possible); but in case this saves anyone some time -- seems to simplify things at any rate as one has to deal only with integers not char arrays.

the main thing that seems to be missing from SD is some straightforward way to get at the index. so what i did was adding a function to Sdfile.cpp, resp. to the SD class, which returns the index for a given filename. that'll do for generating an array of indices during setup().

ie
Code:
(while there's files...)
{
...
 _file = root.openNextFile(); 
if_this_actually_is_a_wav_file : 
{
    _index[filecount] = SD.returnFileIndex(_file.name());
    filecount++;
}
_file.close()
...
}


more specifically, in SD.h/cpp:

Code:
uint16_t SDClass::returnFileIndex(char *filepath) {

  return root.return_index(filepath);

}


and in SdFile (based on ls()) -- there must be a much simpler way without searching the entire root directory, but i didn't get it to return the correct index.)

Code:
uint16_t SdFile::return_index(char* _name) {
  dir_t* p;
  uint16_t index,  _ext;

  rewind();
  while ((p = readDirCache())) {

    index = _ext = 0x0;
    // done if past last used entry
    if (p->name[0] == DIR_NAME_FREE) break;

    // skip deleted entry and entries for . and  ..
    if (p->name[0] == DIR_NAME_DELETED || p->name[0] == '.') continue;

    // only list subdirectories and files
    if (!DIR_IS_FILE_OR_SUBDIR(p)) continue;

    for (uint8_t i = 0; i < 8; i++) { // check filename
      
        if (_name[i] == '.' && p->name[i] == ' ') 
        { 
          _ext = i+1; 
          break; 
        }
        else if (p->name[i] != _name[i]) 
        { 
          index = 0x0; 
          break;
        }
        else index = 0x1;
    }
    if (_ext && index)  // check extension
    {
        for (uint8_t i = 0; i < 3; i++) 
        {
            if (p->name[i + 0x8] != _name[i + _ext]) 
            { 
              index = 0x0; 
              break;
            }
            else index = 0x1;
         }
    }
    if (index) { 
          index = curPosition()/32 - 1; // return index
          break; 
    }
  }
  return index;
}


the SD wrapper doesn't include uint8_t open(SdFile& dirFile, uint16_t index, uint8_t oflag) from SdFat so it has to be added to SD.h/cpp, too; as a quick hack this works for files in the root directory, returning some dummy filename:


Code:
File SDClass::open(uint16_t index, uint8_t mode) {

  SdFile file;
 
  if ( ! file.open(SD.root, index, mode)) {
      // failed to open the file :(
      return File();
  }
   

  if (mode & (O_APPEND | O_WRITE)) 
    file.seekSet(file.fileSize());
  return File(file, "xxx");
}


in play_sd_wav you'd just have to add:


Code:
bool AudioPlaySdWav::play(uint16_t index)
{
	stop();
	__disable_irq();
	AudioStartUsingSPI();
	wavfile = SD.open(index, O_READ);
	__enable_irq();
	if (!wavfile) { 
		AudioStopUsingSPI();
		return false;
	}
	buffer_length = 0;
	buffer_offset = 0;
	state_play = STATE_STOP;
	data_length = 20;
	header_offset = 0;
	state = STATE_PARSE1;
	return true;
}
 
Last edited:
Status
Not open for further replies.
Back
Top