SD.open()?

jim lee

Well-known member
I'm needing to trace through SD.open() and I don't have a very good toolset for finding where it's located. On a Mac and searching through invisible directories with a text terminal and a weak grasp of UNIX commands. I think I've found where these files are, but if anyone could tell me what actual .cpp file it's in, I'd really RELLY appreciate it.

Thanks!
 
Which IDE are you using? I normally use VSCode these days with the PlatformIO plugin. There, you can drill into source code just by doing a "ctrl"-click (or whatever it's actually set to) on the function.

I was under the impression that the Arduino IDE can do this too.
 
You know.. I never tried that. Typcally I use BBEdit and there's no drilling down in that. For compiling I use Arduino IDE 2.x. I'm going to see if I can drill down in that. If I can that would solve a HUGE headache!
 
All I get is SD.h file, but no path to where it is and I can't seem to find where the actual code for it is.
 
On Macintosh, that code lives in /Users/{username}/Library/Arduino15/packages/teensy/hardware/avr/{version}/libraries/SD/src

where {username} is your user name on MacOS, and {version} is 1.60.0 or whatever version of the Teensyduino software you have installed by Arduino IDE's Boards Manager.

The Library folder in your home directory is normally hidden by MacOS Finder. But knowing its name and that it exists should be enough to search for MacOS help. Or in the Terminal app, just type "cd ~/Library".

When you dig into the code, you'll quickly discover it's just a small inline function which actually calls the SdFat library open() function. You can find that code in /Users/{username}/Library/Arduino15/packages/teensy/hardware/avr/{version}/libraries/SdFat/src.
 
Ok, I'm a little lost here.. in FatFile.ccp I find 4 open() methods. No problem. I numbered them I..IV. Now when giving me errors or running fine I see the same path through here.
open II
open III
Jump to an open() method/function that doesn't seem to exist? At least I've not found it.

Here's my code with traces in it. Yes it's a fragment. But I assume you guys have been in here before. Oh, ST is my debugging stack trace thing. Gives you the current method/function name.


//------------------------------------------------------------------------------
C++:
//------------------------------------------------------------------------------
bool FatFile::open(const char* path, oflag_t oflag) {
  ST Serial.println("Open I");
  return open(FatVolume::cwv(), path, oflag);
}
//------------------------------------------------------------------------------
bool FatFile::open(FatVolume* vol, const char* path, oflag_t oflag) {
  ST Serial.println("Open II");
  if (!vol) Serial.println("***** NOT vol! FAIL *****");
  return vol && open(vol->vwd(), path, oflag);
}
//------------------------------------------------------------------------------
bool FatFile::open(FatFile* dirFile, const char* path, oflag_t oflag) {
  ST Serial.println("Open III");
  //Serial.println("Boo!");
  FatFile tmpDir;
  FatName_t fname;

  // error if already open
  if (isOpen() || !dirFile->isDir()) {
    DBG_FAIL_MACRO;
    Serial.println("***** Already open! FAIL *****");
    goto fail;
  }
  if (isDirSeparator(*path)) {
    while (isDirSeparator(*path)) {
      path++;
    }
    if (*path == 0) {
      return openRoot(dirFile->m_vol);
    }
    if (!tmpDir.openRoot(dirFile->m_vol)) {
      DBG_FAIL_MACRO;
      Serial.println("***** Can't open temp root! FAIL *****");
      goto fail;
    }
    dirFile = &tmpDir;
  }
  while (1) {
    if (!parsePathName(path, &fname, &path)) {
      DBG_FAIL_MACRO;
      Serial.println("***** Can't parse path name! FAIL *****");
      goto fail;
    }
    if (*path == 0) {
      break;
    }
    if (!open(dirFile, &fname, O_RDONLY)) {
      Serial.println("***** Can't open directory? FAIL *****");
      DBG_WARN_MACRO;
      goto fail;
    }
    tmpDir = *this;
    dirFile = &tmpDir;
    close();
  }
  Serial.println("So far so good. Calling unknown open(dirFile, &fname, oflag)");
  return open(dirFile, &fname, oflag);

 fail:
  return false;
}
//------------------------------------------------------------------------------
bool FatFile::open(FatFile* dirFile, uint16_t index, oflag_t oflag) {
  ST
  Serial.println("Open IV");
  if (index) {
    // Find start of LFN.
    DirLfn_t* ldir;
    uint8_t n = index < 20 ? index : 20;
    for (uint8_t i = 1; i <= n; i++) {
      ldir = reinterpret_cast<DirLfn_t*>(dirFile->cacheDir(index - i));
      if (!ldir) {
        DBG_FAIL_MACRO;
        goto fail;
      }
      if (ldir->attributes != FAT_ATTRIB_LONG_NAME) {
        break;
      }
      if (ldir->order & FAT_ORDER_LAST_LONG_ENTRY) {
        if (!dirFile->seekSet(32UL*(index - i))) {
          DBG_FAIL_MACRO;
          goto fail;
        }
        break;
      }
    }
  } else {
    dirFile->rewind();
  }
  if (!openNext(dirFile, oflag)) {
    DBG_FAIL_MACRO;
    goto fail;
  }
  if (dirIndex() != index) {
    close();
    DBG_FAIL_MACRO;
    goto fail;
  }
  return true;

 fail:
  return false;
}
//------------------------------------------------------------------------------

My issue is, what's the deal with the seemingly hidden open() function? What am I missing here? It's not marked as a virtual function so it can't be overridden by something else? Right?
 
Last edited:
Before I spend more time on this, can you explain *why* this dive into SD.open() and SdFat code?

Some types of questions, like the directory on MacOS, are quick and easy to answer. SdFat is a fairly large and complicated library. Almost nothing about SdFat's internal code is quick and easy to answer. Even if we had a confirmed and reproducible bug, normally I would put that onto my list of software problems to investigate (later). But without even some indication of an actual bug, I really can't pour much time into diving into SdFat's code, especially if only explain its internal structure.

If there really is a problem to be investigated, a reproducible test case is needed. Or at least I need it, if I'm going to spend time on SdFat. Maybe other people on this forum will take an interest, but if nobody else replies, please assume the lack of response was due to not showing any specific problem, or only describing vaguely without code to reproduce the issue.
 
What's going on is I get an occasional failure on opening a file. Once it starts, everything falls apart and I get lots of file opening errors. It's semi-reproducible. Most likely its me killing/leaking memory somewhere. And that's showing up in the FatLib stuff.

My plan was to step through all this FatLib stuff and see what was getting broken. From that, I'd hoped to get a clue to where/what in my code I'm causing the issue. As you already know, in the Mac here, once I have to get into the hidden code, it becomes a nightmare.

My thinking was that, you guys had to port this to the Teensy, so you must know WAY more about it than I do. So this last question was just kinda' "Does anyone know about this?" If the answer is not an "Oh yeah, we ran into that too." Kinda' thing. Don't worry about it. I don't want you all to put a lot of time into this for me to suddenly go , "Oh duh, look what I did here."

If I can make up something that is simple that reproduces the problem, then I'll post that. But until then, I'll stick to just asking "Has anyone run across this?" Kinda' questions. And it's fine if no one has.
 
Here's another I'm having trouble finding. The dataCachePrepare() method. Can anyone with better tools find what file this one is in?

Thanks a million!
 
dataCachePrepare() method.
Does this help?
1779389025036.png
 
Alright! It ALWAYS fails in uint8_t* FsCache;;prepare(uint32_t sector, uint8_t option) LIne 41 with sector = 16384. Goes and goes and goes then when it hits this POW! But it doens't always hit this. It can go for a long time, lots of .bmp files for 32x32 icons and stuff. Then suddenly 16384 hits and it's all over.

At this point I don't know enough about all this file code to understand why, how or what. Is this maybe just a bad SD card?


EDIT:
Rats I missed something. Sometimes 16384 is fine. But when it fails, it always fails on 16384. With m_curPosition = 0.
 
Last edited:
How about m_cache ? It's not anywhere I've looked.

There are many programs you can use to search all files for specific text.

I usually use "grep" from command line. You can combine it with "find" which produces a list of files. From the SdFat directory, the command syntax would be grep m_cache `find . -type f`. Here's a screenshot of running it MacOS Terminal.

screen.jpg


But these traditional unix commands can seem pretty cryptic if you're not familiar with them.

You can also look at the code on github and click "Search this repository" when viewing any of the files. (if your view doesn't show this left side panel, resize your browser window wider...)

1779394563880.png



1779394998670.png
 
Last edited:
BBEdit really? I didn’t know that. But, If you’re on a Mac. Is there a way to get BBEdit to open a hidden file?
 
You should be able to use the bintools programs to look at the code. The Ardunio IDE will compile your code to a .elf file somewhere in its temp files. Find that and do something like:

.../arm-none-eabi-objdump -S file.elf >output.txt

(Program name prefixes may vary but is will always include "objdump".)

With the usual compile options this should generate mixed source code and assembly. It isn't the same as the source listing but it will show you what is going on without you having to dig around in the source tree.
 
Never heard of these bintools programs.

But..

The CMD-Shift-. Is magic! It even works in BBEdit. Thank you guys so much!!
 
No need to run the binutils programs like objdump, because Teensy's build process does it for you.

When you find the hidden folder with the .HEX and .ELF files, you'll also find corresponding .SYM and .LST files.

Usually the easiest way to discover that hidden folder it turning on verbose output during compile from the Arduino IDE > Settings menu (which is File > Preferences on Windows and Linux). Then look for the full pathname inside the huge amount of info that gets printed during compile.

Whether those .SYM and .LST files are actually helpful is questionable. I have personally used them when writing the early versions of the audio library, to carefully optimize the code to achieve useful audio back in the days of Teensy 3.0 at 48 MHz. But it is a lot of work and requires deep understanding of low level details. Probably not very useful for most people.
 
Back
Top