Teensyduino File System Integration, including MTP and MSC

From the sym files it looks like it uses - in my example -60 Bytes DTCM + some more for code. Not clear how much.
60 Bytes are negletible. Even the RAM gaps and padding are way larger. However, someone should look how much code it needs, and if it can used in FLASH. Didn't want to fiddel too much now (after midnight here) so've just put it into ITCM.

So.. seems to be a reasonable way. And no code-change needed.
 
If we're going to end up with a significant fork of SdFat, we might as well put in malloc(16384) on the initial cluster count. Maybe we'll use less on Teensy 3.2.

Likewise, almost all SdFat usage for reading files via the cache could be expected to improve if we just increase from 1 data sector to 8 or more.
Thought I would play a little more with that test sketch for free space, and also brought in from the PFSLib code we have to
be able to read and write the information sector to try to get the estimated free space...

Code:
Initializing SD card...Free Cluster Count: 976991 dt: 7138
Prototype Free Clusters  976991 dt: 2273
Prototype information sector -1
 dt: 3
Info sector does not match computed, so update
Press any key to run again
Free Cluster Count: 976991 dt: 0
Prototype Free Clusters  976991 dt: 2272
Prototype information sector 976991
 dt: 4
Press any key to run again
For awhile I thought the read was not working as returning -1, which is same as error condition. But that is what we initialize it to when we format...

Code:
  setLe32(fsi->leadSignature, FSINFO_LEAD_SIGNATURE);
  setLe32(fsi->structSignature, FSINFO_STRUCT_SIGNATURE);
  setLe32(fsi->freeCount, 0XFFFFFFFF);
  setLe32(fsi->nextFree, 0XFFFFFFFF);
  if (!m_dev->writeSector(m_relativeSectors + 1, m_secBuf)  ||

The test code sees it is FAT32 and calls the information sector and if the count does not equal the count we computed by the FAT I call the function to update it.

If I run again it now gives me the count I saved out...

Included the sketch...
 

Attachments

  • SDFat32_Free_Cluster_test-211009a.zip
    2.4 KB · Views: 35
No, I'm not so sure, for exactly the reasons you mentioned.

But if Bill isn't willing to improve performance in the many ways we need, or if a major redesign will bring those sorts of changes in the distant future, then we're faced with a very difficult choice, aren't we?
....
Ok, here's a 2nd attempt at bringing format() with SHARED_SPI up to speed. You should get pretty similar speed as DEDICATED_SPI.

https://github.com/PaulStoffregen/SdFat/commit/76b080497d4c0dc47454a04f60c07261151ce16a

This is on a "writeSectorsSame" branch on my fork on github, it just grabbing the whole library is simpler than updating 4 files.

One downside is the dots no longer print during the lengthy write operation. At least right now I'm not feeling so sad about missing that.

Sorry Paul was a bit distracted. I ran one of Kurt's test programs for formatting and it crashes after it asks for the directory which is on format. Are we still using the format in SD library?
Code:
No Crash Data To Report
  Hopefully all is well, but certain types of crashes can't be reported:
	stuck in an infinite loop (technically, hardware still running properly)
	remaining in a low power sleep mode
	access to certain peripherals without their clock enabled (eg, FlexIO)
	change of CPU or bus clock speed without use of glitchless mux

C:\Users\Merli\AppData\Local\Temp\arduino_modified_sketch_273856\Blink.pde Oct  9 2021 19:25:15
Initializing SD card...initialization done.
mtpindex.dat                                        0    00:00

Code:
CrashReport:
  A problem occurred at (system time) 19:25:22
  Code was executing from address 0xA1A
  CFSR: 8200
	(PRECISERR) Data bus error(address in BFAR)
	(BFARVALID) Accessed Address: 0x20070274
  Temperature inside the chip was 40.63 °C
  Startup CPU clock speed is 600MHz
  Reboot was caused by auto reboot after fault or bad interrupt detected

Here's the sketch
Code:
// List files, format, list files
#include <SD.h>

// change this to match your SD shield or module;
// Teensy 2.0: pin 0
// Teensy++ 2.0: pin 20
// Wiz820+SD board: pin 4
// Teensy audio board: pin 10
// Teensy 3.5 & 3.6 & 4.1 on-board: BUILTIN_SDCARD
const int chipSelect = 10; // BUILTIN_SDCARD;

void setup()
{
  //Uncomment these lines for Teensy 3.x Audio Shield (Rev C)
  //SPI.setMOSI(7);  // Audio shield has MOSI on pin 7
  //SPI.setSCK(14);  // Audio shield has SCK on pin 14

  // Open serial communications and wait for port to open:
  Serial.begin(9600);
  while (!Serial) {
    ; // wait for serial port to connect.
  }
  Serial.print(CrashReport);
  Serial.println("\n" __FILE__ " " __DATE__ " " __TIME__);
  delay(1000);
  Serial.print("Initializing SD card...");

  if (!SD.begin(chipSelect)) {
    Serial.println("initialization failed!");
    return;
  }
  Serial.println("initialization done.");

  File root = SD.open("/");

  printDirectory(root, 0);
  root.close();

  Serial.println("Press any key to reformat disk");
  while (Serial.read() == -1);
  while (Serial.read() != -1) ;
  elapsedMillis emFormat = 0;
  SD.format(0,'*', Serial);
  Serial.printf("Format complete %u\n", (uint32_t)emFormat);

  Serial.println("\nNow try listing files again");
//  while (Serial.read() == -1);
//  while (Serial.read() != -1) ;
  root = SD.open("/");
  printDirectory(root, 0);

  Serial.println("done!");
}

void loop()
{
  // nothing happens after setup finishes.
}

void printDirectory(File dir, int numSpaces) {
  while (true) {
    File entry = dir.openNextFile();
    if (! entry) {
      //Serial.println("** no more files **");
      break;
    }
    printSpaces(numSpaces);
    Serial.print(entry.name());
    if (entry.isDirectory()) {
      Serial.println("/");
      printDirectory(entry, numSpaces + 2);
    } else {
      // files have sizes, directories do not
      unsigned int n = log10(entry.size());
      if (n > 10) n = 10;
      printSpaces(50 - numSpaces - strlen(entry.name()) - n);
      Serial.print("  ");
      Serial.print(entry.size(), DEC);
      DateTimeFields datetime;
      if (entry.getModifyTime(datetime)) {
        printSpaces(4);
        printTime(datetime);
      }
      Serial.println();
    }
    entry.close();
  }
}

void printSpaces(int num) {
  for (int i = 0; i < num; i++) {
    Serial.print(" ");
  }
}

void printTime(const DateTimeFields tm) {
  const char *months[12] = {
    "January", "February", "March", "April", "May", "June",
    "July", "August", "September", "October", "November", "December"
  };
  if (tm.hour < 10) Serial.print('0');
  Serial.print(tm.hour);
  Serial.print(':');
  if (tm.min < 10) Serial.print('0');
  Serial.print(tm.min);
  Serial.print("  ");
  Serial.print(months[tm.mon]);
  Serial.print(" ");
  Serial.print(tm.mday);
  Serial.print(", ");
  Serial.print(tm.year + 1900);
}

EDIT: Tried it from within mtp_test to see if format worked and yes it formated the drive - don't have the time though. Rechecking the sketch to see what the problem is.

Commented out the first printDirectory and the it worked:
Code:
[B]Format complete 4342[/B]
I miss the progress the bar :) may have to put a LED on the T4 now.

EDIT2:
Reran the test sketch with the first print directory back and this time round it worked:
Code:
D:\Users\Merli\Documents\Arduino\foo_kurt\foo_kurt.ino Oct  9 2021 19:54:16
Initializing SD card...initialization done.
System Volume Information/
  WPSettings.dat                                   12    19:55  October 9, 2021
  IndexerVolumeGuid                                76    19:55  October 9, 2021
22AT Commands.pdf                              133488    06:32  July 15, 2021
Picture1.png                                  1037303    12:56  August 7, 2021
Arduino Expansion V5 SCH.pdf                    22023    11:39  July 16, 2021
card11a_rev4_web.pdf                           377551    07:01  September 21, 2021
card11b_rev4_web.pdf                           396353    07:00  September 21, 2021
Press any key to reformat disk
Writing FAT 
Format Done
Format complete 4343

Now try listing files again
done!
Must have been something got corrupted on the card during MTP testing - yes I was abusing it to high heaven

So bottom line looks like its working.
 
Last edited:
PFsLib - yep - lots of good stuff in there.

Just gave you sketch a run with my 32GB card and got this
Code:
Initializing SD card...Free Cluster Count: 973582 dt: 4855
Prototype Free Clusters  973582 dt: 2199
Prototype information sector -1
 dt: 1
Info sector does not match computed, so update
Press any key to run again
Free Cluster Count: 973582 dt: 0
Prototype Free Clusters  973582 dt: 2202
Prototype information sector 973582
 dt: 2
Press any key to run again
First pass fails then it updates the infosector and all is good and rigth. Maybe we should update format to do this automatically on format complete?

And yes by default the info sectors free clusters are set to 0xffffffff unless the application is maintaining it from what I remember.
 
Will test more tomorrow.

I have 5 more micoSD cards arriving I think Tuesday. A 3 pack of 34GB a 64GB and a 128GB...

If I integrated some of these functions and plan to use them I would build in short cuts.

Like: FatPartition code does not hold onto the information of where the first sector is of the partition (From the MBR).
Also it does not hold onto the data from the BpbFat32_t sector, which holds the offset value...

So right now I have to do 3 reads each time to get the value. The volume setup code can probably cache this data
and/or I would change these functions, that the first call would compute the sector number and then remember it
and use for subsequent calls.
 
Maybe we can get the initial free cluster count to run even faster with another extension in BlockDeviceInterface to read all the sectors as a single operation?
 
Maybe we can get the initial free cluster count to run even faster with another extension in BlockDeviceInterface to read all the sectors as a single operation?
By all I assume the smaller buffer like up to 16 sectors worth?

As on my 32GB SD card: Sectors Per Fat: 7633 which comes out to almost 3908096 bytes of data...
 
No, by "all" I mean ALL. ;)

Like as in extending BlockDevice with a special readSectors() function which reads them all in 1 shot and does the work inside the BlockDevice class (Bill's not going to like that) or maybe uses a callback as it gets each sector's data (still hard to imagine Bill will accept).

I'm playing with it now. Hope to have something to test shortly...
 
Paul: was wondering if there was a reason, that SD.format() is not simply:
Code:
bool SDClass::format(int type, char progressChar, Print& pr)
{
	return sdfs.format(&pr);
}
Which internally I think does more or less the same thing.
Some differences are: It uses it's own internal cache for the buffer so it does not need the malloc/free
And as such I am pretty sure it as such appears to not have same issue of old FS cache bleeding through...

Which was why you added the code:
Code:
	if (ret) {
		// TODO: Is begin() really necessary?  Is a quicker way possible?
		begin(cspin);
	}
Which has some pluses and minuses.
Some of the minuses include if the SD.begin() was never called, but instead you used some of the examples in SdFat_Usage.ino
Like:
Code:
  ok = SD.sdfs.begin(SdSpiConfig(chipSelect, SHARED_SPI, SD_SCK_MHZ(24)));
or:
Code:
  ok = SD.sdfs.begin(SdioConfig(FIFO_SDIO));

In these cases, the member variable cspin is not set and uses it's initial value of 255...

The plus of calling begin again, may be if the format code changes the formatting of the card. Two cases come to mind:
a) User had something like a 32GB card and old code took so long in the get cluster count, the went to PC and manually formatted as exFat... After format will be Fat32...
b) Suppose you have a 64GB card you have used on RPI, there is a smaller Fat partition, on this card, when you format this wipes the entire SDCard out to one partition, which will format as exFat...
So Which part of the SDFS object is used should change... Note: I have not fully walked through the begin code to make sure it sets the other side to NULL in this case and/or we should call end() before calling begin again...
 
Another minor issue that comes to mind while playing inside SdFat is the use of 32 bit integers for the sector number. Pretty sure nobody makes any SD cards larger than 2TB yet, but reusing SdFat's filesystem code for USB storage will limit us to 2TB drive size.
 
No, by "all" I mean ALL. ;)

Like as in extending BlockDevice with a special readSectors() function which reads them all in 1 shot and does the work inside the BlockDevice class (Bill's not going to like that) or maybe uses a callback as it gets each sector's data (still hard to imagine Bill will accept).

I'm playing with it now. Hope to have something to test shortly...

Sounds good... Actually sounds like what we added to USBMSCFat (MSC) code in the PFSVolume code...

Where we were calling:
Code:
    succeeded = m_usmsci->readSectorsWithCB(first_sector,sectors_to_write, &_getfreeclustercountCB, (uint32_t)&gfcc);
I arbitrarily restricted the code to only call for up to 256 sectors per call in case we thought that if other things on USB Host might get starved...

Code to handle that was in the USBHost MassStorageDriver code...

So yes it makes total sense to me...
 
Some of the minuses include if the SD.begin() was never called, but instead you used some of the examples in SdFat_Usage.ino

Yes, that problem has been on my mind lately. After we get past these format() and freeClusterCount() performance problems, I want to look more carefully as SdFat's initialization code and see if we can find (or add) a way to restart with the same config. Then we wouldn't need to store the pin number at all in SDClass.
 
Yes, that problem has been on my mind lately. After we get past these format() and freeClusterCount() performance problems, I want to look more carefully as SdFat's initialization code and see if we can find (or add) a way to restart with the same config. Then we wouldn't need to store the pin number at all in SDClass.

Think format is working fairly well with the changes you made. This morning I updated the MTP code to remove all the deferred formatting and the 8gb test that was put to try improve performance. There are still a couple of issues that I can't see to isolate in the MTP code. Probably something obvious that I am missing. I did describe the issue in the PR back to MTP_Teensy this morning. Been playing with it more this morning but can't see to trace it down.

Also I changed the format in SD to what @KurtE suggested:
Code:
bool SDClass::format(int type, char progressChar, Print& pr)
{
	return sdfs.format(&pr);
}
really no difference in format timing for a 32GB card - 4.4 seconds (rounded up). But put the back to it original form to avoid the minuses that were described.

So I think the last issue is with freeClusterCount and maintaining it FSINFO for Fat32 drives. Don't think you are going to see that same issue with exFAT. In addition there is no FSINFO sector in exFAT.

Another minor issue that comes to mind while playing inside SdFat is the use of 32 bit integers for the sector number. Pretty sure nobody makes any SD cards larger than 2TB yet, but reusing SdFat's filesystem code for USB storage will limit us to 2TB drive size.
Is that so bad. To really access a 2TB file you are going to need way more performance on data transfer. Never mind this is a whole different discussion that don't think we need to get side tracked on right now. :)
 
Another minor issue that comes to mind while playing inside SdFat is the use of 32 bit integers for the sector number. Pretty sure nobody makes any SD cards larger than 2TB yet, but reusing SdFat's filesystem code for USB storage will limit us to 2TB drive size.

Yes - could at some point become an issue. Not sure of things like MSC reads in blocks:
Code:
// Read Sectors (Multi Sector Capable)
uint8_t msController::msReadBlocks(
									const uint32_t BlockAddress,
									const uint16_t Blocks,
									const uint16_t BlockSize,
									void * sectorBuffer)
	{
	println("msReadBlocks()");
#ifdef DBGprint
	Serial.printf("<<< msReadBlocks(%x %x %x)\n", BlockAddress, Blocks, BlockSize);
#endif
	uint8_t BlockHi = (Blocks >> 8) & 0xFF;
	uint8_t BlockLo = Blocks & 0xFF;
Which again are 32 bits. But probably as the disk gets bigger so do the block sizes...

With the MSC code and the PFS stuff we currently look at MBR for partitions, so again maybe the 32 sectors may limit us to 2TB partitions on these drives.

I don't remember if we supported the more the 4 partitions or not... I am thinking NOT, but I remember looking at them.

But again what I wonder is if I have a 2TB SD card, am I typically going to use it all as one partition, or may I want to have multiple partitions... If so our SD card code will only show you the first one,
and Format will effect all of them... i.e. it ignores the MBR.

Which in previous conversations.... Was by design. I was informed that partitions are BAD as SD manufactures, (may) put in different hardware to handle the FAT in their drives, for wear leveling.... And adding partitions works against that...

Also I am pretty sure we only support MBR and not GPT partition setups. Could be wrong, I have not looked in awhile.
 
I don't remember if we supported the more the 4 partitions or not... I am thinking NOT, but I remember looking at them.

But again what I wonder is if I have a 2TB SD card, am I typically going to use it all as one partition, or may I want to have multiple partitions... If so our SD card code will only show you the first one,
and Format will effect all of them... i.e. it ignores the MBR.

Which in previous conversations.... Was by design. I was informed that partitions are BAD as SD manufactures, (may) put in different hardware to handle the FAT in their drives, for wear leveling.... And adding partitions works against that...

Also I am pretty sure we only support MBR and not GPT partition setups. Could be wrong, I have not looked in awhile.

Right now we only support 4 partitions per drive.

And yes we only support MBR not GPR or NTFS. Have to remember the criteria on card or even partition sizes. Why would we support a single 2TB FAT32 partition - it would by necessity be formatted to either exFAT or NTFS. Believe @KurtE showed this before:
Capture.PNG
 
Added a readSectorsCallback() function.

https://github.com/PaulStoffregen/SdFat/commit/ac5d35d3e27b1cad301b6c4b7822637342d85183

On my Samsung Evo 32GB it makes SHARED_SPI as fast as DEDICATED_SPI.

Code:
SD Card freeClusterCount Speed Test
Free Cluster Count: 976511, took [B][COLOR="#B22222"]5.07 sec[/COLOR][/B]
TestFreeClusterCount(8 sectors): 976511, took [B][COLOR="#B22222"]1.79 sec[/COLOR][/B]
  num_sectors = 7630
  7630 callbacks
TestCallback: 976511, took [B][COLOR="#B22222"]1.45 sec[/COLOR][/B]

Here's the test program which tries all 3 ways.

Code:
#include <SD.h>

const int csPin = 10;
//const int csPin = BUILTIN_SDCARD;


#define CCSECTORS_PER_READ 8
uint8_t fcc_buffer[CCSECTORS_PER_READ * 512];


void setup() {
  Serial.begin(9600);
  while (!Serial) ; // wait
  Serial.println("SD Card freeClusterCount Speed Test");
  bool ok = false;
  //ok = SD.begin(csPin);
  //ok = SD.sdfs.begin(SdSpiConfig(csPin, DEDICATED_SPI, SD_SCK_MHZ(25)));
  ok = SD.sdfs.begin(SdSpiConfig(csPin, SHARED_SPI, SD_SCK_MHZ(25)));
  if (ok) {
    elapsedMillis msec = 0;
    uint32_t n = SD.sdfs.freeClusterCount();
    Serial.printf("Free Cluster Count: %u, took %.2f sec\n",
                  n, (float)msec * 0.001);
    msec = 0;
    n = TestFreeClusterCount();
    Serial.printf("TestFreeClusterCount(%u sectors): %u, took %.2f sec\n",
                  CCSECTORS_PER_READ, n, (float)msec * 0.001);


    msec = 0;
    n = TestCallback();
    Serial.printf("TestCallback: %u, took %.2f sec\n",
                  n, (float)msec * 0.001);
  } else {
    Serial.println("could not initialize SD card");
  }
}

uint32_t sectorsPerFat()
{
  switch (SD.sdfs.fatType()) {
    case FAT_TYPE_FAT16:
      return (SD.sdfs.clusterCount() + 2 + 255) / 256;
      break;
    case FAT_TYPE_FAT32:
      return (SD.sdfs.clusterCount() + 2 + 127) / 128;
      break;
    default: return 0;
  }
}

int32_t TestFreeClusterCount() {
  uint32_t clusters_per_sector;
  switch (SD.sdfs.fatType()) {
    default: return -1; // not one we handle.
    case FAT_TYPE_FAT16: clusters_per_sector = 512 / 2; break;
    case FAT_TYPE_FAT32: clusters_per_sector = 512 / 4; break;
  }

  uint32_t free_count = 0;
  uint32_t first_sector = SD.sdfs.fatStartSector();
  uint32_t sectors_left = sectorsPerFat();
  SdCard *card = SD.sdfs.card();

  int32_t clusters_to_do = SD.sdfs.clusterCount() + 2;

  while (sectors_left) {
    uint32_t sectors_to_read = (sectors_left < CCSECTORS_PER_READ) ? sectors_left : CCSECTORS_PER_READ;
    if (!card->readSectors(first_sector, fcc_buffer, sectors_to_read)) {
      Serial.printf("Failed to read sectors: %u cnt: %u\n", first_sector, sectors_to_read);
      return -1;
    }
    // now lets process the data that we read in.
    uint16_t cnt = clusters_per_sector * sectors_to_read;
    if (cnt > clusters_to_do) cnt = clusters_to_do;
    clusters_to_do -= cnt; // update count here...

    if (clusters_per_sector == 512 / 2) {
      // fat16
      uint16_t *fat16 = (uint16_t *)fcc_buffer;
      while (cnt-- ) {
        if (*fat16++ == 0) free_count++;
      }
    } else {
      uint32_t *fat32 = (uint32_t *)fcc_buffer;
      while (cnt-- ) {
        if (*fat32++ == 0) free_count++;
      }
    }

    // update counts of sectors left to read and
    // starting position i
    sectors_left -= sectors_to_read;
    first_sector += sectors_to_read;
  }
  return free_count;
}


struct FreeClusterCountStruct {
  uint32_t clusters_to_do;
  uint32_t free_count;
  uint32_t callbacks;
};

void callback_fat16(uint32_t sector, uint8_t *buf, void *context)
{
  struct FreeClusterCountStruct *state = (struct FreeClusterCountStruct *)context;
  uint16_t *p = (uint16_t *)buf;
  unsigned int n = state->clusters_to_do;
  if (n > 256) n = 256;
  uint16_t *e = p + n;
  while (p < e) {
    if (*p++ == 0) state->free_count++;
  }
  state->clusters_to_do -= n;
  state->callbacks++;
}
void callback_fat32(uint32_t sector, uint8_t *buf, void *context)
{
  struct FreeClusterCountStruct *state = (struct FreeClusterCountStruct *)context;
  uint32_t *p = (uint32_t *)buf;
  unsigned int n = state->clusters_to_do;
  if (n > 128) n = 128;
  uint32_t *e = p + n;
  while (p < e) {
    if (*p++ == 0) state->free_count++;
  }
  state->clusters_to_do -= n;
  state->callbacks++;
}


int32_t TestCallback() {

  struct FreeClusterCountStruct state;

  state.free_count = 0;
  state.clusters_to_do = SD.sdfs.clusterCount() + 2;
  state.callbacks = 0;

  uint32_t first_sector = SD.sdfs.fatStartSector();
  uint32_t num_sectors;

  //num_sectors = SD.sdfs.m_fVol->sectorsPerFat(); // edit FsVolume.h for public
  //Serial.printf("  num_sectors = %u\n", num_sectors);

  num_sectors = sectorsPerFat();
  Serial.printf("  num_sectors = %u\n", num_sectors);

  SdCard *card = SD.sdfs.card();
  uint8_t buf[512];
  if (SD.sdfs.fatType() == FAT_TYPE_FAT32) {
    if (!card->readSectorsCallback(first_sector, buf, num_sectors, callback_fat32, &state)) return -1;
  } else {
    if (!card->readSectorsCallback(first_sector, buf, num_sectors, callback_fat16, &state)) return -1;
  }
  Serial.printf("  %u callbacks\n", state.callbacks);
  return state.free_count;
}

void loop() {
}

So far I haven't touched FatPartition freeClusterCount(). Need to take a break... might be back for a while this afternoon, or might not be until late. Non-code stuff must get done today. If anyone wants to rework freeClusterCount() to use this, please send a pull request or just post the freeClusterCount() edits here.
 
In hindsight, maybe I should have made writeSectorsCallback() rather than writeSectorsSame(). Then the callback function could do the dot printing. :)

Who knows, maybe Bill might even be willing to consider two add callcack-based functions into the official SdFat library?
 
Same here have to take a break been at it too long this morning but was curious so ran the sketch with a Sansdisk 32GB ultra card:
Code:
SD Card freeClusterCount Speed Test
Free Cluster Count: 973583, took 3.68 sec
TestFreeClusterCount(8 sectors): 973583, took 1.70 sec
  num_sectors = 7607
  7607 callbacks
TestCallback: 973583, took 1.44 sec
So have similar results with the callback metod.
 
Paul,

I will take a look. Should be reasonably simple as similar to the stuff for MSC, where we added the Read multiple sectors with CB...

The other option I wondered about with the block device code, was adding some simple logical idea of "Transaction"? or maybe another way of saying it is:
Temporary call to logically say I am DEDICATED... So at start of the current format code, say setForcedDedicated(true); and end say setForced...(false);
Probably better names then forced and dedicated. Maybe again like transaction, or group...

Now to play!
 
I tested 6 different 32GB cards. All complete the free cluster count in just under 1.5 seconds using 25 MHz SHARED_SPI with the callbacks read. Smaller cards with smaller FAT are faster, and larger cards without huge FAT to read are fast.

If 1.5 seconds is good enough for MTP on Windows, how would you feel about just making 25 MHz the default SPI clock and we can move on to other work? (eg, not worry about reading the info sector and having to keep it updated)

Code:
Samsung EVO Select
Free Cluster Count: 976511, took 6.56 sec
TestFreeClusterCount(8 sectors): 976511, took 2.04 sec
  num_sectors = 7630
TestCallback: 976511, took 1.48 sec


Sandisk Ultra
Free Cluster Count: 971555, took 4.37 sec
TestFreeClusterCount(8 sectors): 971555, took 1.79 sec
  num_sectors = 7607
TestCallback: 971555, took 1.44 sec


Kingston Canvas Select Plus
Free Cluster Count: 945343, took 3.05 sec
TestFreeClusterCount(8 sectors): 945343, took 1.61 sec
  num_sectors = 7386
TestCallback: 945343, took 1.42 sec


Samsung EVO Plus
Free Cluster Count: 976511, took 4.87 sec
TestFreeClusterCount(8 sectors): 976511, took 1.76 sec
  num_sectors = 7630
TestCallback: 976511, took 1.45 sec


PNY Elite
Free Cluster Count: 943839, took 3.09 sec
TestFreeClusterCount(8 sectors): 943839, took 1.62 sec
  num_sectors = 7374
TestCallback: 943839, took 1.42 sec


Gigastone Gaming Plus
Free Cluster Count: 974783, took 3.18 sec
TestFreeClusterCount(8 sectors): 974783, took 1.67 sec
  num_sectors = 7616
TestCallback: 974783, took 1.47 sec
 
I sent you a PR with your code merged in to replace the free cluster count...

Could maybe do things like, grab the cache buffer to use instead of stack space...

I think 25mhz would be fine for what I have seen... Speed is probably fast enough to keep MTP reasonably happy...
 
Good Morning all
For the last couple of days been playing with MTP and trying to get it to work with our latest set of changes. Some luck. For testing at least today I went the simple route and used the Simple_t4_audio_sd sketch but modified to use 2 flash and a Adafruit ext Card reader (want to use the Audio shield on the T3.6). So my storage list looks like:
Code:
Dump Storage list(5)
store:0 storage:10001 name:RAM fs:20006f40
store:1 storage:20001 name:SD_Card fs:20006724
store:2 storage:30001 name:Program fs:20006e78
store:3 storage:40001 name:Flash1 fs:20006d98
store:4 storage:50001 name:Flash2 fs:20006cc0

1. If I format say Flash2. Format completes but files are still shown in Win Explorer but if I do a 'l'ist files from the IDE format completed and no files are shown.
Code:
 Space Used = 8192
Filesystem Size = 67108864
Directory
---------
If I do a 'r'eset and refresh in windows the storage list is updated and no files are shown.

2. Using a 16GB SD Card in the ext reader. Same results as doing a format as on Flash2.

Now switching to a 32Gb card
1. copying a 16mb wav file from the desktop to the 32gb card fails
Code:
171562 CMD: 100c(SEND_OBJECT_INFO)l: 20 T:58 : 20001 ffffffff
171562 DATA:100c(SEND_OBJECT_INFO)l: 164 T:58 : 0 3000 1002a5e 3000 0
SendObjectInfo: 131073 4294967295 20004100: 0 3000 0 1002a5e 3000 0 0 0 0 0 0 0 0 0 0 : SDTEST1.wav
Read DateTime string: 20211011T080026.0
>> date/Time: 6163ef1a 10/11/2021 8:0:26
Created: 6163ef1a
Read DateTime string: 20210101T000000.0
>> date/Time: 5fee6600 1/1/2021 0:0:0
Modified: 5fee6600
171565 RESP:2001(RSP:OK)l: 24 T:58 : 20001 ffffffff a
171565 CMD: 100d(SEND_OBJECT)l: 12 T:59
MTPD::SendObject: len:16788062

MTPD::SendObject *** USB Read Timeout ***
 # USB Packets: 25972 total: 393 avg ms: 0 max: 1
 # Write: 1624 total:84475 avg ms: 52 max: 854
>>>Total Time: 85530668
*MTPD::send_Event(4001)
257096 RESP:2007(RSP:INCOMPLETE_TRANSFER)l: 12 T:59
, Then you loose sd1 and effectively MTP in windows and you have to power cycle. However if you do a list from you can still see the dirve:
Code:
 Space Used = 14548992
Filesystem Size = 32014073856
Directory
---------
22AT Commands.pdf                     133488
Picture1.png                          1037303
SDTEST1.wav                           13298164
You can even still save the datalog to it. So it has to be MTP.

2. Formatting. Format completes but you can still see the files in windows. However, this time if I send a "r"est I loose all drives in the explorer but everything still works from the IDE:
Code:
47307 CMD: 100f(FORMAT_STORE)l: 20 T:3c : 20001 0
 MTPD::formatStore2523 called
*** Start Interval Timer ***
Writing FAT 
Format Done
Format Complete(1 4482) ***
Return Response OK51789 RESP:1l: 20 T:3c : 20001 0
*** end Interval Timer ***
Reset
*MTPD::send_Event(400b)

Dump Storage list(5)
store:0 storage:10001 name:RAM fs:20006f40
store:1 storage:20001 name:SD_Card fs:20006724
store:2 storage:30001 name:Program fs:20006e78
store:3 storage:40001 name:Flash1 fs:20006d98
store:4 storage:50001 name:Flash2 fs:20006cc0

Dump Index List
Storage Index 1 Name: SD_Card Selected

 Space Used = 32768
Filesystem Size = 32014073856
Directory
---------


Logging Data!!!
1007,15,10
1006,15,9
1006,15,9
1006,15,9
1006,15,9
1006,15,9
1006,15,9
1006,15,9
1006,15,9
1006,16,9
1006,15,9
1006,15,9
1006,15,9
1006,15,9
1006,15,9
1006,15,9
1006,15,9

Stopped Logging Data!!!
Records written = 17
*MTPD::send_Event(400b)

Haven't been able to see where the problem is to be honest.
 
Back
Top