Many TLAs: MTP MSC FS SD SDFat LittleFS UsbMSCFat to work with each other 8)

[KurtE]

Aargh:

Not sure what is going on: But that is not too unusual!

We asked to read 15050 sectors, and timed out with 8182 remaining sectors to read, so it read 6868 sectors.
If we look at starting address plus numbers read it feels like it went to about sector 2D40 (11584)
Which does not look like anything that interesting...

Wondering if should go back to drawing board and get rid of the callback and let it take a minute to read in the free clusters.
You might try it and see if it still dies:
Simplest way is to go to sketch in processMSDrive

The line: if (partVols[count_partVols].begin((USBMSCDevice*)msc.usbDrive(), true, i + 1)) {
remove the cast: if (partVols[count_partVols].begin(msc.usbDrive(), true, i + 1)) {
It won't have the USBMSCdevice so it will fall back to BlockDevice which does not have the callback and so the code will short circuit out and read one sector at a time...

It that works, then maybe I can try a hybrid approach, like maybe only read in smaller groups like 256 sectors at a time...

 

[mjs513]

Aargh is right.

That changed, removing the cast worked:

Waiting up to 5 seconds for a USB drive 
Initialize USB drive...UsbBase::mscBegin called 200067a0 1 1
    After usbDriveBegin

msc # Partition Table
	part,boot,bgnCHS[3],type,endCHS[3],start,length
FAT32:	1,0,0x20,0x21,0x0,0xC,0xFE,0xFF,0xFF,2048,61675520
FAT32:	2,0,0xFE,0xFF,0xFF,0xC,0xFE,0xFF,0xFF,61677568,60817408
exFAT:	3,0,0xFE,0xFF,0xFF,0x7,0xFE,0xFF,0xFF,122494976,68661248
FAT32:	4,0,0xFE,0xFF,0xFF,0xC,0xFE,0xFF,0xFF,191156224,50868224
drive 0 Partition 0 valid:1
drive 0 Partition 1 valid:1
drive 0 Partition 2 valid:1
drive 0 Partition 3 valid:1

Initialize SDIO SD card...initialization failed.


Initialize SPI SD card...initialization failed.


***** Partition List *****
0(0:1):>> Fat32: Volume name:(FAT32) Partition Total Size:31561089024 Used:163840 time us: 5819998
1(0:2):>> Fat32: Volume name:(FAT32) Partition Total Size:31121735680 Used:114688 time us: 5565498
2(0:3):>> ExFat: Volume name:(exFAT) Partition Total Size:35151413248 Used:786432 time us: 25875
3(0:4):>> Fat32: Volume name:(FAT32) Partition Total Size:26036142080 Used:131072 time us: 2400373
done...
Enter command:

A slower than this way though.

 

[KurtE]

Still a WIP:

Reading 256 at a time:

***** Partition List *****
0(0:1):>> Fat32: Volume name:(V1) Partition Total Size:31440502784 Used:49152 time us: 952123
1(0:2):>> Fat32: Volume name:(V2) Partition Total Size:31440502784 Used:49152 time us: 952123
2(0:3):>> Fat32: Volume name:(V3) Partition Total Size:31440502784 Used:49152 time us: 952123
3(0:4):>> Fat32: Volume name:(V4) Partition Total Size:25647120384 Used:49152 time us: 777873
done...

1024 at a time

***** Partition List *****
0(0:1):>> Fat32: Volume name:(V1) Partition Total Size:31440502784 Used:49152 time us: 941373
1(0:2):>> Fat32: Volume name:(V2) Partition Total Size:31440502784 Used:49152 time us: 941248
2(0:3):>> Fat32: Volume name:(V3) Partition Total Size:31440502784 Used:49152 time us: 941248
3(0:4):>> Fat32: Volume name:(V4) Partition Total Size:25647120384 Used:49152 time us: 769123
done...

16

***** Partition List *****
0(0:1):>> Fat32: Volume name:(V1) Partition Total Size:31440502784 Used:49152 time us: 1171748
1(0:2):>> Fat32: Volume name:(V2) Partition Total Size:31440502784 Used:49152 time us: 1171748
2(0:3):>> Fat32: Volume name:(V3) Partition Total Size:31440502784 Used:49152 time us: 1171998
3(0:4):>> Fat32: Volume name:(V4) Partition Total Size:25647120384 Used:49152 time us: 958623

That part of the code in PFsVolume.cpp

typedef struct {
  uint32_t free;
  //uint32_t not_free;
  uint32_t todo;
  uint32_t clusters_per_sector;
  uint32_t sectors_left_in_call;
} _gfcc_t;


static void _getfreeclustercountCB(uint32_t token, uint8_t *buffer) 
{
  //digitalWriteFast(1, HIGH);
//  Serial.print("&");
  _gfcc_t *gfcc = (_gfcc_t *)token;
  uint16_t cnt = gfcc->clusters_per_sector;
  if (cnt > gfcc->todo) cnt = gfcc->todo;
  gfcc->todo -= cnt; // update count here...
  gfcc->sectors_left_in_call--;
  if (gfcc->clusters_per_sector == 512/2) {
    // fat16
    uint16_t *fat16 = (uint16_t *)buffer;
    while (cnt-- ) {
      if (*fat16++ == 0) gfcc->free++;
    }
  } else {
    uint32_t *fat32 = (uint32_t *)buffer;
    while (cnt-- ) {
      if (*fat32++ == 0) gfcc->free++;
      //else gfcc->not_free++;
    }
  }

  //digitalWriteFast(1, LOW);
}

//-------------------------------------------------------------------------------------------------
uint32_t PFsVolume::freeClusterCount()  {
  // For XVolume lets let the original code do it.
//  Serial.println("PFsVolume::freeClusterCount() called");
  if (m_xVol) return m_xVol->freeClusterCount();

  if (!m_fVol) return 0;

  if (!m_usmsci) return m_fVol->freeClusterCount();

  // So roll our own here for Fat16/32...
  _gfcc_t gfcc; 
  gfcc.free = 0;
  //gfcc.not_free = 0;

  switch (m_fVol->fatType()) {
    default: return 0;
    case FAT_TYPE_FAT16: gfcc.clusters_per_sector = 512/2; break;
    case FAT_TYPE_FAT32: gfcc.clusters_per_sector = 512/4; break;
  }
  gfcc.todo = m_fVol->clusterCount() + 2;
#if 0    
    Serial.printf("###PFsVolume::freeClusterCount: FT:%u\n", m_fVol->fatType());
    Serial.printf("    m_sectorsPerCluster:%u\n", m_fVol->sectorsPerCluster());
    //Serial.printf("    m_clusterSectorMask:%u\n", m_fVol->clusterSectorMask());
    Serial.printf("    m_sectorsPerClusterShift:%u\n", m_fVol->sectorsPerClusterShift());
    Serial.printf("    m_rootDirEntryCount:%u\n", m_fVol->rootDirEntryCount());
    //Serial.printf("    m_allocSearchStart:%u\n", m_fVol->allocSearchStart());
    Serial.printf("    m_sectorsPerFat:%u\n", m_fVol->sectorsPerFat());
    Serial.printf("    m_dataStartSector:%u\n", m_fVol->dataStartSector());
    Serial.printf("    m_fatStartSector:%u\n", m_fVol->fatStartSector());
    //Serial.printf("    m_lastCluster:%u\n", m_fVol->lastCluster());
    Serial.printf("    m_rootDirStart:%u\n", m_fVol->rootDirStart());
    Serial.printf("    m_bytesPerSector:%u\n", m_fVol->bytesPerSector());
#endif

//  digitalWriteFast(0, HIGH);
//  Serial.println("    Using readSectorswithCB");
  #define CNT_FATSECTORS_PER_CALL 16
  uint32_t first_sector = m_fVol->fatStartSector();
  uint32_t sectors_left = m_fVol->sectorsPerFat();
  bool succeeded = true;

  while (sectors_left) {
    uint32_t sectors_to_write = (sectors_left < CNT_FATSECTORS_PER_CALL)? sectors_left : CNT_FATSECTORS_PER_CALL;
    gfcc.sectors_left_in_call = sectors_to_write;

    succeeded = m_usmsci->readSectorsWithCB(first_sector,sectors_to_write, &_getfreeclustercountCB, (uint32_t)&gfcc);
    if (!succeeded) break;
    sectors_left -= sectors_to_write;
    first_sector += sectors_to_write;
  }

//  digitalWriteFast(0, LOW);
  if(!succeeded) gfcc.free = (uint32_t)-1;
  //Serial.printf("    status: %u free cluster: %x not free:%x\n", succeeded, gfcc.free, gfcc.not_free);
  return gfcc.free;
}

uint32_t PFsVolume::getFSInfoSectorFreeClusterCount() {
  uint8_t sector_buffer[512];
  if (fatType() != FAT_TYPE_FAT32) return (uint32_t)-1;

  // We could probably avoid this read if our class remembered the starting sector number for the partition...
  if (!m_blockDev->readSector(0, sector_buffer)) return (uint32_t)-1;
  MbrSector_t *mbr = reinterpret_cast<MbrSector_t*>(sector_buffer);
  MbrPart_t *pt = &mbr->part[m_part - 1];
  BpbFat32_t* bpb;
  if ((pt->type != 11) && (pt->type != 12))  return (uint32_t)-1;

  uint32_t volumeStartSector = getLe32(pt->relativeSectors);
  if (!m_blockDev->readSector(volumeStartSector, sector_buffer)) return (uint32_t)-1;
  pbs_t *pbs = reinterpret_cast<pbs_t*> (sector_buffer);
  bpb = reinterpret_cast<BpbFat32_t*>(pbs->bpb);
  
  //Serial.println("\nReadFat32InfoSectorFree BpbFat32_t sector");
  //dump_hexbytes(sector_buffer, 512);
  uint16_t infoSector = getLe16(bpb->fat32FSInfoSector); 

  // I am assuming this sector is based off of the volumeStartSector... So try reading from there.
  //Serial.printf("Try to read Info sector (%u)\n", infoSector); Serial.flush(); 
  if (!m_blockDev->readSector(volumeStartSector+infoSector, sector_buffer)) return (uint32_t)-1;
  //dump_hexbytes(sector_buffer, 512);
  FsInfo_t *pfsi = reinterpret_cast<FsInfo_t*>(sector_buffer);

  // check signatures:
  if (getLe32(pfsi->leadSignature) !=  FSINFO_LEAD_SIGNATURE) Serial.println("Lead Sig wrong");
  if (getLe32(pfsi->structSignature) !=  FSINFO_STRUCT_SIGNATURE) Serial.println("struct Sig wrong");    
  if (getLe32(pfsi->trailSignature) !=  FSINFO_TRAIL_SIGNATURE) Serial.println("Trail Sig wrong");    
  uint32_t free_count = getLe32(pfsi->freeCount);
  return free_count;


}

bool PFsVolume::setUpdateFSInfoSectorFreeClusterCount(uint32_t free_count) {
  uint8_t sector_buffer[512];
  if (fatType() != FAT_TYPE_FAT32) return (uint32_t)false;

  if (free_count == (uint32_t)-1) free_count = freeClusterCount();

  // We could probably avoid this read if our class remembered the starting sector number for the partition...
  if (!m_blockDev->readSector(0, sector_buffer)) return (uint32_t)-1;
  MbrSector_t *mbr = reinterpret_cast<MbrSector_t*>(sector_buffer);
  MbrPart_t *pt = &mbr->part[m_part - 1];
  BpbFat32_t* bpb;
  if ((pt->type != 11) && (pt->type != 12))  return (uint32_t)-1;

  uint32_t volumeStartSector = getLe32(pt->relativeSectors);
  if (!m_blockDev->readSector(volumeStartSector, sector_buffer)) return (uint32_t)-1;
  pbs_t *pbs = reinterpret_cast<pbs_t*> (sector_buffer);
  bpb = reinterpret_cast<BpbFat32_t*>(pbs->bpb);
  
  //Serial.println("\nReadFat32InfoSectorFree BpbFat32_t sector");
  //dump_hexbytes(sector_buffer, 512);
  uint16_t infoSector = getLe16(bpb->fat32FSInfoSector); 

  // OK we now need to fill in the the sector with the appropriate information...
  // Not sure if we should read it first or just blast out new data...
  FsInfo_t *pfsi = reinterpret_cast<FsInfo_t*>(sector_buffer);
  memset(sector_buffer, 0 , 512);

  // write FSINFO sector and backup
  setLe32(pfsi->leadSignature, FSINFO_LEAD_SIGNATURE);
  setLe32(pfsi->structSignature, FSINFO_STRUCT_SIGNATURE);
  setLe32(pfsi->freeCount, free_count);
  setLe32(pfsi->nextFree, 0XFFFFFFFF);
  setLe32(pfsi->trailSignature, FSINFO_TRAIL_SIGNATURE);
  if (!m_blockDev->writeSector(volumeStartSector+infoSector, sector_buffer)) return (uint32_t) false;
  return true;
}

 

[mjs513]

Looking good.

Doesn't look like much time difference between 256 and 1024

 

[KurtE]

Question is will it work on yours

 

[mjs513]

Question is will it work on yours

Sorry made the change and then got called away.

The answer is yes it works!

***** Partition List *****
0(0:1):>> Fat32: Volume name:(FAT32) Partition Total Size:31561089024 Used:163840 time us: 1176373
1(0:2):>> Fat32: Volume name:(FAT32) Partition Total Size:31121735680 Used:114688 time us: 1160123
2(0:3):>> ExFat: Volume name:(exFAT) Partition Total Size:35151413248 Used:786432 time us: 25250
3(0:4):>> Fat32: Volume name:(FAT32) Partition Total Size:26036142080 Used:131072 time us: 485498
done...
Enter command:

Another issue bites the dust.

 

[KurtE]

- Pushed up changes to github...

At some point maybe we should migrate the changes back into the "Master" branch ?

 

[mjs513]

- Pushed up changes to github...

At some point maybe we should migrate the changes back into the "Master" branch ?

I agree. A lot has been done to the USBHost, MSC and MTP. Need to get the baseline reset before we destroy something

 

[wwatson]

@KurtE @mjs513 - You guy's did it!

Everything seems to be rock solid. Nothing at this point is failing. Now to setup blank or existing drives MBR's and partitions for formatting.

1) Check for a valid MBR. If not create one.

2) Check for at least one valid partition, If not create 1 or all partitions.

3) Format a partition with volume name and partition type.

Does this make sense? A complete formatting sketch

 

[wwatson]

This is what I am now working on based on existing code.

 

[KurtE]

I have now submitted PRs for MSC and USBHost... so far I have not done it yet for a new one with MTP.

I still have a previous PR on MTP for events support from January...

As for MBR, I might play with our current sketch some, to get an idea... I think we can get most of the pieces.

 

[mjs513]

I have now submitted PRs for MSC and USBHost... so far I have not done it yet for a new one with MTP.

I still have a previous PR on MTP for events support from January...

As for MBR, I might play with our current sketch some, to get an idea... I think we can get most of the pieces.

Good Morning @KurtE

Yes most of the info is contained in the MBR. Think the biggest piece is what you solved yesterday is getting the free space. Not sure what else you are looking for?

 

[wwatson]

@Kurte - PR done.

Thanks

Edit: I went through some of the examples to make sure they still worked. I need to update one or two of them otherwise they are good.

Also, 'PfsFatFormatter.h' needs to be renamed to 'PFsFatFormatter.h'. Causes an error when compiling under Linux. I fixed it on my GitHub.

 

[KurtE]

@all - Playing around this morning,

And I took a 1GB thumb drive, went back to windows and deleted the partition, and then run the formatter.

Note, I turned on some debug stuff and

void processMSDrive(uint8_t drive_number, msController &msDrive, UsbFs &msc)
{
  Serial.printf("Initialize USB drive...");
  //cmsReport = 0;
  if (!msc.begin(&msDrive)) {
    Serial.println("");
    msc.errorPrint(&Serial);
    Serial.printf("initialization drive %u failed.\n", drive_number);[CODE]

[/CODE]
The msc.begin will fail. It will initialize the drive but fail on volume...

Waiting up to 5 seconds for a USB drive 
Initialize USB drive...UsbBase::mscBegin called 20006140 1 1
    After usbDriveBegin
<<< msReadBlocks(0 1 200)
<<< msReadBlocks(0 1 200)

Check USB drive format.
initialization drive 0 failed.

So for the fun it if I tchange that code above to include:

    Serial.printf("initialization drive %u failed.\n", drive_number);

    // see if we can read capacity of this device
    msSCSICapacity_t mscap;
    uint8_t status = msDrive.msReadDeviceCapacity(&mscap);
    Serial.printf("Read Capacity: status: %u Blocks: %u block Size: %u\n", status, mscap.Blocks, mscap.BlockSize);
    Serial.printf("From object: Blocks: %u Size: %u\n", msDrive.msCapacity.Blocks, msDrive.msCapacity.BlockSize);
    return;
  }

Check USB drive format.
initialization drive 0 failed.
Read Capacity: status: 0 Blocks: 2014270 block Size: 512
From object: Blocks: 2014270 Size: 512

So I verified that it already read in that information and that looks valid... Not sure if we can detect when it fails for other reasons. But by this time we have already claimed the device so we assume it is valid...

So maybe at this point, not hard to put something dummy up.

 

[mjs513]

Actually, you are seeing unallocated sectors so technically if you can pass the sector start and the partition size you can format either the whole thumb drive or create partitions. I see we are going down the rabbit hole I mentioned Figured we would.

 

[KurtE]

Yep, I think that is the rabbit hole he is talking about...

My impression of the SDFat code formatter probably does this? That is I don't think it even preserves the partitions, but simply blasts out a whole new one... Could be wrong, but that was my impression of it.

Sort of like the PC app SDFormatter - if you set the option to allow resize, it will undo all of the partitions.

 

[mjs513]

Yep, I think that is the rabbit hole he is talking about...

My impression of the SDFat code formatter probably does this? That is I don't think it even preserves the partitions, but simply blasts out a whole new one... Could be wrong, but that was my impression of it.

Sort of like the PC app SDFormatter - if you set the option to allow resize, it will undo all of the partitions.

Your impression is correct. It just blasts the drive and reformats as a single drive. Partition info is destroyed. SDFat and specified starting sector for the volume (relative sectors). Actually that is relatively easy to implement if you want just to destroy all the partition info Problem comes in if you want to create partitions from unallocated space on the drive. Well not issue but challenge

 

[KurtE]

Yes, I can see several rabbit holes we could go down

a) kiss - Format the whole disk like you mentioned. reasonably easy.
b) add a new partition on to end of a drive that has free space. Probably not hard:
I hacked my current version of the sketch that mbrDMp, remembers the max of start+total and returns it:

uint32_t mbrDmp(BlockDeviceInterface *blockDev) {
  MbrSector_t mbr;
  uint32_t next_free_sector = 0;
...
    uint32_t starting_sector = getLe32(pt->relativeSectors);
    uint32_t total_sector = getLe32(pt->totalSectors);
    Serial.print(starting_sector, DEC); Serial.print(',');
    Serial.println(total_sector);

    // Lets get the max of start+total
    starting_sector += total_sector;
    if (starting_sector > next_free_sector) next_free_sector = starting_sector;
  }
  return next_free_sector;

And in:

void processMSDrive(uint8_t drive_number, msController &msDrive, UsbFs &msc)
...
  uint32_t next_free_sector = mbrDmp( msc.usbDrive() );
  Serial.printf("Device blocks:%u next free:%u\n",  msDrive.msCapacity.Blocks, next_free_sector);

And on my full fill 8GB I see:

msc # Partition Table
	part,boot,bgnCHS[3],type,endCHS[3],start,length
FAT16:	1,0,0x20,0x21,0x0,0x6,0x1A,0x9A,0xFA,2048,3072000
FAT16:	2,0,0x1A,0x9B,0xFA,0x6,0xFE,0xFF,0xFF,3074048,3993600
FAT32:	3,0,0xEF,0x79,0xB7,0xC,0xE6,0xB0,0xB6,7067648,4096000
FAT32:	4,0,0xE6,0xB1,0xB6,0xC,0xFE,0xFF,0xD2,11163648,4564992
Device blocks:15728639 next free:15728640

So no free space.
Now I am going to delete 2 and 4 to see what it shows up as:

msc # Partition Table
	part,boot,bgnCHS[3],type,endCHS[3],start,length
FAT16:	1,0,0x20,0x21,0x0,0x6,0x1A,0x9A,0xFA,2048,3072000
FAT32:	2,0,0xEF,0x79,0xB7,0xC,0xE6,0xB0,0xB6,7067648,4096000
pt_#0:	3,0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0,0
pt_#0:	4,0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0,0
Device blocks:15728639 next free:11163648
drive 0 Partition 0 valid:1
drive 0 Partition 1 valid:1

Again sort of interesting:
It compressed out the hole for 2nd one in sector 0:
But it does show: I can create a partition at the end:
Staring at 11163648 and length could be: 15728639 -11163648 = 4564991 add 1 to get back to the previous one above...

c) Fill in holes:
from above: Partition 0: 2048+3072000 = 3074048
Partition 1 starts at: 7067648 so we have 7067648-3074048) = 3993600 unused sectors.
That we can detect and create partition...
Only question would be, if the MBR is in sector order... i.e. what is now shown as 2, needs to be 3?

Edit: Yes it moved the partition numbers down in sector order:

msc # Partition Table
	part,boot,bgnCHS[3],type,endCHS[3],start,length
FAT16:	1,0,0x20,0x21,0x0,0x6,0x1A,0x9A,0xFA,2048,3072000
FAT16:	2,0,0x59,0x1B,0xBF,0xE,0xCF,0x58,0xB7,3074048,3991552
FAT32:	3,0,0xEF,0x79,0xB7,0xC,0xE6,0xB0,0xB6,7067648,4096000
pt_#0:	4,0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0,0
Device blocks:15728639 next free:11163648

 

[mjs513]

@KurtE - nice job. Knew you couldn't resist. Not sure if this would fit into MTP wrapper but for MSC nice. I can see a graphical interface to control it - guess that is now on the to do list.

 

[KurtE]

Yep - playing some, but was waiting for Instacart delivery, which came, now to clean everything...

I hacked up:

//----------------------------------------------------------------
uint32_t mbrDmp(BlockDeviceInterface *blockDev, uint32_t device_sector_count) {
  MbrSector_t mbr;
  uint32_t next_free_sector = 8192;  // Some inital value this is default for Win32 on SD...
  // bool valid = true;
  if (!blockDev->readSector(0, (uint8_t*)&mbr)) {
    Serial.print("\nread MBR failed.\n");
    //errorPrint();
    return (uint32_t)-1;
  }
  Serial.print("\nmsc # Partition Table\n");
  Serial.print("\tpart,boot,bgnCHS[3],type,endCHS[3],start,length\n");
  for (uint8_t ip = 1; ip < 5; ip++) {
    MbrPart_t *pt = &mbr.part[ip - 1];
    uint32_t starting_sector = getLe32(pt->relativeSectors);
    uint32_t total_sector = getLe32(pt->totalSectors);
    if (starting_sector > next_free_sector) {
      Serial.printf("\t < unused area starting at: %u length %u >\n", next_free_sector, starting_sector-next_free_sector);
    }
    switch (pt->type) {
    case 4:
    case 6:
    case 0xe:
      Serial.print("FAT16:\t");
      break;
    case 11:
    case 12:
      Serial.print("FAT32:\t");
      break;
    case 7:
      Serial.print("exFAT:\t");
      break;
    case 0xf:
      Serial.print("Extend:\t");
      break;
    case 0x83: Serial.print("ext2/3/4:\t"); break; 
    default:
      Serial.print("pt_#");
      Serial.print(pt->type);
      Serial.print(":\t");
      break;
    }
    Serial.print( int(ip)); Serial.print( ',');
    Serial.print(int(pt->boot), HEX); Serial.print( ',');
    for (int i = 0; i < 3; i++ ) {
      Serial.print("0x"); Serial.print(int(pt->beginCHS[i]), HEX); Serial.print( ',');
    }
    Serial.print("0x"); Serial.print(int(pt->type), HEX); Serial.print( ',');
    for (int i = 0; i < 3; i++ ) {
      Serial.print("0x"); Serial.print(int(pt->endCHS[i]), HEX); Serial.print( ',');
    }
    Serial.print(starting_sector, DEC); Serial.print(',');
    Serial.println(total_sector);

    // Lets get the max of start+total
    if (starting_sector && total_sector)  next_free_sector = starting_sector + total_sector;
  }
  if ((device_sector_count != (uint32_t)-1) && (next_free_sector < device_sector_count)) {
    Serial.printf("\t < unused area starting at: %u length %u >\n", next_free_sector, device_sector_count-next_free_sector);
  } 
  return next_free_sector;
}

And in the MSC case I called it:

  mbrDmp( msc.usbDrive(),  msDrive.msCapacity.Blocks);

The other two places I passed -1 for now...

Now with the disk that I deleted partitions 2 and 4 and then later recreated 2, this outputs:

msc # Partition Table
	part,boot,bgnCHS[3],type,endCHS[3],start,length
FAT16:	1,0,0x20,0x21,0x0,0x6,0x1A,0x9A,0xFA,2048,3072000
FAT16:	2,0,0x59,0x1B,0xBF,0xE,0xCF,0x58,0xB7,3074048,3991552
	 < unused area starting at: 7065600 length 2048 >
FAT32:	3,0,0xEF,0x79,0xB7,0xC,0xE6,0xB0,0xB6,7067648,4096000
pt_#0:	4,0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0,0
	 < unused area starting at: 11163648 length 4564991 >
drive 0 Partition 0 valid:1
drive 0 Partition 1 valid:1
drive 0 Partition 2 valid:1

Which is sort of interesting... showing the gap between the two FAT16 and the FAT32...

Will be interesting to see if our partitions tend to walk, that is I think the
FAT16 may start in SDFat at 128 and Fat32 at 8192...

But with partitions, does it keep incrementing each time through leaving more of a hole?

 

[mjs513]

Which is sort of interesting... showing the gap between the two FAT16 and the FAT32...

Will be interesting to see if our partitions tend to walk, that is I think the
FAT16 may start in SDFat at 128 and Fat32 at 8192...

But with partitions, does it keep incrementing each time through leaving more of a hole?

Not sure but don't think so. When I playing, oops, I mean testing the Start Sector always remained the same. According ot the FAT spec there are supposedly hidden sectors before the start of the actual partition which I think is what you are seeing. Windows does the same thing when you format different partitions

 

[wwatson]

@KurtE - Yes nice job. On my 128G thumb stick I created four partitions and recorded the start sector and length for each one. Then in Linux I deleted the last two partitions. The next free sector matched the start sector of the newly created third partition. Spot on

@mjs513 - A graphical interface would be nice. Even a command line type.

 

[KurtE]

Afternoon/evening/... all

Did not get much done on it today. Was wondering about partition start address sizes with alignment...

That is if I take my 1GB Drive, clear out everything and create just Fat partition on Window

Blocks: 2014270 Size: 512
msc # Partition Table
	part,boot,bgnCHS[3],type,endCHS[3],start,length
FAT16:	1,0,0x2,0x3,0x0,0xE,0xFE,0x3F,0x7C,128,2014080
pt_#0:	2,0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0,0
pt_#0:	3,0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0,0
pt_#0:	4,0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0,0
	 < unused area starting at: 2014208 length 62 >
drive 0 Partition 0 valid:1

Formatted as Fat32 appears to give the same values. Wondering if it is the Fat type that determines this, or is it sort of coded around what the SD people decided was the right starting sector and length depending on size of disk...

 

[mjs513]

@KurtE
Think its part of the FAT Specification. Attaching the FAT spec. From what I can see SDFAT is based on the SD spec and the FAT/ExFat specifications

 

[mjs513]

Good Morning @KurtE
Finally got a chance to make your changes and ran them on a older 64GB thumb drive that I was experimenting with before we made all fixes in FAT formatting.

msc # Partition Table
	part,boot,bgnCHS[3],type,endCHS[3],start,length
FAT32:	1,0,0x20,0x21,0x0,0xC,0xFE,0xFF,0xFF,2048,60397568
[COLOR="#FF0000"]	 < unused area starting at: 60399616 length 2048 >
[/COLOR]exFAT:	2,0,0x20,0x21,0x0,0x7,0xFE,0xFF,0xFF,60401664,60743680
pt_#0:	3,0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0,0
pt_#0:	4,0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0,0
drive 0 Partition 0 valid:1
drive 0 Partition 1 valid:1

***** Partition List *****
0(0:1):>> Fat32: Volume name:(fat32) Partition Total Size:30915166208 Used:33521664 time us: 593248
1(0:2):>> ExFat: Volume name:(exFAT) Partition Total Size:31083986944 Used:25427968 time us: 21753

You can see that same GAP.

Now if I reformat in Windows for a baseline:

msc # Partition Table
	part,boot,bgnCHS[3],type,endCHS[3],start,length
FAT32:	1,0,0x20,0x21,0x0,0xC,0xFE,0xFF,0xFF,2048,60397568
FAT32:	2,0,0xFE,0xFF,0xFF,0xC,0xFE,0xFF,0xFF,60399616,60745728

then recreate the FAT32 second partition

msc # Partition Table
	part,boot,bgnCHS[3],type,endCHS[3],start,length
FAT32:	1,0,0x20,0x21,0x0,0xC,0xFE,0xFF,0xFF,2048,60397568
FAT32:	2,0,0xFE,0xFF,0xFF,0xC,0xFE,0xFF,0xFF,60399616,60745728

GAP is not there anymore. If I do the reformat a few times as a check

msc # Partition Table
	part,boot,bgnCHS[3],type,endCHS[3],start,length
FAT32:	1,0,0x20,0x21,0x0,0xC,0xFE,0xFF,0xFF,2048,60397568
FAT32:	2,0,0xFE,0xFF,0xFF,0xC,0xFE,0xFF,0xFF,60399616,60745728
pt_#0:	3,0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0,0
pt_#0:	4,0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0,0

Still no GAP so that is a good thing.