Forum Rule: Always post complete source code & details to reproduce any issue!
Page 12 of 20 FirstFirst ... 2 10 11 12 13 14 ... LastLast
Results 276 to 300 of 477

Thread: Many TLAs: MTP MSC FS SD SDFat LittleFS UsbMSCFat to work with each other 8)

  1. #276
    Senior Member+ mjs513's Avatar
    Join Date
    Jul 2014
    Location
    New York
    Posts
    6,972
    #KurtE
    Just tried changing a 2 partition fat32 to exFAT. Partition 1 reformatted to exFAT no problem. Partition 0 formatted successfully but windows giving me an error about Filesytem not recognized.

    Converting the 2 exFATs back to FAT32's seemed to work.

    Didn't see any hangs. Are you using the updated files in post @273?

  2. #277
    Senior Member+ mjs513's Avatar
    Join Date
    Jul 2014
    Location
    New York
    Posts
    6,972
    Deleted post wrong info

  3. #278
    Senior Member+ KurtE's Avatar
    Join Date
    Jan 2014
    Posts
    9,017
    A little debugging and found it is crashing in the debug code...
    Code:
    ***** Partition List *****
    0(0:1):>> ExFat: Volume name:(VOL_FAT) Partition Total Size:2080374784 Used:284950528 time us: 3753
    1(0:2):>> ExFat: Volume name:(VOL_FAT32) Partition Total Size:3128950784 Used:126615552 time us: 5502
    2(0:3):>> ExFat: Volume name:(Vol_ExFat) Partition Total Size:2791309312 Used:282198016 time us: 5499
    3(ff:1):>> Fat32: Volume name:(VOL_BOOT) Partition Total Size:264289280 Used:47383552 time us: 94947
    4(fe:1):>> ExFat: Volume name:(SDSPI_VOL) Partition Total Size:998768640 Used:239992832 time us: 1388
    done...
    Enter command:
    
     **** Start format partition 1 ****
    
    PFsFatFormatter::format called
        >> After read MBR
        >> After getVolumeLabel
    1
    Volume name:(VOL_FAT32)
    PFsFatFormatter::format................
    Sector Count: 6144000, Sectors/MB: 2048
    Partition Capacity (MB): 3000
    Fat Type: 64
        m_dataStart:4114432
        m_sectorsPerCluster:256
        m_relativeSectors:4098048
    Code:
    bool PFsFatFormatter::format(PFsVolume &partVol, uint8_t* secBuf, print_t* pr) {
      Serial.println("\nPFsFatFormatter::format called"); Serial.flush();
      MbrSector_t mbr;
    
      bool rtn;
      m_secBuf = secBuf;
      m_pr = pr;
      m_dev = partVol.blockDevice();
      m_part = partVol.part()-1;  // convert to 0 biased. 
        
      if (!m_dev->readSector(0, (uint8_t*)&mbr)) {
        Serial.print("\nread MBR failed.\n");
        //errorPrint();
        return false;
      }
      Serial.println("    >> After read MBR"); Serial.flush();
      MbrPart_t *pt = &mbr.part[m_part];
      
      m_sectorCount = getLe32(pt->totalSectors);
      m_capacityMB = (m_sectorCount + SECTORS_PER_MB - 1)/SECTORS_PER_MB;
      
      m_part_relativeSectors = getLe32(pt->relativeSectors);
    
    
    /*
      m_dataStart = partVol.dataStartSector();
      m_sectorsPerCluster = partVol.sectorsPerCluster();
      m_relativeSectors = getLe32(pt->relativeSectors);
      m_totalSectors = m_sectorCount;
      m_partType = pt->type;
      m_fatSize = partVol.getFatVol()->sectorsPerFat();
    */
      bool has_volume_label = partVol.getVolumeLabel(volName, sizeof(volName));
    Serial.println("    >> After getVolumeLabel"); Serial.flush();
    #if defined(DBG_Print)
      Serial.println(m_part);
    
      if (has_volume_label) {
    	Serial.printf("Volume name:(%s)", volName); Serial.flush();
      }
      Serial.println("\nPFsFatFormatter::format................");  Serial.flush();
      Serial.printf("Sector Count: %d, Sectors/MB: %d\n", m_sectorCount, SECTORS_PER_MB); Serial.flush();
      Serial.printf("Partition Capacity (MB): %d\n", m_capacityMB); Serial.flush();
      Serial.printf("Fat Type: %d\n", partVol.fatType()); Serial.flush();
      Serial.printf("    m_dataStart:%u\n", partVol.dataStartSector()); Serial.flush();
      Serial.printf("    m_sectorsPerCluster:%u\n",partVol.sectorsPerCluster()); Serial.flush();
      Serial.printf("    m_relativeSectors:%u\n", getLe32(pt->relativeSectors)); Serial.flush();
      Serial.printf("    m_sectorsPerFat: %u\n", partVol.getFatVol()->sectorsPerFat()); Serial.flush();
      Serial.println();
      Serial.flush();
    #endif
    Notice the code in RED...

  4. #279
    Senior Member+ mjs513's Avatar
    Join Date
    Jul 2014
    Location
    New York
    Posts
    6,972
    Ok makes sense need to get that commented out. and it should work. I had debug turned off for the following.
    EXFAT to FAT32:
    Code:
    Cards up to 2 GiB (GiB = 2^30 bytes) will be formated FAT16.
    Cards larger than 2 GiB and up to 32 GiB will be formatted
    FAT32. Cards larger than 32 GiB will be formatted exFAT.
    
    Waiting up to 5 seconds for USB drive 1
    Initialize USB drive...UsbBase::mscBegin called 20004100 1 1
        After usbDriveBegin
    PFsVolume::begin USBmscInterface(20002584, 1)
    PFsVolume::begin(20002584, 1)
    USB drive 1 is present.
    
    msc # Partition Table
    	part,boot,bgnCHS[3],type,endCHS[3],start,length
    FAT32:	1,0,0x20,0x21,0x0,0xC,0xFE,0xFF,0xFF,2048,30044160
    FAT32:	2,0,0xFE,0xFF,0xFF,0xC,0xFE,0xFF,0xFF,30046208,32505856
    pt_#0:	3,0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0,0
    pt_#0:	4,0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0,0
    PFsVolume::begin USBmscInterface(20002584, 1)
    PFsVolume::begin(20002584, 1)
    Partition 0 valid:1
    PFsVolume::begin USBmscInterface(20002584, 2)
    PFsVolume::begin(20002584, 2)
    Partition 1 valid:1
    PFsVolume::begin USBmscInterface(20002584, 3)
    PFsVolume::begin(20002584, 3)
    Partition 2 valid:0
    PFsVolume::begin USBmscInterface(20002584, 4)
    PFsVolume::begin(20002584, 4)
    Partition 3 valid:0
    0:>> Fat32: Volume name:(v1) Partition Total Size:15378415616 Used:131072 time us: 229623
        Free Clusters: API: 469308 by CB:469308 time us: 229741
        Free Clusters: Info: 4294967295 time us: 1743
    1:>> Fat32: Volume name:(v2) Partition Total Size:16638803968 Used:98304 time us: 248623
        Free Clusters: API: 507773 by CB:507773 time us: 248617
        Free Clusters: Info: 4294967295 time us: 4368
    done...
    Enter 0, 1 , 2, or 3 for Partition or Enter to Bypass
    Initialize USB drive...UsbBase::mscBegin called 20004100 1 1
        After usbDriveBegin
    PFsVolume::begin USBmscInterface(20002584, 1)
    PFsVolume::begin(20002584, 1)
    USB drive 1 is present.
    PFsVolume::begin(20002584, 1)
    Partition 0 valid:1
        m_relativeSectors:2048
        m_totalSectors:30044160
    Writing Partition Boot Sector
    Write eight Extended Boot Sectors
    Write OEM Parameter Sector and reserved sector
    Write Boot CheckSum Sector
    Writing exFAT .................................
    Write cluster two, bitmap
    Writing upcase table
    Writing root
    Format done
    PFsVolume::begin(20002584, 1)
    free clusters after begin on partVol: 117142
    Press any key to run again
    Initialize USB drive...UsbBase::mscBegin called 20004100 1 1
        After usbDriveBegin
    PFsVolume::begin USBmscInterface(20002584, 1)
    PFsVolume::begin(20002584, 1)
    USB drive 1 is present.
    
    msc # Partition Table
    	part,boot,bgnCHS[3],type,endCHS[3],start,length
    exFAT:	1,0,0x20,0x21,0x0,0x7,0xFE,0xFF,0xFF,2048,30044160
    FAT32:	2,0,0xFE,0xFF,0xFF,0xC,0xFE,0xFF,0xFF,30046208,32505856
    pt_#0:	3,0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0,0
    pt_#0:	4,0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0,0
    PFsVolume::begin USBmscInterface(20002584, 1)
    PFsVolume::begin(20002584, 1)
    Partition 0 valid:1
    PFsVolume::begin USBmscInterface(20002584, 2)
    PFsVolume::begin(20002584, 2)
    Partition 1 valid:1
    PFsVolume::begin USBmscInterface(20002584, 3)
    PFsVolume::begin(20002584, 3)
    Partition 2 valid:0
    PFsVolume::begin USBmscInterface(20002584, 4)
    PFsVolume::begin(20002584, 4)
    Partition 3 valid:0
    0:>> ExFat: Volume name:(v1) Partition Total Size:15365832704 Used:11796480 time us: 22251
        Free Clusters: API: 117142 by CB:0 time us: 0
        Free Clusters: Info: 4294967295 time us: 0
    1:>> Fat32: Volume name:(v2) Partition Total Size:16638803968 Used:196608 time us: 248623
        Free Clusters: API: 507770 by CB:507770 time us: 248616
        Free Clusters: Info: 507770 time us: 4368
    done...
    Enter 0, 1 , 2, or 3 for Partition or Enter to Bypass
    Initialize USB drive...UsbBase::mscBegin called 20004100 1 1
        After usbDriveBegin
    PFsVolume::begin USBmscInterface(20002584, 1)
    PFsVolume::begin(20002584, 1)
    USB drive 1 is present.
    PFsVolume::begin(20002584, 2)
    Partition 1 valid:1
        m_relativeSectors:30046208
        m_totalSectors:32505856
    Writing Partition Boot Sector
    Write eight Extended Boot Sectors
    Write OEM Parameter Sector and reserved sector
    Write Boot CheckSum Sector
    Writing exFAT ................................
    Write cluster two, bitmap
    Writing upcase table
    Writing root
    Format done
    PFsVolume::begin(20002584, 2)
    free clusters after begin on partVol: 126752
    Press any key to run again
    Initialize USB drive...UsbBase::mscBegin called 20004100 1 1
        After usbDriveBegin
    PFsVolume::begin USBmscInterface(20002584, 1)
    PFsVolume::begin(20002584, 1)
    USB drive 1 is present.
    
    msc # Partition Table
    	part,boot,bgnCHS[3],type,endCHS[3],start,length
    exFAT:	1,0,0x20,0x21,0x0,0x7,0xFE,0xFF,0xFF,2048,30044160
    exFAT:	2,0,0x20,0x21,0x0,0x7,0xFE,0xFF,0xFF,30046208,32505856
    pt_#0:	3,0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0,0
    pt_#0:	4,0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0,0
    PFsVolume::begin USBmscInterface(20002584, 1)
    PFsVolume::begin(20002584, 1)
    Partition 0 valid:1
    PFsVolume::begin USBmscInterface(20002584, 2)
    PFsVolume::begin(20002584, 2)
    Partition 1 valid:1
    PFsVolume::begin USBmscInterface(20002584, 3)
    PFsVolume::begin(20002584, 3)
    Partition 2 valid:0
    PFsVolume::begin USBmscInterface(20002584, 4)
    PFsVolume::begin(20002584, 4)
    Partition 3 valid:0
    0:>> ExFat: Volume name:(v1) Partition Total Size:15365832704 Used:12189696 time us: 22501
        Free Clusters: API: 117139 by CB:0 time us: 0
        Free Clusters: Info: 4294967295 time us: 0
    1:>> ExFat: Volume name:(v2) Partition Total Size:16626221056 Used:12976128 time us: 23753
        Free Clusters: API: 126749 by CB:0 time us: 0
        Free Clusters: Info: 4294967295 time us: 0
    done...
    Enter 0, 1 , 2, or 3 for Partition or Enter to Bypass
    FAT32 to EXFAT
    Code:
    Press any key to run again
    Initialize USB drive...UsbBase::mscBegin called 20004100 1 1
        After usbDriveBegin
    PFsVolume::begin USBmscInterface(20002584, 1)
    PFsVolume::begin(20002584, 1)
    USB drive 1 is present.
    
    msc # Partition Table
    	part,boot,bgnCHS[3],type,endCHS[3],start,length
    exFAT:	1,0,0x20,0x21,0x0,0x7,0xFE,0xFF,0xFF,2048,30044160
    exFAT:	2,0,0x20,0x21,0x0,0x7,0xFE,0xFF,0xFF,30046208,32505856
    pt_#0:	3,0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0,0
    pt_#0:	4,0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0,0
    PFsVolume::begin USBmscInterface(20002584, 1)
    PFsVolume::begin(20002584, 1)
    Partition 0 valid:1
    PFsVolume::begin USBmscInterface(20002584, 2)
    PFsVolume::begin(20002584, 2)
    Partition 1 valid:1
    PFsVolume::begin USBmscInterface(20002584, 3)
    PFsVolume::begin(20002584, 3)
    Partition 2 valid:0
    PFsVolume::begin USBmscInterface(20002584, 4)
    PFsVolume::begin(20002584, 4)
    Partition 3 valid:0
    0:>> ExFat: Volume name:(v1) Partition Total Size:15365832704 Used:12189696 time us: 22251
        Free Clusters: API: 117139 by CB:0 time us: 0
        Free Clusters: Info: 4294967295 time us: 0
    1:>> ExFat: Volume name:(v2) Partition Total Size:16626221056 Used:12976128 time us: 23753
        Free Clusters: API: 126749 by CB:0 time us: 0
        Free Clusters: Info: 4294967295 time us: 0
    done...
    Enter 0, 1 , 2, or 3 for Partition or Enter to Bypass
    Initialize USB drive...UsbBase::mscBegin called 20004100 1 1
        After usbDriveBegin
    PFsVolume::begin USBmscInterface(20002584, 1)
    PFsVolume::begin(20002584, 1)
    USB drive 1 is present.
    PFsVolume::begin(20002584, 1)
    Partition 0 valid:1
    ExFatFormatter - WIP
    m_dataStart: 16384, m_fatSize: 3666, r: 15533
    Writing FAT ................................
    free clusters after format: 117139
    PFsVolume::begin(20002584, 1)
    free clusters after begin on partVol: 469308
    Format Done
    Press any key to run again
    Initialize USB drive...UsbBase::mscBegin called 20004100 1 1
        After usbDriveBegin
    PFsVolume::begin USBmscInterface(20002584, 1)
    PFsVolume::begin(20002584, 1)
    USB drive 1 is present.
    
    msc # Partition Table
    	part,boot,bgnCHS[3],type,endCHS[3],start,length
    FAT32:	1,0,0x20,0x21,0x0,0xC,0xFE,0xFF,0xFF,2048,30044160
    exFAT:	2,0,0x20,0x21,0x0,0x7,0xFE,0xFF,0xFF,30046208,32505856
    pt_#0:	3,0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0,0
    pt_#0:	4,0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0,0
    PFsVolume::begin USBmscInterface(20002584, 1)
    PFsVolume::begin(20002584, 1)
    Partition 0 valid:1
    PFsVolume::begin USBmscInterface(20002584, 2)
    PFsVolume::begin(20002584, 2)
    Partition 1 valid:1
    PFsVolume::begin USBmscInterface(20002584, 3)
    PFsVolume::begin(20002584, 3)
    Partition 2 valid:0
    PFsVolume::begin USBmscInterface(20002584, 4)
    PFsVolume::begin(20002584, 4)
    Partition 3 valid:0
    0:>> Fat32: Volume name:(v1) Partition Total Size:15378415616 Used:131072 time us: 229623
        Free Clusters: API: 469308 by CB:469308 time us: 229742
        Free Clusters: Info: 4294967295 time us: 1743
    1:>> ExFat: Volume name:(v2) Partition Total Size:16626221056 Used:12976128 time us: 23628
        Free Clusters: API: 126749 by CB:0 time us: 0
        Free Clusters: Info: 4294967295 time us: 0
    done...
    Enter 0, 1 , 2, or 3 for Partition or Enter to Bypass
    Initialize USB drive...UsbBase::mscBegin called 20004100 1 1
        After usbDriveBegin
    PFsVolume::begin USBmscInterface(20002584, 1)
    PFsVolume::begin(20002584, 1)
    USB drive 1 is present.
    PFsVolume::begin(20002584, 2)
    Partition 1 valid:1
    ExFatFormatter - WIP
    m_dataStart: 16384, m_fatSize: 3967, r: 16135
    Writing FAT ................................
    free clusters after format: 126749
    PFsVolume::begin(20002584, 2)
    free clusters after begin on partVol: 507773
    Format Done
    Press any key to run again
    Initialize USB drive...UsbBase::mscBegin called 20004100 1 1
        After usbDriveBegin
    PFsVolume::begin USBmscInterface(20002584, 1)
    PFsVolume::begin(20002584, 1)
    USB drive 1 is present.
    
    msc # Partition Table
    	part,boot,bgnCHS[3],type,endCHS[3],start,length
    FAT32:	1,0,0x20,0x21,0x0,0xC,0xFE,0xFF,0xFF,2048,30044160
    FAT32:	2,0,0xFE,0xFF,0xFF,0xC,0xFE,0xFF,0xFF,30046208,32505856
    pt_#0:	3,0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0,0
    pt_#0:	4,0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0,0
    PFsVolume::begin USBmscInterface(20002584, 1)
    PFsVolume::begin(20002584, 1)
    Partition 0 valid:1
    PFsVolume::begin USBmscInterface(20002584, 2)
    PFsVolume::begin(20002584, 2)
    Partition 1 valid:1
    PFsVolume::begin USBmscInterface(20002584, 3)
    PFsVolume::begin(20002584, 3)
    Partition 2 valid:0
    PFsVolume::begin USBmscInterface(20002584, 4)
    PFsVolume::begin(20002584, 4)
    Partition 3 valid:0
    0:>> Fat32: Volume name:(v1) Partition Total Size:15378415616 Used:131072 time us: 229623
        Free Clusters: API: 469308 by CB:469308 time us: 229742
        Free Clusters: Info: 4294967295 time us: 1743
    1:>> Fat32: Volume name:(v2) Partition Total Size:16638803968 Used:98304 time us: 248623
        Free Clusters: API: 507773 by CB:507773 time us: 248617
        Free Clusters: Info: 4294967295 time us: 4368
    done...
    Enter 0, 1 , 2, or 3 for Partition or Enter to Bypass
    Right now just a hack in my MSCFormatter. Don't know what you have. But looks like it should work.

  5. #280
    Senior Member+ mjs513's Avatar
    Join Date
    Jul 2014
    Location
    New York
    Posts
    6,972
    Just as a note. SectorCount and RelativeSectors come from the MBR not from getFatVol or getExFatVol so shouldn't have an issue with formatting unknow formats - I hope. Probably have to delete any prints that call those functions.

  6. #281
    Senior Member+ KurtE's Avatar
    Join Date
    Jan 2014
    Posts
    9,017
    You might try my version, I was trying decide if to add it as different or replacement

    But it gives me a list of valid partitions from up to 3 usb devices (have not tried more than 1 so far), plus the SDIO, plus I have one SD (SPI) currently on pin 10... Will make it a list as well...

    Then you can enter command like: f 0 3
    which says format the first partition for Fat32... 1 is for 16 although not sure if we can force either yet, and likewise f 0 e is for EX...

    you can also do: l 0
    and it will do an LS on the first partition...
    or v 0 New_label
    and it should set the label...
    Attached Files Attached Files

  7. #282
    Senior Member+ mjs513's Avatar
    Join Date
    Jul 2014
    Location
    New York
    Posts
    6,972
    Well - downloaded and gave it a try and seems to work.

    And no can't force Fat16 unless its smaller than 2GB. Its hardcoded.

  8. #283
    Senior Member+ mjs513's Avatar
    Join Date
    Jul 2014
    Location
    New York
    Posts
    6,972
    @KurtE
    I tested the changes and it appears that if the partition shows as EXTENDED you can reformat to any of the formats. However, if its shows a ext4 you can't get to it because its not in the partition list:
    Code:
    Initialize USB drive...UsbBase::mscBegin called 20006340 1 1
        After usbDriveBegin
    PFsVolume::begin USBmscInterface(20002c18, 1)
    PFsVolume::begin(20002c18, 1)
    
    msc # Partition Table
    	part,boot,bgnCHS[3],type,endCHS[3],start,length
    exFAT:	1,0,0x20,0x21,0x0,0x7,0xFE,0xFF,0xFF,2048,30044160
    ext2/3/4:	2,0,0xFE,0xFF,0xFF,0x83,0xFE,0xFF,0xFF,30046208,32505856
    pt_#0:	3,0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0,0
    pt_#0:	4,0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0,0
    PFsVolume::begin USBmscInterface(20002c18, 1)
    PFsVolume::begin(20002c18, 1)
    drive 0 Partition 0 valid:1
    PFsVolume::begin USBmscInterface(20002c18, 2)
    PFsVolume::begin(20002c18, 2)
    PFsVolume::begin USBmscInterface(20002c18, 3)
    PFsVolume::begin(20002c18, 3)
    PFsVolume::begin USBmscInterface(20002c18, 4)
    PFsVolume::begin(20002c18, 4)
    
    Initialize SDIO SD card...initialization failed.
    
    
    Initialize SPI SD card...initialization failed.
    
    
    ***** Partition List *****
    0(0:1):>> ExFat: Volume name:(v1) Partition Total Size:15365832704 Used:12582912 time us: 23876
    done...
    As a kludge tried it from hacked up MSCFormatter sketch and it formatted it as an exFAT so think the answer is yes it will reformat any partition so far.

  9. #284
    Senior Member+ KurtE's Avatar
    Join Date
    Jan 2014
    Posts
    9,017
    Yep,

    I was thinking of adding code to get to those... We probably could initialize a PFsVolume, which would have the partition number and the blockdevice.
    Neither the Fat or Exfat would be valid. But everything else could be read from the MBR...

    I was also thinking that we might have some additional options that we could ponder.

    For example: Should we

    a) On some disks that have unused space at end unused partitions, we could create new one...

    That is we can ask the device for number of sectors: Serial.printf(" Device sector count: %u\n", m_dev->sectorCount());
    Also suppose we could emulate the other format program, which maybe allows whole disk to be reformatted as one partition?

    Again probably beyond our needs.

  10. #285
    Senior Member+ mjs513's Avatar
    Join Date
    Jul 2014
    Location
    New York
    Posts
    6,972
    Quote Originally Posted by KurtE View Post
    Yep,

    I was thinking of adding code to get to those... We probably could initialize a PFsVolume, which would have the partition number and the blockdevice.
    Neither the Fat or Exfat would be valid. But everything else could be read from the MBR...

    I was also thinking that we might have some additional options that we could ponder.

    For example: Should we

    a) On some disks that have unused space at end unused partitions, we could create new one...

    That is we can ask the device for number of sectors: Serial.printf(" Device sector count: %u\n", m_dev->sectorCount());
    Also suppose we could emulate the other format program, which maybe allows whole disk to be reformatted as one partition?

    Again probably beyond our needs.
    Was thinking about that but then I went to something that @wwatson was working on a "add partitions" for MSC? Think if we are going down that path we are going to be going down the proverbial rabbit hole. I do know that we like rabbit holes though. Probably easy enough to format the whole disk though - would just need the total sectors and the starting sector

    The question is now how you going to put this all into MTP?

    Remember that we that other list as well to work on FS additions, date-time etc.

  11. #286
    Senior Member+ KurtE's Avatar
    Join Date
    Jan 2014
    Posts
    9,017
    Pulled in your changes

    What do you think, should I just update with my version of the sketch?

    As for MTP, probably will look into doing some stuff soon. Like maybe supporting SD Cards to format.

  12. #287
    Senior Member+ mjs513's Avatar
    Join Date
    Jul 2014
    Location
    New York
    Posts
    6,972
    Yep - go ahead and merge your version. More versatile now we made the changes to formatted classes

  13. #288
    Senior Member+ KurtE's Avatar
    Join Date
    Jan 2014
    Posts
    9,017
    Changes pushed up...

  14. #289
    Senior Member+ mjs513's Avatar
    Join Date
    Jul 2014
    Location
    New York
    Posts
    6,972
    Cool - just updated my fork and Github desktop - so we should be even with changes.

  15. #290
    Senior Member+ KurtE's Avatar
    Join Date
    Jan 2014
    Posts
    9,017
    @ good morning all...

    I am playing some more with the Fat formatter to see if something can be sped up...

    But I am pretty sure I have run into an issue, that investigating:

    I am hacking up the initFatDir function, so far I have done some stuff brute force, but was wondering what the speed up might be if I told the system
    to write 32 sectors at a time for this... Code below:

    Code:
    #define CSECTORS_PER_WRITE 32
    bool PFsFatFormatter::initFatDir(uint8_t fatType, uint32_t sectorCount) {
      Serial.printf("PFsFatFormatter::initFatDir(%u, %u)\n", fatType, sectorCount);
      size_t n;
      uint32_t fat_sector = 1;
      writeMsg("Writing FAT ");
      if (sectorCount >= CSECTORS_PER_WRITE) {
        uint8_t *large_buffer = (uint8_t *)malloc(BYTES_PER_SECTOR * CSECTORS_PER_WRITE);
        if (large_buffer) {
          memset(large_buffer, 0, BYTES_PER_SECTOR * CSECTORS_PER_WRITE);
          uint32_t sectors_remaining = sectorCount;
          uint32_t loops_per_dot = sectorCount/(32*CSECTORS_PER_WRITE);
          uint32_t loop_count = 0;
          while (sectors_remaining >= CSECTORS_PER_WRITE) {
            if (!m_dev->writeSectors(m_fatStart + fat_sector, large_buffer, CSECTORS_PER_WRITE)) {
               return false;
            }
            fat_sector += CSECTORS_PER_WRITE;
            sectors_remaining -= CSECTORS_PER_WRITE;
            if (++loop_count == loops_per_dot) {
              writeMsg(".");
              loop_count = 0;
            }
          }
          if (sectors_remaining) {
            if (!m_dev->writeSectors(m_fatStart + fat_sector, large_buffer, sectors_remaining)) {
               return false;
            }
            fat_sector += sectors_remaining;
          }
          free(large_buffer);
        }
      }
      if (fat_sector < sectorCount) {
        memset(m_secBuf, 0, BYTES_PER_SECTOR);
        for (; fat_sector < sectorCount; fat_sector++) {
          if (!m_dev->writeSector(m_fatStart + fat_sector, m_secBuf)) {
             return false;
          }
          if ((fat_sector%(sectorCount/32)) == 0) {
            writeMsg(".");
          }
        }
      }
      writeMsg("\r\n");
      // Allocate reserved clusters and root for FAT32.
      m_secBuf[0] = 0XF8;
      n = fatType == 16 ? 4 : 12;
      for (size_t i = 1; i < n; i++) {
        m_secBuf[i] = 0XFF;
      }
      return m_dev->writeSector(m_fatStart, m_secBuf) &&
             m_dev->writeSector(m_fatStart + m_fatSize, m_secBuf);
    }
    But I think there is another issue, in how many sectors to write... That is on a successful run, the ending parts of output show:
    Code:
    ***** Partition List *****
    0(0:1):>> Fat16:  Partition Total Size:2096857088 Used:1802240 time us: 16500
    1(0:2):>> Fat32: Volume name:(VOL_FAT32) Partition Total Size:3141533696 Used:98304 time us: 47623
    2(0:3):>> ExFat: Volume name:(Vol_ExFat) Partition Total Size:2791309312 Used:282198016 time us: 5499
    3(ff:1):>> Fat32: Volume name:(VOL_BOOT) Partition Total Size:264289280 Used:47383552 time us: 94949
    4(fe:1):>> ExFat: Volume name:(SDSPI_VOL) Partition Total Size:998768640 Used:239992832 time us: 1387
    done...
    Enter command:
    
     **** Start format partition 1 ****
    m_dataStart: 16384, m_fatSize: 749, r: 9699
    PFsFatFormatter::initFatDir(32, 1562)
    Writing FAT ................................................
    free clusters after format: 95865
    free clusters after begin on partVol: 95865
    Format Done
    Press any key to run again
    But in another run that was taking forever...

    Code:
    ***** Partition List *****
    0(0:1):>> Fat16:  Partition Total Size:2096857088 Used:140737488322560 time us: 16500
    1(0:2):>> Fat32: Volume name:(VOL_FAT32) Partition Total Size:3141533696 Used:98304 time us: 47623
    2(0:3):>> ExFat: Volume name:(Vol_ExFat) Partition Total Size:2791309312 Used:282198016 time us: 5499
    3(ff:1):>> Fat32: Volume name:(VOL_BOOT) Partition Total Size:264289280 Used:47383552 time us: 94953
    4(fe:1):>> ExFat: Volume name:(SDSPI_VOL) Partition Total Size:998768640 Used:239992832 time us: 1387
    done...
    Enter command:
    
     **** Start format partition 0 ****
    PFsFatFormatter::initFatDir(16, 4294966015)
    Writing FAT
    Notice how many Fat sectors it was trying to write: 8)

  16. #291
    Senior Member+ mjs513's Avatar
    Join Date
    Jul 2014
    Location
    New York
    Posts
    6,972
    @KurtE
    Not sure why without some debugging. But it looks like the MBR is getting messed up some how. Sector count is changing from run to run and it shouldn't be. So not really sure whats going on.

    Other thing is that on your first run FAT type is 32 but on the second run its 16?

  17. #292
    Senior Member+ KurtE's Avatar
    Join Date
    Jan 2014
    Posts
    9,017
    Code:
    ***** Partition List *****
    0(0:1):>> Fat16:  Partition Total Size:2096857088 Used:140737488322560 time us: 16500
    1(0:2):>> Fat32:  Partition Total Size:3141533696 Used:229376 time us: 47625
    2(0:3):>> ExFat: Volume name:(Vol_ExFat) Partition Total Size:2791309312 Used:282198016 time us: 5874
    3(ff:1):>> Fat32: Volume name:(VOL_BOOT) Partition Total Size:264289280 Used:47383552 time us: 94945
    4(fe:1):>> ExFat: Volume name:(SDSPI_VOL) Partition Total Size:998768640 Used:239992832 time us: 1389
    done...
    Enter command:
    
     **** Start format partition 0 ****
    0
    
    PFsFatFormatter::format................
    Sector Count: 4096000, Sectors/MB: 2048
    Partition Capacity (MB): 2000
    Fat Type: 16
        m_dataStart:2581
        m_sectorsPerCluster:64
        m_relativeSectors:2048
    
     MAKEFAT16
    m_relativeSectors: 235, m_fatSize: 250, m_dataStart: 768
    partType: 6, fatStart: 236, totalSectors: 4095765
    Updated partType: 6, dataStart: 768 fatStart: 2049, totalSectors: 4096000
    PFsFatFormatter::initFatDir(16, 4294966015)
    Writing FAT
    There appears to be an issue in Fat16... I turned on debug...
    I then noticed you again updated fat start after you printed it, so I printed again,
    Code:
    #if defined(DBG_Print)
      Serial.printf("partType: %d, fatStart: %d, totalSectors: %d\n", m_partType, m_fatStart, m_totalSectors);
    #endif
    
    	//Added to keep relative sectors straight
    	m_relativeSectors = m_part_relativeSectors;
    	m_fatStart = m_relativeSectors + m_reservedSectorCount;
    	m_totalSectors = m_sectorCount;
    
      // write MBR
      if (!writeMbr()) {
    	return false;
      }
    #if defined(DBG_Print)
      Serial.printf("Updated partType: %d, dataStart: %d fatStart: %d, totalSectors: %d\n", m_partType, m_dataStart, m_fatStart, m_totalSectors);
    #endif
    
      initPbs();
      setLe16(pbs->bpb.bpb16.rootDirEntryCount, FAT16_ROOT_ENTRY_COUNT);
      setLe16(pbs->bpb.bpb16.sectorsPerFat16, m_fatSize);
      pbs->bpb.bpb16.physicalDriveNumber = 0X80;
      pbs->bpb.bpb16.extSignature = EXTENDED_BOOT_SIGNATURE;
      setLe32(pbs->bpb.bpb16.volumeSerialNumber, 1234567);
      
      for (size_t i = 0; i < sizeof(pbs->bpb.bpb16.volumeLabel); i++) {
        pbs->bpb.bpb16.volumeLabel[i] = ' ';
      }
      pbs->bpb.bpb16.volumeType[0] = 'F';
      pbs->bpb.bpb16.volumeType[1] = 'A';
      pbs->bpb.bpb16.volumeType[2] = 'T';
      pbs->bpb.bpb16.volumeType[3] = '1';
      pbs->bpb.bpb16.volumeType[4] = '6';
      if (!m_dev->writeSector(m_relativeSectors, m_secBuf)) {
        return false;
      }
      
      return initFatDir(16, m_dataStart - m_fatStart);
    So if you look at the updated output:
    Code:
    Updated partType: 6, dataStart: 768 fatStart: 2049, totalSectors: 4096000
    You will see fatStart > dataStart so ... We have lots of sectors... Not sure which is wrong, Maybe dataStart needs to be recalculated?

    Or maybe because the partition is: 0(0:1):>> Fat16: Partition Total Size:2096857088 Used:140737488322560 time us: 16500
    Maybe without the using old data it wants to be Fat32...

  18. #293
    Senior Member+ mjs513's Avatar
    Join Date
    Jul 2014
    Location
    New York
    Posts
    6,972
    @KurtE

    Think I know what is happening. If the partition is less <= 2GB its going to format it as FAT16, if greater than 2GB and its going to be FAT32. See the uncommented line.
    Code:
      rtn = m_sectorCount < 0X400000 ? makeFat16() :makeFat32();
      
      //if(partVol.fatType() == 16) {
    //	writeMsg("format makeFAT16\r\n");  
    //	rtn = makeFat16();
    //  } else if(partVol.fatType() == 32) {
    //	writeMsg("format makeFAT2\r\n");  
    //	rtn = makeFat32();
    //  }	else {
    //	  rtn = false;
    //  }
    Think we going to have to rethink that test.

  19. #294
    Senior Member+ KurtE's Avatar
    Join Date
    Jan 2014
    Posts
    9,017
    Yep - we probably either need to always rethink the test to use the test... And/or need to detect that it was Fat16 before and if so reuse what it was using before...

    Not sure if that makes sense?

  20. #295
    Senior Member+ KurtE's Avatar
    Join Date
    Jan 2014
    Posts
    9,017
    Note: I have tried commenting out the lines:
    Code:
    	//Added to keep relative sectors straight
    #if 0  
    	m_relativeSectors = m_part_relativeSectors;
    	m_fatStart = m_relativeSectors + m_reservedSectorCount;
    	m_totalSectors = m_sectorCount;
    #endif
    And it runs now... Not sure of the side effects. But figured may be it was in here for when things were hard coded?

  21. #296
    Senior Member+ mjs513's Avatar
    Join Date
    Jul 2014
    Location
    New York
    Posts
    6,972
    @KurtE
    Before going further I decided on a retest of what we have with a single 32GB partition:
    Click image for larger version. 

Name:	Capture0.PNG 
Views:	5 
Size:	10.1 KB 
ID:	24172
    Our output shows:
    Code:
    	part,boot,bgnCHS[3],type,endCHS[3],start,length
    exFAT:	1,0,0x20,0x21,0x0,0x7,0xFE,0xFF,0xFF,2048,122126336
    pt_#0:	2,0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0,0
    pt_#0:	3,0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0,0
    pt_#0:	4,0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0,0
    drive 0 Partition 0 valid:1
    ...
    ***** Partition List *****
    0(0:1):>> ExFat: Volume name:(v1Test) Partition Total Size:62525538304 Used:786432 time us: 58500
    Now lets change it to a FAT32 doing a f 0 32:
    Code:
    Enter command:
    
     **** Start format partition 0 ****
    m_dataStart: 16384, m_fatSize: 7454, r: 23109
    m_dataStart: 24576, m_fatSize: 7453, r: 23107
    Writing FAT ................................
    free clusters after format: 477032
    free clusters after begin on partVol: 953981
    Format Done
    Press any key to run again
    ...................
    	part,boot,bgnCHS[3],type,endCHS[3],start,length
    FAT32:	1,0,0x20,0x21,0x0,0xC,0xFE,0xFF,0xFF,2048,122126336
    pt_#0:	2,0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0,0
    pt_#0:	3,0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0,0
    pt_#0:	4,0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0,0
    drive 0 Partition 0 valid:1
    
    Initialize SDIO SD card...initialization failed.
    
    
    Initialize SPI SD card...initialization failed.
    
    
    ***** Partition List *****
    0(0:1):>> Fat32: Volume name:(v1Test) Partition Total Size:0 Used:0 time us: 466248
    
    It shows that it went from exFAT to FAT32 successfully but the partition list is showing total size 0 and used 0? If I put it in windows:
    Click image for larger version. 

Name:	Capture.PNG 
Views:	4 
Size:	10.2 KB 
ID:	24173
    Its showing free and used space and windows has no problems recognizing it. If I turn power on and off still shows 0?

  22. #297
    Senior Member+ mjs513's Avatar
    Join Date
    Jul 2014
    Location
    New York
    Posts
    6,972
    Quote Originally Posted by KurtE View Post
    Note: I have tried commenting out the lines:
    Code:
    	//Added to keep relative sectors straight
    #if 0  
    	m_relativeSectors = m_part_relativeSectors;
    	m_fatStart = m_relativeSectors + m_reservedSectorCount;
    	m_totalSectors = m_sectorCount;
    #endif
    And it runs now... Not sure of the side effects. But figured may be it was in here for when things were hard coded?
    One big side effect is going to be when you do any partition other than Partition 0. Bill has m_relativeSectors hardcoded as 8192 for FAT32 and 128 for FAT16.

  23. #298
    Senior Member+ mjs513's Avatar
    Join Date
    Jul 2014
    Location
    New York
    Posts
    6,972
    Going from exFAT to FAT32 - by the way I updated initFatDir to your revised code - works:
    Code:
    msc # Partition Table
    	part,boot,bgnCHS[3],type,endCHS[3],start,length
    FAT32:	1,0,0x20,0x21,0x0,0xC,0xFE,0xFF,0xFF,2048,61675520
    exFAT:	2,0,0x20,0x21,0x0,0x7,0xFE,0xFF,0xFF,61677568,60817408
    exFAT:	3,0,0x20,0x21,0x0,0x7,0xFE,0xFF,0xFF,122494976,68661248
    exFAT:	4,0,0x20,0x21,0x0,0x7,0xFE,0xFF,0xFF,191156224,50868224
    drive 0 Partition 0 valid:1
    drive 0 Partition 1 valid:1
    drive 0 Partition 2 valid:1
    drive 0 Partition 3 valid:1
    
    Initialize SDIO SD card...initialization failed.
    
    
    Initialize SPI SD card...initialization failed.
    
    
    ***** Partition List *****
    0(0:1):>> Fat32: Volume name:(V1) Partition Total Size:31569477632 Used:31064064 time us: 471123
    1(0:2):>> ExFat: Volume name:(V2) Partition Total Size:31121735680 Used:23592960 time us: 22253
    2(0:3):>> ExFat: Volume name:(V3) Partition Total Size:35121004544 Used:27131904 time us: 25250
    3(0:4):>> ExFat: Volume name:(v4) Partition Total Size:26027753472 Used:20447232 time us: 18875
    ok now with your revised code reformatting part 0:1 to fat32 again works:
    Code:
    msc # Partition Table
    	part,boot,bgnCHS[3],type,endCHS[3],start,length
    FAT32:	1,0,0x20,0x21,0x0,0xC,0xFE,0xFF,0xFF,2048,61675520
    exFAT:	2,0,0x20,0x21,0x0,0x7,0xFE,0xFF,0xFF,61677568,60817408
    exFAT:	3,0,0x20,0x21,0x0,0x7,0xFE,0xFF,0xFF,122494976,68661248
    exFAT:	4,0,0x20,0x21,0x0,0x7,0xFE,0xFF,0xFF,191156224,50868224
    drive 0 Partition 0 valid:1
    drive 0 Partition 1 valid:1
    drive 0 Partition 2 valid:1
    drive 0 Partition 3 valid:1
    
    Initialize SDIO SD card...initialization failed.
    
    
    Initialize SPI SD card...initialization failed.
    
    
    ***** Partition List *****
    0(0:1):>> Fat32: Volume name:(V1) Partition Total Size:31569477632 Used:31064064 time us: 471123
    1(0:2):>> ExFat: Volume name:(V2) Partition Total Size:31121735680 Used:23592960 time us: 22253
    2(0:3):>> ExFat: Volume name:(V3) Partition Total Size:35121004544 Used:27131904 time us: 25250
    3(0:4):>> ExFat: Volume name:(v4) Partition Total Size:26027753472 Used:20447232 time us: 18875

  24. #299
    Senior Member+ mjs513's Avatar
    Join Date
    Jul 2014
    Location
    New York
    Posts
    6,972
    Quote Originally Posted by mjs513 View Post
    One big side effect is going to be when you do any partition other than Partition 0. Bill has m_relativeSectors hardcoded as 8192 for FAT32 and 128 for FAT16.
    FAT16 is failing period end of story have to figure out why. Has to be some other difference I am not seeing.

  25. #300
    Senior Member+ mjs513's Avatar
    Join Date
    Jul 2014
    Location
    New York
    Posts
    6,972
    @KurtE

    Think there is a problem when trying to get relative sectors for FAT16. For instance if I try to format partition 3 it returns 157? My T4 just died so can't go further for a little bit.

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •