New Teensy MicroMods defective

Rezo

Well-known member
I've been working with a friend on a product development for the past two years now, and as soon as the Teensy MicroMod was released we made the transition and implemented it into our test jigs as the footprint was perfect for us and it has all the pins we need.
We utilized I2C, FlexIO2, SDIO, Digital IOs and Analog IOs

A year later, we finally have everything done but we have observed a major issue when it comes to the Teensy MicroMods.
We're seeing on some units at least one GPIO that does not have contact between the RT1062 chip and the MicroMod PCB - this is on brand new, unused units.
Sometimes, pushing down on the RT1062 will create contact and the GPIO works, but then letting go - it goes back to the disconnected state.

I've built a test sketch that does the following:
1. Sets all pins to inputs with internal pull-ups
2. I ground all the pins via a MicroMod ATP board
3. Read each pin's state

Code:
#define MM_PINS 46

void setup() {
  Serial.begin(9600);
  while (!Serial) {
    yield();
  }
  if(CrashReport){Serial.print(CrashReport);}
  for(int i=0; i< MM_PINS;i++){
    pinMode(i, INPUT_PULLUP);
    }
  Serial.println("Pins set to input");
  delay(1000);
  for(int j=0; j< MM_PINS;j++){
    Serial.printf("Pin %d state: %d \n", j, digitalReadFast(j));
    }
  Serial.println("Test Completed");
}

void loop() {


}

Any pin that returns high does not have contact.

So far out of six Teensy MicroMod's I've purchased, four have had at least one GPIO disconnected.
My friend and business partner has ordered over 25 Teensy MicroMods and at least 8 or so have demonstrated the same issues.


Has anyone else experienced similar issues?
 
That sounds like a pain. So far, I have not noticed that, but I don't have that many of them. I have a couple sitting around, that I have not used yet. But again, I mainly just play and could have missed it.
Are there any pins that, appear more likely to fail?

I assume you have contacted Sparkfun?
 
I have a ton of Teensy micromods sitting around, and the ATP board, so I can help do this test. Is there an easy chart showing the pin number relation to the MMOD ATP board location? i.e. the program is reporting Pin 22 and Pin 28 are high, which locations on the ATP board would those be so I can check whether its the MMOD board or something like the jumper / solderless breadboard that's the issue.

EDIT: nevermind, found it. Still drinking coffee and feeling a little under the weather.
 
I can't speak for Sparkfun. I can only comment on PJRC's role. And I suppose I could dabble in some guesswork...

I can tell you every MicroMod Teensy is tested on a test fixture PJRC made for Sparkfun, which is basically an adapted version of the Teensy 4.1 test fixture. It does test every I/O pin, using a large 74HC595 shift register chain and series resistors to weakly drive every I/O pin, and then the IMXRT chip's JTAG boundary scan is used to read them all. Each pin gets tested driven low while all the other pins are driven high, then driven high while all the other pins are low. All pins are read and the test fails if any isn't at expected, and of course it's repeated for driving each individual pin differently from all the others. A few special pins (like USB signals) can't be read with boundary scan, but they too are tested by temporarily loading special code. Many other hardware tests are also done, like using the CLKOUT feature to measure both the 24 MHz and 32.768 kHz crystals. The test fixture only assigns a unique mac address and programs the bootloader after all the hardware tests have passed. If the board appears to your PC as a MicroMod Teensy recognized by Teensy Loader, that's a sure indication the hardware did initially pass a pretty extensive set of tests. The same is true for every Teensy, they're all tested in a similar way where the bootloader is written only after all the hardware tests pass (however, normal Teensy does through a 2nd test where a USB cable is plugged and the operator presses the pushbutton, but clearly that doesn't apply to the MicroMod boards which don't have a USB connector or pushbutton).

The test fixture can only confirm electrical connectivity of the pins at that moment of the initial testing. It can't check whether the solder whetted properly to the PCB versus metals just barely touching and susceptible to disconnect with temperature change. It's also impossible to check whether the soldering will be durable when the PCBs are handled.

The test fixture also can't know whether the operator is pressing their finger on top of the BGA chip during the test. With normal Teensy, I can tell you we always handle them by their edges while testing. When I made the test fixture for Sparkfun, I handed the several MicroMod boards they provided by the edges. I guess I just assumed Sparkfun would do the same. I haven't visited Sparkfun since before the pandemic, so there's been no opportunity to actually meet in person and directly observe how they're operating. I'll send Sparkfun an email today to ask how they're handling the boards while testing.

Please do keep in mind this is really a matter for Sparkfun to address. The MicroMod boards are manufactured by Sparkfun. PJRC did provide the test fixture, so I can comment about how the boards are electrically tested, but everything about the PCBs they're buying, how they're soldered, and how they handle the materials is all outside of PJRC's control or even ability to observe. If the physical hardware is having failures, that's really an issue only Sparkfun can address.
 
my T4 micromod (7/13/21) and ATP carrier show all ATP edge pins are good. I just probed ATP pins with GND wire and this Loop()
Code:
void loop() {
  for (int j = 0; j < MM_PINS; j++) {
    if (digitalRead(j) == 0) Serial.printf("%d  val %d\n", j, digitalRead(j));
  }
  delay(2000);
}
 
Last edited:
@Kurt, so far I have two units, each has one bad pin on FlexIO2 (can't remember which one, I think Teensy pin 42) Another has a bad CAN3 pin, and a brand new one I opened two days ago has a bad pin 14.
Of course, some slight finger pressure on the chip fixes this, but that's not durable ;)
I have contacted Sparkfun twice, 1st time they said they would pass it on to engineers. 2nd time they said they have sold hundreds but have not come across this type of compliant

@brtaylor thanks for stepping up to help! There is Kurt's mapping here and Sparkfun's mapping here under MicroMod Pinout
Just to be clear, you have to ground all the GPIOs on the ATP.
I've done this with a set of male pin headers all soldered together, connected to ground, and inserted into the relevant female header on the ATP board.

@Paul thanks for the drilldown into detail! I do not have any complaints toward PJRC, as this is a Sparkfun product. But I am posting to see if anyone else has experienced the same behavior as well - and this is the right place to find those people

@manitou thanks for confirming!
 
Note: One thing I meant to mention, is that there have been a few times, I thought something was not working properly. But then after I removed the MMOD from the board and put it back in, it worked fine.

So, wondering if the issue you are seeing is with the chip soldering or maybe the contact within the matching connector on the board?
 
I keep a couple MicroMod boards on my workbench for software testing. I'm not seen any pins failing. But most of the tests I usually run are meant to test the more complex libraries like Audio, USBHost_t36, ILI9341_t3, so I usually end up exercising only a small set of the I/O pins. Almost all the stuff I do on a regular basis is more targeted at testing software rather than the hardware.
 
Last edited:
@brtaylor thanks for stepping up to help! There is Kurt's mapping here and Sparkfun's mapping here under MicroMod Pinout
Just to be clear, you have to ground all the GPIOs on the ATP.
I've done this with a set of male pin headers all soldered together, connected to ground, and inserted into the relevant female header on the ATP board.

So far, your program is just reporting Pin 22 and Pin 28 as high. Looking at this image, I can't find those on the ATP board:
https://forum.pjrc.com/attachment.php?attachmentid=25533&d=1628355409
 
Pin 22 I believe is MMOD pin 49 (BATT_VIN3)
Pin 28 is MMOD 4 (3V Enable pin I think)
 
Here are some fresh results from the test:
MM1:
Code:
Pin 0 state: 0 
Pin 1 state: 0 
Pin 2 state: 0 
Pin 3 state: 0 
Pin 4 state: 0 
Pin 5 state: 0 
Pin 6 state: 0 
Pin 7 state: 0 
Pin 8 state: 0 
Pin 9 state: 0 
Pin 10 state: 0 
Pin 11 state: 0 
Pin 12 state: 0 
Pin 13 state: 0 
Pin 14 state: 0 
Pin 15 state: 0 
Pin 16 state: 0 
Pin 17 state: 0 
Pin 18 state: 0 
Pin 19 state: 0 
Pin 20 state: 0 
Pin 21 state: 0 
Pin 22 state: 1 
Pin 23 state: 0 
Pin 24 state: 0 
Pin 25 state: 0 
Pin 26 state: 0 
Pin 27 state: 0 
Pin 28 state: 1 
Pin 29 state: 0 
Pin 30 state: 0 
Pin 31 state: 1 
Pin 32 state: 0 
Pin 33 state: 0 
Pin 34 state: 0 
Pin 35 state: 0 
Pin 36 state: 0 
Pin 37 state: 0 
Pin 38 state: 0 
Pin 39 state: 0 
Pin 40 state: 0 
Pin 41 state: 0 
Pin 42 state: 1 
Pin 43 state: 0 
Pin 44 state: 0 
Pin 45 state: 0

MM2:
Code:
Pin 0 state: 0 
Pin 1 state: 0 
Pin 2 state: 0 
Pin 3 state: 1 
Pin 4 state: 0 
Pin 5 state: 0 
Pin 6 state: 0 
Pin 7 state: 0 
Pin 8 state: 0 
Pin 9 state: 0 
Pin 10 state: 0 
Pin 11 state: 0 
Pin 12 state: 0 
Pin 13 state: 0 
Pin 14 state: 0 
Pin 15 state: 0 
Pin 16 state: 0 
Pin 17 state: 0 
Pin 18 state: 0 
Pin 19 state: 0 
Pin 20 state: 0 
Pin 21 state: 0 
Pin 22 state: 0 
Pin 23 state: 0 
Pin 24 state: 0 
Pin 25 state: 0 
Pin 26 state: 0 
Pin 27 state: 0 
Pin 28 state: 1 
Pin 29 state: 1 
Pin 30 state: 1 
Pin 31 state: 1 
Pin 32 state: 0 
Pin 33 state: 0 
Pin 34 state: 0 
Pin 35 state: 0 
Pin 36 state: 0 
Pin 37 state: 0 
Pin 38 state: 0 
Pin 39 state: 0 
Pin 40 state: 0 
Pin 41 state: 0 
Pin 42 state: 0 
Pin 43 state: 0 
Pin 44 state: 0 
Pin 45 state: 0

MM3: (I fixed this one with some flux & hot air)
Code:
Pin 0 state: 0 
Pin 1 state: 0 
Pin 2 state: 0 
Pin 3 state: 0 
Pin 4 state: 0 
Pin 5 state: 0 
Pin 6 state: 0 
Pin 7 state: 0 
Pin 8 state: 0 
Pin 9 state: 0 
Pin 10 state: 0 
Pin 11 state: 0 
Pin 12 state: 0 
Pin 13 state: 0 
Pin 14 state: 0 
Pin 15 state: 0 
Pin 16 state: 0 
Pin 17 state: 0 
Pin 18 state: 0 
Pin 19 state: 0 
Pin 20 state: 0 
Pin 21 state: 0 
Pin 22 state: 1 
Pin 23 state: 0 
Pin 24 state: 0 
Pin 25 state: 0 
Pin 26 state: 0 
Pin 27 state: 0 
Pin 28 state: 1 
Pin 29 state: 0 
Pin 30 state: 0 
Pin 31 state: 1 
Pin 32 state: 0 
Pin 33 state: 0 
Pin 34 state: 0 
Pin 35 state: 0 
Pin 36 state: 0 
Pin 37 state: 0 
Pin 38 state: 0 
Pin 39 state: 0 
Pin 40 state: 0 
Pin 41 state: 0 
Pin 42 state: 0 
Pin 43 state: 0 
Pin 44 state: 0 
Pin 45 state: 0

MM4 (Brand new unit)
Code:
Pin 0 state: 0 
Pin 1 state: 0 
Pin 2 state: 0 
Pin 3 state: 1 
Pin 4 state: 0 
Pin 5 state: 0 
Pin 6 state: 0 
Pin 7 state: 0 
Pin 8 state: 0 
Pin 9 state: 0 
Pin 10 state: 0 
Pin 11 state: 0 
Pin 12 state: 0 
Pin 13 state: 1 
Pin 14 state: 0 
Pin 15 state: 0 
Pin 16 state: 0 
Pin 17 state: 0 
Pin 18 state: 0 
Pin 19 state: 0 
Pin 20 state: 1 
Pin 21 state: 0 
Pin 22 state: 1 
Pin 23 state: 0 
Pin 24 state: 0 
Pin 25 state: 0 
Pin 26 state: 0 
Pin 27 state: 1 
Pin 28 state: 1 
Pin 29 state: 0 
Pin 30 state: 0 
Pin 31 state: 0 
Pin 32 state: 0 
Pin 33 state: 0 
Pin 34 state: 0 
Pin 35 state: 0 
Pin 36 state: 1 
Pin 37 state: 0 
Pin 38 state: 0 
Pin 39 state: 0 
Pin 40 state: 0 
Pin 41 state: 0 
Pin 42 state: 1 
Pin 43 state: 0 
Pin 44 state: 1 
Pin 45 state: 0

So lets put 28 and 22 aside for now.
I still have many others that are still reading high :(
 
I tested 19 boards. Most passed the test. Out of the 19, I had a couple that I needed to re-seat in the MMOD connector and then they worked. I have two that won't boot at all (i.e. I can't even get the raw hid to show up to flash the test program to the board). So I guess I'm at 17 working out of 19 total.
 
Amazing, thanks for taking the time and running the tests.
On the two units that won’t boot, if you push on the NXP chip and then plug it into the computer - does it show up?
 
Amazing, thanks for taking the time and running the tests.
On the two units that won’t boot, if you push on the NXP chip and then plug it into the computer - does it show up?

I tried that and it didn't help any. At least from my perspective, everyone that booted worked perfectly except for having to re-seat a few in the connector. I'm not sure what's happening with the two that aren't booting.
 
I wired all the ATP pins to GND and ran it here on both MicroMod boards on my workbench.

atp.jpg

Code:
Pins set to input
Pin 0 state: 0 
Pin 1 state: 0 
Pin 2 state: 0 
Pin 3 state: 0 
Pin 4 state: 0 
Pin 5 state: 0 
Pin 6 state: 0 
Pin 7 state: 0 
Pin 8 state: 0 
Pin 9 state: 0 
Pin 10 state: 0 
Pin 11 state: 0 
Pin 12 state: 0 
Pin 13 state: 0 
Pin 14 state: 0 
Pin 15 state: 0 
Pin 16 state: 0 
Pin 17 state: 0 
Pin 18 state: 0 
Pin 19 state: 0 
Pin 20 state: 0 
Pin 21 state: 0 
Pin 22 state: 1 
Pin 23 state: 0 
Pin 24 state: 0 
Pin 25 state: 0 
Pin 26 state: 0 
Pin 27 state: 0 
Pin 28 state: 1 
Pin 29 state: 0 
Pin 30 state: 0 
Pin 31 state: 0 
Pin 32 state: 0 
Pin 33 state: 0 
Pin 34 state: 0 
Pin 35 state: 0 
Pin 36 state: 0 
Pin 37 state: 0 
Pin 38 state: 0 
Pin 39 state: 0 
Pin 40 state: 0 
Pin 41 state: 0 
Pin 42 state: 0 
Pin 43 state: 0 
Pin 44 state: 0 
Pin 45 state: 0 
Test Completed


Code:
Pins set to input
Pin 0 state: 0 
Pin 1 state: 0 
Pin 2 state: 0 
Pin 3 state: 0 
Pin 4 state: 0 
Pin 5 state: 0 
Pin 6 state: 0 
Pin 7 state: 0 
Pin 8 state: 0 
Pin 9 state: 0 
Pin 10 state: 0 
Pin 11 state: 0 
Pin 12 state: 0 
Pin 13 state: 0 
Pin 14 state: 0 
Pin 15 state: 0 
Pin 16 state: 0 
Pin 17 state: 0 
Pin 18 state: 0 
Pin 19 state: 0 
Pin 20 state: 0 
Pin 21 state: 0 
Pin 22 state: 1 
Pin 23 state: 0 
Pin 24 state: 0 
Pin 25 state: 0 
Pin 26 state: 0 
Pin 27 state: 0 
Pin 28 state: 1 
Pin 29 state: 0 
Pin 30 state: 0 
Pin 31 state: 0 
Pin 32 state: 0 
Pin 33 state: 0 
Pin 34 state: 0 
Pin 35 state: 0 
Pin 36 state: 0 
Pin 37 state: 0 
Pin 38 state: 0 
Pin 39 state: 0 
Pin 40 state: 0 
Pin 41 state: 0 
Pin 42 state: 0 
Pin 43 state: 0 
Pin 44 state: 0 
Pin 45 state: 0 
Test Completed
 
I wired all the ATP pins to GND and ran it here on both MicroMod boards on my workbench.
Did you fully seat the MicroMod on the M.2 connector (so it is parallel to the board), or did you leave it tilted like that? Perhaps the PCB material flexes too much on some of the M.2 connectors, causing the connection issues?
(Granted, I'm too lazy to check where the affected balls are, so I'm definitely just guessing here.)

I, too, will be getting a Teensy MicroMod soon (it is on its way from BerryBase.de via a friend), but I don't have the baseboard yet; I'm going to use it with my parallel ILI9341 display module.
 
I tested with the card tilted, exactly as you see in the photo. I too was lazy about using a screwdriver to secure it. ;)
 
Ok, ran the test again on both boards, this time using the screw to hold them done.

Exactly the same result, 1s on pin 22 & 28 and 0s on all other pins.

atp2.jpg
 
I also use Teensy MM's on a design. I don't know of any that had pins that did not work. However, as with any others above, I've found that seating the MM into the socket is very critical. If you get it wrong at all things go really wonky. The easiest approach to fixing issues is usually to reseat the board then screw it down nice and tight.

I have had a couple that refuse to boot. Mostly that was also fixed by reseating. 1-2 were just plain dead. I don't know if I scuffed my feet on carpet for a mile and a half then touched the exposed pins or what. Mostly the MMs seem pretty reliable once you get them into the socket and make them stay there.
 
Haha, that's pretty much the exact opposite of how I've used the 2 that live on my workbench. I almost never bother to screw them down tight (or at all) and I move them between various carrier boards for testing different software. So far I've never observed any pin not working, but my tests only use maybe 20 of the pins.
 
So far, your program is just reporting Pin 22 and Pin 28 as high. Looking at this image, I can't find those on the ATP board:
https://forum.pjrc.com/attachment.php?attachmentid=25533&d=1628355409

I wired all the ATP pins to GND and ran it here on both MicroMod boards on my workbench.

View attachment 29092

Code:
Pins set to input
Pin 22 state: 1 
Pin 28 state: 1 
Test Completed


Code:
Pins set to input
Pin 22 state: 1 
Pin 28 state: 1 
Test Completed
Again, does not surprise me. Paul - would be better with your own board!

As I mentioned,
Pin 22 I believe is MMOD pin 49 (BATT_VIN3)
Pin 28 is MMOD 4 (3V Enable pin I think)

Screenshot1.png

Where the Enable is one one side or the other of jumper
Screenshot.jpg
 
I'm Rezo's friend. I have in total bought 44x micromods from Sparkfun. I do not have the exact number of non working MM's but it's around 15ish. 3 of which we're completely dead and I sent those to Paul many months ago.

I just want to underline that I do not want to put blame PJRC or Sparkfun for that matter. For me it's just a matter of solving the issue at hand since the Micromods are important to me and Rezo. I have spend many hours playing around and the conclusion I come to is that it must be a quality thing from Sparkfun.
I want to underline that Sparkfun have replaced every Micromod that has been faulty so I'm very satisfied with that. But I am also curious specially since Paul wrote above about that cool jig that he made for Sparkfun (very interesting read btw). It seems like no MM's could be faulty since that jig checks everything. Yet I have had several completely dead Micromods, I've had a few that gets warm as hell (shorted out), and I've had MM's with random non-working pins. Me and Rezo use almost all the pins on the Micromod so for us, even if one pin is bad, it's a broken Micromod for us.

Using flux and heating up the NXP does solve the issue in many of the cases. I have tried with quite a few. Since I got those to work 100% I did not report them to Sparkfun. Those are not counted with the 15ish I mention above.
I have just drawn up a test PCB which will be ordered ASAP, this will make it super easy for me to test all the Micromods I order.

I plan to order 50 - 100pcs Micromods from Sparkfun to restock but I am unfortunately scared to place such a large order at once with the risk of many being faulty.
It would be awesome if we as the community perhaps could make Sparkfun implement another step into the production. A test step where they test all the pins with some sort of test board, for example the board I made here.

TEST.jpg
 
Last edited:
Ouch...

So far, I have not run into that. For example, I put a new MMOD in the breakout board that Paul made.
Screenshot.jpg

And I used the test program that @defragster and myself hacked on:
Code:
#ifdef ESP_PLATFORM
#define digitalWriteFast digitalWrite
#define digitalReadFast digitalRead
#endif

void setup()
{
  Serial.begin(115200);
  while (!Serial && millis() < 4000 );
  Serial.println("Compile Time:: " __FILE__ " " __DATE__ " " __TIME__);
  Serial.printf("Num Digital Pins: %d\n", NUM_DIGITAL_PINS);

  testForShorts();
  
}

uint32_t cnt = 0;
void loop() {
  cnt++;
    allPinTest( cnt );
}

uint32_t pinLast[NUM_DIGITAL_PINS];
void allPinTest( uint32_t cnt ) {
  uint32_t ii, SET;
  Serial.print("PULLDOWN Start Vals:\n  ");
  SET = 0;
  Serial.print("PULLDOWN :: TEST to 3.3V\n  ");
  for ( ii = 0; ii < NUM_DIGITAL_PINS; ii++) {
    pinMode( ii, INPUT_PULLDOWN );
    delayMicroseconds( 5 );
    pinLast[ii] = digitalReadFast( ii );
    if (pinLast[ii]) {
      Serial.print("\nd#=");
      Serial.print( ii );
      Serial.print( " val=" );
    }
    Serial.print( pinLast[ii] );
    Serial.print(',');
  }
  Serial.println();
  Serial.println();
  while ( 1 ) {
    uint32_t jj, dd = 0, cc = 0, ee=4;
    cc = 0;
    for ( ii = 0; ii < NUM_DIGITAL_PINS; ii++) {
      jj = digitalReadFast( ii );
      if ( jj != pinLast[ii] ) {
        dd = 1;
        cc++;
        pinLast[ii] = jj;
        Serial.print("d#=");
        Serial.print( ii );
        if ( pinLast[ii] ) Serial.print( "\t" );
        Serial.print( " val=" );
        Serial.print( pinLast[ii] );
        Serial.print(',');
      }
      if ( cc > 1 && ee ) {
        Serial.println(">>> MULTI CHANGE !!");
        ee--;
      }
      if ( Serial.available() ) {
        while ( Serial.available() ) Serial.read();
        if ( 0 == SET ) {
          SET = 1;
          Serial.print("PULLUP :: TEST TO GND\n  ");
        }
        else {
          SET = 0;
          Serial.print("PULLDOWN :: TEST to 3.3V\n  ");
        }
        for ( ii = 0; ii < NUM_DIGITAL_PINS; ii++) {
          if ( 0 == SET )
            pinMode( ii, INPUT_PULLDOWN );
          else
            pinMode( ii, INPUT_PULLUP );
          delayMicroseconds( 20 );
          pinLast[ii] = digitalReadFast( ii );
          if (SET != pinLast[ii]) {
            Serial.print("d#=");
            Serial.print( ii );
            Serial.print( " val=" );
            Serial.println( pinLast[ii] );
          }
        }
      }
    }
    if ( dd ) {
      dd = 0;
      Serial.println();
      delay( 50 );
    }
  }
}

void testForShorts() {
  uint32_t ii;
  Serial.print("Quick Test for Shorts to adjacent pin");
  Serial.println("First pull pins down and see if the next one follows");
  for ( ii = 0; ii < NUM_DIGITAL_PINS-1; ii++) {
    pinMode( ii+1, INPUT_PULLDOWN );
    pinMode( ii, OUTPUT);
    digitalWrite(ii, HIGH);
    delayMicroseconds( 5 );
    if (digitalRead(ii+1)) {
      Serial.printf("%d:%d ", ii, ii+1);
    }
  }
  Serial.println("\n Now try Pull up and see if setting low follow");
  for ( ii = 0; ii < NUM_DIGITAL_PINS-1; ii++) {
    pinMode( ii+1, INPUT_PULLUP );
    pinMode( ii, OUTPUT);
    digitalWrite(ii, LOW);
    delayMicroseconds( 5 );
    if (!digitalRead(ii+1)) {
      Serial.printf("%d:%d ", ii, ii+1);
    }
  }
  Serial.println();  
}

And I rang out all of the IO pins on it and it, all were found.

Note the green board is my own breakout, that I had partially assembled. Earlier I rang out all of the pins on that one.
 
There is another older thread where some fewer intermittent issues were noted. I got out a new one or two {from early production} and had some odd pins - but they went away swapping across boards so none seemed faulty in the end.
 
Back
Top