Adafruit_nrf8001 and SDFat problem on teensy 3.1

Status
Not open for further replies.

lynton

Active member
Hi,

I have an issue when using Adafruit_nrf8001 and SdFat libraries together.

Both work fine on there own but when SDFat is used when bluetooth is in use then the Bloothtooth connection is lost and does not come back until the program is restarted. ie the device stops advertising and scanning fails to rediscover the device.

Hardware
Teensy 3.1
Adafruit nRF8001 Bluefruit breakout.
WIZ820 & SD Adapter (without the wiz820io installed)

tested with UART v2.0 on android nexus 7 tablet.

I have edited the adafruit callback echo demo to read filenames of an sd card when a 'K' is sent over the bluetooth uart.

Once the SdFat library is called the bluetooth connection is lost , the tablet disconnects and advertising is not restarted.

I am using the latest Adafruit_nRF8001 lib as updated by Paul recently as per this thread
http://forum.pjrc.com/threads/26804-connecting-nRF8001-to-Teensy-3-1?p=57983&viewfull=1#post57983

I don't know whether this is an issue with the SDFat library ( I have seen conversations between Paul and the author about SPI transactions for Teensy ) or perhaps just my lack of understanding of SPI code on Teensy.

I have the latest SdFat lib from here https://github.com/greiman/SdFat/

The SD adapter is not piggybacked to the teensy but just connected to the relevant pins, the adapter itself appears to work fine with or without nrf8001 so I presume this is not a wiring issue.

Any advise appreciated,

thanks,
lynton


Code:
#include <SPI.h>
#include "Adafruit_BLE_UART.h"
#include <SdFat.h>

#define ADAFRUITBLE_REQ 10
#define ADAFRUITBLE_RDY 2
#define ADAFRUITBLE_RST 9

Adafruit_BLE_UART uart = Adafruit_BLE_UART(ADAFRUITBLE_REQ, ADAFRUITBLE_RDY, ADAFRUITBLE_RST);

const uint8_t chipSelect = 4; //SS;
SdFat sd;
SdFile file;

/**************************************************************************/
/*!
    This function is called whenever select ACI events happen
*/
/**************************************************************************/
void aciCallback(aci_evt_opcode_t event)
{
  switch(event)
  {
    case ACI_EVT_DEVICE_STARTED:
      Serial.println(F("Advertising started"));
      break;
    case ACI_EVT_CONNECTED:
      Serial.println(F("Connected!"));
      break;
    case ACI_EVT_DISCONNECTED:
      Serial.println(F("Disconnected or advertising timed out"));
      break;
    default:
      break;
  }
}

/**************************************************************************/
/*!
    This function is called whenever data arrives on the RX channel
*/
/**************************************************************************/
void rxCallback(uint8_t *buffer, uint8_t len)
{
  Serial.print(F("Received "));
  Serial.print(len);
  Serial.print(F(" bytes: "));
  for(int i=0; i<len; i++)
   Serial.print((char)buffer[i]); 

  Serial.print(F(" ["));

  for(int i=0; i<len; i++)
  {
    Serial.print(" 0x"); Serial.print((char)buffer[i], HEX); 
  }
  Serial.println(F(" ]"));

  /* Echo the same data back! */
  uart.write(buffer, len);
  
  // Triger card read if 'K" is in the buffer.
  if (*buffer == 'K') {
    get_sd_card_files();
  }
}

/**************************************************************************/
/*!
    Configure the Arduino and start advertising with the radio
*/
/**************************************************************************/
void setup(void)
{ 
  Serial.begin(9600);
  while(!Serial); // Leonardo/Micro should wait for serial init
  Serial.println(F("Adafruit Bluefruit Low Energy nRF8001 Callback Echo demo"));

  uart.setRXcallback(rxCallback);
  uart.setACIcallback(aciCallback);
  // uart.setDeviceName("NEWNAME"); /* 7 characters max! */
  uart.begin();
}

/**************************************************************************/
/*!
    Constantly checks for new events on the nRF8001
*/
/**************************************************************************/
void loop()
{
  uart.pollACI();
}


void get_sd_card_files()
{
  char name[13];
  
  Serial.println("Opening SD card to generate list");

  if (!sd.begin(chipSelect, SPI_HALF_SPEED)) sd.initErrorHalt();

  // open next file in root.  The volume working directory, vwd, is root
  while (file.openNext(sd.vwd(), O_READ)) {
    file.printFileSize(&Serial);
    Serial.write(' ');
    file.printModifyDateTime(&Serial);
    Serial.write(' ');    
    file.printName(&Serial);
    if (!file.isDir()) {
      // Indicate a directory.
      file.getFilename(name);               
      Serial.println(name);        
    }
    file.close();
  }

  return;
}
 
Yup, these 2 libraries are probably conflicting, due SPI usage from an interrupt and lack of SPI transactions.

I've added SPI transactions support here:

https://github.com/PaulStoffregen/Adafruit_nRF8001

Bill's SdFat library can use SPI transactions, but you have to edit a file to enable it. Details are here:

https://github.com/greiman/SdFat/blob/master/SPI_Transactions.txt


Please let me know if this solves the problem for you? I usually wait until at least one other person has confirmed the code before I submit the changes back to Adafruit.
 
Yup, these 2 libraries are probably conflicting, due SPI usage from an interrupt and lack of SPI transactions.

I've added SPI transactions support here:

https://github.com/PaulStoffregen/Adafruit_nRF8001

Bill's SdFat library can use SPI transactions, but you have to edit a file to enable it. Details are here:

https://github.com/greiman/SdFat/blob/master/SPI_Transactions.txt


Please let me know if this solves the problem for you? I usually wait until at least one other person has confirmed the code before I submit the changes back to Adafruit.

Tried but seemed to make no difference, as per the document I modified these 2 defines in sdfatconfig.h

#define ENABLE_SPI_TRANSACTION 1
#define ENABLE_SPI_YIELD 1

I also then tried this as well as it looked a spossible candiate
#define TEENSY3_SOFT_SPI 1

none made any difference,

The behaviour is at least consistent though, the first connect succeeds then is always followed by an immediate disconnect after about 5 seconds. A second scan /connect succeeds and stays connected through multiple tx rx . After triggering the sd card read ( by sending a 'K' in the sample sketch ), the nRF8001 no longer responds, there is no disconnect / advertise or anything in the output.

Happy to try something else if you have other suggestions.
 
Tried but seemed to make no difference
some extra info that may help

With the defines in place the SD card functionality was much more flaky and mostly failed to read the card, ( i think it may have succeeded once or twice after adding the defines although not certain )

I also tried the sdfat quickstart demo and this consistently failed to get the card size, with the SPI_TRANSACTION defines removed the demo worked perfectly every time.

So at face value it appears that the SDFat lib may be the problem it seems not to like transactions :().

oh, and also the adafruit_nrf8001 lib consistently disconnects the first connection after exactly 8 seconds every single time, and then works fine after scanning and connecting a second time for quite long stretches after that.

hope this may be of some help
 
sorry Paul but still the same behaviour, sd card access still kills nrf.

If
#define ENABLE_SPI_TRANSACTION 1
#define ENABLE_SPI_YIELD 1

then SDFat cannot read the card and attempting to kills nrf lib.

If
#define ENABLE_SPI_TRANSACTION 1
#define ENABLE_SPI_YIELD 0

Then SDFat can read the card but still kills the nrf.
 
I'm working on this now. It's a tough problem. I hope to have a real fix soon....

nrf8001.jpg

scope.png
 
At the moment I'm working with a modified program using SD instead of SdFat.

It's looking like the solution is going to touch a lot of stuff.... a fix inside SD, fixes in SPI.h, an addition in avr_emulation.h, and of course edits to Adafruit's library.

This SPI transaction stuff is all still pretty new. Really digging into this has exposed a number of small problems, which weren't apparent when working on CC3000 and RadioHead (the other 2 interrupt-based libs that have been ported to use SPI transactions).
 
I'm still working on a complete fix. It's ending up touching a lot of code. I've just pushed what will hopefully be the last edit needed in nRF8001.

Fixes are needed in SPI and SD. I'm testing those fixes now. This will also need EIMSK support in avr_emulation.h on Teensy 3.1, which I hope to do within the next day or two.

Installing all parts of the fix will require an error-prone process of updating many files. I might just make a Teensyduino 1.21-test1 installer with all the stuff.
 
Ok, I think I now have a truly effective fix. So far, I've tested with SD, not SdFat, but it seems to be working. That's the good news.

The bad news is a lot of stuff needs to be updated. Fixes were needed in all 3 libraries.

https://github.com/PaulStoffregen/Adafruit_nRF8001
https://github.com/PaulStoffregen/SD
https://github.com/PaulStoffregen/SPI

For use on Teensy 3.1, you'll also need to update avr_emulation.h & avr_emulation.cpp

https://github.com/PaulStoffregen/cores/blob/master/teensy3/avr_emulation.h
https://github.com/PaulStoffregen/cores/blob/master/teensy3/avr_emulation.cpp

These updates will really only work on Arduino 1.0.6.

I've sent the SD fixes to the Arduino devs, and the SPI fixes are being discussed now, so hopefully Arduino 1.5.9 or 1.6.0 will "just work" with the updated Adafruit_nRF8001. Of course, exactly what Arduino publishes is beyond my control, so trying to contribute the changes back is the best I can do.

If you're able to apply these many fixes, please let me know how they work for you?
 
What are the changes on the SD library? Im planning to use the nrf8001 with other SPI device, what do I need to consider?
 
The other SPI device needs to use SPI transactions. SD, Ethernet, ILI9341_t3 and a few others do. Most SPI libraries have not yet added SPI transaction support.

SD has SPI transactions, but there was a bug. This update fixes the bug.
 
Arduino has merged the SD library fix, so it'll definitely be part of Arduino 1.5.9 (or whatever version number they use).
 
Ok, I think I now have a truly effective fix. So far, I've tested with SD, not SdFat, but it seems to be working. That's the good news.
If you're able to apply these many fixes, please let me know how they work for you?

Didn't work with SDFat, but initial test does work with SD now :).
Will do some more testing soon (no time right now) and report back.
 
ok, had more time to test and it looks pretty good now.

Still lose the first connection after 8 seconds every single time, but after reconnecting then connection seems stable for long periods of time.
Now able to read files using SD, without killing NRF, so this fix appears to work.

As an aside I don't quite understand why polling in a loop is necessary ( uart.pollACI(); ) for the nrf callbacks to work, to my mind that seems to defeat the purpose of assigning the callback methods in the first place.
I'm starting to think that I may be better off with the simpler, cheaper, more widely supported, easier to use bluetooth 2.0 adapter instead of BLE now.

But I digress...

Great effort Paul, I think you have nailed it now.
 
Now able to read files using SD, without killing NRF, so this fix appears to work.

Good. Thanks for testing.

ok, had more time to test and it looks pretty good now.

Still lose the first connection after 8 seconds every single time, but after reconnecting then connection seems stable for long periods of time.

Any chance you could check if this 8 second issue happens with Adafruit's original library running on Arduino 1.0.6 without these extra changes?

This is important. We really need to be sure these changes didn't add a new problem, before trying to submit them to Adafruit to become part of their official library.


As an aside I don't quite understand why polling in a loop is necessary ( uart.pollACI(); ) for the nrf callbacks to work, to my mind that seems to defeat the purpose of assigning the callback methods in the first place.

Quite a few libraries work this way. It's really only a convenience, to avoid some fairly straightforward code to parse the events and call functions.

I've heard many people wish for interrupt-context callbacks. While that's great for minimizing latency, it has absolutely terrible usability for anything processing data. First, you have to successfully deal with volatile variables and assuming atomic access, which is simple in concept but difficult to get right even for experts. Second, many Arduino features like String and even malloc aren't interrupt context safe. Those are slowly improving, but it's a minefield. Third, while you get good performance from ISR callbacks, it becomes very easy to wreck the performance of other stuff if your code blocks other interrupts for too long. Already too many combinations of things don't work together nicely, which is also slowly improving.

Arduino as a platform lacks infrastructure for generic non-interrupt callback. I've been considering expanding the stuff in yield() to do this. But that's one of hundreds of little things I'm considering. Even when I do stuff like that, it usually takes years to become accepted and get merged back into Arduino. Luckily, the SPI transaction stuff has happened at lightning speed (only proposed early this year, fully accepted a couple months ago).

But I digress...

Yeah, I digress too.

At least the nRF8001 & SD issue seems to be solved.

If you can look into whether than 8 second disconnect issue is new, or something that's always been present, it would really help towards getting this fix published to everyone.

At some point, I guess we'll need to look at why SPI transactions in SD works (at least, after I fixed the bug) but SdFat doesn't. I wonder if Bill G will see this thread?
 
Good. Thanks for testing.

Any chance you could check if this 8 second issue happens with Adafruit's original library running on Arduino 1.0.6 without these extra changes?

This is important. We really need to be sure these changes didn't add a new problem, before trying to submit them to Adafruit to become part of their official library.
I think the 8 second thing has been happening all along, but I will test tomorrow with a clean install of arduino 1.06 and the Adafruit version of nrf.

I'll also have another look at SdFat, I only tried that very briefly and then moved to SD, but since SD is just a wrapper for SdFat it seems odd that it should fail.
 
SD uses a very old version of SdFat.

SPI transaction support has been added to both only recently. Obviously I got it wrong in SD, which is why that update is needed.

I'd be surprised if SdFat has exactly the same issue, because Bill & I did these updates separately. Bill is also the original author of SdFat, so he's a lot more familiar with it's control flow than I am.
 
I'll also have another look at SdFat, I only tried that very briefly and then moved to SD, but since SD is just a wrapper for SdFat it seems odd that it should fail.

SdFat still does not work, tested again with all of Pauls updates and with various settings for

#define ENABLE_SPI_TRANSACTION 1
#define ENABLE_SPI_YIELD 1

setting these on or off makes no noticeable difference, the registered nrf callbacks stop working after the SD card is accessed.
tested using the original posted code but with different pins posted again below for completeness.
Code:
#include <SPI.h>
#include "Adafruit_BLE_UART.h"
#include <SdFat.h>

#define ADAFRUITBLE_REQ 22
#define ADAFRUITBLE_RDY 23
#define ADAFRUITBLE_RST 9

Adafruit_BLE_UART uart = Adafruit_BLE_UART(ADAFRUITBLE_REQ, ADAFRUITBLE_RDY, ADAFRUITBLE_RST);

const uint8_t chipSelect = 3; //SS;
SdFat sd;
SdFile file;

/**************************************************************************/
/*!
    This function is called whenever select ACI events happen
*/
/**************************************************************************/
void aciCallback(aci_evt_opcode_t event)
{
  switch(event)
  {
    case ACI_EVT_DEVICE_STARTED:
      Serial.println(F("Advertising started"));
      break;
    case ACI_EVT_CONNECTED:
      Serial.println(F("Connected!"));
      break;
    case ACI_EVT_DISCONNECTED:
      Serial.println(F("Disconnected or advertising timed out"));
      break;
    default:
      break;
  }
}

/**************************************************************************/
/*!
    This function is called whenever data arrives on the RX channel
*/
/**************************************************************************/
void rxCallback(uint8_t *buffer, uint8_t len)
{
  Serial.print(F("Received "));
  Serial.print(len);
  Serial.print(F(" bytes: "));
  for(int i=0; i<len; i++)
   Serial.print((char)buffer[i]); 

  Serial.print(F(" ["));

  for(int i=0; i<len; i++)
  {
    Serial.print(" 0x"); Serial.print((char)buffer[i], HEX); 
  }
  Serial.println(F(" ]"));

  /* Echo the same data back! */
  uart.write(buffer, len);
  
  // Triger card read if 'K" is in the buffer.
  if (*buffer == 'K') {
    get_sd_card_files();
  }
}

/**************************************************************************/
/*!
    Configure the Arduino and start advertising with the radio
*/
/**************************************************************************/
void setup(void)
{ 
  Serial.begin(9600);
  while(!Serial); // Leonardo/Micro should wait for serial init
  Serial.println(F("Adafruit Bluefruit Low Energy nRF8001 Callback Echo demo"));

  uart.setRXcallback(rxCallback);
  uart.setACIcallback(aciCallback);
  // uart.setDeviceName("NEWNAME"); /* 7 characters max! */
  
  delay(1000);
  
  uart.begin();
}

/**************************************************************************/
/*!
    Constantly checks for new events on the nRF8001
*/
/**************************************************************************/
void loop()
{
  uart.pollACI();
}


void get_sd_card_files()
{
  char name[13];
  
  Serial.println("Opening SD card to generate list");

  if (!sd.begin(chipSelect, SPI_HALF_SPEED)) sd.initErrorHalt();

  // open next file in root.  The volume working directory, vwd, is root
  while (file.openNext(sd.vwd(), O_READ)) {
    file.printFileSize(&Serial);
    Serial.write(' ');
    file.printModifyDateTime(&Serial);
    Serial.write(' ');    
    file.printName(&Serial);
    if (!file.isDir()) {
      // Indicate a directory.
      file.getFilename(name);               
      Serial.println(name);        
    }
    file.close();
  }

  return;
}
 
I think the 8 second thing has been happening all along, but I will test tomorrow with a clean install of arduino 1.06 and the Adafruit version of nrf.

Ok, so on another machine I did a fresh install of Arduino 1.06 and teensyduino 1.20 and downloaded the latest adafruit_nrf lib from here
https://github.com/adafruit/Adafruit_nRF8001/archive/master.zip

I then ran the "callbackEcho" demo.

I can confirm that the '8 second' problem exists in the adafruit_nrf8001 lib *without* any of Paul's recent changes, so I think it is ok to release it Paul, this problem was already there.

or the record here is the output, the first Disconnect occurs after 7-8 seconds every time the program is started, after that it appears fine, It seems like a timeout or reset is called after 7-8 seconds.

Code:
Adafruit Bluefruit Low Energy nRF8001 Callback Echo demo
Advertising started
Connected!
Disconnected or advertising timed out
Advertising started
Connected!
Received 11 bytes: hello world [ 0x68 0x65 0x6C 0x6C 0x6F 0x20 0x77 0x6F 0x72 0x6C 0x64 ]
	Writing out to BTLE: 0x68 0x65 0x6C 0x6C 0x6F 0x20 0x77 0x6F 0x72 0x6C 0x64
 
Thanks for double checking that.

I didn't see this 8 second issue in any of my testing, but I did have plenty of time where I did things wrong and the app froze up when the nRF8001 wasn't talking properly.
 
Thanks for double checking that.

I didn't see this 8 second issue in any of my testing, but I did have plenty of time where I did things wrong and the app froze up when the nRF8001 wasn't talking properly.

That interesting because I see it every single time. First connection is always dumped 7-8 seconds after connecting, Perhaps it's to do with the Android device then, it was a Nexus 7 running Android 4.4.4
 
Status
Not open for further replies.
Back
Top