qwazzerman
Member
I have a project that involves multiple (6) Teensys with audio boards connected via USB to a single computer running Ubuntu. Each Teensy is individually controlled by a Python process, and is set up to stream audio from the Python process (i.e., Teensy acts like a sound card). The Python scripts are written to run for an extended period of time (ideally days to weeks at a time). Teensys are in Serial + MIDI + Audio mode. Audio is being played at a sampling rate of 44.1 kHz.
Anywhere between an hour and a day or two after starting a Python script, one of the Teensys will stop receiving sensor input, or at least the attached sensors no longer trigger any reaction from the control program. It's not immediately noticeable because the attached sensors get triggered at irregular intervals that can range from a few seconds to hours. I don't know if that means the Teensy is not sending the information to the computer, or if the computer has stopped accepting the input for some reason. If I stop the Python process and restart it, the Teensy comes back up right away and the problem is resolved. Python doesn't report any errors, though, so I don't have much additional information.
I've used usbtop to see if there's still communication with the Teensys, and it looks like the computer is reporting communication in both directions, with an average PC->Teensy rate of 300 kb/s and an average Teensy->PC rate of 130 kb/s. I noticed that the Teensys that aren't responding seem to have an output rate of only 110 kb/s.
I have some theories, but not sure about the best way to proceed:
Does this kind of issue sound familiar for anyone? At the moment my solution is just to restart each Python script every day, but I'd like to come up with a more permanent solution.
I would post the Python code but it spans several dozen files...
For reference, here's the code uploaded to each Teensy:
Anywhere between an hour and a day or two after starting a Python script, one of the Teensys will stop receiving sensor input, or at least the attached sensors no longer trigger any reaction from the control program. It's not immediately noticeable because the attached sensors get triggered at irregular intervals that can range from a few seconds to hours. I don't know if that means the Teensy is not sending the information to the computer, or if the computer has stopped accepting the input for some reason. If I stop the Python process and restart it, the Teensy comes back up right away and the problem is resolved. Python doesn't report any errors, though, so I don't have much additional information.
I've used usbtop to see if there's still communication with the Teensys, and it looks like the computer is reporting communication in both directions, with an average PC->Teensy rate of 300 kb/s and an average Teensy->PC rate of 130 kb/s. I noticed that the Teensys that aren't responding seem to have an output rate of only 110 kb/s.
I have some theories, but not sure about the best way to proceed:
- Since it resolves itself when I restart the Python process, is that a good indication that it's a problem on the Python side, or is there a way this could happen on the Teensy end?
- Another possibility is that it's related to the USB bandwidth issues I reported in an earlier thread, but there's no sound distortion since I got a new USB hub.
- Maybe the serial connection is crapping out for some reason after a length of time? I don't know if that would explain why there is still data coming from the Teensy... Maybe somehow the serial connection is getting intercepted by another program or something?
Does this kind of issue sound familiar for anyone? At the moment my solution is just to restart each Python script every day, but I'd like to come up with a more permanent solution.
I would post the Python code but it spans several dozen files...
For reference, here's the code uploaded to each Teensy:
Code:
#include <Audio.h>
#include <Wire.h>
#include <SPI.h>
#include <SD.h>
#include <SerialFlash.h>
/*
* Revision 1.1
* 2/23/18 by AR
*
*/
/*
Pins used:
Set by Teensyduino:
//6 - Audio shield, MEMCS (NOT USED)
7 - Audio shield, MOSI (SD, shared)
9 - Audio shield, BCLK*
10 - Audio shield, CS (SD)
11 - Audio shield, MCLK*
12 - Audio shield, MISO (SD, shared)
13 - Audio shield, RX*
14 - Audio shield, SCLK (SD)
18 - Audio shield, SDA*
19 - Audio shield, SCL*
22 - Audio shield, TX*
23 - Audio shield, LRCLK*
Set by control program:
1 - trial light
2 - overhead light relay
3 - response light
16 - reinforcement relay
37 - response sensor
38 - start sensor
*/
// -----
static uint8_t myID[8]; //for getting unique Teensy ID
unsigned long serialNum = 0; //for getting Teensy serial number - not sure which to use, serialNum or myID
int inputValue = 0;
int baudRate = 19200;
char ioBytes[2];
int ioPort = 0;
// Audio setup
// GUItool: begin automatically generated code
AudioInputUSB usb1; //xy=200,69 (must set Tools > USB Type to Audio)
AudioOutputI2S i2s1; //xy=365,94
AudioConnection patchCord1(usb1, 0, i2s1, 0);
AudioConnection patchCord2(usb1, 1, i2s1, 1);
AudioControlSGTL5000 sgtl5000_1; //xy=302,184
// GUItool: end automatically generated code
/*
Code for getting Teensy unique ID # (so log files can accurately reflect which unit they came from)
*/
void read_EE(uint8_t word, uint8_t *buf, uint8_t offset) {
noInterrupts();
FTFL_FCCOB0 = 0x41; // Selects the READONCE command
FTFL_FCCOB1 = word; // read the given word of read once area
// launch command and wait until complete
FTFL_FSTAT = FTFL_FSTAT_CCIF;
while(!(FTFL_FSTAT & FTFL_FSTAT_CCIF))
;
*(buf+offset+0) = FTFL_FCCOB4;
*(buf+offset+1) = FTFL_FCCOB5;
*(buf+offset+2) = FTFL_FCCOB6;
*(buf+offset+3) = FTFL_FCCOB7;
interrupts();
}
void read_myID() {
read_EE(0xe,myID,0); // should be 04 E9 E5 xx, this being PJRC's registered OUI
read_EE(0xf,myID,4); // xx xx xx xx
unsigned int ii;
for (ii = 0; ii < sizeof(myID); ii++) {
if ( ii > 3)
serialNum = (serialNum << 8) + myID[ii];
}
}
void setup(){
Serial.begin(baudRate);
while (!Serial) {
; // wait for serial port to connect
}
Serial.printf("Connected to Teensy");
read_myID();
Serial.printf("Teensy ID %d0 \n \n", serialNum);
AudioMemory(12);
sgtl5000_1.enable();
sgtl5000_1.volume(0.4);
}
void loop() {
// All serial communications should be two bytes long
// The first byte specifies the port to act on
// The second byte specifies the action to take
// The actions are:
// 0: Read the specified input
// 1: Write the specified output to HIGH
// 2: Write the specified output to LOW
// 3: Set the specified pin to OUTPUT
// 4: Set the specified pin to INPUT
// 5: Set the specified pin to INPUT_PULLUP
// 6: Return Teensy ID number (pin independent)
// 99: Audio control (pin independent)
// if we get a valid serial message, read the request:
if (Serial.available() >= 2) {
// get incoming three bytes:
Serial.readBytes(ioBytes, 2);
// Serial.println("I received: ");
// Serial.println(ioBytes[0], DEC);
// Serial.println(ioBytes[1], DEC);
// Extract the specified port
ioPort = (int) ioBytes[0];
// if (ioPort == 99) {
// // 99 is for audio functions
// if (ioBytes[1] == 0) {
// //Stop playback
// playSdRaw1.stop();
// break;
// }
// else if (ioBytes[1] == 1) {
// //Selected type is S+, choose file
// //randNumber = random(1,length(s+List))
// //const char *filename = S+filelist[filenumber];
// }
// else if (ioBytes[1] == 2) {
// //Selected type is S-, choose file
// //randNumber = random(1,length(s-List))
// //const char *filename = S-filelist[filenumber];
// }
// //Could add additional types here
//
// playSdRaw1.play(filename);
// break;
// }
//
// else
// Specific pin actions
// Switch case on the specified action
switch ((int) ioBytes[1]) {
case 0: // Read an input
inputValue = digitalRead(ioPort);
Serial.write(inputValue);
break;
case 1: // Write an output to HIGH
digitalWrite(ioPort, HIGH);
break;
case 2: // Write an output to LOW
digitalWrite(ioPort, LOW);
break;
case 3: // Set a pin to OUTPUT
pinMode(ioPort, OUTPUT);
digitalWrite(ioPort, LOW);
break;
case 4: // Set a pin to INPUT
pinMode(ioPort, INPUT);
break;
case 5: // Set a pin to INPUT_PULLUP
pinMode(ioPort, INPUT_PULLUP);
break;
case 6: // Return Teensy ID
Serial.write("Teensy ID: ");
Serial.write(serialNum);
break;
//}
}
}
}