Teensy LC and Teensy 4.0 output delay.

Status
Not open for further replies.

Hypocaust

New member
Absolute beginner here, so please bear with me if I don't really know what I'm talking about. I am attempting to control an N64 via mouse and keyboard inputs on my PC. I began this project following the guide I found here: https://medium.com/james-reads-developer-projects/n64-microcontroller-controller-12c76acde194
I was using the Teensy LC to communicate between the PC and the N64 using the code provided in the URL for the Teensy, and I wrote some code in Python to convert keyboard and mouse inputs into 4bytes that the Teensy would take and convert to a signal that the N64 understands. This actually worked, but there was a good 1 second delay between my PC inputs and the Teensy LC outputs. Thinking this was likely due to python being quite slow I tried to minimise this by converting my code to C# (in Unity) instead. This got the delay down to about 0.3 seconds, but that's far too much delay for controlling things in real time.
Maybe the Teensy was the bottleneck, I thought, so I got a Teensy 4.0 thinking that the worst that could happen is no difference. What actually happened was the delay went from 0.3 seconds to about 15 seconds.
I've now got to the point in my troubleshooting where I need some help. I'm thinking that maybe the difference in the size of the memory is causing this and that all the data I send via the USB is getting stored, but when the N64 asks for the controller inputs (100 times per second) it pulls from the bottom of the stack, so the latest inputs have to wait until all the other data is pulled out. Can I reduce the available memory and might that help? What's going on here?

C# code I wrote:
Code:
using System.Collections;
using System.Collections.Generic;
using UnityEngine;
using System.Linq;
using System.IO.Ports;

public class SerialChatter4 : MonoBehaviour
{
    public SerialPort serialPort = new SerialPort("COM4", 115200);//, Parity.None, 8, StopBits.One);
    public float xaxissum, yaxissum, mousefactor;
    public int axisx, axisy;
    private float starttime;
    private int i;

    public byte butA, butB, butZ, butS, butdU, butdD, butdL, butdR;
    public byte butL, butR, butcU, butcD, butcL, butcR;
    public byte antiA, antiB, antiZ, antiS, antidU, antidD, antidL, antidR;
    public byte antiL, antiR, anticU, anticD, anticL, anticR;
    public byte gatheredFirstByte, gatheredSecondByte;
    public byte bufferedFirstByte, bufferedSecondByte;

    void Start()
    {
        //Define binaries
        butA = 0b0000_0001; antiA = 0b1111_1110;
        butB = 0b0000_0010; antiB = 0b1111_1101;
        butZ = 0b0000_0100; antiZ = 0b1111_1011; butL = 0b0000_0100; antiL = 0b1111_1011;
        butS = 0b0000_1000; antiS = 0b1111_0111; butR = 0b0000_1000; antiR = 0b1111_0111;
        butdU = 0b0001_0000; antidU = 0b1110_1111; butcU = 0b0001_0000; anticU = 0b1110_1111;
        butdD = 0b0010_0000; antidD = 0b1101_1111; butcD = 0b0010_0000; anticD = 0b1101_1111;
        butdL = 0b0100_0000; antidL = 0b1011_1111; butcL = 0b0100_0000; anticL = 0b1011_1111;
        butdR = 0b1000_0000; antidR = 0b0111_1111; butcR = 0b1000_0000; anticR = 0b0111_1111;
        gatheredFirstByte = 0b0000_0000; gatheredSecondByte = 0b0000_0000;
        bufferedFirstByte = 0b0000_0000; bufferedSecondByte = 0b0000_0000;

        serialPort.Open();
        starttime = Time.time;
        mousefactor = 40.0f;
        Cursor.lockState = CursorLockMode.Locked;
    }

    void Update()
    {
        gatheredFirstByte = 0b0000_0000; gatheredSecondByte = 0b0000_0000;
        if (Input.GetMouseButton(1)) { gatheredFirstByte |= butA; bufferedFirstByte |= butA; }
        if (Input.GetMouseButton(2)) { gatheredFirstByte |= butB; bufferedFirstByte |= butB; }
        if (Input.GetMouseButton(0)) { gatheredFirstByte |= butZ; bufferedFirstByte |= butZ; }
        if (Input.GetKey(KeyCode.BackQuote)) { gatheredFirstByte |= butS; bufferedFirstByte |= butS; }//That one below escape
        if (Input.GetKey(KeyCode.LeftControl)) { gatheredFirstByte |= butdD; bufferedFirstByte |= butdD; }
        if (Input.GetKey(KeyCode.Tab)) { gatheredFirstByte |= butdR; bufferedFirstByte |= butdR; }

        if (Input.GetKey(KeyCode.Space)) { gatheredSecondByte |= butR; bufferedSecondByte |= butR; ; }
        if (Input.GetKey(KeyCode.W)) { gatheredSecondByte |= butcU; bufferedSecondByte |= butcU; }
        if (Input.GetKey(KeyCode.S)) { gatheredSecondByte |= butcD; bufferedSecondByte |= butcD; }
        if (Input.GetKey(KeyCode.A)) { gatheredSecondByte |= butcL; bufferedSecondByte |= butcL; }
        if (Input.GetKey(KeyCode.D)) { gatheredSecondByte |= butcR; bufferedSecondByte |= butcR; }
        xaxissum += Input.GetAxis("Mouse X");
        yaxissum -= Input.GetAxis("Mouse Y");
        if (Time.time - starttime >= 0.005)
        {
            axisx = Clamp((int)(xaxissum * mousefactor));
            axisy = Clamp((int)(yaxissum * mousefactor));
            if (Input.GetAxis("Mouse X") == 0) { axisx = 0; }
            if (Input.GetAxis("Mouse Y") == 0) { axisy = 0; }
            WriteToSerialPort(axisx, axisy, bufferedFirstByte, bufferedSecondByte);
            yaxissum *= 0.5f; xaxissum *= 0.5f;
            starttime = Time.time;
            bufferedFirstByte = gatheredFirstByte;
            bufferedSecondByte = gatheredSecondByte;
        }
    }

    public int Clamp(int value)
    {
        return (value <= -90) ? -90 : (value >= 90) ? 90 : value;
    }

    public void WriteToSerialPort(int axisx, int axisy, byte firstByte, byte secondByte)
    {
        byte[] byteArray = new byte[4];
        byteArray[0] = firstByte; byteArray[1] = secondByte;
        byteArray[2] = (byte)axisx; byteArray[3] = (byte)axisy;
        serialPort.DiscardOutBuffer();
        serialPort.Write(byteArray, 0, byteArray.Length);
    }
}

Processing code I'm using:
Code:
#include <Arduino.h>

// pin on Teensy through which communications happen
const int pin = 12;

/* emptyAction and seq represent 32bit responses from the controller 
    representing controller input for a given frame.
    Each entry of emptyAction and seq represents a microsecond of high or 
    low voltage over the data wire. Each bit is represented by 4 microseconds 
    of voltage.
    0:    LOW LOW  LOW  HIGH
    1:    LOW HIGH HIGH HIGH
    STOP: LOW LOW  HIGH HIGH */
#define SEQLEN 132
bool seq[SEQLEN] = {false};  // input to be sent to console
bool emptyAction[SEQLEN] = {false};  // input of no button presses

// Buffer variables to deal with nonsense request from console
#define BUFFLEN 4

volatile int8_t bttn1;
volatile int8_t bttn2;
volatile int8_t xAxis;
volatile int8_t yAxis;
// keeps track of when to queue the next input to send to the console
// helps deal with nonsense request from console
volatile bool sentLast;
// interrupt for queueing the next command, also deals with nonsense request
IntervalTimer setCommand;

// Button refs
#define A 0
#define B 1
#define Z 2
#define S 3
#define dU 4
#define dD 5
#define dL 6
#define dR 7
#define rst 8
#define L 10
#define R 11
#define cU 12
#define cD 13
#define cL 14
#define cR 15
#define X 16
#define Y 24


void setup() {
    init();
    // start connection to Serial port
    Serial.begin(115200);
    Serial.print("Starting Program");
    Serial.println();
    // set up interrupts for console input request + command queueing
    attachInterrupt(digitalPinToInterrupt(pin), writeSeq, FALLING);
    setCommand.priority(0);
    setCommand.begin(nextCommand, 400);
}

void init() {
    // 32 0 bits
    for (int i=3 ; i < SEQLEN - 4 ; i = i + 4) {
        emptyAction[i] = true;
    }
    // stop bit
    emptyAction[128] = false;
    emptyAction[129] = false;
    emptyAction[130] = true;
    emptyAction[131] = true;
    resetSeq();

    // on start always queue next command
    sentLast = true;

    // on start default buttons to queue are an empty command
    bttn1 = 0;
    bttn2 = 0;
    xAxis = 0;
    yAxis = 0;

    // set pin mode for FALLING interrupt - listen for console request
    pinMode(pin, INPUT);
}

void loop() {
    // Buffer next command
    int count = 0;
    int8_t buff[BUFFLEN];
    while (count<BUFFLEN) {
        if (Serial.available() && sentLast) {
            buff[count++] = Serial.read();
        }
    }
    bttn1 = buff[0];
    bttn2 = buff[1];
    xAxis = buff[2];
    yAxis = buff[3];
    sentLast = false;
}

void writeSeq() {
    // communicate command in seq to the console
    noInterrupts();
    setCommand.end();
    delayMicroseconds(32);
    pinMode(pin, OUTPUT);
    for (int i=0 ; i<SEQLEN ; i++) {
        if (seq[i]) {
            digitalWrite(pin, HIGH);
        } else {
            digitalWrite(pin, LOW);
        }
    }
    pinMode(pin, INPUT);
    attachInterrupt(digitalPinToInterrupt(pin), writeSeq, FALLING);
    setCommand.begin(nextCommand, 4000);
    interrupts();
}

void nextCommand() {
    // set seq to be the next command in the queue
    setCommand.end();
    resetSeq();

    for (int i=0 ; i<8 ; i++){
        if ((1<<i) & bttn1) {
            pressButton(i);
        }
        if ((1<<i) & bttn2) {
            pressButton(i + 8);
        }
    }

    setAxis(X, xAxis);
    setAxis(Y, yAxis);

    // default buttons to queue are an empty command
    bttn1 = 0;
    bttn2 = 0;
    xAxis = 0;
    yAxis = 0;

    sentLast = true;
}

void pressButton(int button) {
    // set analog button values in seq
    seq[button * 4 + 1] = true;
    seq[button * 4 + 2] = true;
}

void setAxis(int axis, int8_t val) {
    // set joystiq values in seq
    for (int i=0 ; i<8 ; i++) {
        bool bit = (1<<(7-i)) & val;
        if (bit) {
            pressButton(axis + i);
        }
    }
}

void resetSeq() {
    // set all seq values to controller input with no button presses
    for (int i=0 ; i<SEQLEN ; i++) {
        seq[i] = emptyAction[i];
    }
}
 
Very difficult to say what's really going wrong here. But with a quick look at the Teensy code, this looks like trouble:

Code:
void loop() {
    // Buffer next command
    int count = 0;
    int8_t buff[BUFFLEN];
    while (count<BUFFLEN) {
        if (Serial.available() && sentLast) {
            buff[count++] = Serial.read();
        }
    }

There are a couple ways of looking at this issue.

The detail-oriented way is this code keep writing to an increasing location in the buffer, but the rest of your code has no awareness of where new data is written because "count" is a local variable only seen in loop().

But taking a step back to look at the big picture here, I believe you should redesign this code to not use interrupts. And do not use while() inside of loop(). Instead, make "count" a static or global variable, so it retains its value every time loop() runs. Design your loop() function to always complete quickly, never waiting for anything to happen.

With that sort of design, you can use an elapsedMicros variable instead of IntervalTimer. Again, do not wait. Just check if it has incremented past 4000, and do whatever needs to be done in that case.

Likewise for responding to pin 12, just check it with digitalRead() somewhere inside loop(). If you've avoided delays or other code that waits for something to happen, then just checking it once inside loop() should be fine. Even Teensy LC will run loop() hundreds of thousands of times per second. You will need a static or global boolean variable to remember if it was high or low the previous time, so you can do specific actions when it has changed.

I know this isn't very specific, and it may not solve all your problems, but I have a feeling you're hitting really thorny problems that tend to happen with interrupts. I'm pretty sure you'll have much better success if you eliminate all the interrupt stuff and just craft code in loop() which repeatedly checks if things have happened and responds.
 
Thanks for this. Having a direction to go in is exactly what I need. I'll go back and do a redesign of the code cutting out the interrupts and see if that does the job.
The learning curve is steep, but the climb is fun.
 
The sending loop in writeSeq callback has no control over the bit timings. So on the Teensy 4 the bitrate will be much higher than for the LC, possibly/probably causing issues for the N64 to read the bitstream. I think some delay, using delayNanoseconds(), should be added after each bit in the sequence.
 
You've hit on something I noticed myself. I couldn't figure out how the pulses got properly timed, but the inputs I give it do seem to be recognised as valid inputs on the N64, just valid inputs 15 seconds after I send them, so I figure the timing is working in some way. I'm waiting for a signal analyser on order so I can see exactly what the Teensy is being made to output.
This is the problem with using code I didn't write myself I think, so cutting that out and getting proper control over it will be my first step. Then I can figure out exactly what's going on, and this time it'll all be my own fault.
Thanks mlu.
 
Status
Not open for further replies.
Back
Top