Best / fastest method of communication between teensy 3.1 and Touch Designer?

Status
Not open for further replies.

visualSound

Well-known member
I have been interested in streaming video from a piece of VJ software called touch designer to array's of led's run from a teensy 3.1.

Hoping for some help or at least guidance on what teensy prefers best in terms of communication, but also what might be easiest for someone with higher level coding experience only to grasp.
I don't mind digging deeper, but I want to make sure I dig in the right direction!


Touch designer has several communication nodes / protocols:

1) dmx in/out
2) midi in/out
3) serial in/out
4) pipe In / out (their way of saying TCP/IP and network communication)
5) OSC in/out


I've managed to put together a homegrown solution using midi and teensy's support for midi usb. It works great, but there's a bottle neck that I start to notice passing 64 led's already and that's cutting frame rate and performance down. On anything significantly bigger it just wouldn't be doable.

Here's a video of my setup using midi.



Other than midi, I am having some difficulty sending data of other types to teensy, and specifically data that can be sent over usb.
Osc seems like a good option, but can osc be transmitted through usb? or only network?


In the end, I want to be able to send decent amounts of data optimally to teensy, that I can parse in some way.
I'm having trouble establishing a ground 0, so to speak, some working example that I can build off of.

Any help would be greatly appreciated!!
 
I can't comment on what really is the fastest method but if you are looking for a cabled solution then I am suspecting it is going to be USB/Serial.
Again, I also cannot comment what method is best for streaming video but you can send OSC through the USB cable using the OSCuino library written by the creators of the OSC protocol.

I see in your video that you are using TouchOSC and I am assuming it does not communicate with your teensy directly but through Touch Designer on the Computer ?
If you are interested in controlling a Teensy directly with TouchOSC (WiFi) I have some code for you.

Should you prefer a wireless solution I'd suggest Ethernet using a WIZ820io Ethernet Module connected to a little pocket router such as a TP-Link TL WR703n (or 702n and others). You can also use that to transport OSC or reaw UDP etc..
 
How many pixels do you need to control ?
We've been working on artnet support and it works quite well, can touchdesigner send artnet ?
 
Thanks for the reply, I should have been more clear about the intentions behind my video:

Yes touch designer is controlling the teensy / transmitting data via usb and the touchOsc setup on my phone was just something to control it with for the video.
In the end all my devices would preferably be wired for reliability, so looking to stay within that realm.
I'll read into the OSCuino library! I didn't know it could work over wired as well.

I managed to get serial communication working pretty well but there is a very strange and very repeatable visual artifact that happens.

You can see it in action here

The first half shows the weird streak pattern that happens, and the second half shows the actual table of values that's being sent out as serial bytes.
My code on both teensy and the side of touch are mostly the same but this did't happen with the midi route strangely enough.

On the plus side, this does seem MUCH more robust and quick on the touch designer side, so I'd like to go with it if I can but I have no idea how to work out that visual bug.
It does slow down when I slow down the timing in touch designer... however combing through print statements in the debug window reveals no unusual values in touch so I am assuming this is happening somewhere in the serial transmission, or in side of teensy's sketch.

Here's my code for teensy, it's really short right now, trying to keep it small and robust but maybe I'm not treating the serial buffer correctly?:





Code:
// ------------------- Some variables -------------------- //
#include <Adafruit_NeoPixel.h>
#define PIN 2
Adafruit_NeoPixel strip = Adafruit_NeoPixel(64, PIN, NEO_GRB + NEO_KHZ800);
const int numOfLeds = 64;

byte byteRead;
int serialCounter = 0;
int singleLedArray[4];

int maxBright = 32;


// ------------------- Setup -------------------- //
void setup() {
  strip.begin(); // init strip.
  strip.show(); // push dark pixels to all.
  Serial.begin(9600);
  
  for (int j = 0; j < 4; j++) {
    singleLedArray[j] = 0;
  }
  delay(200);
}

// ------------------- Main Loop -------------------- //
void loop() {
  
  serialCounter = 0;
  while (Serial.available()) {
    byteRead = Serial.read();
    if(byteRead==44){
      serialCounter = 0;
      singleLedArray[1] = constrain(singleLedArray[1], 0, maxBright);
      singleLedArray[2] = constrain(singleLedArray[2], 0, maxBright);
      singleLedArray[3] = constrain(singleLedArray[3], 0, maxBright);
      strip.setPixelColor(singleLedArray[0], singleLedArray[1], singleLedArray[2], singleLedArray[3]);
    }
    else{
        singleLedArray[serialCounter] = byteRead;
        serialCounter += 1;
    }
  }
  
  strip.show();
}
 
Thanks for the info there!

I haven't looked into art-net. I suppose I'll need an Ethernet shield to make use of that and a router that connects both teensy and pc?
 
Yes, paul just released an adapter for the wiznet820io which works very nicely, yesterday I made a test with 492 leds and was able to go at 220fps !
you can either use a router or connect your computer directly via ethernet to the wiz820io
 
This 1-byte-at-a-time style consumes a lot of CPU time:

Code:
  serialCounter = 0;
  while (Serial.available()) {
    byteRead = Serial.read();
    if(byteRead==44){

Using Serial.readBytes(array, length) is dramatically more efficient.

But readBytes() is a bit more complex, since you have to know how many bytes you're expecting. You have to check the return value and deal with the case where Serial.readBytes() gives less than the expected length. It works together with Serial.setTimeout(), which usually makes things easier if you plan your communication around some sort of message where you can predict the length and the amount of time it should normally take.

Here is Arduino's documentation on these 2 functions:

http://arduino.cc/en/Reference/StreamReadBytes
http://arduino.cc/en/Reference/StreamSetTimeout

If you can switch to readBytes() for a big block of data instead of available() and read() for every individual byte, your code will use a lot less CPU time on Teensy. That may or may not translate into a huge speed gain, depending on whether the USB bandwidth, PC's speed, or other factors are limiting the overall speed. But making the code more efficient certainly can't hurt.
 
Thanks for the tips everyone, going to try and optimize / fix serial communication but if I can't work out that bug I'm experiencing with the flickering I'll go with one of the other two options and keep the thread updated with what happens!
 
Seemingly "random" flickering on large LED projects is often related to signal quality issues. If you're already using the OctoWS2811 adaptor and your power supplies are close to the LEDs with the power and Teensy grounds meeting at the LED inputs, then this probably isn't an issue.

But if your hardware isn't following these best practices, I'd highly recommend putting the work into locating the power LED supplies close with short wires and avoid the signal+power traveling together for any substantial distance before the first LED.

Many people have wasted countless hours fiddling with software, and especially second guessing the WS2811 timing parameters. The tricky part is those changes can make small differences when your hardware has signal quality issues, which leads to a never-ending quest for just the right setting that will fix everything.

One other quick and easy thing to try is using digitalWriteFast(pin, HIGH) and digitalWriteFast(pin, LOW), where "pin" is a number and not a variable, to set a pin high only when your code is waiting for data or otherwise not doing something useful. You can watch it with a 'scope or even just a multimeter (in DC volts or frequency mode) to get an idea of whether your code is fast enough.
 
After my last lengthy struggle with what turned out to be using the wrong type of logic level converter I've been sure to follow the setup guides to a T :D

Currently, this thread's project resides on a breadboard, I have a 100 ohm resistor close to the pin, and very short lengths of wire going to the neoPixel board. Everything is being run from usb for the time being and I am clamping the values that get written to the led's to a lower number to protect from current over draw.

Paul, thanks for sharing the .readBytes method, I've gotten a bit more comfy with this serial business, I will try and get that implemented!

DSCF0712-web.jpg

I would be knowing how many bytes to expect: 320 with 64 led's (64x5) 1 byte to indicate the led number (although this would need revisiting past 256 led's) 3 bytes to indicate RGB brightness, and a fifth byte for a comma separator. I suppose I could ditch the comma since i'm parsing the incoming byte stream in fixed chunks of 4, so I may optimize that eventually.


In touch, the python line that does the actual sending is really simple, looks like this:
op("serialin1").sendBytes(0, 1, 1, 1, 44, 1, 1, 1, 1, 44, 2, 1, 1, 1, 44, 3, 1, 1, 1, 44, 4, 1, 1, 1, 44)
Where "serialin1" is a node where the port and what not is specified.
the above line is sending rgb values of 1 to led's 0-4 for example.

Touch designer has a limit to how much serial data can be send in one command, so I have my code programatically split up the byte array into smaller chunks then batch send them at the end.
I've changed that amount though and can verify it's not causing the flicker either..




1) So I should in theory have no problems using multiple teensy boards, with those Ethernet adapters if connected to my computer through an Ethernet switch / hub?

2) I eventually want to switch over to the OctoWS2811 Adaptor setup once I start scaling things up, from first glance it seems these don't collide physically.. Should it be possible to use the OctoWS2811 Adaptor, the WIZ820io and the PJRC WIZ820io SD card adapter all at once?

3) If I were to go the route of serial communication, could a dedicated usb hub of some kind allow me to control more teensy's through one usb port on my computer?

4) I downloaded the most recent library from the OSCuino gitHub. Looking through the examples I'm mostly getting it although I'm still not sure how to retrieve a value that I can pass onto another method. Any recommendations on a good example sketch to start from? This would probably also be a prime candidate since it can stay within usb communication and work with a pretty robust / straight forward protocol in touch designer.

I think I'm leaning towards serial OR osc through usb merely for cost and simplicity (at least on the hardware side).. Thoughts / recommendations on that?
 
You should think of sending a single frame at a time rather than sending info about pixel numbers.
Say you have 60 leds, you would send 180 bytes, then use readbytes to store that frame, then use a for loop to update your pixels and after the loop you call leds.show()
 
Should it be possible to use the OctoWS2811 Adaptor, the WIZ820io and the PJRC WIZ820io SD card adapter all at once?

Pin 4 is reserved by OctoWS2811, but that adaptor routes the SD card's CS signal to pin (edited) 4. You'll need to solder a wire from "pin 4" on the Wiz820+SD to some other available pin on the Teensy 3.1.
 
Last edited:
Paul:
It's working! Your tip about the .readBytes() did the trick. I'm sure it's faster now too, but for whatever reason the strange flicker is gone as well. Not going to worry too much about why, I'm just fairly certain my logic was a bit flawed before, maybe something to do with the commas. I have since removed them.


nlecaude:
Yep, I'm doing that now, I ditched the led number byte that gets sent since everything happens sequentially anyways there is no need to identify the led number. Doing so would also eliminated the need for the extra byte to store led numbers higher than 255, or extra 2 bytes for higher than 512, etc.

Massive improvements! FPS is capped out at touch's default of 60. I have a feeling the teensy is doing much better than that too.
Thanks again for the help and info.
I'm going to stick with serial for now, it seems to be pretty solid.

Any ideas on the usb hub / multiple teensy's receiving serial data?



For those of you interested, here is the most recent iteration of the teensy side code. It's really light weight at the moment:

Code:
// ------------------- Variables - These need to be set -------------------- //
#define PIN 2
const int numOfLeds = 64;
const int dataChunkSize = 3;

int maxBright = 32;

// ------------------- Variables - These just need to be declared -------------------- //
#include <Adafruit_NeoPixel.h>
Adafruit_NeoPixel strip = Adafruit_NeoPixel(numOfLeds, PIN, NEO_GRB + NEO_KHZ800);

const int numOfBytes = numOfLeds * dataChunkSize;
int byteReturnLen = 0;
int byteReturnCounter = 0;
char inputBuffer[numOfBytes];
int led = 0;
int led_r = 0;
int led_g = 0;
int led_b = 0;

// ------------------- Setup -------------------- //
void setup() {
  strip.begin(); // init strip.
  strip.show(); // push dark pixels to all.
  Serial.begin(9600);
  Serial.setTimeout(500);
}

// ------------------- Gama Lookup table -------------------- //
const byte dim_curve[] = {
    0,   1,   1,   2,   2,   2,   2,   2,   2,   3,   3,   3,   3,   3,   3,   3,
    3,   3,   3,   3,   3,   3,   3,   4,   4,   4,   4,   4,   4,   4,   4,   4,
    4,   4,   4,   5,   5,   5,   5,   5,   5,   5,   5,   5,   5,   6,   6,   6,
    6,   6,   6,   6,   6,   7,   7,   7,   7,   7,   7,   7,   8,   8,   8,   8,
    8,   8,   9,   9,   9,   9,   9,   9,   10,  10,  10,  10,  10,  11,  11,  11,
    11,  11,  12,  12,  12,  12,  12,  13,  13,  13,  13,  14,  14,  14,  14,  15,
    15,  15,  16,  16,  16,  16,  17,  17,  17,  18,  18,  18,  19,  19,  19,  20,
    20,  20,  21,  21,  22,  22,  22,  23,  23,  24,  24,  25,  25,  25,  26,  26,
    27,  27,  28,  28,  29,  29,  30,  30,  31,  32,  32,  33,  33,  34,  35,  35,
    36,  36,  37,  38,  38,  39,  40,  40,  41,  42,  43,  43,  44,  45,  46,  47,
    48,  48,  49,  50,  51,  52,  53,  54,  55,  56,  57,  58,  59,  60,  61,  62,
    63,  64,  65,  66,  68,  69,  70,  71,  73,  74,  75,  76,  78,  79,  81,  82,
    83,  85,  86,  88,  90,  91,  93,  94,  96,  98,  99,  101, 103, 105, 107, 109,
    110, 112, 114, 116, 118, 121, 123, 125, 127, 129, 132, 134, 136, 139, 141, 144,
    146, 149, 151, 154, 157, 159, 162, 165, 168, 171, 174, 177, 180, 183, 186, 190,
    193, 196, 200, 203, 207, 211, 214, 218, 222, 226, 230, 234, 238, 242, 248, 255,
};

// ------------------- Main Loop -------------------- //
void loop() {
  if(Serial.available() > 0) {
    byteReturnCounter = Serial.readBytes(inputBuffer, numOfBytes);
    byteReturnLen += byteReturnCounter;
  }
  
  if(byteReturnLen >= numOfBytes) {
    for (int j = 0; j < numOfLeds; j++) {
      led_r = constrain(dim_curve[inputBuffer[(j*dataChunkSize)]], 0, maxBright);
      led_g = constrain(dim_curve[inputBuffer[(j*dataChunkSize)+1]], 0, maxBright);
      led_b = constrain(dim_curve[inputBuffer[(j*dataChunkSize)+2]], 0, maxBright);
      strip.setPixelColor(j, led_r, led_g, led_b);
    }
    strip.show();
    byteReturnLen = 0;
  }
}




Here's the touch designer code for anyone who's curious - it runs the function anytime the incoming table of rgb values changes:

Code:
def tableChange(dat):
	
	debugCharLength = 1 # when this is set to one, we build our debug array.
	
	zigZagCounter = 1

	# Lets get the dimensions of our table. 
	colNum = dat.numCols
	rowNum = dat.numRows
	
	# Declare our holding array for our entire led array.
	colorTable_RGB = []
	
	# Parse our DAT table into a python friendly array.
	for x in range(int(rowNum / 3)):
		colorRow_RGB = []
		for y in range(colNum):
			colorSingle_RGB = []
			colorSingle_RGB.append(int(dat[(x*3), y]))
			colorSingle_RGB.append(int(dat[((x*3)+1), y]))
			colorSingle_RGB.append(int(dat[((x*3)+2), y]))
			colorRow_RGB.append(colorSingle_RGB)
		colorTable_RGB.append(colorRow_RGB)


	debugLengthArray = []
	ledCounter = 0
	executeArray = []
	subExArray = []
	firstValueMarker = 1
	for row in colorTable_RGB:
		if(zigZagCounter == 1):
			for ledColor in row:
				if((len(subExArray) + 5) < 255): # if the sub array hasn't reached  touch's maximum allowed length, continue adding...
					subExArray.append(ledColor[0])
					subExArray.append(ledColor[1])
					subExArray.append(ledColor[2])
					if(debugCharLength == 1):
						debugLengthArray.append(ledColor[0])
						debugLengthArray.append(ledColor[1])
						debugLengthArray.append(ledColor[2])
				else: # if the sub array will be too big after next iteration, assign to master, and clear sub for new chunk.
					executeArray.append(subExArray)
					subExArray = []
					subExArray.append(ledColor[0])
					subExArray.append(ledColor[1])
					subExArray.append(ledColor[2])
					if(debugCharLength == 1):
						debugLengthArray.append(ledColor[0])
						debugLengthArray.append(ledColor[1])
						debugLengthArray.append(ledColor[2])
				ledCounter += 1
			zigZagCounter = -1
		elif(zigZagCounter == -1):
			row[::-1] # reverse "Row".
			for ledColor in row:
				if((len(subExArray) + 5) < 255): # if the sub array hasn't reached  touch's maximum allowed length, continue adding...
					subExArray.append(ledColor[0])
					subExArray.append(ledColor[1])
					subExArray.append(ledColor[2])
					if(debugCharLength == 1):
						debugLengthArray.append(ledColor[0])
						debugLengthArray.append(ledColor[1])
						debugLengthArray.append(ledColor[2])
				else: # if the sub array will be too big after next iteration, assign to master, and clear sub for new chunk.
					executeArray.append(subExArray)
					subExArray = []
					subExArray.append(ledColor[0])
					subExArray.append(ledColor[1])
					subExArray.append(ledColor[2])
					if(debugCharLength == 1):
						debugLengthArray.append(ledColor[0])
						debugLengthArray.append(ledColor[1])
						debugLengthArray.append(ledColor[2])
				ledCounter += 1
			zigZagCounter = 1
	executeArray.append(subExArray) # append the rest of the values gathered to the master execute array.
	print(len(debugLengthArray))
	for subArray in executeArray:
		print(subArray)
		exec('op("serialin1").sendBytes(%s)'%(str(subArray).strip('[]'))) # converts array to comma separated string that can be executed at once.
	return


I have noticed some of the colors don't match up perfectly to what I see on screen, even with a general gamma correction table, so I will try and get that looking better with some per channel custom gamma tables.
 
Do I just choose different com ports with multiple teensy's running?

Yes, each Teensy will show up as a different COM port.

Do I need to setup anything in the arduino sketch to indicate which one's which?

How you figure out which is which is up to you!

OctoWS2811's VideoDisplay example uses a scheme where the PC queries each Teensy to learn how many LEDs are connected, what their configuration is, and which portion of the overall video image those LEDs are meant to show. Early (never published) versions had mode PC-side configuration, but it needed to be recreated on each PC. Linux assigns the names in the order the cables are plugged in. Windows assigns COM ports on a machine-specific basis, so the results are stable on 1 machine, but vary between computers. Putting the info inside each Teensy with way for the PC to query it made the system much more robust to use with different computers.

Perhaps you're moving towards a similar approach?

There are certainly many possible ways to accomplish these things, each with its own trade-offs.
 
Last edited:
Thanks again! I picked one of those up. I didn't even know that was a thing with usb hubs, good to know.

I've been making decent headway with the teensy / touch designer combo, currently I have a program setup that allows me to map the led panel to a portion of a video stream (in much the same way as laying out uv's on 3d objects) in realtime, visually.
After the translates / rotates / scales are applied the data is translated into that byte stream from earlier and sent to the arduino.

I threw together a 10 minute video on the touch designer side as it is thus far, and a shorter video with out me rambling on about nodes here :D
I'm building some ad hoc led panels from strips tonight to add to the teensy side to see how touch does with more variety in shape and size.
 
Status
Not open for further replies.
Back
Top