Audio Library for Linear Timecode (LTC)?

[PaulStoffregen]

Agreed, even Teensy 3.2 probably has enough CPU power for this task.

But the real question is whether the project is even useful with approx 6 ms latency (using the default block size). That's about 1/5 of a 30 Hz video frame.

[vjmuzik]

This line from Wikipedia suggests that latency is well within spec.

In general, it is not possible to know the linear timecode (LTC) of the current frame until the frame has already gone by, by which time it is too late to make an edit.

[Frank B]

Came home from work, edited a few lines and have the BMC decoding now working:


you can see the sync-word nicely - here, it's in the middle, because the sync-detection has still to be written.
The most "complicated" is done.. the remaining rest will be easy.

Now to decode the bitstream.

[Frank B]

..done How looks that?



Ok, current status is:
PC plays the wav-file (Thanks to defragster!!) , a Teensy 3.6 is connected via usb-audio.
The code can decode backwards played files, and adapts to different speeds or (in theory, untested) fast-forward or fast-rewind.
It's a "ino" now (which displays the timecode above), and I have to convert it to a audio-library object.
I'll do this in the next days.

I did not use any existing code. License will be MIT.

Maybe I do encoding, too (after you have tested the decoder)

[Frank B]

Tested now with T3.6 @ 24 MHZ. Still works flawlessly, so yes, a 3.2 will be enough

[defragster]

..done How looks that?

...

Ok, current status is:
PC plays the wav-file (Thanks to defragster!!) , a Teensy 3.6 is connected via usb-audio.
The code can decode backwards played files, and adapts to different speeds or (in theory, untested) fast-forward or fast-rewind.
It's a "ino" now (which displays the timecode above), and I have to convert it to a audio-library object.
I'll do this in the next days.

I did not use any existing code. License will be MIT.

Maybe I do encoding, too (after you have tested the decoder)

FrankB back in action and doing impressive good work in short order! Glad to help - just a simple BING search and the [BEST] link was the first result!

[tomfrisch]

Wow Frank. I'm super impressed! You made it seem easy.

I'm wanting to make a minimal parts count camera sync device that would jam sync to the incoming LTC signal at the start of the day, then output the same signal for the rest of the day to each camera the device is attached to.


Questions for you:
Will this work with sampling audio from onboard ADC inputs instead of usb-audio?

Does the default buffer size mean a constant ~6ms offset from the received LTC timestamp until teensy decodes it? Or does it vary within that window? I'm assuming you are filling the input buffer and then decoding the timecode from the signal, so when you are done, your decoded data is in at that point delayed by the length of the buffer (plus any processing overhead that is bigger than the buffer).

If I was using RTC with a cystal, TXCO or GPS, would it make sense to timestamp the beginning of the sample buffer, and then (assuming LTC is constant and not speeding or slowing) use that to cancel out the time offset from the buffer length?


thanks very much.

-tom

[Frank B]

Hi Tom,
is "Frisch" a German name?

Yes, it will be a normal part of the Teensy Audio Library, and you can use any input which is supported by the library.

The latency is constant between frames, but will vary if it has to re-sync. This initial latency is not predictable. In additon, the new new data are (of course) valid after all 80 Bits are received. So you have to substract this time - which varys with the fps.
Not sure about your 3rd question.. i guess you need a frame-start (pin-toggle?) signal to calculate the latency.

[tomfrisch]

Ah, I think i understand. The latency is constant, but the amount is set at initial sync and is unpredictable because we don't know where in the 80bits of data for each timestamp the audio stream happens to be starting from?

I can't wait to play around with the code and see how it works.


-Tom

p.s. my last name is German, but family was originally named Smith and it somehow got changed to Frisch when they emigrated to US. We used to joke that the customs officer must have gotten "fresh" with our ancestors.

[Frank B]

Ok, i've uploaded a first version here:

https://github.com/FrankBoesing/LinearTimecode-Decoder

The usage is very easy. Here is the example-code:

/* Linear Timecode for Audio Library for Teensy 3.x / 4.x

   USB-Audio Example

   Copyright (c) 2019, Frank Bösing, f.boesing (at) gmx.de

   Please support PJRC's efforts to develop open source software by purchasing 
   Teensy or other PJRC products.

   Permission is hereby granted, free of charge, to any person obtaining a copy
   of this software and associated documentation files (the "Software"), to deal
   in the Software without restriction, including without limitation the rights
   to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
   copies of the Software, and to permit persons to whom the Software is
   furnished to do so, subject to the following conditions:

   The above copyright notice, development funding notice, and this permission
   notice shall be included in all copies or substantial portions of the Software.

   THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
   IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
   FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
   AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
   LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
   OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
   THE SOFTWARE.
*/

/*

 https://forum.pjrc.com/threads/41584-Audio-Library-for-Linear-Timecode-(LTC)

 LTC example audio at: https://www.youtube.com/watch?v=uzje8fDyrgg
  or : http://elteesee.pehrhovey.net/

 Howto: Set USB-Type to USB-Audio, on PC: Play LTC Audio and set audio-output to use Teensy as audio-device.
 
*/

#include <Audio.h>
#include <analyze_ltc.h>

// GUItool: begin automatically generated code
AudioInputUSB            usb1;           //xy=386,450
AudioAnalyzeLTC          ltc1;         //xy=556,396
AudioOutputI2S           i2s1;           //xy=556,469
AudioConnection          patchCord1(usb1, 0, i2s1, 0);
AudioConnection          patchCord2(usb1, 0, ltc1, 0);
AudioConnection          patchCord3(usb1, 1, i2s1, 1);
// GUItool: end automatically generated code

ltcframe_t ltcframe;

void setup() {
  AudioMemory(12);
}

void loop() {
  if (ltc1.available()) {
    ltcframe = ltc1.read();
    Serial.printf("%02d:%02d:%02d.%02d\n", ltc1.hour(&ltcframe), ltc1.minute(&ltcframe), ltc1.second(&ltcframe), ltc1.frame(&ltcframe));  
  }
}

Currently, this is the only documentation.
If available() returns true, a new frame is available.
read() returns all 80Bits of the frame in the following struct:

struct ltcframe_t {
     uint64_t data;
     uint16_t sync;
};

The additional functions hour(), minute(), second(), frame() decode the frame.
If you need more, for the userdata or flags, feel free to add it Pullrequests are welcome.
I'm thinking of adding a function that returns micros() of the last received bit. Would that be useful?

Have fun, and PLEASE: Test it!
Frank.

[wcalvert]

Clever implementation, great job Frank.

[defragster]

Clever implementation, great job Frank.

I was going to say the same thing after looking at the code - except I was going to add sarcasm and suggest for SO LITTLE CODE - I can't believe it took so long for him to get that posted … nearly 24 hours ?

[tomfrisch]

Well, I'm excited. I'll be testing with several devices that generate LTC in the next week or so to see if it successfully can read the timecode. (just need to schedule some time with a guy who owns the equipment). I'll try various framerates. I don't think I have a way to test backwards or ff/rew that a tape-based system would generate, so will leave that up to others who are interested in some legacy application.

I think returning micros() of last the received bit would be a good start to helping set the proper latency compensation values.

[tomfrisch]

Well, I'm excited. I'll be testing with several devices that generate LTC in the next week or so to see if it successfully can read the timecode. (just need to schedule some time with a guy who owns the equipment). I'll try various framerates. I don't think I have a way to test backwards or ff/rew that a tape-based system would generate, so will leave that up to others who are interested in some legacy application.

I think returning micros() of last the received bit would be a good start to helping set the proper latency compensation values.

I did a bunch of searching for software on the PC that will read an incoming stream of audio and pull LTC from it, but none of them worked on my windows 10 machine. I bet it's because they used DirectX that isn't supported the same way in Win10. But who knows. It's a bit of a niche field, so am not too surprised.

I downloaded demos and tried the following:
LTCreader https://www.videotoolshed.com/produc...aderconverter/
Timecode Reader http://www.al-ga.jp/
TCode http://timecodesoftware.com/tcode

[Frank B]

I have added some aux functions and the timestamp.
Since nobody is already using the lib, I thought I could modify the API a bit.
The return of read() changed, it is now the following struct:

struct ltcframe_t {
  uint64_t data;
  uint16_t sync;
  uint32_t timestampfirstedge;
};

The timestamp is the micros() of the first edge of the new frame.
I noticed - if I calc the framerate from the timestamps, it's a little bit too fast. I don't know the reason for this.. any ideas? Or, a bug in my code? Or perhaps a problem on the PC side (Playing the audio too fast? or weird USB-Timing?)

Here are the aux. functions - self-explanatory(?):

    //Auxiliary functions:

    int hour(ltcframe_t * ltc)   {
      return 10 * ((int) (ltc->data >> 56) & 0x03)  + ((int) (ltc->data >> 48) & 0x0f);
    }
    int minute(ltcframe_t * ltc) {
      return 10 * ((int) (ltc->data >> 40) & 0x07)  + ((int) (ltc->data >> 32) & 0x0f);
    }
    int second(ltcframe_t * ltc) {
      return 10 * ((int) (ltc->data >> 24) & 0x07)  + ((int) (ltc->data >> 16) & 0x0f);
    }
    int frame(ltcframe_t * ltc)  {
      return 10 * ((int) (ltc->data >>  8) & 0x03)  + ((int) (ltc->data >>  0) & 0x0f);
    }
    bool bit10(ltcframe_t * ltc) {
      return (int) (ltc->data >> 10) & 0x01;
    }
    bool bit11(ltcframe_t * ltc) {
      return (int) (ltc->data >> 11) & 0x01;
    }
    bool bit27(ltcframe_t * ltc) {
      return (int) (ltc->data >> 27) & 0x01;
    }
    bool bit43(ltcframe_t * ltc) {
      return (int) (ltc->data >> 43) & 0x01;
    }
    bool bit58(ltcframe_t * ltc) {
      return (int) (ltc->data >> 58) & 0x01;
    }
    bool bit59(ltcframe_t * ltc) {
      return (int) (ltc->data >> 59) & 0x01;
    }

    uint32_t userdata(ltcframe_t * ltc) {
      return  ((int) (ltc->data >>  4) & 0x0000000fUL) | ((int) (ltc->data >>  8) & 0x000000f0UL) |
              ((int) (ltc->data >> 12) & 0x00000f00UL) | ((int) (ltc->data >> 16) & 0x0000f000UL) |
              ((int) (ltc->data >> 20) & 0x000f0000UL) | ((int) (ltc->data >> 24) & 0x00f00000UL) |
              ((int) (ltc->data >> 28) & 0x0f000000UL) | ((int) (ltc->data >> 32) & 0xf0000000UL);
    }

I'm going to pause it now until I get some feedback.

[tomfrisch]

I’ve got parts on order for testing. Hoping to get it today or tomorrow since Paul and I live in the same town.

[Frank B]

If you ever meet Robin or Paul, please buy them a cup of coffee or a beer. I'll give you the money back

[vjmuzik]

Using your example program, slightly modified to use the ADC on a Teensy 3.6 here's the results of the difference in time between my generator and what the serial monitor shows in slow motion.

[Frank B]

Using your example program, slightly modified to use the ADC on a Teensy 3.6 here's the results of the difference in time between my generator and what the serial monitor shows in slow motion.

Wow, cool
Well, at least it works.
There is a difference of more than 3 frames.
This is not easily explained to me because the code does not add any additional latency. The library is responsible for a maximum of 6 milliseconds. Where does the rest come from? USB data transfer to the PC? The Arduino serial monitor?
We'd have to use a different display to test it..

A big "thank you" for the test! - So, the "automatic volume adaption" works.

[vjmuzik]

I have a couple of SPI displays, I could try one of those.

[Frank B]

would be great. The spi will add a small delay, too, so make it fast, please

[Frank B]

..or just a led that flashes when the frame-number is ==0

[vjmuzik]

I did both and they both seem to be behind by one frame, which I believe is within spec if my hardware only starts to transmit the current frame when it is being displayed like I would expect it to. Any receiving devices should be behind one frame since by the time it receives the whole 80 bit frame the current frame would have already passed on to the next on. I'm pretty sure in Pro Tools you can set a latency offset for this hardware since it's expected to be off it was being generated by a tape machine, so all things considered I believe the LTC reader is complete.
With display:

With LED, on on frame 0, off on frame 15:

[Frank B]

OK
Thank you

[tomfrisch]

While I haven't yet got running with my Teensy, I was able to get some windows software working on my Win7 PC that can both read LTC from audio input, and also send LTC out the audio output. https://www.videotoolshed.com/produc...aderconverter/

vjmuzik- Did you have to do anything special to change the input from usb to ADC?

Frank- what is the volume auto leveler you referenced to in your last post? Is that in your code already?

-Tom