Modify audio on the fly, can do?

N

notahardwareguy

Active member
Background, I have experience with Arduino(s), STM32Fx(s), and ESP32(s), all under the Arduino IDE development environment and have done a number of different types of projects involving I2C, optical sensors, interrupts, etc. but until now never any audio work. I have been playing with both the STM32F411 and ESP32 for reading MP3 files from a micro-SD card and looking at/decoded the data. I want to change that data and then send it on to a DAC (I guess you would call that a custom filter?). I am having some difficulties doing this with those two platforms (mostly supporting the output formats) and so am looking at the Teeney and this audio library as a simpler way to get where I want to go.

I do not know much about this platform and so figured I would up front ask here if I can get "there" from here. For this project I would like to be able to accept a number if possible "music" inputs like SD cards and USB Audio 1 & 2. I would like to push that digital data to an external DAC using SPDIF (near term and possibly I2S and USB Audio 1 & 2 in the future). While I have the data I would like to perform custom filter work on the "raw" numerical information as it comes in and is on its way out.

Can the Teeny and this library get me there? Based on my "playing" with the visual designer it looks like I need(?) a Teeny 4.0 or 4.1. I would like to support at least CD quality audio.

Sorry of this is all "obvious" but I have been banging my head for over a week on working out a few of the interface issues with the other platforms and this looks to be a quicker way to get to "done".

Thanks all.
 
OK, this is working out pretty well, much better than my efforts with the STM32 platform (each according to their abilities). Using the Teensy 4.1, audio board, and audio library, I have been able to framework my effect and get it running off of SD card wave files to the audio board head-phone connection. Tweaking the effect is pretty easy allowing me to try different algorithms.

Now onto the target input/output interfaces. I am looking to do S/PDIF (toslink) for both. Will this do the trick or are there better/cheaper/easier answers? I did read an older forum post(s) where an S/PDIF interface was built using the connector, circuit board, and a few components. It was for an earlier generation of the Teensy and have not come across an update for use with the Teensy 4.1 though.
 
Last edited:
notahardwareguy said:
Now onto the target input/output interfaces. I am looking to do S/PDIF (toslink) for both.
Click to expand...
I've played briefly with S/PDIF on a Teensy 4.1. I received audio generated by a Teensy 3.2, connected electrically (3.2 pin 22 to 4.1 pin 15); and output from pin 14, via a TOSLINK transmitter, to a cheap S/PDIF to analogue converter. I did get a TOSLINK receiver, but haven't yet tried it out, though I've no reason to suppose it won't work. In this process I found and fixed a bug with AudioOutputSPDIF3: the fix got rolled into Teensyduino 1.57, so if you go this route be sure to update.

The TOSLINK devices I got were the Cliff FCR684208T and FCR684208R, which work from 2.7-5.5V, so a good match for a Teensy 4.1. I'm sure others are available (at least in principle - you can never be sure these days...).
 
Did not see that first thread. Did see the board link and then lost that link so thanks for posting it. That board looked very good/interesting. I don't know enough to know if that board is compatible with the Teensy 4.1 but also it only gets me half way there with inexpensive output but I would still need an input answer. The e?ay board I linked to covers in and out for cheap if it is a "good fit". Would like something much more compact (like Paul's board), don't know if I am asking too much (small, good, cheap, and works with 4.1).

I currently have two 4.1s for play/prototyping, one of which I installed the audio board on so I could get right to working on the audio code. Was planning on using the other for S/PDIF in/out dev once the base audio code is completed and I actually found a viable hardware solution for the in/out.

The S/PDIF to I2S and I2S to S/PDIF all on one board looks good on paper for what I want to do but again, don't know enough to be sure it is "my" answer. For that price I guess I will take a chance and the sooner I order, the sooner it gets here. Thanks for your help!
 
Thanks for that info. Ideally I would like both a Toslink input and output solution(s) that have I2S on their other ends and can run on the 4.1 Teensy' voltage. The S/PDIF would get its own (not powered from the Teensy) but at least the same voltage to simplify the power side. The board I found above would need me to drive it at 12v and step down the voltage for the Teensy. Can do but would rather not. Might just have to do that "for now" so that I can keep moving forward with the project and swap out that S/PDIF I/O solution for something more appropriate (size and voltage) as I progress.

Thanks for the transmitter/receiver links. Sounds like you are using them as part of a "tool chain" and not a "single board" answer. I don't mind assembling "stuff" but I am very far from being able to design my own solution for something this complex. I was hoping for an input and output solution(s) that are similar to Paul's in that they run on 3.3v, are small, are "simple", but can also be used with the 4.1. That is an "old" link for the 3.1 and has not been updated in a while so I don't know if it will work with the 4.1 nor have I been able to find a similar solution for S/PDIF input handling. Search goes on! Thanks.
 
Project update and question(s)...

Project update...

OK, I am pretty much all done with pulling this project together and the Teensy platform, Paul, and this forum have been just outstanding in me getting started and going with this project. I have Arduino experience with other micro-controllers but this was my first go with the Teensy platform and I am impressed.

Currently I am using the Teensy 4.1, running both a SPDIF3 input and a SPDIF3 output through the audio library that contains a mix of existing and custom effects. The project also contains an I2C display and a set of pushbuttons for interaction with the system as well as EEPROM storage of configuration values for retention while powered off.

I would still like to add low-power mode/stand-by when no input signal is present, with auto-wake up on a signal being received. I have to dig into that some.

I still have a lot to learn/explore with this platform but I don't think that is too bad for ~5 weeks hobbying around with all of this.

The only real "open" issue I have is (and others more knowledgeable than I may say it does not matter) the AsyncAudioInputSPDIF3 input is working fine but when I use AudioInputSPDIF3 in I get distortion/interference on the output even if I disable all of my manipulations. My goal with this project was to create a device that sits between an audio digital source and a DAC allowing me to manipulate the digital data. I [maybe falsely] assumed that AudioInputSPDIF3 would be preferable due to the resampling done by AsyncAudioInputSPDIF3 thus making the input data less "pure" to the original data from the source. Is this an invalid assumption?

I have been switching between the AudioInputSPDIF3 and AsyncAudioInputSPDIF3 objects. Again AudioInputSPDIF3 results in noticeable distortion and AsyncAudioInputSPDIF3 does not. Am I being overly concerned about the resampling? Is there a way (if it is even needed) to configure the resampling to minimize input changes?

Thanks much!

P.S. added a photo of the current "prototype"...

TeensyProject.jpg
 
Last edited:
Struggling to reproduce this on a Teensy 4.1, using the following code:
Code: 
#include "Audio.h"

// select bertween async and sync S/PDIF input
#define noUSE_ASYNC 

#if defined(USE_ASYNC)
#define AudioInputSPDIF3 AsyncAudioInputSPDIF3
#define sampleRate getInputFrequency
#endif // defined(USE_ASYNC)

// GUItool: begin automatically generated code
AudioInputSPDIF3         spdif3in;       //xy=354,279
AudioRecordQueue         queueL;         //xy=541,219
AudioRecordQueue         queueR; //xy=548,339
AudioOutputSPDIF3        spdif3out;       //xy=573,278

AudioConnection          patchCord1(spdif3in, 0, spdif3out, 0);
AudioConnection          patchCord2(spdif3in, 0, queueL, 0);
AudioConnection          patchCord3(spdif3in, 1, spdif3out, 1);
AudioConnection          patchCord4(spdif3in, 1, queueR, 0);

// GUItool: end automatically generated code


void setup() {
  while (!Serial)
    ;
  
  AudioMemory(10);

  queueL.begin();
  queueR.begin();
}

#define DISCARD(q) while (q.available()) {q.readBuffer(); q.freeBuffer();}

enum state_e {wait,graph} state;
int32_t next;

/*
 * Use Serial Plotter to look at snapshots of audio data
 */
void loop() {
  switch (state)
  {
    case wait:
      DISCARD(queueL);
      DISCARD(queueR);
      if (millis() > next)
      {
        state = graph;
        next = 4;
      }
      break;

    case graph:
      while (queueL.available())
      {
        int16_t* pl,*pr;        
        uint32_t srate = spdif3in.sampleRate();

        pl = queueL.readBuffer();
        pr = queueR.readBuffer();

        for (int i=0;i<128;i++)
          Serial.printf("%d %d %lu\n",*pl++,*pr++,(srate - 44100)*1000);

        queueL.freeBuffer();
        queueR.freeBuffer();

        next--;
      }
      
      if (next <= 0)
      {
        state = wait;
        next = millis() + 1000;
      }
      break;
  }
}

  • I'm using a Teensy 3.2 as a signal generator, so the input is at 44117Hz sample rate which should be a good test
  • You must run the Serial Plotter to get the sketch running: it shows a snapshot of the waveforms every second

Note that the Audio System Design Tool does state in red text that "[The AudioInputSPDIF3] input is incompatible with most other inputs and outputs which run at a speed controlled by Teensy's internal sample rate."

In order to avoid further random guesswork, it would help if you post simple code and wiring to allow someone else to reproduce the issue. If the process of simplifying the code results in a solution, please tell us what it was - future generations will thank you (at least in their minds...).

EDIT: OK, I do have very occasional glitches using synchronous S/PDIF: every 14.57s or so, it appears...
 
Last edited:
Can you try something?

In libraries/Audio/output_spdif3.cpp, change line 266 thus:
Code: 
SPDIF_SCR_TXSEL([COLOR="#FF0000"]1[/COLOR]) |  // 0:off and output 0, 1:Feed-though SPDIFIN, 5:Tx Normal operation
The original code sets SCR_TXSEL to 5 for "normal operation", whereas with this change the Tx clock follows the incoming Rx clock, as far as I can make out. I don't seem to get the glitches any more, and the audio still seems to be be passing through OK.

This will almost certainly fail horribly if you try to use any other audio I/O in combination with it, including I2S or USB. If you need those you'll probably have to fall back to using the AsyncAudioInputSPDIF3 object for input.
 
After a bit of modification and testing, I've put in a pull request to make AudioInputSPDIF3 and AudioOutputSPDIF3 mutually compatible, so long as they're the only audio I/O objects in a system. The PR includes updates to the design tool so the I/O clashes (should) show up correctly, and the info pane says a bit more about what should work together.

I suspect this won't be pulled in any time soon, as Paul has a fair bit of work on Teensyduino 1.58 by the looks of it, so you may need to grab the changes directly if you want to try them out.

Going back to your post #7, as far as I can tell AsyncAudioInputSPDIF3 works very well, though I've not tested it thoroughly (not really qualified to do so!). "Obviously" it's better to avoid resampling if you can, and I think it does take a chunk of CPU time to do it which I'm sure you can put to use making the effects even better...
 
Thanks for all of the replies/info! Much appreciated!

I reviewed a lot of the forum posts on the SPDIF interfaces (software and hardware) and did read in other posts about others making use of AudioInputSPDIF3 and AudioOutputSPDIF3 together so assumed that information was more accurate than the design tool. Gave a quick look at the file changes and yes, I might just pull those three files and give them a try.

Again, thanks for the info and assistance!
 
Hi, just a short remark: If you manage to avoid the resampling and use the incoming clock for the spdif output, you might still encounter problems. Filters and effects normally depend on the sampling rate of the signal and at the algorithms and filters of the audio library it is assumed the the sample rate is 44.1 kHz (AUDIO_SAMPLE_RATE_EXACT resp. AUDIO_SAMPLE_RATE). If the incoming sample rate is not 44.1kHz they won't work as expected.
 
Curious to know how much practical difference this will make? I'd be the first to admit I've no idea about how filters and effects work, but it seems to me that if the sample rate is "a little bit out" due to crystal tolerances etc, the only consequence is that the computations happen a bit faster or slower. A filter corner frequency going from 4410Hz to 4411.7Hz (if fed from a Teensy which can't hit 44100Hz exactly) surely won't be noticeable? And a waveform at 441.17Hz instead of 441.00Hz is only 0.69 cents out of tune, which is imperceptible. It might make a difference for precision engineering or metrology work, but for an effects box it surely won't matter.

Obviously it would be a bit different if fed at 48kHz - not sure if it would work when that far off-target!
 
Ah sorry. I missed the information that the sample rate of the input sample rate will always be 44.1kHz.
 
h4yn0nnym0u5e said:
Can you try something?

In libraries/Audio/output_spdif3.cpp, change line 266 thus:
Code: 
SPDIF_SCR_TXSEL([COLOR="#FF0000"]1[/COLOR]) |  // 0:off and output 0, 1:Feed-though SPDIFIN, 5:Tx Normal operation
The original code sets SCR_TXSEL to 5 for "normal operation", whereas with this change the Tx clock follows the incoming Rx clock, as far as I can make out. I don't seem to get the glitches any more, and the audio still seems to be be passing through OK.

This will almost certainly fail horribly if you try to use any other audio I/O in combination with it, including I2S or USB. If you need those you'll probably have to fall back to using the AsyncAudioInputSPDIF3 object for input.
Click to expand...

OK, finally had a chance to try this out and it does something "strange" for me. When I make that change, the "interference" is resolved BUT none of the sound manipulations seem to be involved at all. I put together a small example to demonstrate. If I make that one edit and load the below code, I hear no effects at all. If I revert back to the original value and compile/load again, I can hear the effect changing the audio but with the additional sound artifacts that got me hear in the first place.

Code: 
// Freeverb - High quality reverb effect

#include <Audio.h>

// GUItool: begin automatically generated code
// Audio Processing Nodes
AudioInputSPDIF3      spdif3_in; //xy=177.5,245
AudioEffectReverb     reverb_right; //xy=480,222.5
AudioEffectReverb     reverb_left; //xy=485,290
AudioOutputSPDIF3     spdif3_out; //xy=748.75,252.5

// Audio Connections (all connections (aka wires or links))
AudioConnection       patchCord1(spdif3_in, 0, reverb_right, 0);
AudioConnection       patchCord2(spdif3_in, 1, reverb_left, 0);
AudioConnection       patchCord3(reverb_right, 0, spdif3_out, 0);
AudioConnection       patchCord4(reverb_left, 0, spdif3_out, 1);
// GUItool: end automatically generated code

void setup( )
{
  AudioMemory(50);

  reverb_right.reverbTime( 2 );
  reverb_left.reverbTime( 3 );
}

void loop( )
{
}

// eof

Did I maybe do something else wrong?

I have not yet had a chance to pull the three files you speak of above but will try to get to that this week. My recent efforts have been greatly impacted by "real life".
 
Right, so after a titanic struggle with the NXP datasheet I may have a thing worth trying! It's not as simple as before, so I think the best option is to pull the relevant branch from my fork of the Audio library. It seems to me to be working, and it may have the benefit that you could use the audio adaptor along with S/PDIF input, without resampling, as everything is synced to the input data rate. Please note, I have not tested this.

If Real Life lets you, please give this a go and let me know how it works out.

Technical note: this change sets IOMUXC_GPR_GPR1 to feed the S/PDIF Rx clock back to the MCLK[3] source of SAI1 (as well as the existing setting to send PLL4 to MCLK[1]); SAI1 is then switched to use its MCLK[3] source if the S/PDIF Rx is in use, as it is when an AudioInputSPDIF3 object is created. The S/PDIF Tx clock is taken as usual from PLL4 / SPDIF_CLOCK_ROOT if no Rx object exists, otherwise it's taken from SAI1. A bit of fiddling with the clock divisors is needed to get the correct clock rate. I've tried to make it immune to the order in which the various objects are created - this is untested at the moment.
 
Will give this a go for sure! Just to verify, I would need to replace the entire audio library with yours at the link, yes? If not what do I need to do? Once you confirm I will apply your code, recompile and test. Thanks much!

Update, OK downloaded/replaced the "stock" audio lib with all of yours and same result. No other changes to my code. Here is a simple test prog:

Code: 
// simplest possible S/PDIF input/output program, test
// https://forum.pjrc.com/threads/58539-Teensy-4-SPDIF-input-ASRC?p=242050&viewfull=1#post242050

#include <SPI.h>
#include <Audio.h>
#include <Wire.h>
#include <SD.h>
#include <SerialFlash.h>

AsyncAudioInputSPDIF3   spdif3In( false, false, 100, 20, 80 );
//AudioInputSPDIF3        spdif3In;       //xy=193,232
AudioOutputSPDIF3       spdif3Out;       //xy=633,316
AudioConnection         patchCord1( spdif3In, 0, spdif3Out, 0 );
AudioConnection         patchCord2( spdif3In, 1, spdif3Out, 1 );

#define LED             13
void setup( )
{
  AudioMemory( 20 );
  
  pinMode( LED, OUTPUT );
}


void loop()
{
  digitalWrite( LED, HIGH );    // turn the LED on (HIGH is the voltage level)
  delay( 200 );                 // wait for a second
  digitalWrite( LED, LOW );     // turn the LED off by making the voltage LOW
  delay( 800 );                 // wait for a second
}

// eof

Only difference is commenting in/out the async and non-async SPDIF in. I was worried that my SPDIF connector might be part of the issue but I don't think that the async sampling would do away with the "static" I hear when using the non-async SPDIF. Again, thanks much for your efforts and let me know if I am/did something wrong to cause this.

Thanks MUCH!
 
Last edited:
For reference, the easiest way to replace the Audio (or any) library is to put it in the "libraries" sub-folder of your sketchbook folder. So my current sketchbook is D:\swd\git\!dynamic, my test audio library is thus in D:\swd\git\!dynamic\libraries\Audio. If you switch verbose compiling on then you get confirmation in the last few lines of output that the replacement is being picked up, such as
Code: 
Multiple libraries were found for "Audio.h"
 Used: D:\swd\git\!dynamic\libraries\Audio
 Not used: D:\swd\arduino-1.8.19\hardware\teensy\avr\libraries\Audio
I may be being dim, but I'm a bit confused! In your sketch as posted, you use the async S/PDIF input, which I haven't re-tested* but should always work, albeit with a CPU overhead penalty. If you comment that out, and the AudioInputSPDIF3 back in, that's what I think I've made work. I torture-tested that by putting the internal sample rate way off 44100Hz, which made an awful racket, and cleared up with the new code enabled. My S/PDIF source is a Teensy 3.2 which runs at 44117Hz.

BUT you say first you get the "same result" with the new code, which I take to mean your "static" hasn't been fixed by it; BUT you ALSO say you "don't think" using async will do away with it, which I think it should if there's only a sample rate mis-match. And it's very kind to thank me "MUCH", but that sort of sounds as if I've fixed something :D.

Maybe an even simpler test at your end would be to output an internally-generated sine wave to the S/PDIF output, and check it sounds glitch-free, just to eliminate everything downstream of your Teensy. As you're "notahardwareguy" I'm guessing a 'scope trace isn't on the cards...

*EDIT: have now re-tested with AsyncAudioInputSPDIF3, and that still works OK. I've also used a separate USB S/PDIF input to my PC, recorded the Teensy output for 1 minute, and it looks and sounds clean.
 
Last edited:
Sorry for the confusion.

The above code was used both ways, with async SPDIF and without. The "with async SPDIF" works as expected and the "with out" causes the static in the output. I used the posted code both ways.

Currently the only SPDIF in/out devices I have are a TV and sound bar so for testing I have been plugging my device between the two. I do have two other T4.1 boards. One has an audio shield installed (this was the first Teensy I acquired). I then got another to do my prototype and finally I got another with a spare audio shield. I guess I am going to have to setup the first one with the audio shield to support SPDIF out, setup the spare to support SPDIF in and an audio shield. I should then be able to play from the one audio shield to SPDIF out, to my prototype, and then to the second audio shield using SPDIF in. That should allow me to use the one audio shield to generate any audio (or wav from SD card) to feed my prototype via SPDIF. I can then either hear the final results on the other audio shield or/and record it to an SD card for "sharing".

That would work, yes?

I do have a "toy" scope as I am trying to outgrow my "handle". It is a cheap FNIRSI-1013D and has been fine for the few things I have needed a scope for. I can give it a go with that once I get the "rig" setup (if the above will work).

The "much thanks" is for your time, effort, and comments. It is appreciated!
 
No problem, it's just easier to make sure I'm not solving a problem you don't have!

You don't need an audio shield to do S/PDIF out, so it may be easiest to use your third Teensy as the source - you also don't need any hardware, just make an electrical connection from S/PDIF out to in. It might also be possible to use a PC to feed the TV with test audio (via HDMI?), not sure. Another test is just to feed the soundbar with a test tone from your prototype. All sorts of possibilities, which may help to pinpoint where the issue is appearing.

I think an audio shield will only co-exist happily with synchronous S/PDIF in if you use my new library. Shield + S/PDIF out should be fine.

The various S/PDIF inputs provide (inconsistently named) functions to tell you the sample rate they're seeing, which may be informative.

Take care with recording to SD card, it's fraught with issues (apart from a new set of SD objects I'm working on! And that still needs a decent card...). You will probably get better results recording via USB to a PC, though again it's worth doing a dry run with a sine wave source to check that glitches aren't being caused by a sample rate mismatch with the USB audio driver.

It's quite possible your scope will be helpful, if you can capture the "static" in a quantifiable way - I'll let you decide if you think it's worth trying. My glitches are very infrequent but repeatable, whereas your description sounds frequent but random, so we may be seeing different things.

Happy to help, it's one of those intriguing problems whose solution may have wider benefits... :D
 
Having discovered my TV has a TOSLINK output, I've just tried my test code using that as a source. Seems to be fine, but I note that its output sample rate is 48kHz so I can foresee definite nasties if the updated library isn't in use.
 
In a new branch of my cores fork I've added a function AudioStream::actual_sample_rate() which returns the sample rate the audio engine is currently running at (uint32_t for T3.x, float for T4.x). You only need to update cores/teensy4/AudioStream.cpp and .h, and similar for /teensy3/. Here's a minimal example for Teensy 3.x, easily adapted to Teensy 4.x:
Code: 
#include "Audio.h"
AudioOutputI2S dummy;

void setup() {
  while (!Serial)
    ;
}

void loop() {
  Serial.printf("Sample rate: %lu\n",AudioStream::actual_sample_rate());
  delay(500);
}
This may help, along with AudioInputSPDIF3::sampleRate() and AsyncAudioInputSPDIF3::getSampleRate(), to see whether there are any sample rate mismatches within the Teensy itself. "Obviously" this shouldn't matter if you're using AsyncAudioInputSPDIF3...
 
Great, will do that update!

To confirm what you said [farther] above, I can take the T4.1 w/audio shield (does not even really need to have the shield) and load a sketch that outputs a fixed frequency sign wave. Connect the SPDIF out of that board to the SPDIF in of another T4.1. On the second T4.1 load a sketch that reads SPDIF in and sends it directly to SPDIF out. Connect my scope to the second board's SPDIF in and out pins and view/compare the results?

Thanks!
 
You must log in or register to reply here.
Back
Top