Phase distortion/modulation

C

ChrisKelly

Member
Hi folks

I've been working on a project to synthesise cymbal sounds. The idea and process is from a great youtube video where someone models cymbals using a variety of audio plug-ins on their DAW.

One plug-in that I've not been able to replicate is 'Phase Distortion' by KiloHearts. The effect seems to be a type of "self phase modulation" (and not the typical phase distortion used in classic Casio CZ keyboards).

An example of the effect is shown here:-
https://www.desmos.com/calculator/vmeqdw1blt

For a sinewave I imagine you could just scan through a wavetable and have the rate/direction be determined by the current amplitude. I'm trying to figure out how this might be applied to an arbitrary/ever-changing input waveform.

Up until now I've only ever edited existing Audio Objects and the prospect of working on this is a bit daunting, so I would welcome input on how to approach the problem.

I've been using the excellent OpenAudio 32bit library (chipaudette) and there are examples of phase modulation using a sine lookup table. Is it feasible to instead apply the same approach but using previous audioblock data rather than a lookup table?

Thanks
Chris
 
So far, the strategy is to receive an audioblock, scale each sample (add 32768, multiply by AUDIO_BLOCK_SAMPLES and divide by 65536) so that the unipolar amplitude of each sample determines how much phase shift is applied to the incoming audio. Then estimate the 'phase' by interpolating between the nearest samples. This means that low amplitudes correspond to low phase shift (near the start of the audioblock) and high amplitudes correspond to high phase shift. Then copy these values to a buffer to be output.
I've realised I can't do this with just one audioblock so will need a ring buffer to interpolate between the end of one block and the start of the next.

If I receive two audioblocks at a time, do I then just transmit one block (overwritten) and release the other?

How does the rest of the audio chain react when this AudioObject receives two blocks at a time? What happens during the update when other objects work on just one block per update?
This is honestly confusing me!
 
You can’t “receive two audio blocks at a time”. Given a single input, your update() method will get a pointer to one block of 128 samples every 2.9ms, no more and no less*

If you need more than 128 samples to create your effect, you can implement your ring buffer simply by not calling release(), storing the block pointer, and making use of it during some later update(). Obviously you have to release it at some point, or you’ll use up all the audio blocks in the pool (set up using AudioMemory()). One it’s up and running you’d be releasing one block for every one received.

If you don’t have enough valid blocks in your ring buffer to produce output, you’ll have to make up something to transmit(). The simplest thing is to output silence, next is to regard empty slots as silence, and so on. You’ll have to figure out for yourself which option gives the most useful result.

Judging by the link in your first post, you may want to take inspiration from the AudioEffectDelay code. It outputs from multiple fixed-offset taps, but could potentially be tweaked a bit to have a single moving / modulated tap position instead. The tap position will usually be fractional; don’t be tempted to just pick the nearest sample, it’ll sound terrible. A linear interpolation between two samples is easy and cheap to do, but I’ve seen code which uses more samples and polynomial interpolation.

*well, you can get a null pointer - that should be interpreted as a block of zeros
 
Thank you - that makes alot more sense now.
To skip the first block (release a silent block) could I just use a boolean in the .h file

Code: 
AudioEffectPhaseDistortion() : AudioStream(1, inputQueueArray) {
        first_block = true;
        memset(queue, 0, sizeof(queue));
    }

And then in the .cpp file use something like
if (first_block == true)
{
transmit(block);
release(block);
first_block == false;
}
so that the first update is always silent?
 
The AudioStream constructor sets all input queue pointer entries to null, so you don't have to do it.

You'd need to provide more context as to where block is coming from - maybe a call to AudioStream::receive()? If so, it won't necessarily be silent.

The following is a very incomplete skeleton, omitting checks for being unable to allocate txBlock, any other input signals you're handling etc. Note you don't have to transmit() anything - the receiver will automatically get a null (silent) block if you don't. DO NOT release() a null block - for some unknown reason the check for this is commented out in AudioStream, and thus mayhem will ensure if you do it.
Code: 
audioblock_t* block, txBlock = nullptr;

block = receiveReadOnly();
ringBufferPush(block); // must cope with null block, and release() excess blocks
if (first_block)
{
    first_block = false;
}
else
{
    txBlock = allocate();
    computeOutput(txBlock); // compute from ring buffer contents
    transmit(txBlock);
    release(txBlock);
}
 
Finished my first draft. It compiles but when uploaded to the Teensy 4.1, the program doesn't start properly (repeatedly disconnects the USB serial port). Perhaps a memory issue? I don't have any experience with this kind or problem.

To quickly summarise the idea:-
- establish a ring buffer with sufficient audioblocks (based on the depth of the effect and up to a pre-determined max size)
- once the buffer is full enough, iterate through the latest block one sample at a time, and use the amplitude of each sample to determine how far from the midpoint of the buffer to grab a 'new' sample from
- take the 'new' sample and the one next to it, and interpolate to get a more accurate value
- write this interpolated value into the new output block
- transmit the output

Here is my code:-

AudioEffectPhaseDistortion.h
Code: 
#ifndef effect_phase_distortion_h_
#define effect_phase_distortion_h_
#include <Arduino.h>     
#include <AudioStream.h>

#define MAX_BUFFER_QUEUE_SIZE (4128/AUDIO_BLOCK_SAMPLES)

class AudioEffectPhaseDistortion : public AudioStream
{
public:
    AudioEffectPhaseDistortion() : AudioStream(1, inputQueueArray) {
        headindex = 0;
        tailindex = 0;
        maxblocks = 0;
        buffer_ready_flag = false;
        memset(queue, 0, sizeof(queue));
    }
    
    void drive(float drive_amt){
        drive_amount = drive_amt;
        if (drive_amount < 0.0f) drive_amount = 0.0f;
        if (drive_amount > 1.0f) drive_amount = 1.0f;
    }

    void intensity(uint16_t level){
        uint32_t nmax = AUDIO_BLOCK_SAMPLES * (MAX_BUFFER_QUEUE_SIZE -1);
        dist_amount = level;
        if (dist_amount <= 0) dist_amount = 0;
        if (dist_amount > nmax) dist_amount = nmax;
        maxblocks = (dist_amount + (AUDIO_BLOCK_SAMPLES-1)) / AUDIO_BLOCK_SAMPLES + 1;
    };
    
    virtual void update(void);

private:
    uint16_t headindex;
    uint16_t tailindex;
    uint16_t position;
    uint16_t maxblocks;
    uint16_t dist_amount;
    float drive_amount;
    boolean buffer_ready_flag;
    audio_block_t *queue[MAX_BUFFER_QUEUE_SIZE];
    audio_block_t *inputQueueArray[1];

};

#endif

AudioEffectPhaseDistortion.cpp
Code: 
#include <Arduino.h>
#include "effect_phase_distortion.h"

void AudioEffectPhaseDistortion::update(void)
{
    audio_block_t *output;

    uint32_t head, tail;
    int16_t *lower_ptr, *upper_ptr, *sample_ptr;
    int16_t lower_amp, upper_amp, sample_amp, interpolated_amp;
    uint16_t block_index;
    uint32_t midpoint, ring_midpoint, lower_sample, sample_offset;
    float real_number, frac_component, scaled_amp;
            
// ring buffer code - taken from AudioEffectDelay

    head = headindex;
    tail = tailindex;
    if (++head >= maxblocks) head = 0;
    if (buffer_ready_flag = true) ++tail;
    if (head == tail) {
        if (queue[tail] != NULL) release(queue[tail]);
        if (++tail >= maxblocks) tail = 0;
        buffer_ready_flag = true;
    }
    
    queue[head] = receiveReadOnly();
    headindex = head;
    tailindex = tail;

    // end of the ring buffer code

    if (buffer_ready_flag == true)
    {
    // determine the midpoint of the phase-distortion range
        if (dist_amount % 2 == 0) midpoint = dist_amount / 2;
        else midpoint = (dist_amount - 1) / 2;   
    // find the midpoint of the ring buffer
        ring_midpoint = midpoint + (headindex * AUDIO_BLOCK_SAMPLES);
        if ((ring_midpoint + midpoint) > (maxblocks * AUDIO_BLOCK_SAMPLES) // wrap-around the buffer
        {
            ring_midpoint -= (maxblocks * AUDIO_BLOCK_SAMPLES);
        }       

        output = allocate();
    
        for (int i = 0 ; i < AUDIO_BLOCK_SAMPLES; i++)
        {
            sample_ptr = queue[head]->data + i; // take sample from block
            sample_amp = *sample_ptr; // copy value at ptr address to variable
            scaled_amp = sample_amp / 32768.0; // scale to -1.0 to +1.0
            scaled_amp *= drive_amount; // scale by parameter 'drive amount'
            real_number = ring_midpoint + (scaled_amp * midpoint); // calculate distance from midpoint
        // shift real_number to sit within bounds of buffer   
            if (real_number < 0)
            {
                 real_number = (maxblocks * AUDIO_BLOCK_SAMPLES) + ring_midpoint + (scaled_amp * midpoint);
            }
            if (real_number > (maxblocks * AUDIO_BLOCK_SAMPLES))
            {
                real_number -= (maxblocks * AUDIO_BLOCK_SAMPLES);
            }
            lower_sample = floor(real_number); // take the integer part
            frac_component = real_number - lower_sample; // take the fractional part
            
            block_index = lower_sample / AUDIO_BLOCK_SAMPLES; // which block is the sample in
            sample_offset = lower_sample % AUDIO_BLOCK_SAMPLES; // index of sample in audioblock
        
            lower_ptr = queue[block_index]->data + (sample_offset - 1);
            lower_amp = *lower_ptr;
        // check whether the next sample will be in the next audio block in the queue
            if (lower_sample % AUDIO_BLOCK_SAMPLES == 31)
            {
                block_index++;
                if (block_index >= maxblocks) block_index = 0;
                upper_ptr = queue[block_index]->data;
            } else {
                upper_ptr = queue[block_index]->data + (sample_offset);       
            }
            upper_amp = *upper_ptr;
            interpolated_amp = lower_amp + ((upper_amp - lower_amp) * frac_component); // LERP between samples
            output->data[i] = interpolated_amp;
        }

        transmit(output);
        if (output != NULL) release(output);
    }
}

I suspect I've misused pointers somehow but it's not clear to me yet.

I'm using Arduino IDE so if anyone can recommend a better IDE to help de-bug my code I would appreciate it.

Thanks
Chris
 
I finally discovered Crash Report and addr2line and was able to pinpoint an offending pointer.

I decided to re-write the code after all since it didn't look correct anyway.
I'm at the stage now where nothing crashes, but there must be an inherent flaw since the output is silent. My input is a WAVEFORM_SINE object.
I suspect it's something wrong with the way I'm pointing back through the audio queue array to extract values at different indices.

Can I ask whether I'm using pointers correctly -

Code: 
s_ptr = myqueue[sample_block]->data;
s_ptr += sample_offset;
lower_value = *s_ptr;

By doing the above, am I setting the pointer address to the first sample of the block stored at the index 'sample_block'? If so, then I thought that increasing the value of s_ptr will shift along the array by that many samples.
If I've misunderstood it, then I must be moving the pointer by entire blocks at a time.

In the AudioEffectDelay code, the queue looks as though it is built up a whole block at a time.
Later on it seems to handle partial blocks by incrementing pointers in the same way:-
Code: 
if (queue[prev]) {
                end = queue[prev]->data + AUDIO_BLOCK_SAMPLES;
                src = end - offset;
                while (src < end) {
                    *dst++ = *src++; // TODO: optimize
                }
This is from around line 108 in effect_delay.cpp

If anyone can spare the time to give my code a quick check it would really help:-

.cpp code
Code: 
#include <Arduino.h>
#include "effect_phase_distortion.h"

void AudioEffectPhaseDistortion::update(void)
{
    audio_block_t *output;
    uint32_t head, tail, blocks_in_use, count, channel;
    int16_t dist_blocks_reqd, interpolated_amp, real_number, upper_value, lower_value, midpoint_offset, midpoint_block, sample_offset, sample_block;
    int16_t *midpoint_ptr, *s_ptr, *h_ptr, current_sample;
    float sample_shift, frac_component, scaled_amp;
        
// ring buffer code - taken from AudioEffectDelay

    head = head_index;
    tail = tail_index;
    if (++head >= MAX_BUFFER_QUEUE_SIZE) head = 0;
    if (head == tail) {
        if (myqueue[tail] != NULL) release(myqueue[tail]);
        if (++tail >= MAX_BUFFER_QUEUE_SIZE) tail = 0;
    }

    myqueue[head] = receiveReadOnly();
    if (myqueue[head] != NULL) blocks_in_use++;
    else blocks_in_use = 0;
    
    if (blocks_in_use > MAX_BUFFER_QUEUE_SIZE) blocks_in_use = maximumblocks;

    head_index = head;

    // discard unneeded blocks from the queue
    if (head >= tail) {
        count = head - tail;
    } else {
        count = MAX_BUFFER_QUEUE_SIZE + head - tail;
    }
    if (count > maximumblocks) {

        count -= maximumblocks;
        do {
            if (myqueue[tail] != NULL) {
                release(myqueue[tail]);
                myqueue[tail] = NULL;
            }
            if (++tail >= MAX_BUFFER_QUEUE_SIZE) tail = 0;
        } while (--count > 0);
    }
    tail_index = tail;
    
    if (blocks_in_use >= maximumblocks)
    {
    
        midpoint_offset = (dist_amount/2) % AUDIO_BLOCK_SAMPLES;
        if (midpoint_offset > 0) dist_blocks_reqd = (dist_amount/AUDIO_BLOCK_SAMPLES) + 1;
        else dist_blocks_reqd = (dist_amount/AUDIO_BLOCK_SAMPLES);

        midpoint_block = head - dist_blocks_reqd;
        if (midpoint_block < 0) midpoint_block = MAX_BUFFER_QUEUE_SIZE + head - dist_blocks_reqd;
        midpoint_ptr = myqueue[midpoint_block]->data;
        midpoint_ptr += midpoint_offset;
 
        if (myqueue[head]) h_ptr = myqueue[head]->data;
        output = allocate();
        if (!output) return;
    
        for (int i = 0 ; i < AUDIO_BLOCK_SAMPLES ; i++)
        {
            current_sample = *h_ptr;
            scaled_amp = current_sample/32768 * drive_amount; // amplitude of sample * drive_amount (scaling factor set in .h)
            h_ptr++;
            sample_shift = ((float)scaled_amp) * (dist_amount/2); // how many samples to deviate from the midpoint as a float
            if (sample_shift < 0) real_number = (int16_t)ceil(sample_shift); // use this number to traverse the myqueue array
            else real_number = (int16_t)floor(sample_shift); // use this number to traverse the myqueue array
            frac_component = (float)(sample_shift - real_number);
        
            if (real_number == 0) s_ptr = midpoint_ptr; // no offset required - sample is at the midpoint
        
            if (real_number > 0) // we offset towards head
            {
                sample_block = real_number / AUDIO_BLOCK_SAMPLES;
                sample_offset = real_number % AUDIO_BLOCK_SAMPLES;
                sample_block += midpoint_block; // positive block shift towards head
                sample_offset += midpoint_offset;
                if (sample_offset > AUDIO_BLOCK_SAMPLES)
                {
                    sample_offset -= AUDIO_BLOCK_SAMPLES;
                    sample_block++;
                }
                s_ptr = myqueue[sample_block]->data;
                s_ptr += sample_offset;
                lower_value = *s_ptr; // first sample before LERP
            
                if (sample_offset % AUDIO_BLOCK_SAMPLES == 31) // then next sample is first from next block
                {
                    sample_block++;
                    s_ptr = myqueue[sample_block]->data;
                    upper_value = *s_ptr;
                } else {
                    s_ptr++;
                    upper_value = *s_ptr; // the next sample is just one along
                }
            } // end of positive real_number conditions
                    
            if (real_number < 0) // we offset towards tail
            {

                sample_block = (real_number / AUDIO_BLOCK_SAMPLES) - 1;
                real_number *= -1;                    // make real_number +ive before mod
                sample_offset = real_number % AUDIO_BLOCK_SAMPLES;
                real_number *= -1;                    // no need for this line?
                sample_block += midpoint_block; // block shift towards tail (since real_number is negative)
                sample_offset = midpoint_offset - sample_offset;
                if (sample_offset < 0 ) sample_offset += AUDIO_BLOCK_SAMPLES;
                s_ptr = myqueue[sample_block]->data;
                s_ptr += sample_offset;
                lower_value = *s_ptr; // first sample before LERP
                if (sample_offset % AUDIO_BLOCK_SAMPLES == 0) // then next sample is last one from previous block
                {
                    sample_block--;
                    s_ptr = myqueue[sample_block]->data;
                    upper_value = *s_ptr + (AUDIO_BLOCK_SAMPLES - 1);
                } else {
                    s_ptr--;
                    upper_value = *s_ptr; // the next sample is just one along
                }
            } // end of negative real_number conditions

        // LERP

            interpolated_amp = (int16_t)((lower_value * (1.0 - frac_component)) + (upper_value * frac_component));
            output->data[i] = (int16_t)interpolated_amp;
        
        } // end of for output->data loop
    

        if (output) {
            transmit(output);
            release(output);
        }
    } // end of if 'blocks_in_use'

    
    
} // end of update function

.h file
Code: 
#ifndef effect_phase_distortion_h_
#define effect_phase_distortion_h_
#include <Arduino.h>     
#include <AudioStream.h>

#define MAX_BUFFER_QUEUE_SIZE (4096/AUDIO_BLOCK_SAMPLES)

class AudioEffectPhaseDistortion : public AudioStream
{
public:
    AudioEffectPhaseDistortion() : AudioStream(1, inputQueueArray) {
        head_index = 0;
        tail_index = 0;
        maximumblocks = 0;
        memset(myqueue, 0, sizeof(myqueue));
    }
    
    void drive(float drive_amt){
        drive_amount = drive_amt;
        if (drive_amount < 0.0f) drive_amount = 0.0f;
        if (drive_amount > 1.0f) drive_amount = 1.0f;
    }

    void distort(uint16_t distort){

        if (distort < 0) distort = 0;
        dist_amount = distort;
        uint32_t numb_max = AUDIO_BLOCK_SAMPLES * (MAX_BUFFER_QUEUE_SIZE -1);
        if (dist_amount > numb_max) dist_amount = numb_max;
        if (distort % AUDIO_BLOCK_SAMPLES == 0) blocks_calc = distort / AUDIO_BLOCK_SAMPLES;
        else blocks_calc = (distort / AUDIO_BLOCK_SAMPLES) + 1;
        maximumblocks = blocks_calc;
    }



    virtual void update(void);

private:

    uint16_t head_index;
    uint16_t tail_index;
    uint32_t blocks_calc;
    uint16_t maximumblocks;
    uint16_t dist_amount;
    float drive_amount;
    audio_block_t *myqueue[MAX_BUFFER_QUEUE_SIZE];
    audio_block_t *inputQueueArray[1];
};

#endif

thanks
Chris
 
You must log in or register to reply here.
Back
Top