I was working on a pitch shifter for the T3.6 that implements the Ocean's pitch shifting algorithm. It is a spin on the concept of the phase vocoder. The basic process is
1) Use overlapping STFT to get the initial frequency/phase bins.
2) Move each bin from it's original index to a new index based on the pitch scaling factor. For example, for octave up (scale factor 2). This will cause a sinusoid component to be created at the new desired frequency. Scale factors < 1.0 pitch down, > 1.0 pitch up.
newBinIndex = round(oldBinIndex * 2.0f)
3) The shifted sinusoid will have incorrect instantaneous phase so the phase is corrected based on the phase for the original bin, it's frequency and the scaling factor.
4) Perform inverse STFT to get back to time domain, apply a synthesis windowing (hanning) function, then overlap add to the output.
The main difference versus a traditional phase vocoder is you are not achieving pitch shifting by resampling, you are simply scaling the complex sinusoid components directly. Pitch shifting is very difficult, this method has serious drawbacks just like every other method.
Here's why I paused the project.
- The T3.6 is stuck using the old CMSIS library which has poor selection of FFT sizes. E.g. 512 and 2048 where 1024 is probably the best size for this application. My understanding is the T4 will use a much newer version of CMSIS that offers better FFT selection sizes.
- The T3.6 is more than fast enough to perform the FFTs in real time. The problem is the phase correction steps require calculating a lot of cos() and sin() values which even using the CMSIS library is very slow. The T4 has a newer FPU with double precision that I'm hoping will allow far more acceleration of these computations.
- The T3.6 code I had actually working was using > 90% of the T3.6 and the algorithm still need more tweaking (like IIR filtering to enhance tone, etc.).
The T4 should provide more than enough power to implement the Ocean's algorithm with lots of processing power left to spare. I'm looking forward to completing the pitchshifter using the T4 once I get it working on my
TGA Pro.
My development branch for my pitch shfiter can be found in the BALibrary on a branch called 'feature/pitchshifter'. You can find the branch
here. Take a look at the AudioEffectPitchShift class.