Lab 8: Sinusoidal plus Residual Modeling

The Sinusoidal plus Residual Modeling assumes that the sinusoids model the stable partials or harmonics of a sound, which could also be called the deterministic component of the sound. The residual models the noisy part of the sound, which could also be called the stochastic component of the sound. The residual is obtained by subtraction the sinusoidal components from the original sound, which then can be modeled as an stochastic signal.

In this lab we will be subtracting the sinusoidal components in the frequency domain. In order to do that we will have to analyze the sound with a Blackman-Harris 92dB window in order to be able to subtract the main lobe of that window in the spectrum.

To do this lab you can start from the harmonicmodel.m using the functions given in lab-7, converting it to sinesplusresmodel.m

8.1 Sinusoidal subtraction

1. Subtract the complex spectrum of the sines obtained by the function genspecsines (ex: Xsines = genspecsines(ihloc,ihmag,ihphase,M,N,bh92lobe);) from the original complex spectrum at each frame, (ex: Xres = X - Xsines;).

2. Synthesize the residual sound by performing the inverse-fft of the residual spectrum and applying the same synthesis window used in the harmonic model. ex:

fftbuffer = real(ifft(Xres)); % inverse FFT
x1((M+1)/2:M) = fftbuffer(1:(M+1)/2);
x1(1:(M-1)/2) = fftbuffer(N-(M-1)/2+1:N);
res(pin+1:pin+M) = res(pin+1:pin+M) + x1(1:M) .* ws;

8.2 Residual approximation

1. Approximante the magnitude of the residual spectrum by line segments given a step size that will be the number of bins to approximate by a single value. ex:

mXres = abs(Xres(1:N2));
l=0;
resenv = zeros(floor(N2/step), 1);
for i = 1:step:(N2-step);
resenv(l)=sum(mXres(i:i+step))/step;
l=l+1;
end

Synthesize the residual approximation by generating noise with the frequency characteristics of the spectral envelope of the residual.

Generate a new residual magnitude spectrum, mXres, by interpolating the approximation, resenv, to the N2 size.
Generate the phase spectrum, pXres, by filling it with random numbers from 0 to 2pi.
Create the complex spectrum by Xres = mXres.*cos(pXres)+i*mXres.*sin(pXres);

8.3 Tunning the model parameters

Analyze several harmonic sounds with the obtained model. The function should be defined by

function [y, sines, res] = sinesplusresmodel(x, fs, w, N, H, mthreshold, nHarms, minf0, maxf0, maxf0error, maxharmdev, resapproxstep)
% x: input array sound
% fs: sampling rate
% w: window (odd size), ex: blackmanharris(1023)
% N: FFT size, ex: 1024
% H: hop size, ex: 512
% mthreshold: threshold in dB used for peak detection, ex: -20
% nHarms: maximum number of harmonics to find, ex: 40
% minf0: minimum frequency for f0, ex: 80
% maxf0: maximim frequency for f0, ex: 500
% maxf0error: maximum error allowed in the f0 detection, ex: 5
% maxharmdev: maximum deviation allowed in detecting harmonics, as a product, ex: .2
% resapproxstep: step increment in bins in spectral noise approximation, ex: 10

Try different parameters, specially window-size, FFT size, hop-size, magnitude threshold, number of harmonics, maximum f0 error, and maximum harmonic deviation.
Describe the effect of these changes

8.4 Trying another model implementation

You can try a complete implementation of the model by using the sms.m code, which does not perform the residual approximation. Compare the two implementations.