Spectral envelop

Let sigma(theta) denote a spectral measure of the stochastic process y_t, the envelop of maximal spectral power is defined as

 rho(theta) := sup_{sigma} left{lim_{epsilon rightarrow 0} (sigma(theta+epsilon)-sigma(theta-epsilon)) ~mid~ dsigma geq0, R=frac{1}{2pi} int_{-pi}^{pi} G(e^{itheta})dsigma(theta) G(e^{itheta})^* right},

where R represents the state covariance and G(z) is the input-to-state filter. In other words, rho(theta) represents the maximal spectral “mass” located at theta which is consistent with the covariance matrix. It can also be shown that

 rho(theta) =maxleft{ rho ~mid~ R-rho G(e^{itheta})G(e^{itheta})^* geq 0right},

where G(e^{itheta})G(e^{itheta})^* represents the state covariance for the sinusoidal input e^{iktheta}. The optimal solution is

 rho(theta)=frac{1}{G(e^{itheta})^* R^{-1}G(e^{itheta})},

and implemented in envlp.m. This generalizes the Capon spectral estimation method which applies to R being Toeplitz matrix and G(z) the tapped delay line transfer function. The Capon method may be motivated by a noise suppression formulation aimed at antenna array applications [P. Stoica, R.L. Moses, 1997].

For real valued processes with a symmetric spectrum with respect to the origin, a symmetrized version of the spectral envelop [1] can be similarly obtained:

 rho_{mbox{real}}(theta)=maxleft{rho ~mid~ R-frac{rho}{2} left(G(e^{itheta})G(e^{itheta})^*+G(e^{-itheta})G(e^{-itheta})^*  right)right}.
% spectral envelop
rhohalf = envlp(R,A,B,th);
rho = rhohalf.^2;