Nonparametric and parametric methods of spectral analysis

. Spectral Analysis is one of the most important methods in signal processing. In practical application, it is critical to discuss the power spectral density estimation of finite data sampled from some stationary time series. A spectral estimator is expected to have good statistical properties such as consistency, high resolution and small variance. For one spectral estimation method, there exists a trade-off between high resolution and small variance. The paper provides a comparison of several popular spectral methods from both theoretical properties and practical applications. We first address several basic nonparametric methods, whose statistical characters are analysed. Then we explain the connections and differences between temporal windowing and lag windowing. Thereafter, the confidence intervals of both windows are given and used to evaluate the estimated results. Besides, several different parametric estimation methods of autoregressive time series are compared, and whose properties and effects are also introduced. Building on our understanding of these studies, we then apply parametric and nonparametric spectral estimation methods on the data of ocean surface wave height.


Introduction
Spectral analysis plays an important role in statistical signal processing and in estimation and detection of random signals. It is an important direction in the field of spectral analysis to develop a spectral density estimator with low variance and high resolution. Many methods have been proposed, which are divided into nonparametric and parametric methods roughly.
The nonparametric methods include conventional periodogram, correlogram, and temporal windowing and lag windowing etc. The temporal and lag windows, which can optimize the estimator and reduce variance, are discussed in [1][2][3][4]. The periodogram has many variants, such as Daniell method [5], Welch method [6], Bartlett method [7,8], Black-Tukey method [9] and so on. The Parametric methods are mainly divided into rational spectrum estimation and line spectrum estimation. In this paper, we focus on the rational spectrum estimation [10,11]. According to the form of rational transfer functions, the researchers use AR, MA, ARMA time series models to describe white noisedriven systems. As the Yule-Walker (Y-W) equation describes the relationship between correlation functions and parameters [12], the parametric estimation methods based on Y-W equation are widely used and extended. Different iterative algorithms, such as Levinson-Durbin (L-D) algorithm [13,14], Delsarte-Genin algorithm [15], and maximum likelihood (ML) algorithm [16] are proposed for the study of the numerical solution for the Y-W equation.
In this paper, the nonparametric estimation and parametric estimation methods are introduced and discussed briefly. Nonparametric methods including periodogram, correlogram, temporal and lag windowing are analysed and compared based on experimental results. The progressive relationship among conventional periodogram, temporal windowing and correlogram is explained. The parametric methods are about rational spectrum estimation with AR time series model driven by white noise. Three different solving algorithms including L-D algorithm, Burg algorithm and ML algorithm are discussed and applied into ocean surface wave height data for comparison. Based on power spectral density (PSD) estimation results of experimental data, the efficiency and reliability of each method are compared and analysed. This paper is divided into five sections. Section II introduces the basic principles of nonparametric spectral methods. Section III summarizes a variety of parametric methods and analyze the statistical property of each algorithm. In section IV, we apply parametric and nonparametric spectral estimation methods on the data of ocean surface wave height and compare the estimated result of each method. Finally, in section V a short conclusion is given.

Foreword
The nonparametric spectral analysis method [17] refers to the method of estimating the spectral density of a random signal without pre-parameter modeling. This method is based on the definition of PSD, and the relationship between the PSD ( ) S f and the autocorrelation sequence ( ) s τ as a Fourier transform pair as shown in below,

The periodogram method
We assume that the sampling time interval is t ∆ , which satisfies the Nyquist condition. Inspired by the definition of power spectral density, we can define spectral density estimation of time series t X , 1, , f is the estimation of PSD as a distribution of frequency f . This method is named as periodogram.
It can be inferred that the estimator satisfies the following relationship is the empirical estimation of autocorrelation sequence, and τ is the lag value. This relationship shows that we can estimate the autocorrelation sequence at first and then obtain the estimation of PSD by discrete time Fourier transform.

Temporal windowing method
To evaluate the periodogram estimator, we first perform an offset analysis and get the expectation of ( ) ( ) is a Fejer kernel. The equation shows that the expectation of the estimator is actually the convolution of the ideal spectral density with a Fejer kernel. The convolution leads to extension and fluctuation resulting in low resolution and high variance.
In order to optimize the estimation, our primary goal is to make the convolution kernel as close as possible to the Dirac function ( ) The confidence interval of the temporal window is analysed. Based on the asymptotic distribution theory of spectral estimation, when N → ∞ , the PSD estimation 2 21 0 log 10 log ,10 log 10 log , 1 In dB, the confidence interval width is only related to b p , and is independent of the shape of the spectral density. The width is

Smoothing method -Lag windowing
To further smooth the estimated ( ) ( ) where ( ) lw s τ is defined as the estimation of autocorrelation sequence after lag windowing, as follows This method is also called smoothing or correlogram. Like temporal windowing, the confidence interval of lag windowing is also analyzed. As the estimator with lag window is actually a local smoothing filter for the temporal window estimator, in accordance with the asymptotic result of the temporal windowing estimator, we assume that 10 10 10 101 0 log 10 log ,10 log 10 log 1 (2.11) Several common used lag windows are briefly summarized in Table 1

Foreword
The rational parameter method treats the signal as a random signal, which is the output of white noise passing through a causal linear time-invariant system. Therefore, the white noise and the random signal satisfy the differential equation corresponding to the rational transfer function. The differential equation reflects that the random signal at current moment is influenced by the previous random signal and noise. The coefficients of the differential equation characterize the PSD of the time series. Therefore the spectral estimation based on the rational parametric method is equivalent to estimation of the coefficients which characterize the time series. We use the AR(p) model with an order of p which is large enough to estimate any continuous spectral density. Y-W method, Burg method and the ML method are applied to estimate the coefficients of AR(p) time series.

Basic principle
The parameter coefficients and autocorrelation functions of the AR(p) time series model satisfy the Y-W equation [12]. Therefore, the estimation of the parameters can be transformed into the estimation of the autocorrelation sequence and solving the Y-W equation. An AR(p) time series { } t X satisfies the differential equation

Levinson-Durbin iterative estimation algorithm
For a stationary time series, we find that the parameters of the Y-W equation satisfying the AR(p) time series are the parameters of the best linear estimator when the autocorrelation sequence { } , 1, , s p τ τ =  is known. This shows that for a stationary time series, we can determine the best estimate from previous k moments to the current moment only by the first k values of the autocorrelation sequence. This also shows that the Y-W equation and its solution can be used to characterize the relationship between parameters of the k -step optimal linear estimator and the auto-correlation sequences of any stationary sequence.
The L-D iterative algorithm are summarized the specific steps in below.
(iv) Calculate the kth order mean square error k P by ( ) (v) Repeat steps ii-iv until , ,1

Burg estimation algorithm
The Burg method is based on minimizing the sum of forward and backward prediction errors of linear predictor to estimate the p Φ iterately. For a model of order 1 k − , the estimator has an estimation error ( ) where ⋅  and ⋅  are denoted as forward ( k t N ≤ ≤ ) and backward ( 1 1 k t N + ≤ ≤ + ) operations respectively. To find the optimal linear estimator, we define the mean square error of the sample as equation ( ) The optimized ,k k φ can be obtained by

Maximum likelihood estimation
The ML estimators are defined to be those values of the parameters that maximize the likelihood function or minimize the log likelihood function equivalently.
For the zero-mean Gaussian AR(p) time series { } t X , the log ML function can be expressed as

Brief introduction to data
In order to compare the performances and characteristics from the practical application, nonparametric and parametric estimation methods are used to estimate the PSD of ocean surface wave height. We consider the wave height model as a stationary time series and estimate its PSD using the methods we have introduced. Fig.1 shows the collected data of ocean surface wave height [18], the sampling frequency is 4Hz, and the total sampling time is 512 seconds.  Firstly, we use the classical periodogram method to estimate the data. Fig.2 draws the PSD estimation amplitude in Nyquist frequency. The black vertical line in the figure represents 95% confidence interval width in the approximate distribution. The estimated spectral density has a peak at 0.16Hz, and has a strong fluctuation rate at all frequencies.  Combining the Slepian temporal window and lag window, we obtain a relatively smooth spectral density curve with the PSD fluctuation further reduced. The results of four types of lag windows including the Parzen window, Daniell window, Bartlett-Priestley window and Gaussian window are demonstrated in Fig.4 to Fig.7. While remaining the black vertical line, we present the confidence interval for each frequency point as an orange shadow.   4 shows the result of Parzen lag window with width m=150. The estimated PSD is effectively smoothed and the spectral structure is maintained. The results in Fig.5, Fig.6 and Fig.7 shows the estimated results of Daniell, Bartlett-Priestley and Gaussian lag window, respectively. These results basically follow the trend of the spectrum, but have a different degree of loss of the details, due to the strong smoothing effect.

Experimental results using parametric methods
We use Y-W equation with three algorithms to estimate the spectral density of wave height. The first step is to determine the order p. The estimated results of the L-D (p=27) and the Burg (p=5) algorithm are shown in Fig.8. Obviously, the performance of the Burg algorithm is significantly better than the L-D. Unlike the result of L-D algorithm, the result of Burg algorithm avoids serious bias and leakage phenomenon. However, it only shows good estimation in low frequency mainly because of the low model order. We increase the order of the AR model to 27. Fig.9 shows a comparison of the estimation results of the Burg and the ML. With the increase of the order, the spectral density estimation preserves the peak information and avoids serious offset phenomenon more effectively. However, the estimated spectral density arises a false peak in ML method (the figure is not shown). When the order is reduced by one, the pseudo frequency peak in ML AR(26) has been eliminated, and we get almost the same result with the Burg AR(27), shown in Fig.9.

Conclusions
The paper introduces non-parametric and parametric methods of PSD estimation and the statistical properties of all methods are analysed. The conventional periodogram are innovatively improved by integrating with temporal window and lag window further. The latter is named correlogram as well.
The non-parametric and parametric estimation methods are compared with the spectral estimation experiments of ocean surface wave height. The estimation results of all methods are compared and analysed qualitatively and quantitatively. To evaluate each estimation method, there are two important indicators. One is the smoothness of the spectral density curve and the other is the preservation of the structure information, such as the trend of the curve, the position and shape of the peak and valley. There is a trade-off between the smoothness and its preservation of the structure information. In the actual estimation, the corresponding estimator can be selected according to the requirement, so as to achieve the desired smoothness of the estimation results and the amount of remained structure information.