Class Web Page



1. Y-W and CSS as intial value

Any numerical optimization requires reasonable initial value from preliminary estimators. For AR parameters, Yule-Walker estimator is one candidate for getting reasonable initial value.

Conditional Sum of Squares Estimators is another candidate that works in general.

CSS

With what is called innovations algorithm, log-likelihood function that MLE is trying to minimization can be written as \[ \ell(\phi, \theta) = \ln\Big( S(\phi,\theta)/n \Big) + \frac{1}{n} \sum_{i=1}^n \ln( r_{j-1} ) \] where \[ S(\phi,\theta) = \sum_{i=1}^n (X_j - \hat X_j)^2 / r_{j-1}, \] and \(\hat \sigma^2 = S(\phi,\theta)/n\).

Conditional Sum of Squares Estimator minimizes \(S(\phi,\theta)\) only. This serves as good starting point for MLE.

Approximation in auto.arima()

When calculating AIC and AICc, uses CSS value to make the computation faster. Use option to turn it off.

2. Ex: Copper MLE and initial values

    D  = read.csv("https://nmimoto.github.io/datasets/copper.csv")
    D1 = ts(D[,2], start=1)    #- extract only second column as time series
    plot(D1, type="o")

    library(forecast)

    Arima(D1, order=c(1,0,3))   # Default method is CSS-ML
## Series: D1 
## ARIMA(1,0,3) with non-zero mean 
## 
## Coefficients:
##          ar1      ma1      ma2      ma3    mean
##       0.8695  -0.0925  -0.2958  -0.1809  1.0607
## s.e.  0.0865   0.1129   0.0921   0.0751  0.1590
## 
## sigma^2 = 0.4809:  log likelihood = -205.24
## AIC=422.47   AICc=422.91   BIC=442.17
    Arima(D1, order=c(1,0,3), method="CSS")    # See what CSS gives
## Series: D1 
## ARIMA(1,0,3) with non-zero mean 
## 
## Coefficients:
##          ar1      ma1      ma2      ma3    mean
##       0.8570  -0.0780  -0.2870  -0.1784  1.1006
## s.e.  0.0889   0.1146   0.0916   0.0738  0.1577
## 
## sigma^2 = 0.479:  log likelihood = -205
    # perform MLE with different initial value
    Arima(D1, order=c(1,0,3), method="ML", init=c(.6, -.1, -.3, .2, 20))
## Series: D1 
## ARIMA(1,0,3) with non-zero mean 
## 
## Coefficients:
## Warning in sqrt(diag(x$var.coef)): NaNs produced
##          ar1      ma1      ma2     ma3     mean
##       0.9998  -0.3018  -0.3959  0.0291  19.9009
## s.e.     NaN   0.0738   0.0618  0.0741      NaN
## 
## sigma^2 = 0.5394:  log likelihood = -219.19
## AIC=450.38   AICc=450.82   BIC=470.08
    # this gives error
    # Arima(D1, order=c(1,0,3), method="ML", init=c(.8, -.5, -.3, .2, 2))  

    Arima(D1, order=c(1,0,3), method="ML", init=c(.7, -.5, -.3, .2, 2))
## Series: D1 
## ARIMA(1,0,3) with non-zero mean 
## 
## Coefficients:
##          ar1      ma1      ma2      ma3    mean
##       0.8695  -0.0926  -0.2958  -0.1809  1.0606
## s.e.  0.0865   0.1129   0.0921   0.0751  0.1590
## 
## sigma^2 = 0.4809:  log likelihood = -205.24
## AIC=422.47   AICc=422.91   BIC=442.17



Summary

  • Since MLE is computed by numerial optimization, it needs good starting point. CSS is used as starting point in and by default.

  • If MLE gives error, try changing the initial value (example code abve.)