Class Web Page



1. SARIMA model

Suppose there’s an annual correlation on monthly data?

(i.e. Aug 2015 will depend on Aug 2014, Aug 2013, and so on)

Jan 2015 will depend on Jan 2014, Jan 2013, and so on…

That’s autocorrelation with lag 12.

Consider MA(12) with only one coefficient, \(\theta_{12} = \Theta\) \[ X_t = \epsilon_t - \Theta e_{t-12}. \]

Calculate ACVF, \[\begin{align*} \gamma(1) \hspace{3mm} &= \hspace{3mm} \mbox{Cov}(X_{t+1}, X_t) \hspace{3mm} = \hspace{3mm} \mbox{Cov}\Big(e_{t+1}-\Theta e_{t-11}, \hspace{3mm} \epsilon_t-\Theta e_{t-12}\Big) = 0 \\\\ \gamma(12) \hspace{3mm} &= \hspace{3mm} \mbox{Cov}(X_{t+12}, X_t) \hspace{3mm} = \hspace{3mm} \mbox{Cov}\Big(e_{t+12}-\Theta e_{t}, \hspace{3mm} \epsilon_t-\Theta e_{t-12}\Big) = - \Theta \sigma^2 . \end{align*}\]

There’s correlation only at lag 12.

Seasonal MA(Q) with \(s=12\)

\[\begin{align*} X_t \hspace{3mm} &= \hspace{3mm} \epsilon_t - \Theta_1 e_{t-12} \hspace{30mm} sMA(Q=1) \hspace{5mm} \mbox{ (s=12)} \\\\ X_t \hspace{3mm} &= \hspace{3mm} \epsilon_t - \Theta_1 e_{t-12} - \Theta_2 e_{t-24} \hspace{15mm} sMA(Q=2) \hspace{5mm} \mbox{ (s=12)} \end{align*}\] \(\gamma(h)\) will be zero except at las \(s\), \(2s\), \(3s\), up to \(Qs\).

Representation with B

sMA(1) s=12 \[\begin{align*} X_t &= \hspace{3mm} \epsilon_t - \Theta_1 e_{t-12}\\\\ &= \hspace{3mm} \underbrace{ (1-\Theta_1 B^{12}) }_{\mbox{seasonal char poly}} \, \epsilon_t \end{align*}\]

sMA(2) s=12 \[\begin{align*} X_t &= \hspace{3mm} \epsilon_t - \Theta_1 e_{t-12} - \Theta_2 e_{t-24} \\\\ &= \hspace{3mm} \underbrace{ (1-\Theta_1 B^{12} - \Theta_2 B^{24}) }_{\mbox{seasonal char poly}} \, \epsilon_t \end{align*}\]

Seasonal AR(P) with \(s=12\)

\[ X_t \hspace{3mm} = \hspace{3mm} \Phi_1 X_{t-s} + \epsilon_t \hspace{30mm} \mbox{sAR(P=1)} \\\\ X_t \hspace{3mm} = \hspace{3mm} \Phi_1 X_{t-s} + \Phi_2 X_{t-2s} + \epsilon_t \hspace{15mm} \mbox{sAR(P=2)} \]

sAR(1) period 12 \[ X_t = \Phi_1 X_{t-12} + \epsilon_t \]


\[\begin{align*} \gamma(0) &= \hspace{3mm} E(X_t, X_t) \hspace{3mm} = \hspace{3mm} E[X_t \cdot (\Phi_1 X_{t-12} + \epsilon_t)] \\ \\ &= \hspace{3mm} \Phi_1 \, \gamma(12) + E(X_t, \epsilon_t) \\ \\ &= \hspace{3mm} \Phi_1 \, \gamma(12) + E[(\Phi_1 X_{t-12} + \epsilon_t) \cdot \epsilon_t] \\ \\ &= \hspace{3mm} \Phi_1 \, \gamma(12) + \sigma^2 \end{align*}\]


For \(k \geq 1\), \[ \gamma(k) \hspace{3mm} = \hspace{3mm} E(X_{t-k} \, X_t) \hspace{3mm} = \hspace{3mm} E[ X_{t-k} \, (\Phi_1 X_{t-12} + \epsilon_t)] \hspace{3mm} = \hspace{3mm} \Phi_1 \gamma(k-12) + 0 \]


\[ \left\{ \begin{array}{ll} \gamma(0) &= \hspace{3mm} \Phi_1 \gamma(12) + \sigma^2 \hspace{30mm} - \hspace{3mm} (2) \\ \gamma(k) &= \hspace{3mm} \Phi_1 \gamma(k-12) \,\, \mbox{ if } k \geq 1 \hspace{20mm} - \hspace{3mm} (1) \\ \end{array} \right. \]


Take Eqn (1), \(\gamma(k) = \Phi_1 \gamma(k-12)\). Letting \(k=1\), we get \[ \gamma(1) = \Phi_1 \gamma(-11) = \Phi_1 \gamma(11). \]

Take (1), \(k=11\) \[ \gamma(11) = \Phi_1 \gamma(-1) = \Phi_1 \gamma(1). \] which can only happen if \[ \gamma(1) = 0, \mbox{ and } \gamma(11) = 0. \]

Eqn (1) says \(\gamma(k) = \Phi_1 \gamma(k-12)\). Repeating for other \(k\), \[\begin{align*} \gamma(2) &= \hspace{3mm} \Phi_1 \gamma(10) = \Phi_1^2 \gamma(2) \hspace{3mm} \Rightarrow \hspace{3mm} \gamma(2) = 0, \gamma(10) = 0 \\ \gamma(3) &= \hspace{3mm} \Phi_1 \gamma(9) = \Phi_1^2 \gamma(3) \hspace{3mm} \Rightarrow \hspace{3mm} \gamma(3) = 0, \gamma(9) = 0 \\ &\vdots& \\ \gamma(5) &= \hspace{3mm} \Phi_1 \gamma(7) = \Phi_1^2 \gamma(5) \hspace{3mm} \Rightarrow \hspace{3mm} \gamma(5) = 0, \gamma(7) = 0 \\ \gamma(6) &= \hspace{3mm} \Phi_1 \gamma(6) = \Phi_1^2 \gamma(6) \hspace{3mm} \Rightarrow \hspace{3mm} \gamma(6) = 0, \gamma(6) = 0 \end{align*}\]

Now take Eqn (2), and Eqn (1) with \(k=12\), \[\begin{align*} \gamma(0) &= \hspace{3mm} \Phi_1 \gamma(12) + \sigma^2, \\\\ \gamma(12) &= \hspace{3mm} \Phi_1 \gamma(0) \end{align*}\]

Substituting in, \[ \gamma(0) \hspace{3mm} = \hspace{3mm} \Phi_1 \Big(\Phi_1 \gamma(0)\Big) + \sigma^2 \\ \\ \hspace{3mm} = \hspace{3mm} \Phi_1^2 \gamma(0) + \sigma^2 \]

Solving, we get \[ \gamma(0) \hspace{3mm} = \hspace{3mm} \frac{\sigma^2}{1-\Phi_1^2}, \hspace{10mm} \gamma(12) \hspace{3mm} = \hspace{3mm} \Phi_1 \gamma(0). \]

ACVF for sAR(1) with s=12

For \(k=1,2,3,\ldots\) \[ \left\{ \begin{array}{l} \gamma(0) = \frac{\sigma^2}{1-\Phi_1^2} \\ \gamma(k12) = \Phi_1^k \gamma(0) \\ \mbox{ otherwise } \gamma(h) = 0 \\ \end{array} \right. \]

2. sARIMA(p,d,q) x (P,D,Q)[s] model

Consider sMA(1) with \(s=12\) with innovation \(X_t\), where \(X_t \sim\) MA(1), i.e. \[\begin{align*} Y_t &= X_t - \Theta_1 X_{t-12} \\ X_t &= \epsilon_t - \theta_1 \epsilon_{t-1} \end{align*}\]

That means we have

\[\begin{align*} Y_t &= (1-\Theta_1 B^{12}) X_t \\\\ &= (1-\Theta_1 B^{12}) (1-\theta_1 B) \, \epsilon_t \end{align*}\]

This is called ARMA\((0,1) \times (0,1)_{12}\) model.

a) Seasonal ARIMA model

\(Y_t \sim \mbox{ARIMA}(p,d,q) \times (P,D,Q)_s\) means \[ \bigtriangledown^d \bigtriangledown^D_s Y_t \hspace{3mm} = \hspace{3mm} X_t \sim \mbox{ARMA}(p,q) \times (P,Q)_s \]


\(Y_t \sim \mbox{ARMA}(p,q) \times (1,1)_{12}\) means \[ \Big(1-\phi_1 B -\cdots - \phi_p B^p\Big)\Big(1-\Phi_1 B^{12}\Big) Y_t = \Big(1-\theta_1 B -\cdots - \theta_q B^q \Big)\Big(1-\Theta_1 B^{12}\Big) \epsilon_t \]

b) Causality and Invertibility

Causality : \[ (1-\Phi_1 z^s) ( 1 - \phi_1 z - \cdots - \phi_p z^p) = 0 \] must have all the roots outside of the unit circle.

\[ (1-\Phi_1 z^s) \hspace{3mm} = \hspace{3mm} 0 \\ ( 1 - \phi_1 z - \cdots - \phi_p z^p) \hspace{3mm} = \hspace{3mm} 0 \]

Similar in invertibility.

c) Theoretical ACF

Suppose you have ARMA\((1,1) \times (0,1)_{12}\) model \[ \big(1-\phi_1 B\big) Y_t = \big(1-\theta_1 B\big)\big( 1- \Theta_1 B^{12}\big) \epsilon_t \] Then this is same as \[ \big(1-\phi_1 B\big) Y_t = \big(1-\theta_1 B - \Theta_1 B^{12} + \theta_1 \Theta_1 B^{13} \big) \epsilon_t \] \[ \big(1- .7 B\big) Y_t = \big(1 + .7 B + .5 B^{12} + (.7)(.5) B^{13} \big) \epsilon_t \]

  rho = ARMAacf(ar=c(.7), ma=c( .7,0,0,0,0,0,0,0,0,0,0,.5,.35), lag.max=40)
  plot(rho, col="red")



Summary

  • SMA(1) means that there’s MA effect from 12 month ago (when s=12).

  • ACF and PACF of SMA(1) is similar to that of MA(1), but 12 month apart.

  • SAR(1) means that there’s AR effect from 12 month ago (when s=12).

  • ACF and PACF of SAR(1) is similar to that of AR(1), but 12 month apart.