Class Web Page



1. MA(1) Model

First order moving average process is defiend as: \[ X_t \hspace3mm = \hspace3mm \epsilon_t - \theta_1 \epsilon_{t-1}, \hspace15mm \epsilon_t \sim WN(0, \sigma^2) \] and \(\theta_1\) is real valued constant.


Compare this to the first order Autoregressive process \[ X_t - \phi_1 X_{t-1} \hspace3mm = \hspace3mm \epsilon_t \]


Using the backward operator, MA(1) can be written as \[ X_t \hspace3mm = \hspace3mm \underbrace{ (1 - \theta_1 B) }_{\Theta(B) } \, \epsilon_t. \]


Watch the sign in front of \(\theta_1\)! This is Cryer’s convension. See ARIMA functions page of the website.

Mean of MA(1)

\[ \begin{align} E(X_t) &= \hspace3mm E(\epsilon_t - \theta_1 \epsilon_{t-1}) \\\\ &= \hspace3mm E(\epsilon_t) - \theta_1 E(\epsilon_{t-1}) &= \hspace3mm 0 \end{align} \]

ACVF at lag 0

\[ \begin{align} \mbox{Var}(X_t) &= \hspace3mm \mbox{Var}(\epsilon_t - \theta_1 \epsilon_{t-1})\\ \\ &= \hspace3mm \mbox{Var}(\epsilon_t) + \theta_1^2 \mbox{Var}(\epsilon_{t-1})\\ \\ &= \hspace3mm \sigma^2 + \theta_1^2 \sigma^2 \\ \\ &= \hspace3mm (1+\theta_1^2) \sigma^2 \end{align} \]

ACVF at lag 1

\[ \begin{align} \gamma(1) &= \hspace3mm \mbox{ Cov }(X_t, X_{t+1} ) \\ \\ &= \hspace3mm \mbox{ Cov }(\epsilon_t- \theta_1 \epsilon_{t-1}, \hspace3mm e_{t+1}-\theta_1 e_{t} )\\ \\ &= \hspace3mm \mbox{ Cov }(\epsilon_t , \, e_{t+1} ) - \mbox{ Cov }(\epsilon_t , \, \theta_1 e_{t} ) - \mbox{ Cov }(\theta_1 \epsilon_{t-1}, \, e_{t+1} ) + \mbox{ Cov }(\theta_1 \epsilon_{t-1}, \, \theta_1 e_{t} ) \\ \\ &= \hspace3mm -\theta_1 \mbox{ Cov }(\epsilon_t , \, e_{t} ) \\ \\ &= \hspace3mm -\theta_1 \, \sigma^2 \end{align} \]

ACVF at lag 2

\[ \begin{align} \gamma(2) &= \hspace3mm \mbox{ Cov }(Y_t, Y_{t+2} ) \\ \\ &= \hspace3mm \mbox{ Cov }(\epsilon_t- \theta_1 \epsilon_{t-1}, \hspace3mm e_{t+2}-\theta_1 e_{t+1} )\\ \\ &= \hspace3mm 0 \end{align} \]

ACVF and ACF of MA(1)

ACVF \[ \gamma(h) = \left\{ \begin{array}{ll} \sigma^2(1+\theta_1^2) & \mbox{ if } h=0 \\ -\sigma^2 \theta_1 & \mbox{ if } h= \pm 1 \\ 0 & \mbox{ if } |h|>1 \\ \end{array} \right. \]

ACF \[ \rho(h) = \left\{ \begin{array}{ll} 1 & \mbox{ if } h=0 \\ -\theta_1/ (1+\theta^2_1) & \mbox{ if } h= \pm 1 \\ 0 & \mbox{ if } |h|>1 \\ \end{array} \right. \]

2. MA(q) Model

Moving Average process of order \(q\) is \[ X_t = \epsilon_t - \theta_1 \epsilon_{t-1} - \theta_2 \epsilon_{t-2} - \cdots - \theta_q e_{t-q}, \] where \(\epsilon_t \sim WN(0,\sigma^2)\) and \(\theta_1, \ldots, \theta_q\) is real valued constant.

Using the backward operator, we write this as \[ X_t = (1 - \theta_1 B - \theta_2 B^2 - \cdots - \theta_q B^q) \, \epsilon_t. \]

We shorten the notation further, and write \[ X_t = \Theta(B) Z_t. \] where Characteristic Polinomial, \(\Theta(z) = (1 - \theta_1 z - \theta_2 z^2 - \cdots - \theta_q z^q)\).

Mean and Var of MA(q)

Mean \[ E(X_t) \hspace3mm = \hspace3mm E(\epsilon_t - \theta_1 \epsilon_{t-1} - \theta_2 \epsilon_{t-2} - \cdots - \theta_q e_{t-q}) \hspace3mm = \hspace3mm 0. \]

Variance \[ \begin{align} \mbox{Var}(X_t) &= \hspace3mm \mbox{Var}(\epsilon_t - \theta_1 \epsilon_{t-1} - \theta_2 \epsilon_{t-2} - \cdots - \theta_q e_{t-q}) \\\\ &= \hspace3mm \sigma^2 + \theta_1 \, \sigma^2 + \theta_2 \, \sigma^2 + \cdots + \theta_q \, \sigma^2 \\\\ &= \hspace3mm \Big(1 + \theta_1 + \theta_2 + \cdots + \theta_q \Big) \, \sigma^2 \end{align} \]

ACVF and ACF of MA(q)

\[ \begin{align} \gamma(h) &= \hspace3mm \mbox{ Cov }(X_t, \hspace3mm X_{t+h} )\\ &= \hspace3mm \mbox{ Cov }\Big(\epsilon_t - \theta_1 \epsilon_{t-1} - \theta_2 \epsilon_{t-2} - \cdots - \theta_q e_{t-q}, \hspace5mm e_{t+h} - \theta_1 e_{t-1+h} - \theta_2 e_{t-2+h} - \cdots - \theta_q e_{t-q+h}\Big) \end{align} \]

Theoretical ACVF: \[ \gamma(h) = \left\{ \begin{array}{ll} \sigma^2 (1+\theta_1^2 \cdots + \theta_q^2) & \mbox{ if } h=0 \\ \sigma^2 (-\theta_1 + \theta_2 \theta_1 + \theta_3 \theta_2 + \cdots + \theta_{q-1} \theta_{q}) & \mbox{ if } h=1 \\ \sigma^2 (-\theta_2 + \theta_3 \theta_1 + \cdots + \theta_{q-2} \theta_{q}) & \mbox{ if } h=2 \\ \vdots & \\ \sigma^2 (-\theta_q ) & \mbox{ if } h=q \\ 0 & \mbox{ if } |h|>q\\ \end{array} \right. \]


Theoretical ACF: \[ \rho(h) = \left\{ \begin{array}{ll} 1 & \mbox{ if } h=0 \\ \gamma(h) / \gamma(0) & \mbox{ if } -q \leq h \leq q \\ 0 & \mbox{ if } |h|> q \\ \end{array} \right. \]

Theorem

Every stationary q-corr TS with mean 0 can be represented as MA(\(q\)) process.

3. Ex: MA(3)

\[ X_t = \epsilon_t - \theta_1 \epsilon_{t-1} - \theta_2 \epsilon_{t-2} - \theta_3 \epsilon_{t-3} \]

\[ \gamma(h) = \left\{ \begin{array}{ll} \sigma^2 (1+\theta_1^2 +\theta_2^2 + \theta_3^2 ) & \mbox{ if } h=0 \\ \sigma^2 (-\theta_1 + \theta_2 \theta_1 + \theta_3 \theta_2 ) & \mbox{ if } h=1 \\ \sigma^2 (-\theta_2 + \theta_3 \theta_1 ) & \mbox{ if } h=2 \\ \sigma^2 (-\theta_3 ) & \mbox{ if } h=3 \\ 0 & \mbox{ if } |h|> \\ \end{array} \right. \]

\[ \rho(h) = \gamma(h) / \gamma(0) \]

Watch the Sign!

#   [Cryer]'s sign convention is (1-phi B) Y_t = (1-theta) \epsilon_t.
#   [Brockwell]'s             is (1-phi B) Y_t = (1+theta) \epsilon_t.
#
#   For AR(1) (1-.5 B) Y_t = \epsilon_t,          ACF should be all positive.
#             (1+.5 B) Y_t = \epsilon_t,          ACF should be alternating in sign.
#
#   For MA(1)          Y_t = (1-.5 B) \epsilon_t  ACF at lag 1 should be negative
#                      Y_t = (1+.5 B) \epsilon_t  ACF at lag 1 should be positive

#--- Theoretical ACF and PACF of MA ---
#     ARMAacf() uses  sign of MA parameters like [Brockwell]:  Y_t = (1+theta B) \epsilon_t
    Theta = c(.2,  .3, .2,  .2)

    MArho1 = ARMAacf(ma = Theta, lag.max=20, pacf=FALSE)
    MApacf1<- ARMAacf(ma = Theta, lag.max=20, pacf=TRUE)

    layout(matrix(1:2, 1,2))
    plot(0:20, MArho1, type="h", col="red");   abline(h=0)
    plot(1:20, MApacf1, type="h", col="red");   abline(h=0)

#--- Theoretical ACVF of MA
#
#   tacvfARMA() uses sign of MA parameters like [Cryer]:  Y_t = (1 - theta B) \epsilon_t
#

  library(ltsa)

  MAgam2 = tacvfARMA(theta= c(.2, .3, .2, .2), maxLag=20, sigma2=1)  #- Theoretical ACVF
  MArho2 = MAgam2/MAgam2[1]                                          #- Theoretical ACF

  plot(0:20, MArho2, type="h", col="red");   abline(h=0)


Simulation: MA(q)

  T1 = c(.5)
  T2 = c(.2, -.3, .2, -.2)
  T3 = c(.2,  .3, .2,  .2)
  T4 = c(-.2,  -.3, -.2,  -.2)

  Theta = T1

  # arima.sim() uses sign of MA parameters like [Brockwell]:  Y_t = (1+theta B) \epsilon_t

  x  = arima.sim( list(ma = c(0.5) ), n=250    )
  MArho1 = ARMAacf(ma = Theta, lag.max=20, pacf=FALSE)
  MApacf1<- ARMAacf(ma = Theta, lag.max=20, pacf=TRUE)

  plot(x, type="o");  abline(h=0)

  layout(matrix(1:2, 1, 2) )
  acf(x);    lines(0:20, MArho1,  type="p", col="red")
  pacf(x);   lines(1:20, MApacf1, type="p", col="red")

  layout(1)


Characteristics of MA(q)

ACF cuts off at \(q\)

PACF tails off

Summary

  • MA(\(q\)) process is defiend as \[ X_t \hspace3mm = \hspace3mm (1 - \theta_1 B - \theta_2 B^2 - \cdots - \theta_q B^q) \epsilon_t. \\ \hspace3mm = \hspace3mm \Theta(B) \epsilon_t \] where \(\Theta(z)\) is the characteristic polynomial of MA(\(q\)), and \(\epsilon_t ~ WN(0,\sigma^2)\).

  • MA(\(q\)) has ACF that cuts-off after lag \(q\), and PACF that decays down.

  • MA(\(q\)) has mean of 0. Theoretical ACVF: \[ \gamma(h) = \left\{ \begin{array}{ll} \sigma^2 (1+\theta_1^2 \cdots + \theta_q^2) & \mbox{ if } h=0 \\ \sigma^2 (-\theta_1 + \theta_2 \theta_1 + \theta_3 \theta_2 + \cdots + \theta_{q-1} \theta_{q}) & \mbox{ if } h=1 \\ \sigma^2 (-\theta_2 + \theta_3 \theta_1 + \cdots + \theta_{q-2} \theta_{q}) & \mbox{ if } h=2 \\ \vdots & \\ \sigma^2 (-\theta_q ) & \mbox{ if } h=q \\ 0 & \mbox{ if } |h|>q\\ \end{array} \right. \] \[ \rho(h) = \left\{ \begin{array}{ll} 1 & \mbox{ if } h=0 \\ \gamma(h) / \gamma(0) & \mbox{ if } -q \leq h \leq q \\ 0 & \mbox{ if } |h|> q \\ \end{array} \right. \]

  • You can find code for Theoretical ACF and ACVF of MA(q) on page .

  • You can find code for Simulating MA(\(q\)) process and looking at Saompe ACF and PACF on page .