Class Web Page



1. Characteristic Polinomal


Alternative Notation for AR(1)

We began with writing AR(1) as \[ X_t = \phi X_{t-1} + \epsilon_t, \]

We could also write AR(1) as \[ X_t - \phi X_{t-1} = \epsilon_t, \] and using the backward operator, write \[ \underbrace{ (1 - \phi B) }_{\Phi(B)} X_t = \epsilon_t. \]

\(\Phi(z)\) is called characteristic polynomial of AR(1).

Backward operator makes it go back a day: \[ B X_t = X_{t-1}. \]

AR(p) process

Autoregressive process of order \(p\) is \[ X_t - \phi_1 X_{t-1} - \phi_2 X_{t-2} - \cdots - \phi_p X_{t-p} \hspace3mm = \hspace3mm \epsilon_t, \\ \\ \mbox{ where } \epsilon_t \sim WN(0,\sigma^2) \] and \(\phi_1, \ldots, \phi_p\) is real valued constant.

Alternative notation using characteristic polynomial is, \[ \underbrace{ (1 - \phi_1 B - \phi_2 B^2 - \cdots - \phi_p B^p) }_{\Phi(B) } \, X_t \hspace3mm = \hspace3mm \epsilon_t. \\\\\\ \Phi(B) \hspace3mm X_t \hspace3mm = \hspace3mm \epsilon_t. \]

2. Causal Representation


For AR(1)

If AR(1) representation is \[ X_t \hspace3mm = \hspace3mm \phi X_{t-1} + \epsilon_t, \] then I can write the same thing for yesterday, \[ X_{t-1} \hspace3mm = \hspace3mm \phi X_{t-2} + \epsilon_{t-1} \]

Now combining the two, I can write \[ \begin{align} X_t &\hspace3mm = \hspace3mm \phi \, X_{t-1} + \epsilon_t \\ \\ &\hspace3mm = \hspace3mm \phi \big\{\phi X_{t-2} + \epsilon_{t-1} \big\} + \epsilon_t \\ \\ &\hspace3mm = \hspace3mm \phi^2 X_{t-2} + \hspace3mm \phi \epsilon_{t-1} + \epsilon_t \end{align} \]


Do it again, this time for \(X_{t-2}\) and we get \[ \begin{align} X_t &\hspace3mm = \hspace3mm \phi^2 X_{t-2} \hspace3mm + \hspace3mm \phi \, \epsilon_{t-1} + \epsilon_t \\ \\ &\hspace3mm = \hspace3mm \phi^2 \big(\phi X_{t-3} + \epsilon_{t-2} \big) \hspace3mm + \hspace3mm \phi \, \epsilon_{t-1} + \epsilon_t \\ \\ &\hspace3mm = \hspace3mm \phi^3 X_{t-3} \hspace3mm + \hspace3mm \phi^2 \, \epsilon_{t-2} + \phi \epsilon_{t-1} + \epsilon_t \end{align} \]

We can keep doing this, and get \[ X_t \hspace3mm = \hspace3mm \phi^k X_{t-k} \hspace3mm + \hspace3mm\phi^{k-1} e_{t-(k-1)} \hspace3mm + \hspace3mm \cdots \hspace3mm + \hspace3mm \phi \, \epsilon_{t-1} \hspace3mm + \hspace3mm \epsilon_t \]

If \(|\phi|<1\), then letting \(k \to \infty\) will yield \[ X_t \hspace3mm = \hspace3mm \epsilon_t + \phi \epsilon_{t-1} + \phi^2 \epsilon_{t-2} + \cdots \hspace3mm = \hspace3mm \sum_{i=0}^\infty \phi^{i} \, \epsilon_{t-i} \]

So hereโ€™s causal representation for AR(1) process \[ X_t \hspace3mm = \hspace3mm \sum_{i=0}^\infty \phi^{i} \, \epsilon_{t-i} \hspace3mm = \hspace3mm \epsilon_t + \phi \, \epsilon_{t-1} + \phi^2 \, \epsilon_{t-2} + \cdots \]

This is called causal representation because we can write \(Y_t\) as infinite sum of past errors (innovations).

Get Mean of AR(1) using Causal Representation

Now it is easy to see that mean of AR(1) \[ E(X_t) \hspace3mm = \hspace3mm \sum_{i=0}^\infty \phi^{i} \, E( \epsilon_{t-i} ) \hspace3mm = \hspace3mm 0 \]

Recall \(\epsilon_t\) are White Noise with mean 0 and variance \(\sigma\).

Or often, we assume \[ \epsilon_t \sim N(0,\sigma^2) \]

Get Var with Causal Representation

So if you write AR(1) in causal way, \[ X_t = \sum_{i=0}^\infty \phi^{i} \, \epsilon_{t-i}, \]

Then we can calculate variance as \[ \begin{align} \mbox{Var}(X_t) \hspace3mm = \hspace3mm \mbox{Var } \Big[ \sum_{i=0}^\infty \phi^i \, \epsilon_{t-i}\Big] \hspace3mm = \hspace3mm \sum_{i=0}^\infty \mbox{Var}\big[\phi^i \, \epsilon_{t-i}\big] \hspace3mm = \hspace3mm \sum_{i=0}^\infty \phi^{2i} \mbox{Var}(\epsilon_{t-i}) \hspace3mm = \hspace3mm \sigma^2 \sum_{i=0}^\infty \phi^{2i} \end{align} \] This does not converge unless \(|\phi|<1\).

When it does converge, the variance is \(\sigma^2/(1-\phi^2)\).

When Char. Poly. can be written with Causal Rep?

Recall yet another way to write AR(1), \[ (1-\phi B) \, X_t \hspace3mm = \hspace3mm \epsilon_t. \]

This means that we can write \[ X_t \hspace3mm = \hspace3mm \frac{1}{(1-\phi \, B)} \, \epsilon_t. \]

Compare this with Causal representation \[ X_t \hspace3mm = \hspace3mm \epsilon_t + \phi \, \epsilon_{t-1} + \phi^2 \, \epsilon_{t-2} + \cdots \hspace3mm = \hspace3mm 1+\phi B+ \phi^2 B^2 + \cdots \, \epsilon_t \]

So we have the equivalence on the right hand side, \[ \frac{1}{(1-\phi B)} \hspace3mm = \hspace3mm 1 + \phi B + \phi^2 B^2 + \phi^3 B^3 + \cdots \]

This is exactly same as the geometric series, \[ \frac{1}{1-x} \hspace3mm = \hspace3mm 1 + x + x^2 + x^3 + \cdots \]

Note that the condition for the geometiric series is \[ |x| < 1 \hspace10mm \mbox{ or } \hspace10mm |\phi| <1 \] Which is same as the causal (stationary) condition.

Writing AR(p) in Causal Rep.

Using the characteristic polynomial, \[ \Phi(B) \, X_t \hspace3mm = \hspace3mm \epsilon_t \\ \\ X_t \hspace3mm = \hspace3mm \frac{1}{\Phi(B)} \,\, \epsilon_t \] That means we must be able to write polynomial as \[ \frac{1}{\big(1 - \phi_1 B - \phi_2 B^2 - \cdots - \phi_p B^p\big) } \hspace3mm = \hspace3mm \big( \psi_0 + \psi_1 B + \psi_2 B^2 + \psi_3 B^3 + \cdots \big) \] When can we do this?

Causal Condition for AR(p)

From operator theory, we know that if \[ \mbox{ (complex) root of } \Phi(z) \mbox{ is outside of the unit circle, } \\ \] we can expand the inverse of \(\Phi(z)\) as \[ \frac{1}{\Phi(z)} \hspace3mm = \hspace3mm \psi_0 + \psi_1 z + \psi_2 z^2 + \psi_3 z^3 + \cdots \]

This is the condition that allows us to write AR(p) in causal representation.

We have seen that causal condition for AR(1) was \(|\phi|<1\).

This was because polynomial \[ \Phi(z) \hspace3mm = \hspace3mm (1-\phi z) \] will have root inside the unit circle if \(|\phi|>1\).

3. Examples


Ex: Checking Causality 1

Check to see of AR(2) model, \[ Y_t \hspace3mm = \hspace3mm .4 Y_{t-1} - .3Y_{t-2} + e_{t} \] is causal (stationary).

We have to look at the root of \[ \Phi(z) \hspace3mm = \hspace3mm 1 - .4z + .3z^2 \] which is \[ \frac{ -b \pm \sqrt{b^2 - 4ac} }{2a} = \frac{ .4 \pm \sqrt{(.4)^2 - 4(.3)(1)} }{2(.3)} = .667 \pm i 1.7 \]

Their distance form the origin is \[ \sqrt{(.667)^2 + (i 1.7)^2 } = 1.826 \] So the roots are outside the complex unit circle. Thus this AR(2) is causal.


Ex: Checking Causality 2

Check to see of AR(2) model, \[ Y_t = -.7 Y_{t-1} - .6Y_{t-2} + e_{t} \] is causal or not.

We have to look at the root of \[ \Phi(z) = 1 + .7z + .6z^2, \] which has roots $-.583 1.15 i $.

## [1] -0.583333+1.15169i -0.583333-1.15169i
## [1] 1.290994 1.290994

They are at at distance \(\sqrt{(-.583)^2+(1.15i)^2} = 1.29\) from origin. Therefore, this AR(2) is causal.

Ex: Checking Causality 3

Given \[ X_t = .7 X_{t-1} + .6 X_{t-2} + e_{t}, \] Check the causality.

we look at the root of \[ \Phi(z) = 1 - .7z - .6z^2, \]

## [1]  0.8333333+0i -2.0000000+0i
## [1] 0.8333333 2.0000000

The polynomial has roots \(.833, .2\). Therefore, this AR(2) is not causal.

Summary

  • AR(\(p\)) is defined as \[ X_t - \phi_1 X_{t-1} - \phi_2 X_{t-2} - \cdots - \phi_p X_{t-p} \hspace3mm = \hspace3mm \epsilon_t, \\ (1 - \phi_1 B - \phi_2 B^2 - \cdots - \phi_p B^p) \, X_t \hspace3mm = \hspace3mm \epsilon_t. \\ \Phi(B) \hspace2mm X_t \hspace3mm = \hspace3mm \epsilon_t. \] where \(\phi_1, \ldots, \phi_p\) is real valued constant, and \(\epsilon_t \sim WN(0,\sigma^2)\).

  • AR(\(p\)) can be written in causal representation, \[ X_t = \sum_{i=0}^\infty \phi^{i} \, \epsilon_{t-i}, \] when its characteristic polynomial \(\Phi(z)\) has all the roots outside of the unit circle on the imaginary plane.

When the AR(\(p\)) can be written as causal process, then it is stationary.

You can use polyroot() function in R to calculate the roots of polynomial, and **Mod()* to calculate their distance from the origin.