Arima More Time Series

So far, regression analysis has taken the form \[Y = f(X) + \epsilon\] This could be rewritten \(Y - f(X) = \epsilon\) and then using the function \(F(a,b) = a - f(b)\) it could be rewritten \[F(X,Y) = \epsilon.\] Here \(\epsilon\) is generally assumed to have a fixed distribution independent of \(X\) or \(Y\) (but it might have heterogeneous error). We then develop various ways to estimate \(\hat F\) and perhaps something about \(\epsilon\) from the data \((x_i,y_i)\).

We could view this as attempting to find something about the relationship between \(X\) and \(Y\) which is invariant. If we had \(\epsilon = 0\) then our relationship would be exact. The next best thing is to make assumptions such as, in increasing order of assumption strenght, that

• \(\epsilon\) has mean \(0\) but otherwise might depend on \(X\)
• \(\epsilon\) has mean \(0\) but constant variance, but we don’t know the distribution (which may or may not depend on \(X\))
• \(\epsilon\) is normally distributed with mean \(0\), constant variance, and is independent of \(X\).

We need to believe something about the relationship between \(X\) and mkY doesn’t change in order to get started with regression modeling. We have to adjust this story a bit for time series. Let’s stick with a time series \(y_t \) without any covariats.

We’d like for there to be some invariance of the time series with time, i.e. some assumption that should hold at any time, perhaps about the relationship between \(Y_t\) and \(Y_{t+1}\).

We say that a time series \(\{y_t\}\) is strictly stationary if the joint probability distribution \(Y_{t_1},Y_{t_2},..., Y_{t_n} \) is equal to that of \(Y_{t_1+\tau}, Y_{t_2 + \tau},...,Y_{t_n+\tau}\) for any \(t_1,...,t_n\), \(\tau\), \(n\). This would imply that the joint distribution only depends on the intervals between \(t_1,...,t_n\) and in particular if \(n=1\) \[E(Y_t) = \mu, \text{Var}(Y_t) = \sigma^2\] are not functions of \(t\). Further if \(n = 2\) the joint distributio of \(Y_{t_1}\)and \(Y_{t_2}\) only depends on \((t_2 - t_1)\) which is called the lag between \(t_1\) and \(t_2\).

Strict stationarity is quite restrictive and we more often define a weaker sense: \[E(Y_t) = \mu, \text{~ and } ~ \text{Cov}(Y_t, Y_{t + \tau}) = \gamma (\tau).\]

• White noise: start with a sequence of random variables \(Y_t\) each iid. Then anything we expect about \(Y_t,...,Y_{t_n}\) we should expect about \(Y_{t+\tau},...,Y_{t+\tau+n}\). In particular, since they are iid, the pdf of the first sequence is

\[f_{Y_1,...,Y_n}(y_1,...,y_n) = \prod_{i=1}^n f_{Y_i}\]

The autocorrelation of a time series is essentially the correlation of that time series with its future observations placed at different lag. In particular, the autocorrelation of a time series at la \(k\) is given by

\begin{align*} r_k &= \text{Corr}(Y_{t+k, Y+t}) \\ &= \frac{\text{Corr}(Y_{t+k}, Y_t)}{\text{StdDev}(Y_{t+k})\text{StdDev}(Y_t)} \end{align*}