Time series data is a sequence of datapoints \[(\vec x_1,y_1),...,(\vec x_t, y_t),...\] where \(\vec x_t\) represents a collection of \(p\) features and \(y_t\) represents a numeric variable of interest at time \(t\). We may or may not have an actual set of features, that is, you might only have the variable \(y_t\) and its time index.
Modeling vs Forecasting
- A time series model is a mathematical description of the underlying data generating process. They allow us to produce distributions for a given time period.
- A forecast method is any method for extrapolating observed data forward through time.
Baseline Forecasts
Both of these baseline models/forecasts are appropriate when the underlying data doesn’t exhibit a trend.
- Example: Google’s stock price would be bad for this, it exhibits an upward trend. The “change since yesterday” data from Google’s stock price would be good for this, since it doesn’t exhibit an upwards or downwards trend (the difference data isn’t stationary, as the stock becomes more valuable, the difference between yesterday and today’s price increases, so there is a change in variability as time progresses. But there isn’t a trend.)
Gaussian white noise model We hypothesize a data generating process of the form \[f(t) = \mu + \epsilon_t\] with \(\mu\) some constant and \(\epsilon_t\sim \mathcal N(0,\sigma^2)\). We call the error terms “Gaussian white noise”. The MLE fitted model will have \[\hat \mu = \frac1n \sum_{i=1}^n y_i, ~ \text{and} ~ \hat\sigma = \frac1n \sum_{i=1}^n(y - \hat \mu)^2.\] This leads to the average forecast: to predict the value of \(y_t\) for \(t > n\) we simply report \(\hat \mu\).
Gaussian random walk model Another baseline data generating hypothesis: \[y_t = y_{t-1} + \epsilon_t\] where \(\epsilon_t \sim \mathcal N(0,\sigma^2)\). We fit this model by computing \(\hat \sigma^2 = \frac1n \sum_{i=1}^n(y_t - y_{t-1})^2\).
This leads to the naive forecast: \(y_t = y_{t-1}\).