time series ch1 summary

Introduction 1.1 Examples of Time Series 1.2 Objectives of Time Series Analysis 1.3 Some Simple Time Series Models 1.4 Stationary Models and the Autocorrelation Function 1.5 Estimation and Elimination of Trend and Seasonal Components 1.6 Testing the Estimated Noise Sequence In this chapter we introduce some basic ideas of time series analysis and stochastic processes. Of particular importance are the concepts of stationarity and the autocovariance and sample autocovariance functions. Some standard techniques are described for the estimation and removal of trend and seasonality (of known period) from an observed time series. These are illustrated with reference to the data sets in Section 1.1. The calculations in all the examples can be carried out using the time series package ITSM, the professional version of which is available at http://extras. springer.com. The data sets are contained in files with names ending in .TSM. For example, the Australian red wine sales are filed as WINE.TSM. Most of the topics covered in this chapter will be developed more fully in later sections of the book. The reader who is not already familiar with random variables and random vectors should first read Appendix A, where a concise account of the required background is given. 1.1 Examples of Time Series A time series is a set of observations xt, each one being recorded at a specific time t. A discrete-time time series (the type to which this book is primarily devoted) is one in which the set T0 of times at which observations are made is a discrete set, as is the case, for example, when observations are made at fixed time intervals. Continuoustime time series are obtained when observations are recorded continuously over some time interval, e.g., when T0 = [0, 1].

1.2 Objectives of Time Series Analysis The examples considered in Section 1.1 are an extremely small sample from the multitude of time series encountered in the fields of engineering, science, sociology, and economics. Our purpose in this book is to study techniques for drawing inferences from such series. Before we can do this, however, it is necessary to set up a hypothetical probability model to represent the data. After an appropriate family of models has been chosen, it is then possible to estimate parameters, check for goodness of fit to the data, and possibly to use the fitted model to enhance our understanding of the mechanism generating the series. Once a satisfactory model has been developed, it may be used in a variety of ways depending on the particular field of application. The model may be used simply to provide a compact description of the data. We may, for example, be able to represent the accidental deaths data of Example 1.1.3 as the sum of a specified trend, and seasonal and random terms. For the interpretation of economic statistics such as unemployment figures, it is important to recognize the presence of seasonal components and to remove them so as not to confuse them with long-term trends. This process is known as seasonal adjustment. Other applications of time series models include separation (or filtering) of noise from signals as in Example 1.1.4, prediction of future values of a series such as the red wine sales in Example 1.1.1 or the population data in Example 1.1.5, testing hypotheses such as global warming using recorded temperature data, predicting one series from observations of another, e.g., predicting future sales using advertising expenditure data, and controlling future values of a series by adjusting parameters. Time series models are also useful in simulation studies. For example, the performance of a reservoir depends heavily on the random daily inputs of water to the system. If these are modeled as a time series, then we can use the fitted model to simulate a large number of independent sequences of daily inputs. Knowing the size and mode of operation of the reservoir, we can determine the fraction of the simulated input sequences that cause the reservoir to run out of water in a given time period. This fraction will then be an estimate of the probability of emptiness of the reservoir at some time in the given period.

1.3 Some Simple Time Series Models An important part of the analysis of a time series is the selection of a suitable probability model (or class of models) for the data. To allow for the possibly unpredictable nature of future observations it is natural to suppose that each observation xt is a realized value of a certain random variable Xt. Definition 1.3.1 A time series model for the observed data {xt} is a specification of the joint distributions (or possibly only the means and covariances) of a sequence of random variables {Xt} of which {xt} is postulated to be a realization. Remark. We shall frequently use the term time series to mean both the data and the process of which it is a realization. A complete probabilistic time series model for the sequence of random variables {X1, X2,...} would specify all of the joint distributions of the random vectors (X1,..., Xn) , n = 1, 2,..., or equivalently all of the probabilities P[X1 ≤ x1,..., Xn ≤ xn], −∞ < x1,..., xn < ∞, n = 1, 2,.... Such a specification is rarely used in time series analysis (unless the data are generated by some well-understood simple mechanism), since in general it will contain far too many parameters to be estimated from the available data. Instead we specify only the first- and second-order moments of the joint distributions, i.e., the expected values EXt and the expected products E(Xt+hXt), t = 1, 2,..., h = 0, 1, 2,..., focusing on properties of the sequence {Xt} that depend only on these. Such properties of {Xt} are referred to as second-order properties. In the particular case where all the joint distributions are multivariate normal, the second-order properties of {Xt} completely determine the joint distributions and hence give a complete probabilistic characterization of the sequence. In general we shall lose a certain amount of information by looking at time series “through second-order spectacles”; however, as we shall see in Chapter 2, the theory of minimum mean squared error linear prediction depends only on the second-order properties, thus providing further justification for the use of the second-order characterization of time series models. Figure 1-7 shows one of many possible realizations of {St,t = 1,..., 200}, where {St} is a sequence of random variables specified in Example 1.3.3 below. In most practical problems involving time series we see only one realization. For example, there is only one available realization of Fort Collins’s annual rainfall for the years 1900–1996, but we imagine it to be one of the many sequences that might have occurred. In the following examples we introduce some simple time series models. One of our goals will be to expand this repertoire so as to have at our disposal a broad range of models with which to try to match the observed behavior of given data sets. 1.3.1 Some Zero-Mean Models Example 1.3.1 iid Noise Perhaps the simplest model for a time series is one in which there is no trend or seasonal component and in which the observations are simply independent and identically distributed (iid) random variables with zero mean. We refer to such a sequence as independent and identically distributed noise

1.3.3 A General Approach to Time Series Modeling The examples of the previous section illustrate a general approach to time series analysis that will form the basis for much of what is done in this book. Before introducing the ideas of dependence and stationarity, we outline this approach to provide the reader with an overview of the way in which the various ideas of this chapter fit together. • Plot the series and examine the main features of the graph, checking in particular whether there is (a) a trend, (b) a seasonal component, (c) any apparent sharp changes in behavior, (d) any outlying observations. • Remove the trend and seasonal components to get stationary residuals (as defined in Section 1.4). To achieve this goal it may sometimes be necessary to apply a preliminary transformation to the data. For example, if the magnitude of the fluctuations appears to grow roughly linearly with the level of the series, then the transformed series {ln X1,...,ln Xn} will have fluctuations of more constant magnitude. See, for example, Figures 1-1 and 1-17. (If some of the data are negative, add a positive constant to each of the data values to ensure that all values are positive before taking logarithms.) There are several ways in which trend and seasonality can be removed (see Section 1.5), some involving estimating the components and subtracting them from the data, and others depending on differencing the data, i.e., replacing the original series {Xt} by {Yt := Xt − Xt−d} for some positive integer d. Whichever method is used, the aim is to produce a stationary series, whose values we shall refer to as residuals. • Choose a model to fit the residuals, making use of various sample statistics including the sample autocorrelation function to be defined in Section 1.4. • Forecasting will be achieved by forecasting the residuals and then inverting the transformations described above to arrive at forecasts of the original series {Xt}. • An extremely useful alternative approach touched on only briefly in this book is to express the series in terms of its Fourier components, which are sinusoidal waves of different frequencies (cf. Example 1.1.4). This approach is especially important in engineering applications such as signal processing and structural design. It is important, for example, to ensure that the resonant frequency of a structure does not coincide with a frequency at which the loading forces on the structure have a particularly large component.

Loosely speaking, a time series  is said to be stationary if it has statistical properties similar to those of the “time-shifted” series, for each integer. Restricting attention to those properties that depend only on the first- and second-order moment.

1.5 Estimation and Elimination of Trend and Seasonal Components The first step in the analysis of any time series is to plot the data. If there are any apparent discontinuities in the series, such as a sudden change of level, it may be advisable to analyze the series by first breaking it into homogeneous segments. If there are outlying observations, they should be studied carefully to check whether there is any justification for discarding them (as for example if an observation has been incorrectly recorded). Inspection of a graph may also suggest the possibility of representing the data as a realization of the process (the classical decomposition model) Xt = mt + st + Yt, (1.5.1) where mt is a slowly changing function known as a trend component, st is a function with known period d referred to as a seasonal component, and Yt is a random noise component that is stationary in the sense of Definition 1.4.2. If the seasonal and noise fluctuations appear to increase with the level of the process, then a preliminary transformation of the data is often used to make the transformed data more compatible with the model (1.5.1). Compare, for example, the red wine sales in Figure 1-1 with the transformed data, Figure 1-17, obtained by applying a logarithmic transformation. The transformed data do not exhibit the increasing fluctuation with increasing level that was apparent in the original data. This suggests that the model (1.5.1) is more appropriate for the transformed than for the original series. In this section we shall assume that the model (1.5.1) is appropriate (possibly after a preliminary transformation of the data) and examine some techniques for estimating the components mt, st, and Yt in the model. Our aim is to estimate and extract the deterministic components mt and st in the hope that the residual or noise component Yt will turn out to be a stationary time series. We can then use the theory of such processes to find a satisfactory probabilistic model for the process Yt, to analyze its properties, and to use it in conjunction with mt and st for purposes of prediction and simulation of {Xt}. Another approach, developed extensively by Box and Jenkins (1976), is to apply differencing operators repeatedly to the series {Xt} until the differenced observations resemble a realization of some stationary time series {Wt}. We can then use the theory of stationary processes for the modeling, analysis, and prediction of {Wt} and hence of the original process. The various stages of this procedure will be discussed in detail in Chapters 5 and 6. The two approaches to trend and seasonality removal, (1) by estimation of mt and st in (1.5.1) and (2) by differencing the series {Xt}, will now be illustrated with reference to the data introduced in Section 1.1.

Method 1: Trend Estimation Moving average and spectral smoothing are essentially nonparametric methods for trend (or signal) estimation and not for model building. Special smoothing filters can also be designed to remove periodic components as described under Method S1 below. The choice of smoothing filter requires a certain amount of subjective judgment, and it is recommended that a variety of filters be tried in order to get a good idea of the underlying trend. Exponential smoothing, since it is based on a moving average of past values only, is often used for forecasting, the smoothed value at the present time being used as the forecast of the next value. To construct a model for the data (with no seasonality) there are two general approaches, both available in ITSM. One is to fit a polynomial trend (by least squares) as described in Method 1(d) below, then to subtract the fitted trend from the data and to find an appropriate stationary time series model for the residuals. The other is to eliminate the trend by differencing as described in Method 2 and then to find an appropriate stationary model for the differenced series. The latter method has the advantage that it usually requires the estimation of fewer parameters and does not rest on the assumption of a trend that remains fixed throughout the observation period. The study of the residuals (or of the differenced series) is taken up in Section 1.6. (a) Smoothing with a finite moving average filter. Let q be a nonnegative integer and consider the two-sided moving average
Method 2: Trend Elimination by Differencing Instead of attempting to remove the noise by smoothing as in Method 1, we now attempt to eliminate the trend term by differencing. We define the lag-1 difference operator ∇ by

1.5.2 Estimation and Elimination of Both Trend and Seasonality The methods described for the estimation and elimination of trend can be adapted in a natural way to eliminate both trend and seasonality in the general model,

1.6 Testing the Estimated Noise Sequence The objective of the data transformations described in Section 1.5 is to produce a series with no apparent deviations from stationarity, and in particular with no apparent trend or seasonality. Assuming that this has been done, the next step is to model the estimated noise sequence (i.e., the residuals obtained either by differencing the data or by estimating and subtracting the trend and seasonal components). If there is no dependence among between these residuals, then we can regard them as observations of independent random variables, and there is no further modeling to be done except to estimate their mean and variance. However, if there is significant dependence among the residuals, then we need to look for a more complex stationary time series model for the noise that accounts for the dependence. This will be to our advantage, since dependence means in particular that past observations of the noise sequence can assist in predicting future values. In this section we examine some simple tests for checking the hypothesis that the residuals from Section 1.5 are observed values of independent and identically distributed random variables. If they are, then our work is done. If not, then we must use the theory of stationary processes to be developed in later chapters to find a more appropriate mode such as the sample autocorrelation function.

chapter 2 stationary processes

Stationary Processes 2.1 Basic Properties 2.2 Linear Processes 2.3 Introduction to ARMA Processes 2.4 Properties of the Sample Mean and Autocorrelation Function 2.5 Forecasting Stationary Time Series 2.6 The Wold Decomposition A key role in time series analysis is played by processes whose properties, or some of them, do not vary with time. If we wish to make predictions, then clearly we must assume that something does not vary with time. In extrapolating deterministic functions it is common practice to assume that either the function itself or one of its derivatives is constant. The assumption of a constant first derivative leads to linear extrapolation as a means of prediction. In time series analysis our goal is to predict a series that typically is not deterministic but contains a random component. If this random component is stationary, in the sense of Definition 1.4.2, then we can develop powerful techniques to forecast its future values. These techniques will be developed and discussed in this and subsequent chapters.