Last modified: June 05, 2022

This article is written in: 🇺🇸

Invertibility in Time Series Models

In time series modeling, invertibility is the property of a model that allows the innovation process (also called the noise or disturbance process) to be expressed as a function of the observed series and its past values. This is particularly relevant for Moving Average (MA) models.

Intuition Behind Invertibility

Invertibility ensures that a Moving Average (MA) model, which expresses the current value of a series as a linear combination of white noise terms, can be transformed into an Autoregressive (AR) form. This is crucial because it allows us to use observed data to infer the underlying white noise, making the model estimable and stable.

An invertible model ensures a unique relationship between the observed values of the time series and the underlying innovations. Without invertibility, the same time series could be explained by multiple different models, which makes interpretation and prediction difficult.

Mathematical Definition of Invertibility

Formally, a process Xt is said to be invertible if the white noise sequence Zt can be expressed as a convergent infinite sum of past observations Xt:

Zt=∞∑k=0πkXt−k

where the series ∑∞k=0|πk| converges.

Why is Invertibility Important?

Invertibility is essential for practical purposes because:

• It ensures that the noise (or shock) sequence Zt can be uniquely recovered from the observed data.
• It prevents model ambiguity by ensuring a one-to-one correspondence between the observed series and the underlying noise.
• It is necessary for model estimation, as non-invertible models may lead to poor or misleading parameter estimates.

Invertibility also allows the series to be expressed in an AR(∞) (AutoRegressive process of infinite order) form, which helps in analysis and forecasting.

General Conditions for Invertibility

For a Moving Average (MA) process to be invertible, the parameters associated with the lagged noise terms must satisfy certain conditions. Specifically, for an MA process of order q:

Xt=Zt+θ1Zt−1+θ2Zt−2+⋯+θqZt−q

the model is invertible if and only if the roots of the associated characteristic equation lie outside the unit circle in the complex plane. The characteristic equation is:

1+θ1z+θ2z2+⋯+θqzq=0

Example: MA(1) Process

Now let's apply the concept of invertibility to a specific case, the Moving Average (MA(1)) process.

Consider an MA(1) process:

Xt=Zt+βZt−1

where Zt is white noise with mean 0 and variance σ2Z, and β is a constant. To determine the invertibility of this model, we need to express Zt in terms of Xt and its past values.

Inversion Using Backward Substitution

We can express Zt in terms of Xt and its lagged values by backward substitution. Starting from:

Zt=Xt−βZt−1

Now substitute Zt−1 from the same equation:

Zt=Xt−β(Xt−1−βZt−2)=Xt−βXt−1+β2Zt−2

Continuing this process:

Zt=Xt−βXt−1+β2Xt−2−β3Xt−3+…

This shows that the MA(1) process can be written as an infinite autoregressive process:

Zt=∞∑k=0(−β)kXt−k

This series converges if |β|<1. Therefore, the MA(1) process is invertible if |β|<1.

Inversion Using the Backward Shift Operator

Alternatively, we can use the backward shift operator to invert the MA(1) process.

Given:

Xt=(1+βB)Zt

where B is the backward shift operator (i.e., BZt=Zt−1), we aim to find Zt in terms of Xt by inverting the operator 1+βB:

Zt=(1+βB)−1Xt

The inverse of 1+βB can be expanded as a power series:

(1+βB)−1=1−βB+β2B2−β3B3+…

Thus, applying this operator to Xt, we get:

Zt=Xt−βXt−1+β2Xt−2−β3Xt−3+…

This is the same result obtained by backward substitution. Again, the series converges if |β|<1, confirming that the MA(1) process is invertible under this condition.

Example: MA(2) Process

Consider an MA(2) process:

Xt=Zt+θ1Zt−1+θ2Zt−2

The characteristic equation for this process is:

1+θ1z+θ2z2=0

For invertibility, the roots of this equation must lie outside the unit circle. Suppose θ1=0.5 and θ2=0.3. The characteristic equation becomes:

1+0.5z+0.3z2=0

Solving this quadratic equation:

z=−0.5±√0.52−4⋅0.3⋅12⋅0.3=−0.5±√0.25−1.20.6

z=−0.5±√−0.950.6

Since the discriminant is negative, the roots are complex numbers. The modulus of these complex roots must be greater than 1 for the process to be invertible. If the modulus of the roots is less than 1, the process is not invertible.

Convergence of the Series in MA Models

When working with Moving Average (MA) models, it's important to ensure that the infinite series we obtain when trying to express the noise term Zt as a function of past observations Xt converges. This is crucial because we want to guarantee that the process remains stable and well-defined over time.

Understanding Mean-Square Convergence

In time series analysis, when we express Zt (the white noise term) as an infinite sum of past values of the observed series Xt, we need this sum to converge in the mean-square sense. This type of convergence is important because it means that, as we include more and more past terms, the series approaches a stable value in terms of the average squared deviation from the true value of the noise term Zt.

In the context of an MA(1) model, this means we can write:

Zt=Xt−βXt−1+β2Xt−2−β3Xt−3+…

This representation involves an infinite series of lagged values of Xt, where each term is weighted by increasing powers of β. For this series to be stable and convergent, the terms βn must decay as n→∞, meaning that as we go further back in time (larger lags), the influence of past values should diminish.

Condition for Convergence: |β|<1

For the series to converge in the mean-square sense, the absolute value of β must be less than 1. This condition ensures that the powers of β become progressively smaller as n increases, which in turn guarantees that the terms in the infinite sum βnXt−n shrink in magnitude. If |β|≥1, the terms would grow or remain constant, and the series would fail to converge, leading to instability in the model.

Let’s break it down:

• If |β|<1: Each successive term βn becomes smaller and smaller as n increases, causing the series to converge to a finite value.
• If |β|=1: The terms do not decay, and the series either oscillates (if β=−1) or remains constant (if β=1).
• If |β|>1: The terms βn grow larger as n increases, causing the series to diverge.

Therefore, the necessary condition for the convergence of the series in an MA(1) process is:

|β|<1

This ensures that the MA(1) model is invertible, meaning we can express the white noise Zt as a finite sum of past values of Xt.

Generalization to Higher-Order MA Models

The concept of convergence extends to higher-order MA models as well. For an MA(q) model:

Xt=Zt+θ1Zt−1+θ2Zt−2+⋯+θqZt−q

We can express the noise Zt as an infinite sum of past values of Xt using a similar process. The condition for convergence in this case is that the roots of the characteristic polynomial:

1+θ1z+θ2z2+⋯+θqzq=0

must lie outside the unit circle in the complex plane. This ensures that the coefficients of the lagged terms (analogous to powers of β in MA(1)) decay as we go further back in time, leading to a convergent series.

Example: MA(1) Process

For an MA(1) process:

Xt=Zt+βZt−1

Using backward substitution, we express Zt as an infinite series:

Zt=Xt−βXt−1+β2Xt−2−β3Xt−3+…

The condition for this series to converge is:

|β|<1

This ensures that the terms βn decay as n→∞, leading to convergence of the series. In practical terms, if |β| is too large, the influence of past values remains strong, and the infinite series becomes unstable and non-convergent.

Thus, for any MA process, ensuring that the absolute values of the coefficients of the lagged noise terms are less than 1 is a key requirement for convergence and invertibility.