Difference Equations

Suppose a quantity retains 70% of its previous value and receives an addition of 2 at every step. The rule determines tomorrow's value from today's value; an initial value determines which particular path follows that rule. A difference equation formalizes this kind of discrete-time evolution.

The recursion and its starting conditions answer different questions. The recursion describes the mechanism, while the starting values locate one path within that mechanism. Two paths can begin far apart yet approach the same level, or begin close together and move farther apart, depending on the recursion's coefficients.

For time-series modeling, the central question is how an initial deviation or a new shock travels through the recursion. If its influence fades, the system forgets its initial conditions. If it persists, oscillates, or grows, forecasts inherit that behavior. Characteristic roots describe these patterns and connect the algebra of recurrences to AR stability and forecast uncertainty.

Worked calculation: a stable recursion

Take

$$ x_t=0.7x_{t-1}+2,\qquad x_0=10. $$

Substitution gives

$$ x_1=0.7(10)+2=9,\qquad x_2=0.7(9)+2=8.3. $$

The equilibrium level $x^\star$ solves $x^\star=0.7x^\star+2$, so

$$ x^\star=\frac{2}{1-0.7}=6.\overline{6}. $$

Subtracting the equilibrium from both sides gives

$$ x_t-x^\star=0.7(x_{t-1}-x^\star), $$

and therefore

$$ x_t=x^\star+0.7^t(x_0-x^\star). $$

Subtracting the equilibrium has removed the constant input 2 and left a simpler recursion for deviations. After $t$ steps, the initial deviation has been multiplied by $0.7^t$, which tends to zero. The level itself does not tend to zero: it tends to $2/(1-0.7)$. A coefficient between $-1$ and 0 would also shrink deviations, but their signs would alternate. A coefficient with magnitude above 1 would enlarge deviations instead.

Recurrence relations and initial conditions

A difference equation, also called a recurrence relation, defines each term of a sequence from one or more earlier terms. An order-$p$ recursion ordinarily needs $p$ initial values to select a particular solution from the family allowed by the equation. An explicit formula such as $a_n=3n+2$ gives $a_n$ directly. A difference equation instead describes how the sequence evolves from its previous values.

For example,

$$ a_n=5a_{n-1}-6a_{n-2} $$

is a second-order difference equation because $a_n$ depends on the two preceding terms.

Solving Difference Equations

For a homogeneous linear difference equation with constant coefficients, we often try a solution of the form

$$ a_n=\lambda^n. $$

Powers are a useful trial solution because shifting their time index changes them by a constant factor: $\lambda^{n-1}=\lambda^n/\lambda$. Substitution therefore turns the recurrence into an algebraic condition on that factor. Each characteristic root supplies one basic time pattern. Linearity then allows those patterns to be combined, with constants chosen to satisfy the initial conditions.

For

$$ a_n=5a_{n-1}-6a_{n-2}, $$

substitute $a_n=\lambda^n$:

$$ \lambda^n=5\lambda^{n-1}-6\lambda^{n-2}. $$

Dividing by $\lambda^{n-2}$ gives the characteristic equation

$$ \lambda^2-5\lambda+6=0. $$

Its roots are

$$ \lambda=2,\qquad \lambda=3. $$

Because the roots are distinct, the general solution is

$$ a_n=c_1 2^n+c_2 3^n, $$

where $c_1$ and $c_2$ are determined by the initial conditions.

Example: Solving with Initial Conditions

Suppose $a_0=4$ and $a_1=10$. Substituting $n=0$ gives

$$ a_0=c_1+c_2=4, $$

and substituting $n=1$ gives

$$ a_1=2c_1+3c_2=10. $$

Thus,

$$ \begin{aligned} c_1+c_2&=4,\\ 2c_1+3c_2&=10. \end{aligned} $$

Solving the system yields

$$ c_1=2,\qquad c_2=2. $$

The sequence therefore has the explicit form

$$ a_n=2\cdot2^n+2\cdot3^n. $$

The calculation has two separate stages. The characteristic equation determines the possible time patterns, here $2^n$ and $3^n$. The initial conditions then determine how much of each pattern appears in this particular sequence. Checking $n=0$ and $n=1$ in the final expression verifies that both starting values are recovered.

Higher-Order Difference Equations

A $k$th-order homogeneous linear difference equation can be written as

$$ a_n=\beta_1a_{n-1}+\beta_2a_{n-2}+\dots+\beta_ka_{n-k}. $$

Its characteristic equation is

$$ \lambda^k-\beta_1\lambda^{k-1}-\dots-\beta_k=0. $$

If the $k$ roots $\lambda_1,\lambda_2,\dots,\lambda_k$ are distinct, the general solution is

$$ a_n=c_1\lambda_1^n+c_2\lambda_2^n+\dots+c_k\lambda_k^n. $$

With distinct roots, the basic solutions are independent powers, and the initial conditions determine their coefficients. A repeated root does not supply enough independent powers on its own: factors such as $n\lambda^n$ supply the additional solutions. Complex-conjugate roots combine into real sine and cosine terms multiplied by a changing amplitude. The modulus controls decay or growth; the angle controls oscillation. This explains why a linear recursion can produce cycles without an explicitly seasonal input.

Example: Fibonacci Sequence

Consider the Fibonacci-type recurrence

$$ a_n=a_{n-1}+a_{n-2} $$

with initial conditions $a_0=2$ and $a_1=3$. Its characteristic equation is

$$ \lambda^2-\lambda-1=0, $$

with roots

$$ \lambda_1=\frac{1-\sqrt5}{2},\qquad \lambda_2=\frac{1+\sqrt5}{2}. $$

The general solution is therefore

$$ a_n=c_1\left(\frac{1-\sqrt5}{2}\right)^n+c_2\left(\frac{1+\sqrt5}{2}\right)^n. $$

The initial conditions imply

$$ c_1+c_2=2 $$

and

$$ c_1\left(\frac{1-\sqrt5}{2}\right)+c_2\left(\frac{1+\sqrt5}{2}\right)=3. $$

Solving gives

$$ c_1=1-\frac{2}{\sqrt5},\qquad c_2=1+\frac{2}{\sqrt5}. $$

Equivalently, because this sequence is the ordinary Fibonacci sequence shifted by three indices,

$$ a_n=\frac{1}{\sqrt5}\left[\left(\frac{1+\sqrt5}{2}\right)^{n+3}-\left(\frac{1-\sqrt5}{2}\right)^{n+3}\right]. $$

The closed form makes a feature less obvious from the recurrence: the positive root $(1+\sqrt5)/2$ exceeds 1, so its contribution eventually dominates. The other root has magnitude below 1 and its alternating contribution fades. This Fibonacci-type example is therefore useful for understanding root behavior, but it is not an example of a stable stationary AR recursion.

Relation to Differential Equations

Linear difference equations play a role in discrete time that is analogous to linear differential equations in continuous time. For example, a $k$th-order linear ordinary differential equation can be written as

$$ y^{(k)}=\beta_1y^{(k-1)}+\dots+\beta_ky. $$

Trying a solution of the form $y(t)=e^{\lambda t}$ produces a characteristic polynomial. Its roots determine the exponential components of the continuous-time solution, just as powers of characteristic roots determine the solution of a difference equation.

The analogy is useful, but the stability conditions use different boundaries. In discrete time, powers $r^t$ decay when $|r|<1$. In continuous time, exponentials $e^{\lambda t}$ decay when the real part of $\lambda$ is negative. Keep the solution form in view when applying either condition.

Stability, initial conditions, and random forcing

For a homogeneous AR(2)-style recursion

$$ x_t=\phi_1x_{t-1}+\phi_2x_{t-2}, $$

try a solution $x_t=r^t$. Substitution gives

$$ r^2-\phi_1r-\phi_2=0. $$

Apply the root condition to the homogeneous deviation, not automatically to every part of the observed series. Roots inside the unit circle make the effect of initial conditions vanish. A constant or changing external input can still keep the level away from zero. With random shocks continually entering, a stable path continues to fluctuate even though the influence of each individual shock fades.

Numerical first-order example

For

$$ x_t=0.7x_{t-1}+2,\qquad x_0=10, $$

the equilibrium is $x^\star=2/(1-0.7)=6.\overline6$. The first values are

$$ x_1=9,\qquad x_2=8.3,\qquad x_3=7.81. $$

The deviation from equilibrium is multiplied by $0.7$ at each step:

$$ x_t-x^\star=0.7^t(x_0-x^\star). $$

The same algebra explains why an AR coefficient near 1 produces persistent responses and why a coefficient with absolute value above 1 is unstable.

Forced recursions and shocks

With shocks,

$$ x_t=c+\phi x_{t-1}+\varepsilon_t, $$

the realized path contains the starting condition, the repeated constant input, and all shocks that have arrived so far. At a finite horizon, repeated substitution gives the initial term $\phi^t x_0$, the constant contribution $c\sum_{j=0}^{t-1}\phi^j$, and the shock contribution $\sum_{j=0}^{t-1}\phi^j\varepsilon_{t-j}$. Separating them prevents the phrase “the system settles” from suggesting that a stochastic path eventually becomes constant.

For $|\phi|<1$, after the initial transient has decayed,

$$ x_t=\frac{c}{1-\phi}+\sum_{j=0}^{\infty}\phi^j\varepsilon_{t-j}. $$

The infinite shock sum describes the stationary past-driven solution when $|\phi|<1$. A process started at a fixed value at a finite date also has an initial transient; it approaches the stationary behavior as that transient decays. The coefficient on a shock $j$ periods ago is $\phi^j$. Those same coefficients appear squared when calculating the uncertainty contributed by shocks that have not yet arrived.

Read the distance between a path and its equilibrium, or between two paths receiving the same inputs. In the stable cases that distance shrinks; in the unstable cases it grows. This comparison isolates the effect of the recursion from variation caused by different shocks, making the root condition visible without relying on the appearance of one noisy path.

A runnable companion is the difference equations notebook.

Formula reference