Last modified: March 28, 2024
This article is written in: πΊπΈ
Sequences and Series
Sequences
A sequence is an ordered list of numbers that can be viewed as a function mapping each natural number to a specific value . More formally, a sequence is a function whose domain is the set of natural numbers, and the values are called the terms of the sequence:
A sequence can either approach a particular value, in which case it is said to converge, or it can increase or oscillate indefinitely, in which case it diverges.
A sequence converges to a limit if, as becomes very large, the values of get arbitrarily close to . Mathematically, this is expressed as:
This means for any small number , there exists a large number such that for all , the distance between and is smaller than . If no such limit exists, the sequence diverges.
Examples of Sequences
- Convergent Sequence:
Consider the sequence . This gives the sequence:
As , the terms of the sequence approach 1:
Therefore, this sequence converges to 1.
- Divergent Sequence:
Now consider the sequence . This gives the sequence:
As , the terms of the sequence grow without bound, meaning this sequence diverges.
- Another Divergent Sequence:
Another example of a divergent sequence is , which gives:
As , the sequence grows indefinitely, and hence it diverges.
- Convergent Sequence:
Consider the sequence , which gives:
As , the terms of this sequence get smaller and smaller, approaching 0:
Hence, this sequence converges to 0.
Partial Sums
The partial sum of a sequence is the sum of the first terms of the sequence:
Partial sums are used to analyze the behavior of series, where the idea is to observe whether the sum of the terms converges to a specific value as more terms are added. For example, consider the following partial sums:
A series formed by a sequence is said to converge if the sequence of partial sums converges as .
Series
A series is the sum of the terms of a sequence. If the sequence of partial sums converges to a limit , the series is said to converge to that limit . Mathematically, the infinite series is written as:
If the partial sums do not approach a finite limit as increases, then the series is said to be divergent.
Geometric Series and Rational Functions
A geometric series is a series of the form:
This result can be used to represent certain rational functions as infinite series. For example:
This is useful in time series analysis when dealing with autoregressive models and other representations involving rational functions.
Examples of Convergent Series
I. Geometric Series:
A geometric series is a series where each term is a constant multiple (called the common ratio) of the previous term. A simple example is the following geometric series:
This series converges to:
II. P-Series (convergent for ):
A p-series is of the form:
For , the series converges to:
III. Alternating Series:
An alternating series is a series where the signs of the terms alternate between positive and negative. A famous example is:
This series converges to:
Examples of Divergent Series
I. Geometric Series with growth factor greater than 1:
Consider the geometric series:
Since the ratio between successive terms is greater than 1, this series diverges as the partial sums grow without bound.
II. Arithmetic Series:
In an arithmetic series, the difference between successive terms is constant. For example:
The terms grow linearly, and the series diverges as .
III. Harmonic Series:
The harmonic series is:
Although the terms approach 0 as , the sum of the terms grows without bound. Hence, the harmonic series diverges.
Absolute Convergence
A series is absolutely convergent if the series of the absolute values of its terms is convergent:
Absolute convergence implies convergence. This is a stronger condition than regular convergence, as a series can converge without being absolutely convergent (for example, alternating series).
Convergence Tests
When dealing with infinite series, itβs important to determine whether the series converges (approaches a finite value) or diverges (grows without bound or oscillates without settling). To do this, mathematicians use various convergence tests, each suited for different types of series. Below are some of the most commonly used tests for determining whether an infinite series converges or diverges:
I. Integral Test:
The integral test compares a series to the integral of a continuous, positive, decreasing function. Suppose is a sequence and is a continuous, positive, decreasing function such that for all . If the integral of from 1 to infinity converges, then the series converges. Otherwise, the series diverges.
Example: Consider the series . Compare this to the integral , which converges to 1. Hence, the series converges.
II. Comparison Test:
The comparison test compares the given series to a known convergent or divergent series. If the terms of the series are smaller than the terms of a known convergent series , and all terms are positive, then the series also converges. Similarly, if the terms are larger than those of a known divergent series, the series diverges.
Example: To check if converges, compare it to , which is a convergent p-series with . Since the terms of are smaller, the series converges by comparison.
III. Limit Comparison Test:
The limit comparison test is useful when the terms of the series are not directly comparable to a known series but have similar behavior as . If the limit of the ratio between the terms of two series is a finite, positive constant, both series either converge or diverge together.
Example: Consider the series . Compare it to . The limit of the ratio of terms is:
Since the comparison series converges, the original series also converges.
IV. Alternating Series Test (Leibniz's Test):
This test is used for series where the terms alternate in sign. For an alternating series , if the absolute value of the terms decreases monotonically (i.e., ) and , then the series converges.
Example: Consider the alternating harmonic series , which is of the form . Since decreases and approaches 0 as , the series converges.
V. Ratio Test:
The ratio test is particularly useful for series involving factorials or exponential terms. To apply the ratio test, compute the limit of the absolute value of the ratio of successive terms:
- If , the series converges.
- If , the series diverges.
- If , the test is inconclusive.
Example: For the series , applying the ratio test gives:
Since , the series converges.
VI. Root Test:
The root test, or Cauchy's root test, involves taking the -th root of the absolute value of the terms of the series. Let:
- If , the series converges.
- If , the series diverges.
- If , the test is inconclusive.
Example: For the series , applying the root test gives:
Since , the series converges.
Mean-Square Convergence
In the context of stochastic processes, mean-square convergence is an important concept for analyzing the behavior of sequences of random variables. Suppose is a sequence of random variables representing a stochastic process. We say that converges to a random variable in the mean-square sense if the expected value of the squared difference between and approaches zero as becomes large. Formally, this is expressed as:
This definition means that as , the random variables become increasingly close to in the sense of the expected value of their squared differences, providing a measure of convergence in a probabilistic sense.
Inverting the MA(1) Model
Consider the MA(1) (Moving Average of order 1) model, which is commonly used in time series analysis. In this model, the process at time is defined as:
where represents white noise, which is a sequence of independent, identically distributed random variables with zero mean and constant variance . The parameter is a constant that defines the relationship between the current observation and the previous white noise term.
We are interested in expressing as an infinite sum involving the past values of . Through algebraic manipulation, we can express as:
This infinite sum can be shown to converge in the mean-square sense, provided certain conditions on are satisfied. Specifically, we need to ensure that the sum of these terms remains bounded as , which ensures that the approximation becomes increasingly close to .
Autocovariance Function of the MA(1) Process
The autocovariance function of a stochastic process measures the covariance between and for different time lags . For the MA(1) process, the autocovariance function is relatively simple due to the limited memory of the process, which only depends on the current and previous white noise terms.
The autocovariance function for the MA(1) process is given by:
For negative values of , the autocovariance function is symmetric, meaning that .
This shows that the MA(1) process has non-zero autocovariances only for the first lag (i.e., between and ), and beyond that, the covariance is zero due to the white noise properties of .
Series Convergence in Mean-Square Sense
To investigate the convergence of the infinite series expression for , consider the partial sum of the first terms:
We want to ensure that this sum converges to in the mean-square sense. In other words, we need:
This expression represents the expected value of the squared difference between the partial sum and the true value of , and we aim to show that this difference diminishes as more terms are added to the sum.
Expanding this expression:
Breaking this down into individual terms: - The first term involves the expected value of the squared partial sum: . - The second term represents the cross terms between the partial sum and . - The third term is the variance of , which is .
To achieve mean-square convergence, the total expression must approach 0 as .
Condition for Convergence
For the series to converge in the mean-square sense, we require that the individual terms involving powers of decay sufficiently fast. Specifically, we need:
This will only occur if:
When , the powers of diminish as increases, ensuring that the sum remains bounded and that the partial sums converge to .
Invertibility Condition
The condition is also known as the invertibility condition for the MA(1) process. This condition guarantees that the moving average process can be expressed as an infinite autoregressive (AR) process. In terms of the polynomial (where is the backshift operator), the invertibility condition states that the root of the polynomial must lie outside the unit circle in the complex plane, meaning:
Thus, the invertibility condition ensures that the MA(1) process can be uniquely represented as an AR(β) process, which is crucial for the identification and estimation of time series models.