Statistics Notes

Multiple Comparisons

October 13, 2026
Statistics Notes
EN

When conducting multiple hypothesis tests, the probability of making at least one Type I error—falsely rejecting a true null hypothesis—increases. This is known as the multiple comparisons problem or, in some contexts, the look-elsewhere effect...

Type i and Type Ii Errors

October 13, 2026
Statistics Notes
EN

Hypothesis testing allows researchers to evaluate claims about a population using sample data. We begin with a null hypothesis, $H_0$, which represents the claim being tested, and an alternative hypothesis, $H_a$, which represents a competing claim. Because decisions are based on sample data, errors...

Analysis of Categorical Data

October 13, 2026
Statistics Notes
EN

The chi-square ($\chi^2$) test is a family of statistical tests for categorical count data. These tests compare observed frequencies with the frequencies expected under a null hypothesis...

Confidence Intervals

October 13, 2026
Statistics Notes
EN

A confidence interval (CI) is a range of plausible values for a population parameter, such as a mean or proportion, calculated from sample data. It supplements a point estimate by showing the uncertainty around that estimate...

Hypothesis Testing

October 13, 2026
Statistics Notes
EN

Hypothesis testing is a statistical tool used to draw conclusions about populations based on sample data. It is widely applied in scientific research, from evaluating new treatments in clinical trials to studying customer behavior in business analytics...

Null Hypothesis

October 13, 2026
Statistics Notes
EN

Statistical hypothesis testing is a method for using sample data to make inferences about a population. Understanding null and alternative hypotheses, along with how p-values are calculated and interpreted, is essential for applying hypothesis tests correctly...

Point Estimation

October 13, 2026
Statistics Notes
EN

A parameter is an unknown feature of a population or statistical model, such as a mean $\mu$, variance $\sigma^2$, probability $p$, or regression coefficient $\beta$. A point estimator is a rule or statistic computed from sample data and used to estimate that parameter...

Stationarity

October 13, 2026
Statistics Notes
EN

Stationarity describes which probabilistic features of a time series remain stable when the time origin is shifted. Weak stationarity focuses on a constant mean and variance together with an autocovariance that depends only on lag, while strict stationarity requires the full joint distribution to be...

Autoregressive Models

October 13, 2026
Statistics Notes
EN

Autoregressive models describe a time series whose current value depends linearly on its own recent history plus a new innovation. The lag coefficients determine how strongly past observations persist into the present and whether the process tends to decay smoothly, alternate, or oscillate...

Frequency Domain Analysis

October 13, 2026
Statistics Notes
EN

Frequency-domain analysis describes a time series by the cycle lengths that contribute to its variation rather than by relationships at individual lags. The periodogram and spectral density provide a complementary view to the ACF, translating repeated temporal structure into peaks at corresponding f...

Stochastic Processes and White Noise

October 13, 2026
Statistics Notes
EN

A stochastic process is a collection of random variables indexed by time, and an observed time series is one realized path from that process. This viewpoint separates the data we actually see from the probabilistic mechanism used to describe possible paths and future uncertainty...

Dynamic Regression

October 13, 2026
Statistics Notes
EN

Dynamic regression combines external predictors with time-series structure so that the conditional mean can respond to explanatory variables without assuming the remaining errors are independent. Lagged predictors can represent delayed effects, while ARMA or ARIMA errors capture serial dependence le...

Multivariate Time Series

October 13, 2026
Statistics Notes
EN

Multivariate time-series models describe several evolving variables jointly so that cross-lag feedback, shared shocks, and predictive relationships can be represented within one system. They are useful when separate univariate models would ignore information carried by the histories of related serie...

Random Walk

October 13, 2026
Statistics Notes
EN

A random walk models a level that changes by accumulating new shocks over time. Each innovation is temporary as an increment but permanent in the level, so the process can drift far from its starting point even when the expected one-step change is zero...

Difference Equations

October 13, 2026
Statistics Notes
EN

A difference equation describes how a discrete-time quantity evolves from its earlier values. Rather than specifying every term directly, it gives a recursive rule, so the behavior of the sequence is determined by the recursion together with its initial conditions...

Financial Time Series Models

October 13, 2026
Statistics Notes
EN

Financial time series often separate naturally into a price level, a return process, and a time-varying scale of uncertainty. Returns may show little predictable movement in their conditional mean while their magnitude clusters over time, producing periods of calm and periods of elevated volatility...

State Space Models

October 13, 2026
Statistics Notes
EN

State-space models separate a time series into an unobserved state that evolves through time and an observation process that measures that state with noise. This framework is flexible enough to represent latent levels, trends, seasonal components, regression effects, missing observations, and many f...

Autocovariance Function

October 13, 2026
Statistics Notes
EN

Autocovariance measures how a time series varies jointly with lagged versions of itself. Lag 0 is the variance, while nonzero lags show the direction and scale of linear dependence between observations separated in time, expressed in the squared units of the original series...

Autocorrelation Function

October 13, 2026
Statistics Notes
EN

The autocorrelation function measures the unit-free linear association between a time series and lagged versions of itself. The partial autocorrelation function asks a narrower question: how much direct linear relationship remains at a given lag after the intervening lags have been accounted for...

Arima Models

October 13, 2026
Statistics Notes
EN

ARMA models describe stationary linear dependence through a combination of autoregressive terms, which use past observations, and moving-average terms, which use current and past innovations. ARIMA extends that structure by differencing a series with a stochastic trend, while SARIMA adds seasonal di...

Seasonality and Trends

October 13, 2026
Statistics Notes
EN

Trend and seasonality are systematic forms of time structure that operate on different scales. A trend is a persistent long-run movement in level or slope, while seasonality is a pattern that repeats at a fixed and known period such as day of week, month of year, or quarter...

Regression with Arma Errors

October 13, 2026
Statistics Notes
EN

Regression with ARMA errors separates two kinds of structure that often appear together in time series. The regression explains how observed predictors shift the conditional mean, while the ARMA component models serial dependence that remains in the unexplained part...

Forecasting

October 13, 2026
Statistics Notes
EN

A time-series forecast is a conditional statement about a future observation given the information available at a specific forecast origin. The horizon, target functional, and information set are part of the forecast itself, so a point prediction without its timing and uncertainty is an incomplete d...

Moving Average Models

October 13, 2026
Statistics Notes
EN

A moving-average model represents the current observation as a finite weighted combination of the current innovation and a fixed number of past innovations. Because each shock enters the model for only a limited number of periods, an MA($q$) process has a finite direct shock response and a theoretic...

Time Series

October 13, 2026
Statistics Notes
EN

A time series is an ordered set of observations together with an information structure: timestamps, sampling rules, and sequence position determine what can depend on what and what information is available for prediction. Treating the same values as an unordered sample would discard much of the stru...

Yule Walker Equations

October 13, 2026
Statistics Notes
EN

The Yule-Walker equations connect an autoregressive model's coefficients to its autocovariances and autocorrelations. They translate a recursion written in terms of lagged observations into a set of moment relationships that can be studied theoretically or estimated from data...

Backward Shift Operator

October 13, 2026
Statistics Notes
EN

The backward shift operator is a compact notation for referring to earlier observations in a time series. Instead of writing each lag separately, it lets lagged values, differences, and model equations be expressed as polynomials in a single operator...

Series

October 13, 2026
Statistics Notes
EN

Sequences and series provide the convergence language behind many time-series representations. A sequence describes an ordered set of terms, while a series asks whether the cumulative effect of those terms approaches a finite limit...

Time Series Modeling

October 13, 2026
Statistics Notes
EN

Time-series modeling is an iterative process of specifying structure, estimating parameters, diagnosing what remains unexplained, and testing forecasts on future-like data. Choosing an equation is only one step; transformations, initial conditions, residual behavior, and the forecast information set...

Statistical Moments and Time Series

October 13, 2026
Statistics Notes
EN

Statistical moments summarize features such as the center, spread, and joint variation of a distribution. In time series, those same quantities acquire a time dimension: the mean and variance may change across the record, and covariance becomes a function of lag as well as scale...

Randomness Tests

October 13, 2026
Statistics Notes
EN

Randomness tests look for specific kinds of structure that should not appear under a stated null model. In time-series work, they are most useful as diagnostics: one test may target linear autocorrelation, another unusual turning-point behavior, another monotone trend, and another dependence in squa...

Axioms of Probability

October 13, 2026
Statistics Notes
EN

Probability theory is built on a small set of principles, called axioms, that define how probability measures behave. These axioms, formalized by the Russian mathematician Andrey Kolmogorov, provide the foundation for the rules used throughout probability theory...

Geometric Probability

October 13, 2026
Statistics Notes
EN

Geometric probability is a fascinating branch of probability theory where outcomes are associated with geometric figures and their measures—such as lengths, areas, and volumes—rather than discrete numerical outcomes. It often deals with continuous random variables and employs integral calculus to ca...

Total Probability

October 13, 2026
Statistics Notes
EN

The law of total probability allows for the computation of the probability of an event A based on a set of mutually exclusive and exhaustive events. It's particularly useful when the overall sample space is divided into several distinct scenarios, or partitions, that cover all possible outcomes. The...

Conditional Probability

October 13, 2026
Statistics Notes
EN

Conditional Probability is the likelihood of an event occurring given that another event has already occurred. It is denoted as $P(A|B)$, representing the probability of event $A$ happening, assuming event $B$ has already taken place. This concept is crucial in understanding dependent events in prob...

Introduction to Probability

October 13, 2026
Statistics Notes
EN

Probability theory offers a structured approach to assessing the probability of events, allowing for logical and systematic reasoning about their likelihood...

Probability Tree

October 13, 2026
Statistics Notes
EN

Probability trees are a visual representation of all possible outcomes of a probabilistic experiment and the paths leading to these outcomes. They are especially helpful in understanding sequences of events, particularly when these events are conditional on previous outcomes...

Descriptive Statistics

October 13, 2026
Statistics Notes
EN

Descriptive statistics summarize the main characteristics of a dataset or sample. They help us understand data by describing its frequency, center, spread, and overall distribution...

Introduction to Statistics

October 13, 2026
Statistics Notes
EN

Statistics is the science of learning from data. It provides methods for collecting, summarizing, analyzing, and interpreting observations so that we can describe patterns, quantify uncertainty, and make informed decisions...

F Distribution

October 13, 2026
Statistics Notes
EN

The F-distribution, also known as the Fisher-Snedecor distribution, is a continuous probability distribution of the ratio $F=(U/d_1)/(V/d_2)$ of two independent chi-square variables $U\sim\chi^2(d_1)$ and $V\sim\chi^2(d_2)$, each divided by its degrees of freedom. It arises in hypothesis testing whe...

Student T Distribution

October 13, 2026
Statistics Notes
EN

Student's t-distribution is a continuous probability distribution that arises when a normally distributed quantity is standardized using an estimated standard deviation rather than a known population standard deviation. Formally, if $Z\sim N(0,1)$ and $V\sim\chi^2(\nu)$ are independent, then $T=Z/\s...

Chi Square Distribution

October 13, 2026
Statistics Notes
EN

A chi-square distribution is a continuous probability distribution of the sum of the squares of k independent standard normal random variables. The chi-square distribution is denoted as $X \sim \chi^2(k)$, where k is the number of degrees of freedom...

Central Limit Theorem

October 13, 2026
Statistics Notes
EN

The Central Limit Theorem (CLT) is a fundamental result in statistics. It explains why the distribution of sample means often approaches a normal distribution as the sample size increases, even when the population itself is not normally distributed...

Standard Error and Lln

October 13, 2026
Statistics Notes
EN

Expected Value (E), also known as the mean, is the long-run average of a random variable, representing the value we anticipate on average from repeated random draws from a population...

Bayesian vs Frequentist

October 13, 2026
Statistics Notes
EN

Bayesian and frequentist statistics are two major approaches to statistical inference. Both use sample data to learn about an underlying population or data-generating process, but they differ in how they interpret probability, represent uncertainty, and draw conclusions about unknown parameters...

Multiple Regression

October 13, 2026
Statistics Notes
EN

Multiple linear regression is a statistical technique used to model the relationship between a single dependent variable and two or more independent variables. It extends the concept of simple linear regression by incorporating multiple predictors to explain the variability in the dependent variable...

Logistic Regression

October 13, 2026
Statistics Notes
EN

Logistic regression is a statistical method used for modeling the probability of a binary outcome based on one or more predictor variables. It is widely used in various fields such as medicine, social sciences, and machine learning for classification problems where the dependent variable is dichotom...

Simple Linear Regression

October 13, 2026
Statistics Notes
EN

Simple linear regression is a statistical method used to model the relationship between a single dependent variable and one independent variable. It aims to find the best-fitting straight line through the data points, which can be used to predict the dependent variable based on the independent varia...

Analysis of Variance

October 13, 2026
Statistics Notes
EN

Does peer assessment enhance student learning...

Statistical Moments

October 13, 2026
Statistics Notes
EN

In both statistics and mechanics the word moment measures how much "leverage" the values of a quantity exert about a chosen reference point. In statistics the leverage is exerted by probability mass, in mechanics by physical mass, but the mathematics is identical: take a distance from the reference ...

Normal Curve and z Score

October 13, 2026
Statistics Notes
EN

A normal distribution (often referred to as the normal curve or Gaussian distribution) is a continuous probability distribution that is symmetric about the mean, where most of the observations cluster around the central peak and taper off symmetrically towards both ends. Many real-world datasets suc...

Introduction to Distributions

October 13, 2026
Statistics Notes
EN

A distribution is a function that describes the probability of a random variable. It helps to understand the underlying patterns and characteristics of a dataset. Distributions are widely used in statistics, data analysis, and machine learning for tasks such as hypothesis testing, confidence interva...

Validation and Model Selection

October 13, 2026
Statistics Notes
EN

A fitted model is optimized using observed data. Model assessment asks a different question: how well will the modeling procedure perform on new data generated under comparable conditions...

Metrics

October 13, 2026
Statistics Notes
EN

Evaluation metrics are essential tools for assessing the performance of statistical and machine learning models. They provide quantitative measures that help us understand how well a model is performing and where improvements can be made. In both classification and regression tasks, selecting approp...

Resampling

October 13, 2026
Statistics Notes
EN

Statistical inference often involves estimating population parameters and constructing confidence intervals based on sample data. Traditional methods rely on assumptions about the sampling distribution of estimators, such as normality and known standard errors. However, these assumptions may not hol...

Geostatistics

October 13, 2026
Statistics Notes
EN

Geostatistics is used when observations are tied to locations and nearby values may be more similar than distant values...

Spatial Weights

October 13, 2026
Statistics Notes
EN

A spatial weights matrix turns a qualitative statement such as...

Point Processes

October 13, 2026
Statistics Notes
EN

A spatial point process is used when the observed data are the event locations themselves...

Spatial Autocorrelation

October 13, 2026
Statistics Notes
EN

Spatial autocorrelation asks whether values attached to locations show a systematic spatial pattern...

Kriging

October 13, 2026
Statistics Notes
EN

Kriging is a method for predicting a spatial variable at unsampled locations using a model of spatial dependence...

Spatial Data and Distance

October 13, 2026
Statistics Notes
EN

Spatial analysis begins before any spatial statistic is calculated...

Spatial Regression

October 13, 2026
Statistics Notes
EN

Regression models the mean relationship between an outcome and its predictors...

Spatial Validation

October 13, 2026
Statistics Notes
EN

Spatial model performance depends not only on which observations are held out, but also on where those observations are located relative to the training data...

Covariance

October 13, 2026
Statistics Notes
EN

Covariance is a fundamental statistical measure that quantifies the degree to which two random variables change together. It indicates the direction of the linear relationship between variables...

Correlation

October 13, 2026
Statistics Notes
EN

Correlation is a statistical measure that quantifies the strength and direction of the linear relationship between two variables. It is a fundamental concept in statistics, enabling researchers and analysts to understand how one variable may predict or relate to another. The most commonly used corre...

Negative Binomial Distribution

October 13, 2026
Statistics Notes
EN

A discrete random variable X follows a negative binomial distribution if it represents the number of trials required to achieve a specified number of successes in a sequence of independent Bernoulli trials. The negative binomial distribution is often denoted as $X \sim \text{NegBinomial}(r, p)$, whe...

Binomial Distribution

October 13, 2026
Statistics Notes
EN

A discrete random variable $X$ follows a binomial distribution if it represents the number of successes in a fixed number of Bernoulli trials with the same probability of success. The binomial distribution is denoted as $X \sim \text{Binomial}(n, p)$, where $n$ is the number of trials and $p$ is the...

Geometric Distribution

October 13, 2026
Statistics Notes
EN

A discrete random variable X follows a geometric distribution if it represents the number of trials needed to get the first success in a sequence of Bernoulli trials. The geometric distribution is denoted as $X \sim \text{Geometric}(p)$, where $p$ is the probability of success on each trial. The sup...

Poisson Distribution

October 13, 2026
Statistics Notes
EN

A discrete random variable X follows a Poisson distribution if the events occur independently and at a constant average rate. The Poisson distribution is denoted as $X \sim \text{Poisson}(\lambda)$, where $\lambda$ is the average rate (or mean) of events occurring in a given interval...

Log Normal Distribution

October 13, 2026
Statistics Notes
EN

A continuous random variable X follows a log-normal distribution if its natural logarithm is normally distributed. The log-normal distribution is useful in modeling continuous random variables that are constrained to be positive. It is denoted as $X \sim \text{LogNormal}(\mu, \sigma^2)$, where $\mu...

Beta Distribution

October 13, 2026
Statistics Notes
EN

A continuous random variable $X$ follows a beta distribution when its support is the interval $[0,1]$ and its shape is governed by two positive parameters. The beta distribution is denoted as $X \sim \text{Beta}(\alpha, \beta)$, where $\alpha$ and $\beta$ are shape parameters. A shifted and scaled b...

Exponential Distribution

October 13, 2026
Statistics Notes
EN

The exponential distribution is a continuous probability distribution that models the time between events in a Poisson point process. The exponential distribution is denoted as $X \sim \text{Exp}(\lambda)$, where $\lambda$ is the rate parameter...

Uniform Distribution

October 13, 2026
Statistics Notes
EN

A continuous random variable $X$ follows a uniform distribution over an interval $[a, b]$ if it has a constant probability density over that interval. The uniform distribution is denoted as $X \sim \text{Uniform}(a, b)$...

Normal Distribution

October 13, 2026
Statistics Notes
EN

A continuous random variable $X$ follows a normal distribution, denoted as $X \sim \mathcal{N}(\mu,\,\sigma^{2})$, when it has the familiar symmetric bell-shaped density. Its support is the entire real line, so extreme values are possible but receive increasingly small probability in the tails. The ...

Gamma Distribution

October 13, 2026
Statistics Notes
EN

A continuous random variable $X$ follows a gamma distribution when it is a positive, right-skewed quantity with the density below. For integer shape $\alpha = k$ it is the waiting time until the $k$-th event of a Poisson process with rate $\beta$ (a sum of $k$ independent exponential waiting times)...

Forecast Evaluation

September 14, 2026
Statistics Notes
EN

Forecast evaluation measures how an entire forecasting procedure performs on observations that were genuinely unknown when each prediction was made. It is therefore different from in-sample fit: transformations, feature construction, parameter estimation, predictor values, and model selection all ha...

Invertibility

September 14, 2026
Statistics Notes
EN

Invertibility is the condition that lets a moving-average or ARMA model recover its unobserved innovations from the observed series in a stable way. It turns the shock representation of the model into a usable past-based filter whose coefficients decay rather than grow without bound...

Bayes Theorem

September 13, 2026
Statistics Notes
EN

Bayes' theorem provides a way to update the probability of an event when new evidence becomes available. It connects conditional probabilities and allows us to revise an initial probability, or prior, in light of additional information...

Sampling Distributions

September 13, 2026
Statistics Notes
EN

A statistic is computed from a sample, but before the sample is observed the statistic is itself a random variable. Its probability distribution over hypothetical repeated samples is called its sampling distribution...

Joint Distributions

September 13, 2026
Statistics Notes
EN

A univariate distribution describes a single random variable. A joint distribution describes several random variables simultaneously and contains the information needed to reason about dependence, conditional behavior, and functions of multiple variables...