Statistics Notes

Multiple Comparisons

EN

When conducting multiple hypothesis tests, the probability of making at least one Type I error—falsely rejecting a true null hypothesis—increases. This is known as the multiple comparisons problem or, in some contexts, the look-elsewhere effect...

Type i and Type Ii Errors

EN

Hypothesis testing allows researchers to evaluate claims about a population using sample data. We begin with a null hypothesis, H₀, which represents the claim being tested, and an alternative hypothesis, H_a, which represents a competing claim. Because decisions are based on sample data, errors are ...

Analysis of Categorical Data

EN

The chi-square (χ²) test is a family of statistical tests for categorical count data. These tests compare observed frequencies with the frequencies expected under a null hypothesis...

Confidence Intervals

EN

A confidence interval is the output of a sample-based procedure designed to cover a fixed population or model parameter at a stated rate over repeated samples. It supplements a point estimate with uncertainty under the sampling model. The interval's interpretation depends on that procedure and its a...

Hypothesis Testing

EN

A hypothesis test compares an observed statistic with its distribution under a specified null model. A difference matters statistically when it is large relative to the variation that model predicts, rather than simply large in raw units. Applications include comparing treatments, checking productio...

Null Hypothesis

EN

Statistical hypothesis testing is a method for using sample data to make inferences about a population. Understanding null and alternative hypotheses, along with how p-values are calculated and interpreted, is essential for applying hypothesis tests correctly...

Point Estimation

EN

A parameter is an unknown feature of a population or statistical model, such as a mean μ, variance σ², probability p, or regression coefficient β. A point estimator is a rule or statistic computed from sample data and used to estimate that parameter...

Axioms of Probability

EN

Probability theory is built on a small set of principles, called axioms, that define how probability measures behave. These axioms, formalized by the Russian mathematician Andrey Kolmogorov, provide the foundation for the rules used throughout probability theory...

Geometric Probability

EN

Geometric probability describes random outcomes represented by locations, directions, lengths, or shapes. Under a uniform model, probability is proportional to the relevant geometric measure, such as length on a line or area in a plane. The sampling mechanism determines that measure: uniform angle, ...

Total Probability

EN

The law of total probability computes an event's overall probability by separating the possible cases that can produce it. The cases must be disjoint and cover the sample space, so every outcome contributes exactly once. Within each case, multiply the event's conditional probability by the chance of...

Bayes Theorem

EN

Bayes' theorem reverses the direction of a conditional probability. It connects the chance of evidence under a proposed event, P(B | A), with the chance of that event after seeing the evidence, P(A | B). These quantities use different reference groups. Updating requires both the evidence rate within...

Conditional Probability

EN

Conditional probability changes the reference set after information is supplied. In P(A | B), we retain outcomes in B and ask what fraction of their probability also lies in A. The vertical bar means “given”; it does not require B to happen earlier in time or cause A. The multiplication rule reverse...

Introduction to Probability

EN

A probability model specifies possible outcomes and assigns probabilities to events made from those outcomes. The sample space tells us what can happen; the probability law tells us how likely those possibilities are. Listing outcomes does not make them equally likely. For example, two fair dice hav...

Probability Tree

EN

A probability tree represents an experiment through successive cases. Each branch probability is conditional on the history at its starting node. Multiplying along a path gives a joint probability, and adding complete, disjoint paths gives an event probability. The tree therefore records both which ...

Descriptive Statistics

EN

Descriptive statistics turn a collection of observations into summaries of frequency, center, spread, and shape. Each summary answers a particular question. A frequency table counts how often values occur; a mean preserves the total when multiplied by the number of observations; a percentile describ...

Introduction to Statistics

EN

Statistics is the science of learning from data. It provides methods for collecting, summarizing, analyzing, and interpreting observations so that we can describe patterns, quantify uncertainty, and make informed decisions...

F Distribution

EN

The F-distribution, also known as the Fisher-Snedecor distribution, is a continuous probability distribution of the ratio F = (U/d₁)/(V/d₂) of two independent chi-square variables U ~ χ²(d₁) and V ~ χ²(d₂), each divided by its degrees of freedom. It arises in hypothesis testing when comparing the va...

Student T Distribution

EN

Student's t-distribution is a continuous probability distribution that arises when a normally distributed quantity is standardized using an estimated standard deviation rather than a known population standard deviation. Formally, if Z ~ N(0,1) and V ~ χ²(ν) are independent, then T = Z/√(V/ν) ~ t(ν)...

Sampling Distributions

EN

A statistic is computed from a sample, but before the sample is observed the statistic is itself a random variable. Its probability distribution over hypothetical repeated samples is called its sampling distribution...

Chi Square Distribution

EN

A chi-square distribution is a continuous probability distribution of the sum of the squares of k independent standard normal random variables. The chi-square distribution is denoted as X ~ χ²(k), where k is the number of degrees of freedom...

Central Limit Theorem

EN

The central limit theorem describes the fluctuations of a sum or average around its expectation. For independent, identically distributed observations with finite, positive variance, the centered and standardized mean approaches a standard normal distribution as sample size grows. The population can...

Standard Error and Lln

EN

A standard deviation describes variability of a specified random quantity. When that quantity is an estimator, its standard deviation is called a standard error. The law of large numbers explains why certain averages concentrate as the amount of suitable information grows. These ideas are connected...

Bayesian vs Frequentist

EN

Bayesian and frequentist statistics are two major approaches to statistical inference. Both use sample data to learn about an underlying population or data-generating process, but they differ in how they interpret probability, represent uncertainty, and draw conclusions about unknown parameters...

Multiple Regression

EN

Multiple linear regression is a statistical technique used to model the relationship between a single dependent variable and two or more independent variables. It extends the concept of simple linear regression by incorporating multiple predictors to explain the variability in the dependent variable...

Logistic Regression

EN

Logistic regression is a statistical method used for modeling the probability of a binary outcome based on one or more predictor variables. It is widely used in various fields such as medicine, social sciences, and machine learning for classification problems where the dependent variable is dichotom...

Simple Linear Regression

EN

Simple linear regression is a statistical method used to model the relationship between a single dependent variable and one independent variable. It aims to find the best-fitting straight line through the data points, which can be used to predict the dependent variable based on the independent varia...

Analysis of Variance

EN

Does peer assessment enhance student learning...

Statistical Moments

EN

A statistical moment is a probability-weighted average of a power of a value or deviation. The chosen reference point matters. Squared distance from zero gives a raw second moment; squared distance from the mean gives variance. Higher powers put progressively more weight on observations far from tha...

Normal Curve and z Score

EN

A z-score expresses a value's distance from a mean in standard-deviation units. It turns a height difference in inches or a score difference in points into a dimensionless comparison. When the original variable is normal, this transformation also gives a shared probability scale: the standard normal...

Introduction to Distributions

EN

A probability distribution assigns probabilities to the possible values of a random quantity and to events involving those values. It describes a model for what could happen, rather than merely listing the values already observed. A dataset is one realization of that model; its relative frequencies ...

Validation and Model Selection

EN

A fitted model is optimized using observed data. Model assessment asks a different question: how well will the modeling procedure perform on new data generated under comparable conditions...

Metrics

EN

An evaluation metric turns predictions and observed outcomes into a numerical answer to a particular question. Accuracy measures correct decisions at a threshold; AUC measures ordering; log loss measures probabilistic predictions; absolute and squared errors assign different costs to numerical misse...

Resampling

EN

Simulation approximates a probability calculation by repeatedly generating data under a specified mechanism. A parametric Monte Carlo experiment samples from an explicit model; a nonparametric bootstrap samples from the empirical distribution of the observed data; a permutation test rearranges data ...

Geostatistics

EN

Geostatistics is used when observations are tied to locations and nearby values may be more similar than distant values. Typical examples are...

Spatial Weights

EN

A spatial weights matrix turns a qualitative statement such as...

Point Processes

EN

A spatial point process is used when the observed data are the event locations themselves. This differs from geostatistics. In geostatistics, locations are usually fixed and a value is observed at each location...

Spatial Autocorrelation

EN

Spatial autocorrelation asks whether values attached to locations show a systematic spatial pattern. The idea is...

Kriging

EN

Kriging is a method for predicting a spatial variable at unsampled locations using a model of spatial dependence. Kriging is often introduced as the best linear unbiased predictor (BLUP). The phrase is useful when each part is interpreted carefully...

Spatial Data and Distance

EN

A map supplies coordinates, but coordinates alone do not define a statistical observation. A dot might represent a sensor reading, an event location, or a district total plotted at its centroid. Those three cases require different probability models. Before calculating Moran's I, fitting a variogram...

Spatial Regression

EN

Regression models the mean relationship between an outcome and its predictors. A basic linear model is...

Spatial Validation

EN

Spatial model performance depends not only on which observations are held out, but also on where those observations are located relative to the training data. This makes spatial validation fundamentally different from ordinary random train/test splitting. A random split often leaves test locations s...

Covariance

EN

Covariance averages products of deviations from two means. When both deviations have the same sign, their product is positive; when their signs differ, it is negative. The average therefore indicates the direction of linear co-variation, while its numerical size also reflects the units and variabili...

Correlation

EN

Correlation is a statistical measure that quantifies the strength and direction of the linear relationship between two variables. It is a fundamental concept in statistics, enabling researchers and analysts to understand how one variable may predict or relate to another. The most commonly used corre...

Joint Distributions

EN

A univariate distribution describes a single random variable. A joint distribution describes several random variables simultaneously and contains the information needed to reason about dependence, conditional behavior, and functions of multiple variables...

Negative Binomial Distribution

EN

The negative binomial distribution extends geometric waiting from the first success to the r-th success. Repeat independent Bernoulli trials with a common success probability p and stop when the required positive integer number r of successes has occurred...

Binomial Distribution

EN

A binomial random variable counts successes in a fixed number of independent trials. Each trial has two possible labels, success and failure, and the same success probability p. “Success” names the event being counted; it need not be desirable. Counting defective products can use exactly the same ma...

Geometric Distribution

EN

The geometric distribution counts trials up to and including the first success. Unlike a binomial experiment, it does not fix the number of trials in advance. Repeat independent Bernoulli trials with a common success probability p until the first success, and call its trial number X...

Poisson Distribution

EN

A Poisson random variable counts events in a specified interval of time, distance, area, or other exposure. Write X ~ Poisson(λ), where λ is the expected count in that interval. Its support is {0,1,2,…}: there is no fixed number of trials that caps the count...

Log Normal Distribution

EN

A positive random variable is log-normal when its natural logarithm is normal...

Beta Distribution

EN

The beta distribution models a continuously varying quantity confined to the unit interval, such as an uncertain probability or an underlying proportion. Write X ~ Beta(α,β), where both shape parameters are positive. Unlike a normal model, it assigns no probability outside the permitted range...

Exponential Distribution

EN

The exponential distribution models the wait to the next event in a constant-rate Poisson process. Write X ~ Exp(λ) with rate λ > 0. If time is measured in years, λ has units of events per year and X has units of years. Larger rates mean shorter typical waits...

Uniform Distribution

EN

A continuous uniform variable assigns equal probability to equal-length subintervals inside a fixed range. Write X ~ Uniform(a,b) with a < b. The density is constant on [a,b], but no individual point has positive probability...

Normal Distribution

EN

The normal distribution is a symmetric continuous model with a center and a scale. Write X ~ N(μ,σ²), where μ is the mean and σ > 0 is the standard deviation. Increasing μ shifts the distribution; increasing σ spreads its probability over a wider range...

Gamma Distribution

EN

The gamma family models positive quantities with flexible right-skewed shapes. Write X ~ Gamma(α,β), where α > 0 is the shape and β > 0 is the rate. This note uses a rate, not a scale; the corresponding scale is θ = 1/β...

Stationarity

EN

Stationarity is a statement about a process across possible realizations, not a demand that one plotted path look flat. Imagine moving an observation window from one date to another. If the relevant probability laws stay the same after that common shift, information from one part of the record can d...

Forecast Evaluation

EN

A backtest recreates forecasting decisions at historical dates. At each date, hide the later observations, run the procedure using the information then available, and score the resulting predictions when their target observations become available. This evaluates a forecasting process rather than how...

Autoregressive Models

EN

An autoregressive model predicts the next observation from observed values of the same series. If high demand tends to be followed by high demand, lagged demand contains predictive information. An AR equation assigns coefficients to that information and adds a new shock for the part the past did not...

Frequency Domain Analysis

EN

A time plot tells us when values rise and fall. Frequency-domain analysis asks which oscillation rates contribute to those movements. A repeating annual pattern in monthly data appears near 1/12 cycles per month; shorter and longer patterns appear at different frequencies. This view can separate ove...

Stochastic Processes and White Noise

EN

Imagine observing daily demand over the same month under many possible sequences of customer arrivals. Each possible sequence is a path. At a fixed date, the different possible demands form a random variable. A stochastic process specifies how those random variables fit together across dates, includ...

Dynamic Regression

EN

Suppose electricity demand depends on temperature, but errors from a temperature-only regression stay positive for several days in a row. The predictor explains one source of movement; the residual runs suggest another. Dynamic regression represents both the predictor relationship and the remaining ...

Invertibility

EN

An MA(1) equation tells us how a current shock and a previous shock combine to form an observation. In practice, however, we observe the combination, not the shocks. To estimate residuals and forecast recursively, we need to work backward from observations to a plausible shock history. Invertibility...

Multivariate Time Series

EN

Related series can carry predictive information for one another. A demand series may respond to earlier prices, while prices respond to earlier demand. Modeling each series separately ignores those cross-lag relationships. A multivariate time-series model writes equations for the variables jointly, ...

Random Walk

EN

A random walk says that the current level is the previous level plus a new increment. If today's shock raises the level by 2, tomorrow starts from that raised level. There is no mean-reverting term that gradually pulls the effect back toward a fixed baseline. Each shock therefore appears once in the...

Difference Equations

EN

Suppose a quantity retains 70% of its previous value and receives an addition of 2 at every step. The rule determines tomorrow's value from today's value; an initial value determines which particular path follows that rule. A difference equation formalizes this kind of discrete-time evolution...

Financial Time Series Models

EN

A price level, its return, and the uncertainty of that return answer different questions. The price records a cumulative level. A return measures proportional change over an interval. A volatility model describes how large an unexpected return is likely to be, even when its sign remains hard to pred...

State Space Models

EN

A noisy measurement need not equal the underlying quantity we want to track. Daily observations can jump because the true level changed, because measurement error changed, or both. A state-space model writes one equation for the hidden state and another for the observation, making those sources of v...

Autocovariance Function

EN

Take two observations from the same series, subtract the appropriate mean from each, and multiply the deviations. A positive product means that both observations are on the same side of their means; a negative product means that they are on opposite sides. Autocovariance is the expectation of this p...

Autocorrelation Function

EN

If today's value is high, does that help predict tomorrow's value? What about the value two periods later? The autocorrelation function (ACF) measures the overall linear association at each separation. The partial autocorrelation function (PACF) asks what remains at a separation after the intermedia...

Arima Models

EN

ARMA, ARIMA, and SARIMA describe related mechanisms at increasing levels of structure. An ARMA model combines feedback from past observations with the effects of current and past shocks around a stable mean. ARIMA first takes differences to remove an integrated level, then models the changes with AR...

Seasonality and Trends

EN

A series can rise over several years while repeatedly dipping at the same time each year. The rise and the repeating dip describe different structure. Trend summarizes persistent movement in level or slope; seasonality describes a pattern associated with a fixed known period. A cycle with an evolvin...

Regression with Arma Errors

EN

A regression can explain the response's relationship with observed predictors while leaving persistent departures around that relationship. Regression with ARMA errors models those departures as a time series. Its two equations distinguish the predictor contribution from the dynamic error; the final...

Forecasting

EN

A forecast answers a dated question: what can be said about a specified future quantity using the information available now? Predicting next month's total sales from today's data is different from predicting tomorrow's sales, and both differ from estimating a past trend with the full record availabl...

Moving Average Models

EN

A moving-average model explains an observation through a short history of shocks. A temporary disruption may affect today's measurement and leave a smaller effect tomorrow, then disappear. An MA(1) encodes that pattern with one current shock and one lagged shock; an MA(q) extends the direct response...

Time Series

EN

A time series records observations in an order that matters. A week of unusually high demand followed by a week of low demand tells us something different from the same values alternating each day. An unordered sample preserves the values but loses that pattern, even though the pattern may determine...

Yule Walker Equations

EN

An AR equation predicts the next value from earlier values. The Yule-Walker equations describe what that prediction rule implies for covariance across lags. Multiply the model by an earlier centered observation, take expectations, and use the innovation's lack of covariance with the past. The result...

Backward Shift Operator

EN

The symbol B means “take the previous observation.” It acts on a sequence: BXₜ = Xₜ₋₁, and applying it twice gives B²Xₜ = Xₜ₋₂. It is useful because a long equation involving many lags can then be written as a polynomial acting on the whole sequence...

Series

EN

A time-series model often says that today's observation contains the effects of many earlier shocks. Writing those effects as an infinite sum is convenient, but it raises a concrete question: does including more history settle on a definite answer, or does the approximation keep changing without lim...

Time Series Modeling

EN

Fitting a time-series model means more than finding coefficients for an equation. The procedure chooses a target, decides how trend and seasonality enter, handles missing observations, estimates dynamic parameters, and produces forecasts from a defined information set. Any of those decisions can aff...

Statistical Moments and Time Series

EN

A mean or variance summarizes a distribution, but a time series adds two questions: which date's distribution, and how are values at different dates related? A single average of the whole record cannot answer both. A rising level can be hidden by that average, and two sequences with identical histog...

Randomness Tests

EN

A noisy-looking plot can still contain structure. Consecutive values may cluster on one side of the mean, changes may alternate unusually often, or large residuals may arrive together. A diagnostic test turns one such observation into a defined statistic and compares it with a reference distribution...