Numerical Methods

Eigenvalues and Eigenvectors

October 31, 2026
Numerical Methods
EN

An eigenvector identifies a direction that a square matrix preserves under multiplication, while its eigenvalue records the scaling along that direction. The scaling may include a sign change or, over the complex numbers, a phase change. These concepts connect matrix transformations to vibration mod...

Systems of Equations

October 31, 2026
Numerical Methods
EN

A linear system of equations is a collection of one or more linear equations involving the same set of variables. Such systems arise in diverse areas such as engineering, economics, physics, and computer science. The main goal is to find values of the variables that simultaneously satisfy all equati...

Gaussian Elimination

October 31, 2026
Numerical Methods
EN

Gaussian elimination solves a linear system by applying elementary row operations to its augmented matrix. Each operation preserves the solution set while eliminating a coefficient below a pivot. The process produces an upper-triangular system, which can be solved from the last equation upward by ba...

Newtons Method

October 31, 2026
Numerical Methods
EN

Newton's method, also called the Newton–Raphson method, approximates a root using the function value and its first derivative at the current point. It replaces the nearby curve by its tangent line and uses that line's zero as the next approximation. This is an open method: one starting guess is suff...

Differentiation

October 31, 2026
Numerical Methods
EN

Differentiation describes how a quantity changes as its input changes. The derivative gives a local rate of change and, geometrically, the slope of the tangent to the function's graph. It also supplies the linear term used to predict the effect of a small input change: f(x+h) ≈ f(x)+hf'(x). This con...

Central Difference

October 31, 2026
Numerical Methods
EN

The central difference method estimates a derivative using function values at equal distances on either side of the target point. It combines the forward and backward difference formulas so that their leading curvature errors cancel. For a sufficiently smooth function, this symmetry gives a second-o...

Convex Optimization

October 31, 2026
Numerical Methods
EN

Convex optimization minimizes a convex objective over a convex feasible set. Its structure provides a useful guarantee before an algorithm is chosen: a point that is locally optimal is also globally optimal. Supporting-hyperplane inequalities and, when applicable, KKT multipliers turn that geometric...

Optimization

October 31, 2026
Numerical Methods
EN

Optimization asks a simple question with many numerical subtleties...

Newton and Quasi Newton

October 31, 2026
Numerical Methods
EN

Newton-type methods use local curvature to transform the gradient into a better-scaled search direction...

Local Optimization

October 31, 2026
Numerical Methods
EN

Local optimization methods refine a starting point using information about the objective near the current iterate. They are the default tools for smooth numerical optimization because they can converge rapidly with far fewer evaluations than global search...

Constrained Optimization

October 31, 2026
Numerical Methods
EN

Constrained optimization adds restrictions to the decision variables...

Global Optimization

October 31, 2026
Numerical Methods
EN

Global optimization seeks the best feasible point across the entire search region rather than only within the basin of a starting point...

Regression

October 31, 2026
Numerical Methods
EN

Regression models the relationship between one or more predictors and a response variable. Unlike interpolation, regression generally does not try to pass exactly through every observation. Instead, it chooses model parameters that balance data fit with a chosen model structure...

Partial Differential Equations

October 22, 2026
Numerical Methods
EN

A partial differential equation (PDE) relates an unknown function of several independent variables to its partial derivatives. PDEs describe fields rather than single trajectories: temperature in space and time, pressure in a fluid, displacement in an elastic body, or the amplitude of a wave...

Ordinary Differential Equations

October 22, 2026
Numerical Methods
EN

An ordinary differential equation (ODE) relates an unknown function of one independent variable to one or more of its derivatives. ODEs are the natural language of dynamical systems: they describe how a state changes when its instantaneous rate of change is known...

Heuns Method

October 22, 2026
Numerical Methods
EN

Heun's method improves Euler's method by using information from both ends of each step. It is a second-order explicit Runge–Kutta method and is also known as the explicit trapezoidal method or improved Euler method...

Picards Method

October 22, 2026
Numerical Methods
EN

Picard iteration turns an initial value problem into a sequence of integral approximations. Each iterate is a function obtained by substituting the previous function into the ODE's integral form. This construction is central to the Picard–Lindelof existence-and-uniqueness theorem and explains how fi...

Runge Kutta

October 22, 2026
Numerical Methods
EN

Runge–Kutta methods advance an ODE solution using several carefully chosen slope evaluations inside each step. They achieve higher-order accuracy without requiring derivatives of f; only evaluations of f itself are needed...

Eulers Method

October 22, 2026
Numerical Methods
EN

Euler's method is the simplest explicit time-stepping method for an initial value problem (IVP). It is useful both as a practical first approximation and as a way to understand how numerical ODE solvers work...

Checksum

October 22, 2026
Numerical Methods
EN

Checksum algorithms are methods for computing a small, fixed-size value from a block of data. They serve as a simple mechanism to detect accidental changes or corruption in data during transmission or storage. While not cryptographically secure, checksums are fundamental to error detection in digita...

Lagrange Evaluation

October 22, 2026
Numerical Methods
EN

Lagrange evaluation is a technique from polynomial interpolation that finds a direct cryptographic application in Shamir's Secret Sharing. It allows a secret to be divided into multiple shares such that only a sufficient number of shares can reconstruct the original secret, while any smaller number ...

Elliptic Curves

October 22, 2026
Numerical Methods
EN

Elliptic curve cryptography (ECC) is a public-key cryptographic approach based on the algebraic structure of elliptic curves over finite fields. The security of ECC relies on the difficulty of the Elliptic Curve Discrete Logarithm Problem (ECDLP), which makes it possible to achieve strong security w...

Monte Carlo

October 22, 2026
Numerical Methods
EN

Monte Carlo integration approximates an integral by averaging function values at randomly selected points and applying the appropriate volume or probability weights. The sampling distribution is part of the method: uniform samples use the domain volume, while nonuniform samples require weights that ...

Simpsons Rule

October 22, 2026
Numerical Methods
EN

Simpson's rule approximates a definite integral by integrating quadratic interpolants through sampled function values. On each panel it uses two endpoints and their midpoint, fitting a parabola through the three points. A composite rule repeats this construction across pairs of equal subintervals an...

Midpoint Rule

October 22, 2026
Numerical Methods
EN

The midpoint rule approximates a definite integral by dividing its interval into smaller pieces and representing each piece by a rectangle. The rectangle's width is the subinterval width, and its height is the function value at that subinterval's midpoint. Adding the signed rectangle areas estimates...

Integration Introduction

October 22, 2026
Numerical Methods
EN

Integration accumulates a quantity over an interval. It can recover total change from a known rate of change, measure area, and express solutions of differential equations. For example, integrating velocity over time gives displacement, including the sign of the motion...

Trapezoidal Rule

October 22, 2026
Numerical Methods
EN

The trapezoidal rule approximates a definite integral by connecting neighboring samples with straight line segments and integrating those segments. When the sampled values are nonnegative, each contribution is the area of a trapezoid; negative values contribute signed area in the same way as the exa...

Matrix Methods

October 22, 2026
Numerical Methods
EN

Matrices organize numbers into rows and columns and represent linear transformations, systems of equations, and collections of data. Their dimensions determine which operations are defined, while their structure determines which numerical methods are appropriate. In applied mathematics and engineeri...

Singular Value Decomposition

October 22, 2026
Numerical Methods
EN

Singular value decomposition (SVD) expresses a matrix as orthogonal or unitary coordinate changes surrounding a rectangular diagonal scaling. It exists for every real or complex matrix, including rectangular and rank-deficient matrices, whereas a diagonal eigenvalue decomposition requires a diagonal...

Qr Method

October 22, 2026
Numerical Methods
EN

The QR method computes the eigenvalues of a square matrix through a sequence of similarity transformations. Each iteration factors the current matrix into an orthogonal factor Q and an upper-triangular factor R, then forms the product in the opposite order. The new matrix has the same eigenvalues bu...

Power Method

October 22, 2026
Numerical Methods
EN

The power method estimates an eigenvalue of largest magnitude and its associated eigendirection by repeatedly multiplying a vector by a matrix and normalizing the result. Each multiplication strengthens the contribution from a strictly dominant eigenvalue relative to smaller ones. For the eigenbasis...

Eigen Value Decomposition

October 22, 2026
Numerical Methods
EN

Eigenvalue decomposition, also called eigendecomposition, expresses a diagonalizable square matrix in a basis of its eigenvectors. In that basis, the transformation acts independently along each eigenvector by multiplying it by the corresponding eigenvalue. Changing back to the original coordinates ...

Inverse Matrix

October 22, 2026
Numerical Methods
EN

The inverse of a square matrix A is denoted by A⁻¹. When it exists, it is the unique matrix whose product with A, on either side, is the identity matrix I. Mathematically, this is expressed as...

Lu Decomposition

October 22, 2026
Numerical Methods
EN

LU decomposition, or LU factorization, stores Gaussian elimination as a product of triangular matrices. In the unpivoted form, A = LU, where L is lower triangular and U is upper triangular; with row pivoting, the convention used here is PA = LU. The factors support linear-system solves, determinant ...

Jacobi Method

October 22, 2026
Numerical Methods
EN

Jacobi iteration approximates the solution of a linear system A x = b by repeatedly updating every component from the previous iterate. Each row is solved for the variable on its diagonal, with all other components held at their old values during that sweep. The resulting vector becomes the input to...

Gauss Seidel

October 22, 2026
Numerical Methods
EN

Gauss–Seidel is an iterative method for a linear system A x = b, with A ∈ ℝ^(n×n), unknown vector x = (x₁,…,xₙ)ᵀ, and given right-hand side b. It updates one component at a time during a sweep through the equations. Each new component is used immediately in the remaining updates, while components no...

Secant Method

October 22, 2026
Numerical Methods
EN

The secant method approximates a root of f(x) = 0 using the two most recent points on the graph. Instead of evaluating the derivative in Newton's formula, it uses the slope of the secant line through those points. The next approximation is the zero of that line. This makes the method useful when an ...

Relaxation Method

October 22, 2026
Numerical Methods
EN

The relaxation method, commonly referred to as the fixed-point iteration method, is an iterative approach used to find solutions (roots) to nonlinear equations of the form f(x) = 0. Instead of directly solving for the root, the method involves rewriting the original equation in the form...

Golden Ratio Search

October 22, 2026
Numerical Methods
EN

Golden-section search, also called golden-ratio search, locates a minimum or maximum of a unimodal function on a specified interval. It compares function values at two interior points and discards a part of the interval while preserving a minimizer under the assumptions stated below. No derivatives ...

Gradient Descent

October 22, 2026
Numerical Methods
EN

Gradient descent is a first-order method for minimizing a differentiable objective in mathematics, statistics, machine learning, and artificial intelligence. At the current point, the gradient gives the direction of steepest local increase in the Euclidean norm. The method moves in the opposite dire...

Bisection Method

October 22, 2026
Numerical Methods
EN

Bisection approximates a real root of a continuous function f by repeatedly halving an interval [a,b] whose endpoints have opposite signs. It is a bracketing method: each update preserves an interval known to contain at least one root, rather than relying on a single guess. Its main advantages are a...

Root Finding

October 22, 2026
Numerical Methods
EN

Root-finding algorithms aim to solve equations of the form...

Backward Difference

October 22, 2026
Numerical Methods
EN

The backward difference method estimates a derivative using the function value at the target point x and a preceding point x-h, with h > 0. It is a finite difference method: the slope over a nonzero interval replaces the limiting slope that defines the derivative. This stencil is useful at the right...

Taylor Series

October 22, 2026
Numerical Methods
EN

Taylor expansion describes a function near a chosen center a using its value and derivatives at that point. The constant term records the function value, the linear term records its slope, and higher powers describe curvature and further local variation. A finite expansion includes a remainder, whic...

Forward Difference

October 22, 2026
Numerical Methods
EN

The forward difference method estimates a derivative from the function value at the target point x and a later point x+h, with h > 0. Dividing the change in the two values by their separation gives a secant slope that approximates the tangent slope at x. The method is useful at the left boundary of ...

Quadratic Programming

October 22, 2026
Numerical Methods
EN

A quadratic program (QP) has a quadratic objective and linear constraints...

Genetic Algorithms

October 22, 2026
Numerical Methods
EN

A genetic algorithm (GA) searches with a population of candidate solutions rather than a single trajectory...

Line Search

October 22, 2026
Numerical Methods
EN

A line search chooses the step length αₖ after a search direction pₖ has been selected...

Linear Programming

October 22, 2026
Numerical Methods
EN

A linear program optimizes a linear objective over linear constraints. A common minimization form is...

Simulated Annealing

October 22, 2026
Numerical Methods
EN

Simulated annealing is a stochastic global-search method that occasionally accepts worse moves so that it can escape local minima...

Lagrange Multipliers and Kkt

October 22, 2026
Numerical Methods
EN

Lagrange multipliers convert constrained first-order optimality into a system of equations involving the objective and constraint gradients...

Thin Plate Spline Interpolation

October 22, 2026
Numerical Methods
EN

Thin-plate spline (TPS) interpolation fits a surface through scattered data while controlling bending. This note develops the classical two-dimensional formulation; higher-dimensional polyharmonic spline constructions depend on the dimension and derivative order, so their kernel is not always the sa...

Interpolation

October 22, 2026
Numerical Methods
EN

Interpolation constructs a function that passes exactly through a set of known data points. If the data are...

Gaussian Interpolation

October 22, 2026
Numerical Methods
EN

Gaussian radial basis function (RBF) interpolation builds a smooth interpolant from Gaussian functions centered at the data sites...

Cubic Spline Interpolation

October 22, 2026
Numerical Methods
EN

A cubic spline interpolates data with a sequence of cubic polynomials rather than one high-degree polynomial across the entire interval...

Least Squares

October 22, 2026
Numerical Methods
EN

Least squares fits a model to data by minimizing the sum of squared residuals...

Linear Interpolation

October 22, 2026
Numerical Methods
EN

Linear interpolation estimates a value between two known points by assuming the function is a straight line over that interval...

Newton Polynomial

October 22, 2026
Numerical Methods
EN

Newton interpolation represents the same unique polynomial obtained by Lagrange interpolation, but in a form that is easier to build incrementally...

Lagrange Polynomial Interpolation

October 22, 2026
Numerical Methods
EN

Lagrange interpolation constructs the unique polynomial of degree at most n that passes through n+1 distinct data points...