Databases Notes

Normalization

EN

Normalization is a way to organize related facts so a change does not require editing many copies of the same information. Begin with the problems in a combined table, then use the normal forms to explain how to separate it safely...

Isolation

EN

Isolation describes what overlapping transactions can observe and how their effects interact. At Serializable, committed transactions have an outcome equivalent to some serial order (not necessarily the order they started). Lower levels permit additional interaction to reduce coordination costs. BEG...

Transactions Intro

EN

A bookstore checkout changes several facts: stock decreases, an order is created, and the purchased items are recorded. Those changes belong together. An order without its items is incomplete; a stock reduction without an order loses inventory...

Consistency

EN

In ACID, consistency means a transaction preserves the invariants that define valid states. The DBMS enforces invariants it knows, such as declared keys and checks; it cannot enforce an unexpressed business rule simply because a transaction commits. Correct application logic and suitable concurrency...

Durability

EN

Durability means that an acknowledged commit survives the failure classes covered by the configured logging, synchronization, replication, and storage assumptions. A successful reply is not a guarantee against simultaneous destruction of all durable copies or against an administrator durably committ...

Atomicity

EN

Atomicity means that the database makes a transaction's effects visible as a committed unit or leaves none of those effects committed. The statements still execute in sequence; atomicity does not mean simultaneous execution, nor does it guarantee that application logic is correct. A statement error ...

Clustered Indexes Heap Tables and Row Locality

EN

A table's primary key does not determine one universal physical layout. This chapter separates the logical key, the physical location of base rows, and the row locator used by secondary indexes. Those decisions affect read locality, key width, and write amplification...

Row Based vs Column Based Databases

EN

A SQL table is a logical relation, not a promise about disk layout. A row store groups many columns of a record together; a column store groups values from the same field across records. This changes the work of reconstructing one row versus scanning a few columns across many rows. Real engines also...

Composite Indexes and Column Order

EN

A composite index orders entries by a tuple of key columns. The leading equalities, range predicates, and requested sort determine which part of that ordering the engine can exploit. The purpose is not to index every filtered column, but to build a useful access path for a specific workload...

Indexing

EN

An index trades extra storage and write maintenance for a potentially cheaper read path. Choose one only after identifying the query's filter, ordering, and columns to return; then compare its plan and measured cost with a scan. This chapter is an overview—the following notes cover individual struct...

Primary Key vs Secondary Key

EN

A key is a logical uniqueness/identity rule; an index is a physical route to rows. Engines often create indexes to enforce keys, but an additional index does not create a new business identifier. This note separates primary and alternate candidate keys from non-unique lookup indexes...

Btree Internals and Page Splits

EN

B-tree-family indexes are commonly used for equality lookup, ranges, and ordered access because they keep keys in a balanced, page-oriented structure. Their cost has two parts: finding an entry and visiting enough leaf or table pages to return all qualifying rows. This chapter follows both search an...

How Tables and Indexes Are Stored on Disk

EN

SQL exposes tables and indexes as logical objects, but the DBMS reads and writes pages, files, and recovery records. This chapter follows a single row lookup through those layers. The key distinction is that rows returned, pages touched, and physical disk reads are different quantities; the cache an...

Index Maintenance Bloat and Online Operations

EN

Indexes have a lifecycle...

Covering Partial and Expression Indexes

EN

A basic index answers...

Specialized Index Structures

EN

B-tree is the default general-purpose index for many relational workloads, but not every predicate is naturally ordered...

Database Pages

EN

SQL presents a table as rows and columns. Storage works with chunks of bytes. In many database engines, a page is a fixed-size unit used to organize data and transfer it between storage and memory. A table or index usually spans many pages...

Query Optimization Techniques

EN

A query can return the correct result and still do far more work than necessary. Optimization means reducing that work while preserving the result the application needs. Start with one measured problem, not a checklist of indexes to add everywhere...

Working with Billion Row Table

EN

A billion rows is not a magic threshold at which one architecture becomes mandatory. A narrow, well-indexed billion-row table with selective queries can be easier to operate than a much smaller table with wide rows, poor access patterns, heavy churn, or global scans. Start from data size, row width...

Pagination and Large Result Sets

EN

A query is not finished when the database finds matching rows. The system still has to order them, transfer them, allocate client memory, deserialize them, and render/process them...

Execution Plans Statistics and Cardinality

EN

A database optimizer estimates costs from table metadata, statistics, predicates, indexes, join algorithms, and its cost model. It does not execute every possible plan and pick the fastest one...

Partitioning Sharding and Data Locality

EN

Large datasets can be divided so queries touch less data and maintenance works on smaller units. The terms are related but not interchangeable...

Connection Pooling Batching and N Plus One

EN

Database performance is not only query execution time...

Write Performance Vacuum and Bloat

EN

Read performance is only half of database performance...

Accessing Database in Code

EN

Application code sits between users and the database. A request can be slow even when the SQL engine executes its statements quickly...

Benchmarking Load Testing and Capacity

EN

A performance claim without a workload definition is incomplete...

Indexing Strategies

EN

Index access methods suit different operations. This note compares B-tree, bitmap, hash, and full-text indexes using named engines. For composite index design, covering queries, and maintenance, see Database Indexing...

Database Caching

EN

A cache keeps a reusable copy of data or a computed result. Reading the copy can avoid repeating a more expensive operation. The difficulty is knowing whether that copy is still appropriate for the request...

Materialized Views

EN

A materialized view stores the result of a query so readers can reuse work that has already been performed...

Introduction to Orm

EN

An object-relational mapper (ORM) is a library that maps application objects to relational tables and generates SQL for common database operations...

Popular Orm Tools

EN

ORMs solve similar problems, but they expose very different programming models. Some are tightly integrated with a web framework, some implement a language ecosystem standard, and some deliberately stay close to SQL...

Schema Migrations

EN

An ORM maps application code to a database schema, but the schema changes over time. Schema migrations are versioned changes that move the database from one known structure to another...

Relationship Loading and Query Performance

EN

ORM performance problems usually come from the SQL workload generated by application code: too many round trips, too many rows, large joins, missing indexes, or transactions that stay open too long...

Transactions Concurrency and Unit of Work

EN

ORMs make database access convenient, but concurrent requests still operate on a shared database. Two application workers can read the same row, make different decisions, and attempt conflicting writes...

Data Control Language Dcl

EN

DCL describes database privileges, such as GRANT and REVOKE. This chapter uses PostgreSQL, not SQLite: table privileges, role membership, object ownership, and application-user authorization are distinct layers. The goal is to trace effective access, not to assume a single grant is the whole securit...

Intro to Sql

EN

SQL (Structured Query Language) expresses database requests and changes declaratively. Start by reading a result set, then work through filters, joins, groups, missing values, and controlled writes. This chapter uses SQLite so the examples can be run locally; it does not claim every syntax choice wo...

Data Manipulation Language Dml

EN

DML changes rows with INSERT, UPDATE, and DELETE; SELECT reads data and is sometimes classified separately as query language. This chapter treats them together because applications must reason about both the targeted rows and the outcome. A successfully executed statement may affect zero rows, which...

Aggregate Functions

EN

An aggregate reduces qualifying rows to a summary, either once for the whole input or once per GROUP BY key. COUNT(*) counts rows, while aggregates such as AVG(column) generally skip NULL inputs. Before writing a report, identify its grain (one result per what?), which rows qualify, and whether miss...

Data Definition Language Ddl

EN

Data Definition Language (DDL) changes database structure, not just the current rows. Schema work must protect existing data and dependent code: CREATE TABLE on an empty database is straightforward, while changing a populated table is a migration with compatibility, validation, and rollback requirem...

Transaction Control Language Tcl

EN

A checkout combines stock reservation, order creation, and line items. A transaction supplies an atomic commit boundary, not a complete checkout algorithm: the caller must still check affected-row counts, handle errors, and coordinate external effects. This lesson distinguishes what COMMIT, ROLLBACK...

Triggers

EN

A trigger runs automatically in response to specified writes; its effects are part of the database write, not a separate background job. This chapter makes the hidden work visible: first log a real price change, then test an unchanged value, rollback, and a rejected state transition. Prefer an ordin...

Stored Procedures and Functions

EN

A stored routine gives database logic a callable interface. Use a function when a query needs a calculated value or set; use a procedure for an explicitly invoked operation. This chapter uses PostgreSQL because routine signatures, privileges, side effects, and transaction behavior are dialect-specif...

Window Functions

EN

A window function calculates a value over related rows without collapsing the result into one row per group. PARTITION BY chooses the group, ORDER BY establishes the calculation order, and the frame (where applicable) controls which neighboring rows contribute. The most common errors come from ambig...

Hierarchical Data

EN

A category hierarchy is a set of parent–child relationships. Adjacency lists store only the immediate parent, while recursive queries reconstruct ancestors or descendants. Later representations cache more traversal information in return for harder updates. The critical invariant is not just a valid ...

Joins Subqueries and Views

EN

The bookstore keeps customers, orders, products, and lines as separate facts. This chapter shows how to combine without copying them: joins pair related rows, subqueries express dependent tests and calculations, and views name reusable queries. The central skill is keeping track of result grain—whet...

Azure Services

EN

Azure provides several managed relational, NoSQL, cache, and distributed data services. The correct service depends on the data model, access pattern, transaction requirements, compatibility needs, recovery objectives, and team expertise...

Clickhouse

EN

ClickHouse is a column-oriented SQL database designed for analytical workloads over large datasets. It is commonly used for...

Elasticsearch and Opensearch

EN

OpenSearch and Elasticsearch are search-oriented distributed data engines built around Apache Lucene concepts such as inverted indexes, analyzers, mappings, shards, and replicas...

Distributed Sql

EN

Distributed SQL databases keep a relational SQL model while distributing data and replication across multiple nodes...

Cassandra

EN

Apache Cassandra is a distributed wide-column database designed for high write throughput, horizontal scale, multi-node availability, and predictable key-oriented access...

Choosing Database

EN

Choosing the right database can significantly influence your project’s reliability, performance, cost, and ability to scale. A good database choice is not only about picking a popular engine; it is about matching the database to your data shape, query patterns, consistency needs, latency requirement...

Sql Server

EN

Microsoft SQL Server is a relational database engine widely used in enterprise applications, business systems, reporting, and Microsoft-centric environments. It supports transactional workloads, analytical features, mature administration tooling, and deep integration with the .NET and Azure ecosyste...

Aws Services

EN

Choose a service by data model, access pattern, transaction requirements, and operational constraints. A keyword such as “financial,” “serverless,” or “high scale” is a clue, not a complete architecture decision. Check service availability and compatibility before designing a new deployment...

Gcp Services

EN

Match the workload to a data model and consistency requirement before selecting a managed product. Relational transactions, large analytical scans, document queries, and cached state have different needs. Managed operations reduce some administrative work but do not remove schema design, capacity pl...

Redis

EN

Redis is an in-memory data store built around specialized data structures such as strings, hashes, lists, sets, sorted sets, streams, JSON, geospatial indexes, probabilistic structures, and newer vector-oriented capabilities...

Data Integrity

EN

Data integrity means the stored data satisfies the rules the system relies on. A constraint is a rule declared in a table's definition so the database can check it whenever relevant data changes...

Indexing Strategies

EN

An index is an additional structure that helps the database find selected rows. It is comparable to a book's index: look up a topic, find its location, and then read the relevant pages. The database still has to retrieve and return the answer...

Denormalization

EN

Normalization separates facts so each has a clear home. Denormalization deliberately stores an additional copy or precomputed result to make a particular read cheaper. It creates a new responsibility: keep that copy correct, or explicitly allow it to lag...

Requirements Analysis

EN

Before choosing tables, find out what the application must do. A requirement is a statement about a needed behavior or rule. “Use PostgreSQL” is a technology choice; “an order must retain the price originally charged” is a requirement that the design must support...

Distributed Database Systems

EN

A distributed database places related data and computation across independently failing nodes while exposing a usable database interface. Distribution is not a single feature: placement, replication, transaction coordination, and failure handling make different promises...

Database Hardening and Patch Management

EN

Hardening reduces the number of ways a database can be attacked or accidentally misused...

Capacity Planning

EN

Capacity planning is the strategic process of determining the necessary resources required to meet current and future demands of an application or system. It involves analyzing workloads, forecasting growth, and ensuring that the infrastructure can handle anticipated loads while maintaining optimal ...

Backup and Recovery Strategies

EN

A backup is useful only if it can restore the required data within the required time. Choose backup types and schedules against recovery point and recovery time objectives, then test restoration. Production impact matters, but a fast backup that cannot be recovered does not meet the goal...

Crash Recovery in Databases

EN

Crash recovery is an important component of database systems that ensures data consistency and durability despite unexpected events like power outages, hardware failures, or software crashes. By design, databases must be capable of returning to a reliable state after a failure occurs. This is largel...

Auditing Compliance and Data Governance

EN

A secure database should answer...

Incident Response and Disaster Recovery Drills

EN

A security or reliability incident is the wrong time to discover that...

Performance Monitoring and Tuning

EN

Performance tuning should start with a measured problem...

Encryption Secrets and Key Management

EN

Database security depends on protecting data in several different states...

Database Security

EN

Database security encompasses a comprehensive set of measures designed to protect database management systems against threats that could compromise their confidentiality, integrity, and availability. As databases often store sensitive and critical information, safeguarding them is important for prot...

Identity Authentication and Access Control

EN

Database security starts with answering three different questions...

Sql Injection

EN

SQL injection occurs when untrusted input changes the structure of a SQL statement...

Database Migration

EN

Database migration is the process of transferring data, schema, and database objects from one database environment to another. This complex undertaking is crucial when organizations aim to upgrade technology, shift infrastructures, enhance performance, or reduce costs. A successful database migratio...

Querying Nosql Databases

EN

There is no universal NoSQL query language. A document database, key-value store, wide-column database, and graph database expose different query shapes because they organize data differently...

Document Modeling

EN

A document database stores records as documents rather than normalized rows spread across many tables. The main modeling question is not "can this be JSON?" but...

Operating Nosql and Polyglot Persistence

EN

NoSQL systems are often introduced through data models, but production success depends just as much on operations. A database choice adds responsibilities for deployment, backups, monitoring, security, capacity, failure recovery, schema evolution, and synchronization with other systems...

Crud in Sql vs Nosql

EN

Comparing common CRUD operations in SQL (relational databases) and MongoDB (a NoSQL document store) provides valuable insights into the differences between relational and non-relational databases. Understanding these differences is crucial for developers and database administrators when designing an...

Types of Nosql Databases

EN

NoSQL databases are commonly grouped by data model: key-value, document, wide-column, and graph. These categories describe how data is represented and accessed; they do not by themselves guarantee horizontal scalability, eventual consistency, low latency, or schema flexibility. Those properties depe...

Key Value Modeling

EN

A key-value store exposes a simple idea...

Graph Modeling

EN

Graph databases model connected data explicitly...

Consistency Transactions and Replication

EN

Choosing a NoSQL data model does not automatically choose a consistency model. Document, key-value, wide-column, and graph describe how data is represented. Replication, transaction scope, and read/write guarantees are separate design choices...

Nosql Databases Intro

EN

NoSQL is an umbrella label for databases built around models such as documents, key-value pairs, wide-column records, and graphs. It is not one query language or one set of guarantees. Some products support transactions, joins, validation, or SQL-like queries; others expose different operations...

Wide Column Modeling

EN

Wide-column databases such as Cassandra and HBase are designed around partitioned, distributed access patterns. The central modeling rule is different from a relational database...

Types of Databases

EN

The first note connected customers and orders through relational keys. Other systems organize access around nested documents, exact keys, partitions, graph edges, or analytical columns. These are model and workload choices, not a ranking of modernity or scalability. Compare a concrete read and its u...

Databases Intro

EN

A bookstore needs to remember its customers, books, orders, and remaining stock. At first, a spreadsheet or a CSV file might be enough. Once a website, a checkout desk, and an import job all change the same information, storing it becomes only part of the problem. The system also needs to answer que...

Glossary

EN

Use this as a lookup reference, not a lesson to memorize linearly. Start with data and keys, then SQL, transactions, storage, distribution, and analytics. A term describes a specific role: a key, constraint, index, and join are not interchangeable simply because all four appear in one query...

Database Management Systems Dbms

EN

A database is stored data plus its definitions; a DBMS is the engine responsible for interpreting requests, enforcing declared rules, executing queries, coordinating concurrent work, and recovering state. Drivers, clients, hosting, and storage services surround that engine, but do not replace its gu...

Data Models

EN

A data model specifies facts, identity, relationships, and invariants before those decisions become SQL tables, documents, or graph edges. A DBMS cannot infer a good model from product choice alone: the same bookstore can be modeled well or poorly in PostgreSQL, MongoDB, or another engine. Start wit...

Spark Sql

EN

Spark SQL is Apache Spark's structured-data engine. It lets you express work with SQL or DataFrame operations while Spark distributes the computation across partitions...

Data Warehousing

EN

A data warehouse is an analytical system built to answer questions across large amounts of historical data. Operational databases are optimized for day-to-day application work such as creating an order, updating inventory, or authenticating a user. Warehouses are optimized for scans, joins, aggregat...

Data Pipelines Orchestration and Quality

EN

A data platform is more than storage and queries. Data has to move from sources to destinations reliably, transformations must run in the correct order, failures must be retried safely, and downstream users need to know whether the resulting tables are fresh and trustworthy...

Streaming Kafka and Cdc

EN

Batch systems process bounded data: "all orders from yesterday." Streaming systems process data that continues arriving: clicks, payments, sensor readings, database changes, and application events...

Hadoop and Hdfs

EN

Hadoop is an ecosystem for storing and processing data across many machines. Its original core components solve different problems...

Data Lakes and Lakehouses

EN

A data lake stores large analytical datasets as files, usually in object storage. A lakehouse keeps that open-file storage model but adds table metadata so readers and writers can treat groups of files more like database tables...

Sharding

EN

Sharding distributes horizontally partitioned rows across separate nodes or clusters. A routing layer decides which shard owns a key; replication separately determines how many copies exist. Sharding can add storage or write capacity, but it also makes cross-shard reads, uniqueness rules, transactio...

Partitioning

EN

Partitioning splits one logical table into smaller physical pieces according to a declared key. Partition pruning can reduce query work when the filter identifies relevant pieces; it is not automatic for every query and does not replace suitable indexes. Partition management can also make retention ...

Cap Theorem

EN

CAP constrains a distributed read/write service during network partitions. When a request reaches a live node that cannot communicate with another portion of the system, maintaining a single linearizable history can require refusing or delaying some operations. This chapter develops the information ...

Eventual Consistency

EN

Eventual consistency is a convergence guarantee: if updates stop and replicas can communicate and repair their states, they eventually agree on the value selected by the system's reconciliation policy. It does not promise a maximum delay, a read-your-writes experience, or preservation of every concu...

Consistent Hashing

EN

Imagine you're organizing books in a vast library with shelves arranged in a circle. Each book’s position is chosen by the first letter of its title, looping back to the beginning after Z. When you install a new shelf or remove one, you’d prefer not to reshuffle every book—only a small, predictable ...

Partitioning vs Sharding

EN

Partitioning divides a logical dataset into pieces. Sharding places horizontal pieces on separate nodes or clusters and introduces distributed routing and coordination. They are not mutually exclusive terms: a sharded system may also use date partitions inside each shard. The operational boundary—on...

Quizes

EN

Try answering each question before opening its answer. SQL exercises identify their dialect; syntax and guarantees vary between engines...

Deadlocks

EN

A deadlock occurs when two or more transactions are waiting for one another in a cycle and none can continue unless one of the transactions is aborted or otherwise forced to release its resources...

Serializable vs Repeatable Read

EN

Imagine two requests hit an application at almost exactly the same time...

Shared vs Exclusive Locks

EN

Locks coordinate concurrent access to database resources...

Two Phase Locking

EN

Two-Phase Locking, usually abbreviated 2PL, is a concurrency-control protocol for making interleaved transactions behave like some serial execution...

Sqlite

EN

SQLite is a self-contained, serverless, and zero-configuration SQL database engine that's known for its simplicity and efficiency. Unlike traditional databases that require a separate server to operate, SQLite operates directly on ordinary disk files. This makes it an ideal choice for small to mediu...

Postgresql

EN

PostgreSQL is an open-source object-relational database maintained by the PostgreSQL Global Development Group. It supports relational queries, extensible types, and transactions for a wide range of applications...

Mongodb

EN

MongoDB is a popular open-source NoSQL database management system that offers a flexible and scalable approach to data storage. Instead of using the traditional table-based relational database structure, MongoDB stores data in flexible, JSON-like documents, which means fields can vary from document ...

Mysql

EN

MySQL is a popular open-source RDBMS known for its reliability, performance, and ease of use. Developed by Oracle Corporation, it powers numerous web applications, content management systems, and enterprise solutions. Its robust architecture efficiently manages large databases, making it a top choic...

Neo4J

EN

Neo4j is a leading open-source graph database management system that specializes in handling data with complex and interconnected relationships. Unlike traditional relational databases that use tables and rows, Neo4j stores data in nodes and relationships, allowing for more natural and efficient mod...

Multi Master Replication

EN

Multi-primary replication—also called multi-master or active-active replication—allows more than one database node to accept writes...

Master Standby Replication

EN

Primary–standby replication is a replication topology in which one database server acts as the authoritative writer and one or more standby servers follow the primary's changes...

Intro to Replication

EN

Replication keeps database data on more than one node by transferring changes from one database instance to another...

Synchronous vs Asynchronous Replication

EN

The fundamental difference between synchronous and asynchronous replication is...

Double Booking Problem

EN

The double-booking problem is a common issue in database systems, particularly in applications like booking platforms, reservation systems, and inventory management. It occurs when multiple transactions simultaneously attempt to reserve or modify the same resource, leading to conflicts and inconsist...