Last modified: November 04, 2024
This article is written in: 🇺🇸
Accessing databases through code is a fundamental skill for developers building applications that rely on data storage and retrieval. Whether you're developing a web application, mobile app, or any software that requires data persistence, understanding how to interact with databases programmatically is essential.
After reading the material, you should be able to answer the following questions:
Establishing a connection to the database is the first step in interacting with it programmatically.
Each programming language and database system requires specific libraries or drivers to facilitate communication.
Examples:
Programming Language | Database | Library/Driver |
Python | MySQL | mysql-connector-python , PyMySQL |
PostgreSQL | psycopg2 , asyncpg |
|
SQLite | Built-in sqlite3 module |
|
Java | JDBC Drivers | mysql-connector-java , postgresql-jdbc |
C#/.NET | SQL Server | System.Data.SqlClient |
MySQL | MySql.Data |
|
JavaScript (Node.js) | MySQL | mysql , mysql2 |
PostgreSQL | pg |
Installing a Python Library for PostgreSQL:
pip install psycopg2-binary
A connection string contains the information required to establish a connection to the database.
Components:
Component | Description |
Hostname | The server's address. |
Port | The port number (default: MySQL is 3306, PostgreSQL is 5432). |
Database Name | The specific database to connect to. |
Username and Password | Credentials for authentication. |
Additional Parameters | SSL mode, timeouts, charset, and other options. |
Example in Python using psycopg2:
import psycopg2
conn = psycopg2.connect(
dbname="my_database",
user="my_user",
password="my_password",
host="localhost",
port="5432"
)
Example in Java using JDBC:
String url = "jdbc:postgresql://localhost:5432/my_database";
Properties props = new Properties();
props.setProperty("user", "my_user");
props.setProperty("password", "my_password");
Connection conn = DriverManager.getConnection(url, props);
Connection pooling manages a pool of database connections, reusing them instead of creating new ones for each request.
Benefits:
Implementing Connection Pooling in Python with psycopg2:
from psycopg2 import pool
db_pool = pool.SimpleConnectionPool(
1, # Minimum number of connections
20, # Maximum number of connections
dbname="my_database",
user="my_user",
password="my_password",
host="localhost",
port="5432"
)
# Getting a connection from the pool
conn = db_pool.getconn()
# Returning the connection to the pool
db_pool.putconn(conn)
Executing SQL queries is the core of database interactions.
Parameterized queries prevent SQL injection by separating SQL code from data.
Example in Python with psycopg2:
cursor = conn.cursor()
sql = "SELECT * FROM users WHERE username = %s;"
username = 'john_doe'
cursor.execute(sql, (username,))
results = cursor.fetchall()
Benefits:
CRUD stands for Create, Read, Update, Delete—fundamental operations in data manipulation.
Inserting Data:
sql = "INSERT INTO users (username, email) VALUES (%s, %s);"
data = ('john_doe', 'john@example.com')
cursor.execute(sql, data)
conn.commit()
Selecting Data:
sql = "SELECT id, username FROM users WHERE active = %s;"
cursor.execute(sql, (True,))
users = cursor.fetchall()
Updating Data:
sql = "UPDATE users SET email = %s WHERE id = %s;"
cursor.execute(sql, ('new_email@example.com', user_id))
conn.commit()
Deleting Data:
sql = "DELETE FROM users WHERE id = %s;"
cursor.execute(sql, (user_id,))
conn.commit()
After executing a SELECT query, you need to retrieve the results.
fetchone()
retrieves the next row of a query result set.fetchmany(size)
retrieves the next set of rows.fetchall()
retrieves all remaining rows.Example:
cursor.execute("SELECT * FROM users;")
all_users = cursor.fetchall()
for user in all_users:
print(user)
Transactions ensure that a series of operations either all succeed or all fail, maintaining data integrity.
Example:
try:
cursor.execute("BEGIN;")
cursor.execute("UPDATE accounts SET balance = balance - 100 WHERE id = 1;")
cursor.execute("UPDATE accounts SET balance = balance + 100 WHERE id = 2;")
conn.commit()
except Exception as e:
conn.rollback()
print(f"Transaction failed: {e}")
ACID Properties:
Proper error handling is crucial for building robust applications.
Use try-except blocks to handle exceptions gracefully.
Example in Python:
try:
cursor.execute("SELECT * FROM non_existing_table;")
except psycopg2.Error as e:
print(f"Database error: {e}")
Logging errors helps in diagnosing issues, especially in production environments.
Example:
import logging
logging.basicConfig(filename='app.log', level=logging.ERROR)
try:
cursor.execute("SELECT * FROM users;")
except Exception as e:
logging.error(f"Error executing query: {e}")
Implement retry logic for transient errors like network issues.
Example:
import time
max_retries = 5
for attempt in range(max_retries):
try:
cursor.execute("SELECT * FROM users;")
break
except psycopg2.OperationalError as e:
if attempt < max_retries - 1:
time.sleep(2 ** attempt) # Exponential backoff
else:
raise
Why:
Example:
Instead of fetching all data and filtering in code:
cursor.execute("SELECT * FROM orders;")
orders = cursor.fetchall()
large_orders = [order for order in orders if order['amount'] > 1000]
Filter directly in SQL:
cursor.execute("SELECT * FROM orders WHERE amount > %s;", (1000,))
large_orders = cursor.fetchall()
Enforce permissions at the database level for consistency and security.
Methods:
Example of a View:
CREATE VIEW active_users AS
SELECT id, username, email FROM users WHERE active = TRUE;
GRANT SELECT ON active_users TO regular_user_role;
ORMs allow you to interact with the database using objects instead of raw SQL.
Benefits:
Popular ORMs:
Programming Language | ORM (Object-Relational Mapping) Tool |
Python | SQLAlchemy, Django ORM |
Java | Hibernate |
C# | Entity Framework |
JavaScript | Sequelize |
Example with SQLAlchemy:
from sqlalchemy import Column, Integer, String, create_engine
from sqlalchemy.ext.declarative import declarative_base
Base = declarative_base()
class User(Base):
__tablename__ = 'users'
id = Column(Integer, primary_key=True)
username = Column(String)
email = Column(String)
engine = create_engine('sqlite:///users.db')
Base.metadata.create_all(engine)
Expose specific database functionalities securely.
Methods:
Example of a Stored Procedure in MySQL:
DELIMITER //
CREATE PROCEDURE GetUserByEmail(IN userEmail VARCHAR(255))
BEGIN
SELECT id, username FROM users WHERE email = userEmail;
END //
DELIMITER ;
Example:
import os
conn = psycopg2.connect(
dbname=os.environ['DB_NAME'],
user=os.environ['DB_USER'],
password=os.environ['DB_PASSWORD'],
host=os.environ['DB_HOST'],
sslmode='require'
)
Ensure your application remains stable under unexpected conditions.
Regularly assess performance to optimize and prevent issues.
EXPLAIN
helps understand how queries are executed and optimized.Example of Query Profiling:
EXPLAIN SELECT * FROM users WHERE email = 'john@example.com';
Maintain code quality and adapt to changing requirements.
Manage database schema changes systematically.
Programming Language | Migration Tool |
Python | Alembic (SQLAlchemy), Django Migrations |
Ruby | ActiveRecord Migrations |
JavaScript | Knex.js Migrations |
Example with Alembic:
alembic revision --autogenerate -m "Added new column to users"
alembic upgrade head