dbbackup

Professional database backup and restore utility for PostgreSQL, MySQL, and MariaDB.

Key Features

• Multi-database support: PostgreSQL, MySQL, MariaDB
• Backup modes: Single database, cluster, sample data
• Restore operations with safety checks and validation
• Automatic CPU detection and parallel processing
• Streaming compression for large databases
• Interactive terminal UI with progress tracking
• Cross-platform binaries (Linux, macOS, BSD)

Installation

Download Pre-compiled Binary

Linux x86_64:

curl -L https://git.uuxo.net/uuxo/dbbackup/raw/branch/main/bin/dbbackup_linux_amd64 -o dbbackup
chmod +x dbbackup

Linux ARM64:

curl -L https://git.uuxo.net/uuxo/dbbackup/raw/branch/main/bin/dbbackup_linux_arm64 -o dbbackup
chmod +x dbbackup

macOS Intel:

curl -L https://git.uuxo.net/uuxo/dbbackup/raw/branch/main/bin/dbbackup_darwin_amd64 -o dbbackup
chmod +x dbbackup

macOS Apple Silicon:

curl -L https://git.uuxo.net/uuxo/dbbackup/raw/branch/main/bin/dbbackup_darwin_arm64 -o dbbackup
chmod +x dbbackup

Other platforms available in bin/ directory: FreeBSD, OpenBSD, NetBSD.

Build from Source

Requires Go 1.19 or later:

git clone https://git.uuxo.net/uuxo/dbbackup.git
cd dbbackup
go build

Quick Start

Interactive Mode

PostgreSQL (peer authentication):

sudo -u postgres ./dbbackup interactive

MySQL/MariaDB:

./dbbackup interactive --db-type mysql --user root --password secret

Menu-driven interface for all operations. Press arrow keys to navigate, Enter to select.

Interactive Features

• Backup Selection: Choose backup type (single, cluster, sample)
• Database Selection: Browse and select database
• Configuration Settings: Set compression, parallelism, CPU workload, performance options
• CPU Workload Profiles: Balanced, CPU-Intensive, or I/O-Intensive (auto-adjusts Jobs/DumpJobs)
• Backup Management: List, restore, verify, and delete backup archives
• Safety Checks: Archive validation, disk space verification
• Progress Tracking: Real-time progress with ETA estimation
• Restore Options: Smart database cleanup detection, safety confirmations

Command Line Mode

Backup single database:

./dbbackup backup single myapp_db

Backup entire cluster (PostgreSQL):

./dbbackup backup cluster

Restore database:

./dbbackup restore single backup.dump --target myapp_db --create

Restore full cluster:

./dbbackup restore cluster cluster_backup.tar.gz --confirm

Commands

Global Flags (Available for all commands)

Flag	Description	Default
-d, --db-type	postgres, mysql, mariadb	postgres
--host	Database host	localhost
--port	Database port	5432 (postgres), 3306 (mysql)
--user	Database user	root
--password	Database password	(empty)
--database	Database name	postgres
--backup-dir	Backup directory	/root/db_backups
--compression	Compression level 0-9	6
--ssl-mode	disable, prefer, require, verify-ca, verify-full	prefer
--insecure	Disable SSL/TLS	false
--jobs	Parallel jobs	8
--dump-jobs	Parallel dump jobs	8
--max-cores	Maximum CPU cores	16
--cpu-workload	cpu-intensive, io-intensive, balanced	balanced
--auto-detect-cores	Auto-detect CPU cores	true
--debug	Enable debug logging	false
--no-color	Disable colored output	false

Backup Operations

Single Database

Backup a single database to compressed archive:

./dbbackup backup single DATABASE_NAME [OPTIONS]

Common Options:

• --host STRING - Database host (default: localhost)
• --port INT - Database port (default: 5432 PostgreSQL, 3306 MySQL)
• --user STRING - Database user (default: postgres)
• --password STRING - Database password
• --db-type STRING - Database type: postgres, mysql, mariadb (default: postgres)
• --backup-dir STRING - Backup directory (default: /var/lib/pgsql/db_backups)
• --compression INT - Compression level 0-9 (default: 6)
• --insecure - Disable SSL/TLS
• --ssl-mode STRING - SSL mode: disable, prefer, require, verify-ca, verify-full

Examples:

# Basic backup
./dbbackup backup single production_db

# Remote database with custom settings
./dbbackup backup single myapp_db \
  --host db.example.com \
  --port 5432 \
  --user backup_user \
  --password secret \
  --compression 9 \
  --backup-dir /mnt/backups

# MySQL database
./dbbackup backup single wordpress \
  --db-type mysql \
  --user root \
  --password secret

Supported formats:

• PostgreSQL: Custom format (.dump) or SQL (.sql)
• MySQL/MariaDB: SQL (.sql)

Cluster Backup (PostgreSQL)

Backup all databases in PostgreSQL cluster including roles and tablespaces:

./dbbackup backup cluster [OPTIONS]

Performance Options:

• --max-cores INT - Maximum CPU cores (default: auto-detect)
• --cpu-workload STRING - Workload type: cpu-intensive, io-intensive, balanced (default: balanced)
• --jobs INT - Parallel jobs (default: auto-detect based on workload)
• --dump-jobs INT - Parallel dump jobs (default: auto-detect based on workload)
• --cluster-parallelism INT - Concurrent database operations (default: 2, configurable via CLUSTER_PARALLELISM env var)

Examples:

# Standard cluster backup
sudo -u postgres ./dbbackup backup cluster

# High-performance backup
sudo -u postgres ./dbbackup backup cluster \
  --compression 3 \
  --max-cores 16 \
  --cpu-workload cpu-intensive \
  --jobs 16

Output: tar.gz archive containing all databases and globals.

Sample Backup

Create reduced-size backup for testing/development:

./dbbackup backup sample DATABASE_NAME [OPTIONS]

Options:

• --sample-strategy STRING - Strategy: ratio, percent, count (default: ratio)
• --sample-value FLOAT - Sample value based on strategy (default: 10)

Examples:

# Keep 10% of all rows
./dbbackup backup sample myapp_db --sample-strategy percent --sample-value 10

# Keep 1 in 100 rows
./dbbackup backup sample myapp_db --sample-strategy ratio --sample-value 100

# Keep 5000 rows per table
./dbbackup backup sample myapp_db --sample-strategy count --sample-value 5000

Warning: Sample backups may break referential integrity.

Restore Operations

Single Database Restore

Restore database from backup file:

./dbbackup restore single BACKUP_FILE [OPTIONS]

Options:

• --target STRING - Target database name (required)
• --create - Create database if it doesn't exist
• --clean - Drop and recreate database before restore
• --jobs INT - Parallel restore jobs (default: 4)
• --verbose - Show detailed progress
• --no-progress - Disable progress indicators
• --confirm - Execute restore (required for safety, dry-run by default)
• --dry-run - Preview without executing
• --force - Skip safety checks

Examples:

# Basic restore
./dbbackup restore single /backups/myapp_20250112.dump --target myapp_restored

# Restore with database creation
./dbbackup restore single backup.dump \
  --target myapp_db \
  --create \
  --jobs 8

# Clean restore (drops existing database)
./dbbackup restore single backup.dump \
  --target myapp_db \
  --clean \
  --verbose

Supported formats:

• PostgreSQL: .dump, .dump.gz, .sql, .sql.gz
• MySQL: .sql, .sql.gz

Cluster Restore (PostgreSQL)

Restore entire PostgreSQL cluster from archive:

./dbbackup restore cluster ARCHIVE_FILE [OPTIONS]

Options:

• --confirm - Confirm and execute restore (required for safety)
• --dry-run - Show what would be done without executing
• --force - Skip safety checks
• --jobs INT - Parallel decompression jobs (default: auto)
• --verbose - Show detailed progress
• --no-progress - Disable progress indicators

Examples:

# Standard cluster restore
sudo -u postgres ./dbbackup restore cluster cluster_backup.tar.gz --confirm

# Dry-run to preview
sudo -u postgres ./dbbackup restore cluster cluster_backup.tar.gz --dry-run

# High-performance restore
sudo -u postgres ./dbbackup restore cluster cluster_backup.tar.gz \
  --confirm \
  --jobs 16 \
  --verbose

Safety Features:

• Archive integrity validation
• Disk space checks (4x archive size recommended)
• Automatic database cleanup detection (interactive mode)
• Progress tracking with ETA estimation

Restore List

Show available backup archives in backup directory:

./dbbackup restore list

System Commands

Status Check

Check database connection and configuration:

./dbbackup status [OPTIONS]

Shows: Database type, host, port, user, connection status, available databases.

Preflight Checks

Run pre-backup validation checks:

./dbbackup preflight [OPTIONS]

Verifies: Database connection, required tools, disk space, permissions.

List Databases

List available databases:

./dbbackup list [OPTIONS]

CPU Information

Display CPU configuration and optimization settings:

./dbbackup cpu

Shows: CPU count, model, workload recommendation, suggested parallel jobs.

Version

Display version information:

./dbbackup version

Configuration

PostgreSQL Authentication

PostgreSQL uses different authentication methods based on system configuration.

Peer/Ident Authentication (Linux Default)

Run as postgres system user:

sudo -u postgres ./dbbackup backup cluster

Password Authentication

Option 1: .pgpass file (recommended for automation):

echo "localhost:5432:*:postgres:password" > ~/.pgpass
chmod 0600 ~/.pgpass
./dbbackup backup single mydb --user postgres

Option 2: Environment variable:

export PGPASSWORD=your_password
./dbbackup backup single mydb --user postgres

Option 3: Command line flag:

./dbbackup backup single mydb --user postgres --password your_password

MySQL/MariaDB Authentication

Option 1: Command line

./dbbackup backup single mydb --db-type mysql --user root --password secret

Option 2: Environment variable

export MYSQL_PWD=your_password
./dbbackup backup single mydb --db-type mysql --user root

Option 3: Configuration file

cat > ~/.my.cnf << EOF
[client]
user=backup_user
password=your_password
host=localhost
EOF
chmod 0600 ~/.my.cnf

Environment Variables

PostgreSQL:

export PG_HOST=localhost
export PG_PORT=5432
export PG_USER=postgres
export PGPASSWORD=password

MySQL/MariaDB:

export MYSQL_HOST=localhost
export MYSQL_PORT=3306
export MYSQL_USER=root
export MYSQL_PWD=password

General:

export BACKUP_DIR=/var/backups/databases
export COMPRESS_LEVEL=6
export CLUSTER_TIMEOUT_MIN=240

Database Types

• postgres - PostgreSQL
• mysql - MySQL
• mariadb - MariaDB

Select via:

• CLI: -d postgres or --db-type postgres
• Interactive: Arrow keys to cycle through options

Performance

Memory Usage

Streaming architecture maintains constant memory usage:

Database Size	Memory Usage
1-10 GB	~800 MB
10-50 GB	~900 MB
50-100 GB	~950 MB
100+ GB	<1 GB

Large Database Optimization

• Databases >5GB automatically use plain format with streaming compression
• Parallel compression via pigz (if available)
• Per-database timeout: 4 hours default
• Automatic format selection based on size

CPU Optimization

Automatically detects CPU configuration and optimizes parallelism:

./dbbackup cpu

Manual override:

./dbbackup backup cluster \
  --max-cores 32 \
  --jobs 32 \
  --cpu-workload cpu-intensive

Parallelism

./dbbackup backup cluster --jobs 16 --dump-jobs 16

• --jobs - Compression/decompression parallel jobs
• --dump-jobs - Database dump parallel jobs
• --max-cores - Limit CPU cores (default: 16)
• Cluster operations use worker pools with configurable parallelism (default: 2 concurrent databases)
• Set CLUSTER_PARALLELISM environment variable to adjust concurrent database operations

CPU Workload

./dbbackup backup cluster --cpu-workload cpu-intensive

Options: cpu-intensive, io-intensive, balanced (default)

Workload types automatically adjust Jobs and DumpJobs:

• Balanced: Jobs = PhysicalCores, DumpJobs = PhysicalCores/2 (min 2)
• CPU-Intensive: Jobs = PhysicalCores×2, DumpJobs = PhysicalCores (more parallelism)
• I/O-Intensive: Jobs = PhysicalCores/2 (min 1), DumpJobs = 2 (less parallelism to avoid I/O contention)

Configure in interactive mode via Configuration Settings menu.

Compression

./dbbackup backup single mydb --compression 9

• Level 0 = No compression (fastest)
• Level 6 = Balanced (default)
• Level 9 = Maximum compression (slowest)

SSL/TLS Configuration

SSL modes: disable, prefer, require, verify-ca, verify-full

# Disable SSL
./dbbackup backup single mydb --insecure

# Require SSL
./dbbackup backup single mydb --ssl-mode require

# Verify certificate
./dbbackup backup single mydb --ssl-mode verify-full

Disaster Recovery

Complete automated disaster recovery test:

sudo ./disaster_recovery_test.sh

This script:

1. Backs up entire cluster with maximum performance
2. Documents pre-backup state
3. Destroys all user databases (confirmation required)
4. Restores full cluster from backup
5. Verifies restoration success

Warning: Destructive operation. Use only in test environments.

Troubleshooting

Connection Issues

Test connectivity:

./dbbackup status

PostgreSQL peer authentication error:

sudo -u postgres ./dbbackup status

SSL/TLS issues:

./dbbackup status --insecure

Out of Memory

Check memory:

free -h
dmesg | grep -i oom

Add swap space:

sudo fallocate -l 16G /swapfile
sudo chmod 600 /swapfile
sudo mkswap /swapfile
sudo swapon /swapfile

Reduce parallelism:

./dbbackup backup cluster --jobs 4 --dump-jobs 4

Debug Mode

Enable detailed logging:

./dbbackup backup single mydb --debug

Common Errors

• "Ident authentication failed" - Run as matching OS user or configure password authentication
• "Permission denied" - Check database user privileges
• "Disk space check failed" - Ensure 4x archive size available
• "Archive validation failed" - Backup file corrupted or incomplete

Building

Build for all platforms:

./build_all.sh

Binaries created in bin/ directory.

Requirements

System Requirements

• Linux, macOS, FreeBSD, OpenBSD, NetBSD
• 1 GB RAM minimum (2 GB recommended for large databases)
• Disk space: 30-50% of database size for backups

Software Requirements

PostgreSQL:

• Client tools: psql, pg_dump, pg_dumpall, pg_restore
• PostgreSQL 10 or later

MySQL/MariaDB:

• Client tools: mysql, mysqldump
• MySQL 5.7+ or MariaDB 10.3+

Optional:

• pigz (parallel compression)
• pv (progress monitoring)

Best Practices

1. Test restores regularly - Verify backups work
2. Monitor disk space - Ensure adequate space
3. Use compression - Balance speed/space (level 3-6)
4. Automate backups - cron or systemd timers
5. Secure credentials - .pgpass/.my.cnf with 0600 permissions
6. Keep versions - Multiple backup versions for point-in-time recovery
7. Off-site storage - Remote backup copies
8. Document procedures - Maintain restore runbooks

Project Structure

dbbackup/
├── main.go                      # Entry point
├── cmd/                         # CLI commands
├── internal/
│   ├── backup/                  # Backup engine
│   ├── restore/                 # Restore engine
│   ├── config/                  # Configuration
│   ├── database/                # Database drivers
│   ├── cpu/                     # CPU detection
│   ├── logger/                  # Logging
│   ├── progress/                # Progress tracking
│   └── tui/                     # Interactive UI
├── bin/                         # Pre-compiled binaries
├── disaster_recovery_test.sh    # DR testing script
└── build_all.sh                 # Multi-platform build

Support

• Repository: https://git.uuxo.net/uuxo/dbbackup
• Issues: Use repository issue tracker

License

MIT License

Recent Improvements

Performance Optimizations

• Parallel Cluster Operations: Worker pool pattern for concurrent database backup/restore (3-5x speedup)
• Memory Efficiency: Streaming command output eliminates OOM errors on large restores
• Optimized Goroutines: Ticker-based progress indicators reduce CPU overhead
• Configurable Concurrency: Control parallel database operations via CLUSTER_PARALLELISM

Interactive UI Enhancements

• CPU Workload Profiles: Configure workload type in settings (auto-adjusts parallelism)
• Backup Management: Delete archives directly from TUI with confirmation
• Better Callbacks: Fixed confirmation dialogs to execute correct actions
• Cleaner Interface: Configuration consolidated in Settings menu

Bug Fixes

• Fixed OOM error during large cluster restores
• Fixed delete backup incorrectly triggering cluster backup
• Fixed signal handler cleanup preventing goroutine leaks
• Improved error handling and user feedback

Why dbbackup?

• Reliable - Comprehensive safety checks, validation, and error handling
• Simple - Intuitive menu-driven interface or straightforward CLI
• Fast - Automatic CPU detection, parallel processing, streaming compression
• Efficient - Minimal memory footprint, even for huge databases (constant ~1GB regardless of size)
• Flexible - Multiple backup modes, compression levels, performance options
• Safe - Dry-run by default, archive verification, smart database cleanup
• Complete - Full cluster backup/restore, multiple formats
• Scalable - Handles databases from megabytes to terabytes

dbbackup is production-ready for backup and disaster recovery operations on PostgreSQL, MySQL, and MariaDB databases of any size.