dbbackup/MYSQL_PITR.md

MySQL/MariaDB Point-in-Time Recovery (PITR)

This guide explains how to use dbbackup for Point-in-Time Recovery with MySQL and MariaDB databases.

Overview

Point-in-Time Recovery (PITR) allows you to restore your database to any specific moment in time, not just to when a backup was taken. This is essential for:

• Recovering from accidental data deletion or corruption
• Restoring to a state just before a problematic change
• Meeting regulatory compliance requirements for data recovery

How MySQL PITR Works

MySQL PITR uses binary logs (binlogs) which record all changes to the database:

1. Base Backup: A full database backup with the binlog position recorded
2. Binary Log Archiving: Continuous archiving of binlog files
3. Recovery: Restore base backup, then replay binlogs up to the target time

┌─────────────┐     ┌──────────────┐     ┌──────────────┐     ┌──────────────┐
│ Base Backup │ --> │ binlog.00001 │ --> │ binlog.00002 │ --> │ binlog.00003 │
│ (pos: 1234) │     │              │     │              │     │  (current)   │
└─────────────┘     └──────────────┘     └──────────────┘     └──────────────┘
       │                   │                    │                    │
       ▼                   ▼                    ▼                    ▼
   10:00 AM            10:30 AM             11:00 AM            11:30 AM
                                                 ↑
                                          Target: 11:15 AM

Prerequisites

MySQL Configuration

Binary logging must be enabled in MySQL. Add to my.cnf:

[mysqld]
# Enable binary logging
log_bin = mysql-bin
server_id = 1

# Recommended: Use ROW format for PITR
binlog_format = ROW

# Optional but recommended: Enable GTID for easier replication and recovery
gtid_mode = ON
enforce_gtid_consistency = ON

# Keep binlogs for at least 7 days (adjust as needed)
expire_logs_days = 7
# Or for MySQL 8.0+:
# binlog_expire_logs_seconds = 604800

After changing configuration, restart MySQL:

sudo systemctl restart mysql

MariaDB Configuration

MariaDB configuration is similar:

[mysqld]
log_bin = mariadb-bin
server_id = 1
binlog_format = ROW

# MariaDB uses different GTID implementation (auto-enabled with log_slave_updates)
log_slave_updates = ON

Quick Start

1. Check PITR Status

# Check if MySQL is properly configured for PITR
dbbackup pitr mysql-status

Example output:

━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
  MySQL/MariaDB PITR Status (mysql)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

PITR Status:     ❌ NOT CONFIGURED
Binary Logging:  ✅ ENABLED
Binlog Format:   ROW
GTID Mode:       ON
Current Position: mysql-bin.000042:1234

PITR Requirements:
  ✅ Binary logging enabled
  ✅ Row-based logging (recommended)

2. Enable PITR

# Enable PITR and configure archive directory
dbbackup pitr mysql-enable --archive-dir /backups/binlog_archive

3. Create a Base Backup

# Create a backup - binlog position is automatically recorded
dbbackup backup single mydb

Note: All backups automatically capture the current binlog position when PITR is enabled at the MySQL level. This position is stored in the backup metadata and used as the starting point for binlog replay during recovery.

4. Start Binlog Archiving

# Run binlog archiver in the background
dbbackup binlog watch --binlog-dir /var/lib/mysql --archive-dir /backups/binlog_archive --interval 30s

Or set up a cron job for periodic archiving:

# Archive new binlogs every 5 minutes
*/5 * * * * dbbackup binlog archive --binlog-dir /var/lib/mysql --archive-dir /backups/binlog_archive

5. Restore to Point in Time

# Restore to a specific time
dbbackup restore pitr mydb_backup.sql.gz --target-time '2024-01-15 14:30:00'

Commands Reference

PITR Commands

pitr mysql-status

Show MySQL/MariaDB PITR configuration and status.

dbbackup pitr mysql-status

pitr mysql-enable

Enable PITR for MySQL/MariaDB.

dbbackup pitr mysql-enable \
  --archive-dir /backups/binlog_archive \
  --retention-days 7 \
  --require-row-format \
  --require-gtid

Options:

• --archive-dir: Directory to store archived binlogs (required)
• --retention-days: Days to keep archived binlogs (default: 7)
• --require-row-format: Require ROW binlog format (default: true)
• --require-gtid: Require GTID mode enabled (default: false)

Binlog Commands

binlog list

List available binary log files.

# List binlogs from MySQL data directory
dbbackup binlog list --binlog-dir /var/lib/mysql

# List archived binlogs
dbbackup binlog list --archive-dir /backups/binlog_archive

binlog archive

Archive binary log files.

dbbackup binlog archive \
  --binlog-dir /var/lib/mysql \
  --archive-dir /backups/binlog_archive \
  --compress

Options:

• --binlog-dir: MySQL binary log directory
• --archive-dir: Destination for archived binlogs (required)
• --compress: Compress archived binlogs with gzip
• --encrypt: Encrypt archived binlogs
• --encryption-key-file: Path to encryption key file

binlog watch

Continuously monitor and archive new binlog files.

dbbackup binlog watch \
  --binlog-dir /var/lib/mysql \
  --archive-dir /backups/binlog_archive \
  --interval 30s \
  --compress

Options:

• --interval: How often to check for new binlogs (default: 30s)

binlog validate

Validate binlog chain integrity.

dbbackup binlog validate --binlog-dir /var/lib/mysql

Output shows:

• Whether the chain is complete (no missing files)
• Any gaps in the sequence
• Server ID changes (indicating possible failover)
• Total size and file count

binlog position

Show current binary log position.

dbbackup binlog position

Output:

━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
  Current Binary Log Position
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

File:     mysql-bin.000042
Position: 123456
GTID Set: 3E11FA47-71CA-11E1-9E33-C80AA9429562:1-1000

Position String: mysql-bin.000042:123456

Restore Scenarios

Restore to Specific Time

# Restore to January 15, 2024 at 2:30 PM
dbbackup restore pitr mydb_backup.sql.gz \
  --target-time '2024-01-15 14:30:00'

Restore to Specific Position

# Restore to a specific binlog position
dbbackup restore pitr mydb_backup.sql.gz \
  --target-position 'mysql-bin.000042:12345'

Dry Run (Preview)

# See what SQL would be replayed without applying
dbbackup restore pitr mydb_backup.sql.gz \
  --target-time '2024-01-15 14:30:00' \
  --dry-run

Restore to Backup Point Only

# Restore just the base backup without replaying binlogs
dbbackup restore pitr mydb_backup.sql.gz --immediate

Best Practices

1. Archiving Strategy

• Archive binlogs frequently (every 5-30 minutes)
• Use compression to save disk space
• Store archives on separate storage from the database

2. Retention Policy

• Keep archives for at least as long as your oldest valid base backup
• Consider regulatory requirements for data retention
• Use the cleanup command to purge old archives:

dbbackup binlog cleanup --archive-dir /backups/binlog_archive --retention-days 30

3. Validation

• Regularly validate your binlog chain:

dbbackup binlog validate --binlog-dir /var/lib/mysql

• Test restoration periodically on a test environment

4. Monitoring

• Monitor the dbbackup binlog watch process
• Set up alerts for:
  • Binlog archiver failures
  • Gaps in binlog chain
  • Low disk space on archive directory

5. GTID Mode

Enable GTID for:

• Easier tracking of replication position
• Automatic failover in replication setups
• Simpler point-in-time recovery

Troubleshooting

Binary Logging Not Enabled

Error: "Binary logging appears to be disabled"

Solution: Add to my.cnf and restart MySQL:

[mysqld]
log_bin = mysql-bin
server_id = 1

Missing Binlog Files

Error: "Gaps detected in binlog chain"

Causes:

• RESET MASTER was executed
• expire_logs_days is too short
• Binlogs were manually deleted

Solution:

• Take a new base backup immediately
• Adjust retention settings to prevent future gaps

Permission Denied

Error: "Failed to read binlog directory"

Solution:

# Add dbbackup user to mysql group
sudo usermod -aG mysql dbbackup_user

# Or set appropriate permissions
sudo chmod g+r /var/lib/mysql/mysql-bin.*

Wrong Binlog Format

Warning: "binlog_format = STATEMENT (ROW recommended)"

Impact: STATEMENT format may not capture all changes accurately

Solution: Change to ROW format (requires restart):

[mysqld]
binlog_format = ROW

Server ID Changes

Warning: "server_id changed from X to Y (possible master failover)"

This warning indicates the binlog chain contains events from different servers, which may happen during:

• Failover in a replication setup
• Restoring from a different server's backup

This is usually informational but review your topology if unexpected.

MariaDB-Specific Notes

GTID Format

MariaDB uses a different GTID format than MySQL:

• MySQL: 3E11FA47-71CA-11E1-9E33-C80AA9429562:1-5
• MariaDB: 0-1-100 (domain-server_id-sequence)

Tool Detection

dbbackup automatically detects MariaDB and uses:

• mariadb-binlog if available (MariaDB 10.4+)
• Falls back to mysqlbinlog for older versions

Encrypted Binlogs

MariaDB supports binlog encryption. If enabled, ensure the key is available during archive and restore operations.

See Also

• PITR.md - PostgreSQL PITR documentation
• DOCKER.md - Running in Docker environments
• CLOUD.md - Cloud storage for archives