feat: Add enterprise DBA features for production reliability

New features implemented:

1. Backup Catalog (internal/catalog/)
   - SQLite-based backup tracking
   - Gap detection and RPO monitoring
   - Search and statistics
   - Filesystem sync

2. DR Drill Testing (internal/drill/)
   - Automated restore testing in Docker containers
   - Database validation with custom queries
   - Catalog integration for drill-tested status

3. Smart Notifications (internal/notify/)
   - Event batching with configurable intervals
   - Time-based escalation policies
   - HTML/text/Slack templates

4. Compliance Reports (internal/report/)
   - SOC2, GDPR, HIPAA, PCI-DSS, ISO27001 frameworks
   - Evidence collection from catalog
   - JSON, Markdown, HTML output formats

5. RTO/RPO Calculator (internal/rto/)
   - Recovery objective analysis
   - RTO breakdown by phase
   - Recommendations for improvement

6. Replica-Aware Backup (internal/replica/)
   - Topology detection for PostgreSQL/MySQL
   - Automatic replica selection
   - Configurable selection strategies

7. Parallel Table Backup (internal/parallel/)
   - Concurrent table dumps
   - Worker pool with progress tracking
   - Large table optimization

8. MySQL/MariaDB PITR (internal/pitr/)
   - Binary log parsing and replay
   - Point-in-time recovery support
   - Transaction filtering

CLI commands added: catalog, drill, report, rto

All changes support the goal: reliable 3 AM database recovery.
This commit is contained in:
2025-12-13 20:28:55 +01:00
parent d0d83b61ef
commit f69bfe7071
34 changed files with 13469 additions and 41 deletions

127
README.md
View File

@@ -16,12 +16,22 @@ Database backup and restore utility for PostgreSQL, MySQL, and MariaDB.
- AES-256-GCM encryption
- Incremental backups
- Cloud storage: S3, MinIO, B2, Azure Blob, Google Cloud Storage
- Point-in-Time Recovery (PITR) for PostgreSQL
- Point-in-Time Recovery (PITR) for PostgreSQL and MySQL/MariaDB
- **GFS retention policies**: Grandfather-Father-Son backup rotation
- **Notifications**: SMTP email and webhook alerts
- Interactive terminal UI
- Cross-platform binaries
### Enterprise DBA Features
- **Backup Catalog**: SQLite-based catalog tracking all backups with gap detection
- **DR Drill Testing**: Automated disaster recovery testing in Docker containers
- **Smart Notifications**: Batched alerts with escalation policies
- **Compliance Reports**: SOC2, GDPR, HIPAA, PCI-DSS, ISO27001 report generation
- **RTO/RPO Calculator**: Recovery objective analysis and recommendations
- **Replica-Aware Backup**: Automatic backup from replicas to reduce primary load
- **Parallel Table Backup**: Concurrent table dumps for faster backups
## Installation
### Docker
@@ -257,6 +267,10 @@ dbbackup backup single mydb --dry-run
| `pitr` | PITR management |
| `wal` | WAL archive operations |
| `interactive` | Start interactive UI |
| `catalog` | Backup catalog management |
| `drill` | DR drill testing |
| `report` | Compliance report generation |
| `rto` | RTO/RPO analysis |
## Global Flags
@@ -478,6 +492,117 @@ dbbackup backup single mydb --notify
- `cleanup_completed`
- `verify_completed`, `verify_failed`
- `pitr_recovery`
- `dr_drill_passed`, `dr_drill_failed`
- `gap_detected`, `rpo_violation`
## Backup Catalog
Track all backups in a SQLite catalog with gap detection and search:
```bash
# Sync backups from directory to catalog
dbbackup catalog sync /backups
# List recent backups
dbbackup catalog list --database mydb --limit 10
# Show catalog statistics
dbbackup catalog stats
# Detect backup gaps (missing scheduled backups)
dbbackup catalog gaps --interval 24h --database mydb
# Search backups
dbbackup catalog search --database mydb --start 2024-01-01 --end 2024-12-31
# Get backup info
dbbackup catalog info 42
```
## DR Drill Testing
Automated disaster recovery testing restores backups to Docker containers:
```bash
# Run full DR drill
dbbackup drill run /backups/mydb_latest.dump.gz \
--database mydb \
--db-type postgres \
--timeout 30m
# Quick drill (restore + basic validation)
dbbackup drill quick /backups/mydb_latest.dump.gz --database mydb
# List running drill containers
dbbackup drill list
# Cleanup old drill containers
dbbackup drill cleanup --age 24h
# Generate drill report
dbbackup drill report --format html --output drill-report.html
```
**Drill phases:**
1. Container creation
2. Backup download (if cloud)
3. Restore execution
4. Database validation
5. Custom query checks
6. Cleanup
## Compliance Reports
Generate compliance reports for regulatory frameworks:
```bash
# Generate SOC2 report
dbbackup report generate --type soc2 --days 90 --format html --output soc2-report.html
# HIPAA compliance report
dbbackup report generate --type hipaa --format markdown
# Show compliance summary
dbbackup report summary --type gdpr --days 30
# List available frameworks
dbbackup report list
# Show controls for a framework
dbbackup report controls soc2
```
**Supported frameworks:**
- SOC2 Type II (Trust Service Criteria)
- GDPR (General Data Protection Regulation)
- HIPAA (Health Insurance Portability and Accountability Act)
- PCI-DSS (Payment Card Industry Data Security Standard)
- ISO 27001 (Information Security Management)
## RTO/RPO Analysis
Calculate and monitor Recovery Time/Point Objectives:
```bash
# Analyze RTO/RPO for a database
dbbackup rto analyze mydb
# Show status for all databases
dbbackup rto status
# Check against targets
dbbackup rto check --rto 4h --rpo 1h
# Set target objectives
dbbackup rto analyze mydb --target-rto 4h --target-rpo 1h
```
**Analysis includes:**
- Current RPO (time since last backup)
- Estimated RTO (detection + download + restore + validation)
- RTO breakdown by phase
- Compliance status
- Recommendations for improvement
## Configuration