feat: Add enterprise DBA features for production reliability
New features implemented: 1. Backup Catalog (internal/catalog/) - SQLite-based backup tracking - Gap detection and RPO monitoring - Search and statistics - Filesystem sync 2. DR Drill Testing (internal/drill/) - Automated restore testing in Docker containers - Database validation with custom queries - Catalog integration for drill-tested status 3. Smart Notifications (internal/notify/) - Event batching with configurable intervals - Time-based escalation policies - HTML/text/Slack templates 4. Compliance Reports (internal/report/) - SOC2, GDPR, HIPAA, PCI-DSS, ISO27001 frameworks - Evidence collection from catalog - JSON, Markdown, HTML output formats 5. RTO/RPO Calculator (internal/rto/) - Recovery objective analysis - RTO breakdown by phase - Recommendations for improvement 6. Replica-Aware Backup (internal/replica/) - Topology detection for PostgreSQL/MySQL - Automatic replica selection - Configurable selection strategies 7. Parallel Table Backup (internal/parallel/) - Concurrent table dumps - Worker pool with progress tracking - Large table optimization 8. MySQL/MariaDB PITR (internal/pitr/) - Binary log parsing and replay - Point-in-time recovery support - Transaction filtering CLI commands added: catalog, drill, report, rto All changes support the goal: reliable 3 AM database recovery.
This commit is contained in:
127
README.md
127
README.md
@@ -16,12 +16,22 @@ Database backup and restore utility for PostgreSQL, MySQL, and MariaDB.
|
||||
- AES-256-GCM encryption
|
||||
- Incremental backups
|
||||
- Cloud storage: S3, MinIO, B2, Azure Blob, Google Cloud Storage
|
||||
- Point-in-Time Recovery (PITR) for PostgreSQL
|
||||
- Point-in-Time Recovery (PITR) for PostgreSQL and MySQL/MariaDB
|
||||
- **GFS retention policies**: Grandfather-Father-Son backup rotation
|
||||
- **Notifications**: SMTP email and webhook alerts
|
||||
- Interactive terminal UI
|
||||
- Cross-platform binaries
|
||||
|
||||
### Enterprise DBA Features
|
||||
|
||||
- **Backup Catalog**: SQLite-based catalog tracking all backups with gap detection
|
||||
- **DR Drill Testing**: Automated disaster recovery testing in Docker containers
|
||||
- **Smart Notifications**: Batched alerts with escalation policies
|
||||
- **Compliance Reports**: SOC2, GDPR, HIPAA, PCI-DSS, ISO27001 report generation
|
||||
- **RTO/RPO Calculator**: Recovery objective analysis and recommendations
|
||||
- **Replica-Aware Backup**: Automatic backup from replicas to reduce primary load
|
||||
- **Parallel Table Backup**: Concurrent table dumps for faster backups
|
||||
|
||||
## Installation
|
||||
|
||||
### Docker
|
||||
@@ -257,6 +267,10 @@ dbbackup backup single mydb --dry-run
|
||||
| `pitr` | PITR management |
|
||||
| `wal` | WAL archive operations |
|
||||
| `interactive` | Start interactive UI |
|
||||
| `catalog` | Backup catalog management |
|
||||
| `drill` | DR drill testing |
|
||||
| `report` | Compliance report generation |
|
||||
| `rto` | RTO/RPO analysis |
|
||||
|
||||
## Global Flags
|
||||
|
||||
@@ -478,6 +492,117 @@ dbbackup backup single mydb --notify
|
||||
- `cleanup_completed`
|
||||
- `verify_completed`, `verify_failed`
|
||||
- `pitr_recovery`
|
||||
- `dr_drill_passed`, `dr_drill_failed`
|
||||
- `gap_detected`, `rpo_violation`
|
||||
|
||||
## Backup Catalog
|
||||
|
||||
Track all backups in a SQLite catalog with gap detection and search:
|
||||
|
||||
```bash
|
||||
# Sync backups from directory to catalog
|
||||
dbbackup catalog sync /backups
|
||||
|
||||
# List recent backups
|
||||
dbbackup catalog list --database mydb --limit 10
|
||||
|
||||
# Show catalog statistics
|
||||
dbbackup catalog stats
|
||||
|
||||
# Detect backup gaps (missing scheduled backups)
|
||||
dbbackup catalog gaps --interval 24h --database mydb
|
||||
|
||||
# Search backups
|
||||
dbbackup catalog search --database mydb --start 2024-01-01 --end 2024-12-31
|
||||
|
||||
# Get backup info
|
||||
dbbackup catalog info 42
|
||||
```
|
||||
|
||||
## DR Drill Testing
|
||||
|
||||
Automated disaster recovery testing restores backups to Docker containers:
|
||||
|
||||
```bash
|
||||
# Run full DR drill
|
||||
dbbackup drill run /backups/mydb_latest.dump.gz \
|
||||
--database mydb \
|
||||
--db-type postgres \
|
||||
--timeout 30m
|
||||
|
||||
# Quick drill (restore + basic validation)
|
||||
dbbackup drill quick /backups/mydb_latest.dump.gz --database mydb
|
||||
|
||||
# List running drill containers
|
||||
dbbackup drill list
|
||||
|
||||
# Cleanup old drill containers
|
||||
dbbackup drill cleanup --age 24h
|
||||
|
||||
# Generate drill report
|
||||
dbbackup drill report --format html --output drill-report.html
|
||||
```
|
||||
|
||||
**Drill phases:**
|
||||
1. Container creation
|
||||
2. Backup download (if cloud)
|
||||
3. Restore execution
|
||||
4. Database validation
|
||||
5. Custom query checks
|
||||
6. Cleanup
|
||||
|
||||
## Compliance Reports
|
||||
|
||||
Generate compliance reports for regulatory frameworks:
|
||||
|
||||
```bash
|
||||
# Generate SOC2 report
|
||||
dbbackup report generate --type soc2 --days 90 --format html --output soc2-report.html
|
||||
|
||||
# HIPAA compliance report
|
||||
dbbackup report generate --type hipaa --format markdown
|
||||
|
||||
# Show compliance summary
|
||||
dbbackup report summary --type gdpr --days 30
|
||||
|
||||
# List available frameworks
|
||||
dbbackup report list
|
||||
|
||||
# Show controls for a framework
|
||||
dbbackup report controls soc2
|
||||
```
|
||||
|
||||
**Supported frameworks:**
|
||||
- SOC2 Type II (Trust Service Criteria)
|
||||
- GDPR (General Data Protection Regulation)
|
||||
- HIPAA (Health Insurance Portability and Accountability Act)
|
||||
- PCI-DSS (Payment Card Industry Data Security Standard)
|
||||
- ISO 27001 (Information Security Management)
|
||||
|
||||
## RTO/RPO Analysis
|
||||
|
||||
Calculate and monitor Recovery Time/Point Objectives:
|
||||
|
||||
```bash
|
||||
# Analyze RTO/RPO for a database
|
||||
dbbackup rto analyze mydb
|
||||
|
||||
# Show status for all databases
|
||||
dbbackup rto status
|
||||
|
||||
# Check against targets
|
||||
dbbackup rto check --rto 4h --rpo 1h
|
||||
|
||||
# Set target objectives
|
||||
dbbackup rto analyze mydb --target-rto 4h --target-rpo 1h
|
||||
```
|
||||
|
||||
**Analysis includes:**
|
||||
- Current RPO (time since last backup)
|
||||
- Estimated RTO (detection + download + restore + validation)
|
||||
- RTO breakdown by phase
|
||||
- Compliance status
|
||||
- Recommendations for improvement
|
||||
|
||||
## Configuration
|
||||
|
||||
|
||||
Reference in New Issue
Block a user