dbbackup/ROADMAP.md

dbbackup Version 2.0 Roadmap

Current Status: v1.1 (Production Ready)

• ✅ 24/24 automated tests passing (100%)
• ✅ PostgreSQL, MySQL, MariaDB support
• ✅ Interactive TUI + CLI
• ✅ Cluster backup/restore
• ✅ Docker support
• ✅ Cross-platform binaries

---

Version 2.0 Vision: Enterprise-Grade Features

Transform dbbackup into an enterprise-ready backup solution with cloud storage, incremental backups, PITR, and encryption.

Target Release: Q2 2026 (3-4 months)

---

Priority Matrix

                    HIGH IMPACT
                         │
    ┌────────────────────┼────────────────────┐
    │                    │                    │
    │  Cloud Storage ⭐  │ Incremental ⭐⭐⭐ │
    │  Verification      │ PITR ⭐⭐⭐       │
    │  Retention         │ Encryption ⭐⭐   │
LOW │                    │                    │ HIGH
EFFORT ─────────────────┼──────────────────── EFFORT
    │                    │                    │
    │  Metrics           │ Web UI (optional)  │
    │  Remote Restore    │ Replication Slots  │
    │                    │                    │
    └────────────────────┼────────────────────┘
                         │
                    LOW IMPACT

---

Development Phases

Phase 1: Foundation (Weeks 1-4)

Sprint 1: Verification & Retention (2 weeks)

Goals:

• Backup integrity verification with SHA-256 checksums
• Automated retention policy enforcement
• Structured backup metadata

Features:

• ✅ Generate SHA-256 checksums during backup
• ✅ Verify backups before/after restore
• ✅ Automatic cleanup of old backups
• ✅ Retention policy: days + minimum count
• ✅ Backup metadata in JSON format

Deliverables:

# New commands
dbbackup verify backup.dump
dbbackup cleanup --retention-days 30 --min-backups 5

# Metadata format
{
  "version": "2.0",
  "timestamp": "2026-01-15T10:30:00Z",
  "database": "production",
  "size_bytes": 1073741824,
  "sha256": "abc123...",
  "db_version": "PostgreSQL 15.3",
  "compression": "gzip-9"
}

Implementation:

• internal/verification/ - Checksum calculation and validation
• internal/retention/ - Policy enforcement
• internal/metadata/ - Backup metadata management

---

Sprint 2: Cloud Storage (2 weeks)

Goals:

• Upload backups to cloud storage
• Support multiple cloud providers
• Download and restore from cloud

Providers:

• ✅ AWS S3
• ✅ MinIO (S3-compatible)
• ✅ Backblaze B2
• ✅ Azure Blob Storage (optional)
• ✅ Google Cloud Storage (optional)

Configuration:

[cloud]
enabled = true
provider = "s3"  # s3, minio, azure, gcs, b2
auto_upload = true

[cloud.s3]
bucket = "db-backups"
region = "us-east-1"
endpoint = "s3.amazonaws.com"  # Custom for MinIO
access_key = "..."  # Or use IAM role
secret_key = "..."

New Commands:

# Upload existing backup
dbbackup cloud upload backup.dump

# List cloud backups
dbbackup cloud list

# Download from cloud
dbbackup cloud download backup_id

# Restore directly from cloud
dbbackup restore single s3://bucket/backup.dump --target mydb

Dependencies:

"github.com/aws/aws-sdk-go-v2/service/s3"
"github.com/Azure/azure-sdk-for-go/sdk/storage/azblob"
"cloud.google.com/go/storage"

---

Phase 2: Advanced Backup (Weeks 5-10)

Sprint 3: Incremental Backups (3 weeks)

Goals:

• Reduce backup time and storage
• File-level incremental for PostgreSQL
• Binary log incremental for MySQL

PostgreSQL Strategy:

Full Backup (Base)
    ├─ Incremental 1 (changed files since base)
    ├─ Incremental 2 (changed files since inc1)
    └─ Incremental 3 (changed files since inc2)

MySQL Strategy:

Full Backup
    ├─ Binary Log 1 (changes since full)
    ├─ Binary Log 2
    └─ Binary Log 3

Implementation:

# Create base backup
dbbackup backup single mydb --mode full

# Create incremental
dbbackup backup single mydb --mode incremental

# Restore (automatically applies incrementals)
dbbackup restore single backup.dump --apply-incrementals

File Structure:

backups/
├── mydb_full_20260115.dump
├── mydb_full_20260115.meta
├── mydb_incr_20260116.dump      # Contains only changes
├── mydb_incr_20260116.meta      # Points to base: mydb_full_20260115
└── mydb_incr_20260117.dump

---

Sprint 4: Security & Encryption (2 weeks)

Goals:

• Encrypt backups at rest
• Secure key management
• Encrypted cloud uploads

Features:

• ✅ AES-256-GCM encryption
• ✅ Argon2 key derivation
• ✅ Multiple key sources (file, env, vault)
• ✅ Encrypted metadata

Configuration:

[encryption]
enabled = true
algorithm = "aes-256-gcm"
key_file = "/etc/dbbackup/encryption.key"

# Or use environment variable
# DBBACKUP_ENCRYPTION_KEY=base64key...

Commands:

# Generate encryption key
dbbackup keys generate

# Encrypt existing backup
dbbackup encrypt backup.dump

# Decrypt backup
dbbackup decrypt backup.dump.enc

# Automatic encryption
dbbackup backup single mydb --encrypt

File Format:

+------------------+
| Encryption Header|  (IV, algorithm, key ID)
+------------------+
| Encrypted Data   |  (AES-256-GCM)
+------------------+
| Auth Tag         |  (HMAC for integrity)
+------------------+

---

Sprint 5: Point-in-Time Recovery - PITR (4 weeks)

Goals:

• Restore to any point in time
• WAL archiving for PostgreSQL
• Binary log archiving for MySQL

PostgreSQL Implementation:

[pitr]
enabled = true
wal_archive_dir = "/backups/wal_archive"
wal_retention_days = 7

# PostgreSQL config (auto-configured by dbbackup)
# archive_mode = on
# archive_command = '/usr/local/bin/dbbackup archive-wal %p %f'

Commands:

# Enable PITR
dbbackup pitr enable

# Archive WAL manually
dbbackup archive-wal /var/lib/postgresql/pg_wal/000000010000000000000001

# Restore to point-in-time
dbbackup restore single backup.dump \
  --target-time "2026-01-15 14:30:00" \
  --target mydb

# Show available restore points
dbbackup pitr timeline

WAL Archive Structure:

wal_archive/
├── 000000010000000000000001
├── 000000010000000000000002
├── 000000010000000000000003
└── timeline.json

MySQL Implementation:

# Archive binary logs
dbbackup binlog archive --start-datetime "2026-01-15 00:00:00"

# PITR restore
dbbackup restore single backup.sql \
  --target-time "2026-01-15 14:30:00" \
  --apply-binlogs

---

Phase 3: Enterprise Features (Weeks 11-16)

Sprint 6: Observability & Integration (3 weeks)

Features:

1. Prometheus Metrics

# Exposed metrics
dbbackup_backup_duration_seconds
dbbackup_backup_size_bytes
dbbackup_backup_success_total
dbbackup_restore_duration_seconds
dbbackup_last_backup_timestamp
dbbackup_cloud_upload_duration_seconds

Endpoint:

# Start metrics server
dbbackup metrics serve --port 9090

# Scrape endpoint
curl http://localhost:9090/metrics

2. Remote Restore

# Restore to remote server
dbbackup restore single backup.dump \
  --remote-host db-replica-01 \
  --remote-user postgres \
  --remote-port 22 \
  --confirm

3. Replication Slots (PostgreSQL)

# Create replication slot for continuous WAL streaming
dbbackup replication create-slot backup_slot

# Stream WALs via replication
dbbackup replication stream backup_slot

4. Webhook Notifications

[notifications]
enabled = true
webhook_url = "https://slack.com/webhook/..."
notify_on = ["backup_complete", "backup_failed", "restore_complete"]

---

Technical Architecture

New Directory Structure

internal/
├── cloud/              # Cloud storage backends
│   ├── interface.go
│   ├── s3.go
│   ├── azure.go
│   └── gcs.go
├── encryption/         # Encryption layer
│   ├── aes.go
│   ├── keys.go
│   └── vault.go
├── incremental/        # Incremental backup engine
│   ├── postgres.go
│   └── mysql.go
├── pitr/              # Point-in-time recovery
│   ├── wal.go
│   ├── binlog.go
│   └── timeline.go
├── verification/      # Backup verification
│   ├── checksum.go
│   └── validate.go
├── retention/         # Retention policy
│   └── cleanup.go
├── metrics/           # Prometheus metrics
│   └── exporter.go
└── replication/       # Replication management
    └── slots.go

Required Dependencies

// Cloud storage
"github.com/aws/aws-sdk-go-v2/service/s3"
"github.com/Azure/azure-sdk-for-go/sdk/storage/azblob"
"cloud.google.com/go/storage"

// Encryption
"crypto/aes"
"crypto/cipher"
"golang.org/x/crypto/argon2"

// Metrics
"github.com/prometheus/client_golang/prometheus"
"github.com/prometheus/client_golang/prometheus/promhttp"

// PostgreSQL replication
"github.com/jackc/pgx/v5/pgconn"

// Fast file scanning for incrementals
"github.com/karrick/godirwalk"

---

Testing Strategy

v2.0 Test Coverage Goals

• Minimum 90% code coverage
• Integration tests for all cloud providers
• End-to-end PITR scenarios
• Performance benchmarks for incremental backups
• Encryption/decryption validation
• Multi-database restore tests

New Test Suites

# Cloud storage tests
./run_qa_tests.sh --suite cloud

# Incremental backup tests
./run_qa_tests.sh --suite incremental

# PITR tests
./run_qa_tests.sh --suite pitr

# Encryption tests
./run_qa_tests.sh --suite encryption

# Full v2.0 suite
./run_qa_tests.sh --suite v2

---

Migration Path

v1.x → v2.0 Compatibility

• ✅ All v1.x backups readable in v2.0
• ✅ Configuration auto-migration
• ✅ Metadata format upgrade
• ✅ Backward-compatible commands

Deprecation Timeline

• v2.0: Warning for old config format
• v2.1: Full migration required
• v3.0: Old format no longer supported

---

Documentation Updates

New Docs

• CLOUD.md - Cloud storage configuration
• INCREMENTAL.md - Incremental backup guide
• PITR.md - Point-in-time recovery
• ENCRYPTION.md - Encryption setup
• METRICS.md - Prometheus integration

---

Success Metrics

v2.0 Goals

• 🎯 95%+ test coverage
• 🎯 Support 1TB+ databases with incrementals
• 🎯 PITR with <5 minute granularity
• 🎯 Cloud upload/download >100MB/s
• 🎯 Encryption overhead <10%
• 🎯 Full compatibility with pgBackRest for PostgreSQL
• 🎯 Industry-leading MySQL PITR solution

---

Release Schedule

• v2.0-alpha (End Sprint 3): Cloud + Verification
• v2.0-beta (End Sprint 5): + Incremental + PITR
• v2.0-rc1 (End Sprint 6): + Enterprise features
• v2.0 GA (Q2 2026): Production release

---

What Makes v2.0 Unique

After v2.0, dbbackup will be:

✅ Only multi-database tool with full PITR support
✅ Best-in-class UX (TUI + CLI + Docker + K8s)
✅ Feature parity with pgBackRest (PostgreSQL)
✅ Superior to mysqldump with incremental + PITR
✅ Cloud-native with multi-provider support
✅ Enterprise-ready with encryption + metrics
✅ Zero-config for 80% of use cases

---

Contributing

Want to contribute to v2.0? Check out:

• CONTRIBUTING.md
• Good First Issues
• v2.0 Milestone

---

Questions?

Open an issue or start a discussion:

• Issues: https://git.uuxo.net/uuxo/dbbackup/issues
• Discussions: https://git.uuxo.net/uuxo/dbbackup/discussions

---

Next Step: Sprint 1 - Backup Verification & Retention (January 2026)