UUXO/dbbackup
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 31 Wiki Activity
263 Commits 1 Branch 56 Tags
b99b05fedb213570148aec8835470dad7a90a1ec
Commit Graph

8 Commits

Author SHA1 Message Date
Alexander Renz
d0d83b61ef feat: add dry-run mode, GFS retention policies, and notifications 
- Add --dry-run/-n flag for backup commands with comprehensive preflight checks
  - Database connectivity validation
  - Required tools availability check
  - Storage target and permissions verification
  - Backup size estimation
  - Encryption and cloud storage configuration validation

- Implement GFS (Grandfather-Father-Son) retention policies
  - Daily/Weekly/Monthly/Yearly tier classification
  - Configurable retention counts per tier
  - Custom weekly day and monthly day settings
  - ISO week handling for proper week boundaries

- Add notification system with SMTP and webhook support
  - SMTP email notifications with TLS/STARTTLS
  - Webhook HTTP notifications with HMAC-SHA256 signing
  - Slack-compatible webhook payload format
  - Event types: backup/restore started/completed/failed, cleanup, verify, PITR
  - Configurable severity levels and retry logic

- Update README.md with documentation for all new features
 2025-12-13 19:00:54 +01:00
A. Renz
914307ac8f ci: add golangci-lint config and fix formatting 
- Add .golangci.yml with minimal linters (govet, ineffassign)
- Run gofmt -s and goimports on all files to fix formatting
- Disable fieldalignment and copylocks checks in govet
 2025-12-11 17:53:28 +01:00
Renz
3ef57bb2f5 polish: Week 2 improvements - error messages, progress, performance 
## Error Message Improvements (Phase 1)
- ✅ Cluster backup: Added database type context to error messages
- ✅ Rate limiting: Show specific host and wait time in errors
- ✅ Connection failures: Added troubleshooting steps (3-point checklist)
- ✅ Encryption errors: Include backup location in failure messages
- ✅ Archive not found: Suggest cloud:// URI for remote backups
- ✅ Decryption: Hint about wrong key verification
- ✅ Backup directory: Include permission hints and --backup-dir suggestion
- ✅ Backup execution: Show database name and diagnostic checklist
- ✅ Incremental: Better base backup path guidance
- ✅ File verification: Indicate silent command failure possibility

## Progress Indicator Enhancements (Phase 2)
- ✅ ETA calculations: Real-time estimation based on transfer speed
- ✅ Speed formatting: formatSpeed() helper (B/KB/MB/GB per second)
- ✅ Byte formatting: formatBytes() with proper unit scaling
- ✅ Duration display: Improved to show Xm Ys format vs decimal
- ✅ Progress updates: Show [%] bytes/total (speed, ETA: time) format

## Performance Optimization (Phase 3)
- ✅ Buffer sizes: Increased stderr read buffers from 4KB to 64KB
- ✅ Scanner buffers: 64KB initial, 1MB max for command output
- ✅ I/O throughput: Better buffer alignment for streaming operations

## Code Cleanup (Phase 4)
- ✅ TODO comments: Converted to descriptive comments
- ✅ Method calls: Fixed GetDatabaseType() -> DisplayDatabaseType()
- ✅ Build verification: All changes compile successfully

## Summary
Time: ~1.5h (2-4h estimated)
Changed: 4 files (cmd/backup_impl.go, cmd/restore.go, internal/backup/engine.go, internal/progress/detailed.go)
Impact: Better UX, clearer errors, faster I/O, cleaner code
 2025-11-26 10:30:29 +00:00
Renz
e059cc2e3a feat: Step 4 - Add --backup-type incremental CLI flag (scaffolding) 
Added CLI integration for incremental backups:

cmd/backup.go:
- Added --backup-type flag (full/incremental)
- Added --base-backup flag for specifying base backup
- Updated help text with incremental examples
- Global vars to avoid initialization cycle

cmd/backup_impl.go:
- Validation: incremental requires PostgreSQL
- Validation: incremental requires --base-backup
- Validation: base backup file must exist
- Logging: backup_type added to log output
- Fallback: warns and does full backup for now

Status: CLI READY but not functional
- Flag parsing works
- Validation works
- Warns user that incremental is not implemented yet
- Falls back to full backup

Next: Implement CreateIncrementalBackup() and RestoreIncremental()
 2025-11-26 06:37:54 +00:00
Renz
bdf9af0650 feat: v2.0 Sprint 3 - Cloud URI Support & Command Integration (Part 1) 
Sprint 3 Implementation - Cloud URI Support:

New Features:
✅ Cloud URI parser (s3://bucket/path)
✅ Backup command with --cloud URI flag
✅ Restore from cloud URIs
✅ Verify cloud backups
✅ Cleanup cloud storage with retention policy

New Files:
- internal/cloud/uri.go - Cloud URI parser
- internal/restore/ - Cloud download module
- internal/restore/cloud_download.go - Download & verify helper

Modified Commands:
- cmd/backup.go - Added --cloud s3://bucket/path flag
- cmd/restore.go - Auto-detect & download from cloud URIs
- cmd/verify.go - Verify backups from cloud storage
- cmd/cleanup.go - Apply retention policy to cloud storage

URI Support:
- s3://bucket/path/file.dump - AWS S3
- minio://bucket/path/file.dump - MinIO
- b2://bucket/path/file.dump - Backblaze B2
- gs://bucket/path/file.dump - Google Cloud Storage

Examples:
  # Backup with cloud URI
  dbbackup backup single mydb --cloud s3://my-bucket/backups/

  # Restore from cloud
  dbbackup restore single s3://my-bucket/backups/mydb.dump --confirm

  # Verify cloud backup
  dbbackup verify-backup s3://my-bucket/backups/mydb.dump

  # Cleanup old cloud backups
  dbbackup cleanup s3://my-bucket/backups/ --retention-days 30

Features:
- Automatic download to temp directory
- SHA-256 verification after download
- Automatic temp file cleanup
- Progress tracking for downloads
- Metadata synchronization
- Retention policy for cloud storage

Sprint 3 Part 1 COMPLETE!
 2025-11-25 20:30:28 +00:00
Renz
20b7f1ec04 feat: v2.0 Sprint 2 - Auto-Upload to Cloud (Part 2) 
- Add cloud configuration to Config struct
- Integrate automatic upload into backup flow
- Add --cloud-auto-upload flag to all backup commands
- Support environment variables for cloud credentials
- Upload both backup file and metadata to cloud
- Non-blocking: backup succeeds even if cloud upload fails

Usage:
  dbbackup backup single mydb --cloud-auto-upload \
    --cloud-bucket my-backups \
    --cloud-provider s3

Or via environment:
  export CLOUD_ENABLED=true
  export CLOUD_AUTO_UPLOAD=true
  export CLOUD_BUCKET=my-backups
  export AWS_ACCESS_KEY_ID=...
  export AWS_SECRET_ACCESS_KEY=...
  dbbackup backup single mydb

Credentials from AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY
 2025-11-25 19:44:52 +00:00
Renz
0cf21cd893 feat: Complete MEDIUM priority security features with testing 
- Implemented TUI auto-select for automated testing
- Fixed TUI automation: autoSelectMsg handling in Update()
- Auto-database selection in DatabaseSelector
- Created focused test suite (test_as_postgres.sh)
- Created retention policy test (test_retention.sh)
- All 10 security tests passing

Features validated:
✅ Backup retention policy (30 days, min backups)
✅ Rate limiting (exponential backoff)
✅ Privilege checks (root detection)
✅ Resource limit validation
✅ Path sanitization
✅ Checksum verification (SHA-256)
✅ Audit logging
✅ Secure permissions
✅ Configuration persistence
✅ TUI automation framework

Test results: 10/10 passed
Backup files created with .dump, .sha256, .info
Retention cleanup verified (old files removed)
 2025-11-25 15:25:56 +00:00
Renz
e361968022 Initial commit: Database Backup Tool v1.1.0 
- PostgreSQL and MySQL support
- Interactive TUI with fixed menu navigation
- Line-by-line progress display
- CPU-aware parallel processing
- Cross-platform build support
- Configuration settings menu
- Silent mode for TUI operations
 2025-10-22 19:27:38 +00:00