- Add 'System Resource Profile' menu item
- Show resource badge in main menu header (🔋 Tiny, 💡 Small, ⚡ Medium, 🚀 Large, 🏭 Huge)
- Display profile summary during backup/restore execution
- Add profile summary to restore preview screen
- Add 'p' shortcut in database selector to view profile
- Add 'p' shortcut in archive browser to view profile
- Create profile view with system info, settings editor, auto/manual toggle
TUI Integration:
- Menu: Shows system category badge (e.g., '⚡ Medium')
- Database Selector: Press 'p' to view full profile before backup
- Archive Browser: Press 'p' to view full profile before restore
- Backup Execution: Shows resources line with workers/pool
- Restore Execution: Shows resources line with workers/pool
- Restore Preview: Shows system profile summary at top
Version bump: 5.7.1
- Fixed separator line to appear UNDER title instead of after it
- Separator now matches title width for clean alignment
Before: Cluster Restore Progress ━━━━━━━━
After: Cluster Restore Progress
━━━━━━━━━━━━━━━━━━━━━━━━
Fixed:
- Native restore now connects to target database correctly (was connecting to source)
- Sequences now properly exported (fixed type mismatch in information_schema query)
- COPY FROM stdin protocol now properly handled using pgx CopyFrom
- Tool verification skipped when --native flag is used
- Fixed slice bounds panic on short SQL statements
Changes:
- internal/engine/native/manager.go: Create engine with target database for restore
- internal/engine/native/postgresql.go: COPY handling, sequence type casting
- cmd/restore.go: Skip VerifyTools in native mode
- internal/tui/restore_preview.go: Native engine mode bypass
Tested: 100k row backup/restore cycle verified working
NEW FEATURES:
- --native flag for cluster backup creates SQL format (.sql.gz) using pure Go
- --native flag for cluster restore uses pure Go engine for .sql.gz files
- Zero external tool dependencies when using native mode
- Single-binary deployment now possible without pg_dump/pg_restore
CLUSTER BACKUP (--native):
- Creates .sql.gz files instead of .dump files
- Uses pgx wire protocol for data export
- Parallel gzip compression with pgzip
- Automatic fallback with --fallback-tools
CLUSTER RESTORE (--native):
- Restores .sql.gz files using pure Go (pgx CopyFrom)
- No psql or pg_restore required
- Automatic detection: native for .sql.gz, pg_restore for .dump
FILES MODIFIED:
- cmd/backup.go: Added --native and --fallback-tools flags
- cmd/restore.go: Added --native and --fallback-tools flags
- internal/backup/engine.go: Native engine path in BackupCluster()
- internal/restore/engine.go: Added restoreWithNativeEngine()
- NATIVE_ENGINE_SUMMARY.md: Complete rewrite with accurate docs
- CHANGELOG.md: v5.5.0 release notes
CRITICAL FIX:
- Progress only updated after DB completed, not during restore
- For 100GB DB taking 4+ hours, TUI showed 0% the whole time
CHANGES:
- Heartbeat now reports estimated progress every 5s (was 15s text-only)
- Time-based estimation: ~10MB/s throughput, capped at 95%
- TUI shows spinner + elapsed time when byte-level progress unavailable
- Better visual feedback that restore is actively running
- Use 1.2x multiplier for cluster .tar.gz (pre-compressed dumps)
- Use 5x multiplier for single .sql.gz files (was 7x)
- New CheckSystemMemoryWithType() for archive-aware estimation
- 119GB archive now estimates ~143GB instead of ~833GB
- Add native engine support for restore (cmd/native_restore.go)
- Integrate native engine restore into cmd/restore.go with fallback
- Fix CPUWorkloadType to auto-detect CPU if CPUInfo is nil
- Replace standard gzip with pgzip in native_backup.go
- All compression now uses parallel pgzip consistently
Bump version to 5.4.2
- Remove compress/gzip import from internal/backup/engine.go
- Use pgzip.NewReader for parallel decompression in archive verification
- All restore paths now consistently use pgzip for parallel gzip operations
Bump version to 5.4.1
P0 Critical:
- Implement PostgreSQL native restore with COPY FROM support
- Implement MySQL native restore with DELIMITER handling
P1 High Priority:
- Fix deprecated strings.Title usage in mysql.go
- Fix unused variable in man.go
- Simplify TrimSuffix patterns in schedule.go
- Remove unused functions and commands
Dashboard:
- Add PITR section with 6 new panels
- Integrate PITR and dedup metrics into exporter
All checks pass: go build, staticcheck, go test -race
- profile.go only had: conservative, balanced, aggressive, potato
- 'turbo' profile returned ERROR and silently fell back to 'balanced'
- 'balanced' has Jobs=0 which became Jobs=1 after default fallback
- Result: --profile turbo was IGNORED, restore ran single-threaded
Added:
- turbo profile: Jobs=8, ParallelDBs=2
- max-performance profile: Jobs=8, ParallelDBs=4
NOW --profile turbo correctly uses pg_restore --jobs=8
ROOT CAUSE FOUND AND FIXED:
- BuildRestoreCommand() had condition 'if options.Parallel > 1'
- This meant --jobs flag was NEVER added when Parallel was 1 or less
- Changed to 'if options.Parallel > 0' so --jobs is ALWAYS set
- This was THE root cause why restores took 12+ hours instead of ~4 hours
- Now pg_restore --jobs=8 is correctly generated for turbo profile
PROBLEM: User's profile Jobs setting was being overridden in multiple places:
1. restoreSection() for phased restores had NO --jobs flag at all
2. Auto-fallback forced Jobs=1 when PostgreSQL locks couldn't be boosted
3. Auto-fallback forced Jobs=1 on low memory detection
FIX:
- Added --jobs flag to restoreSection() for phased restores
- Removed auto-override of Jobs=1 - now only warns user
- User's profile choice (turbo, performance, etc.) is now respected
- This was causing restores to take 9+ hours instead of ~4 hours
- TUI Settings now persist to .dbbackup.conf file (was only in-memory)
- Native Engine (pure Go) is now the default instead of external tools
- Added FallbackToTools=true for graceful degradation
- Environment variables: USE_NATIVE_ENGINE, FALLBACK_TO_TOOLS
CRITICAL BUG FIX: The --jobs flag and profile Jobs setting were completely
ignored for pg_restore. The code had hardcoded Parallel: 1 instead of using
e.cfg.Jobs, causing all restores to run single-threaded regardless of
configuration.
This fix enables restores to match native pg_restore -j8 performance:
- 12h 38m -> ~4h for 119.5GB cluster backup
- Throughput: 2.7 MB/s -> ~8 MB/s
Affected functions:
- restorePostgreSQLDump()
- restorePostgreSQLDumpWithOwnership()
Now logs parallel_jobs value for visibility. Turbo profile with Jobs: 8
now correctly passes --jobs=8 to pg_restore.
- Update version string to 5.1.14
- Update CHANGELOG with v5.1.10-v5.1.14 features
- Update README with new enterprise features
- Remove development files from repository
- Add sensitive files to .gitignore
Sync backups between cloud regions for disaster recovery
- Copy backups from source to destination cloud
- Support S3, MinIO, Azure, GCS providers
- Parallel transfers with configurable concurrency
- Dry-run mode to preview sync plan
- Filter by database name or age
- Delete orphaned files with --delete flag
Implement retention-simulator command to preview retention policy effects
- Simulate simple age-based and GFS retention strategies
- Compare multiple retention periods side-by-side
- Calculate space savings without deleting anything
- Preview which backups would be kept vs deleted
- Analyze backup frequency and provide recommendations
- Implement `dbbackup retention-simulator` command
- Preview retention policy effects without deleting backups
- Compare multiple retention strategies side-by-side
- Support both simple and GFS retention strategies
- Calculate space savings and backup counts
Simulation Features:
- Simulate simple age-based retention (days + min backups)
- Simulate GFS (Grandfather-Father-Son) retention
- Preview which backups would be kept vs deleted
- Calculate space that would be freed
- Show detailed reasoning for each backup
Comparison Mode:
- Compare multiple retention periods (7, 14, 30, 60, 90 days)
- Side-by-side comparison table
- Analyze backup frequency and patterns
- Provide retention recommendations based on backup history
- Show total storage impact
Display Information:
- Total backups and affected count
- Detailed list of backups to delete with reasons
- List of backups to keep (limited to first 10)
- Space savings calculation
- Backup frequency analysis
- Retention recommendations
Strategies Supported:
- Simple: Age-based with minimum backup protection
Example: --days 30 --min-backups 5
- GFS: Grandfather-Father-Son multi-tier retention
Example: --strategy gfs --daily 7 --weekly 4 --monthly 12
Comparison Analysis:
- Average backup interval calculation
- Total storage usage
- Recommendations based on backup frequency
- Daily backups → 7 days or GFS
- Weekly backups → 30 days
- Infrequent → 90+ days
- Multiple retention periods compared in table format
Use Cases:
- Test retention policies before applying
- Understand impact of different retention settings
- Plan storage capacity requirements
- Optimize retention for cost vs safety
- Avoid accidental deletion of important backups
- Compliance planning and validation
Output Formats:
- Text: Human-readable tables and lists
- JSON: Machine-readable for automation
Safety:
- Completely non-destructive simulation
- Clear indication this is preview only
- Instructions for applying policy with cleanup command
This completes Quick Win #14 from TODO_SESSION.md.
Helps users safely plan retention policies.
- Implement `dbbackup catalog dashboard` interactive TUI
- Browse backup catalog in sortable, filterable table view
- View detailed backup information with Enter key
- Real-time statistics (total backups, size, databases)
- Multi-level sorting and filtering capabilities
Interactive Features:
- Sortable columns: date, size, database, type
- Ascending/descending sort toggle
- Database filter with cycle navigation
- Search/filter by database name or path
- Pagination for large catalogs (20 entries per page)
- Detail view for individual backups
Navigation:
- ↑/↓ or k/j: Navigate entries
- ←/→ or h/l: Previous/next page
- Enter: View backup details
- s: Cycle sort mode
- r: Reverse sort order
- d: Cycle through database filters
- /: Enter filter mode
- c: Clear all filters
- R: Reload catalog from disk
- q/ESC: Quit (or return from details)
Display Information:
- List view: Date, database, type, size, status in table format
- Detail view: Full backup metadata including:
- Basic info (database, type, status, timestamp)
- File info (path, size, compression, encryption)
- Performance metrics (duration, throughput)
- Custom metadata fields
Statistics Bar:
- Total backup count
- Total size across all backups
- Number of unique databases
- Current filters and sort mode
Filtering Capabilities:
- Filter by database name (cycle through all databases)
- Free-text search across database names and paths
- Multiple filters can be combined
- Clear all filters with 'c' key
Use Cases:
- Quick overview of all backups
- Find specific backups interactively
- Analyze backup patterns and sizes
- Verify backup coverage per database
- Browse large backup catalogs efficiently
This completes Quick Win #13 from TODO_SESSION.md.
Provides user-friendly catalog browsing via TUI.
- Implement `dbbackup progress-webhooks` command group
- Add ProgressTracker for monitoring long-running operations
- Send periodic progress updates via webhooks/SMTP
- Track bytes processed, tables completed, and time estimates
- Calculate remaining time based on processing rate
- Support configurable update intervals (default 30s)
Progress tracking features:
- Real-time progress notifications during backup/restore
- Bytes and tables processed with percentage complete
- Elapsed time and estimated time remaining
- Current operation phase tracking
- Automatic progress calculation and rate estimation
Command structure:
- status: Show current configuration and backend status
- enable: Display setup instructions for progress tracking
- disable: Show how to disable progress updates
- test: Simulate backup with progress webhooks (5 updates)
Configuration methods:
- Environment variables (DBBACKUP_WEBHOOK_URL, DBBACKUP_PROGRESS_INTERVAL)
- Config file (.dbbackup.conf)
- Supports both webhook and SMTP notification backends
Use cases:
- Monitor long-running database backups
- External monitoring system integration
- Real-time backup progress tracking
- Automated alerting on slow/stalled backups
This completes Quick Win #11 from TODO_SESSION.md.
Enables real-time operation monitoring via webhooks.
Added 'dbbackup forecast' command for capacity planning and growth prediction:
Features:
- Analyzes historical backup growth patterns from catalog
- Calculates daily/weekly/monthly/annual growth rates
- Projects future space requirements (7, 30, 60, 90, 180, 365 days)
- Confidence scoring based on sample size and variance
- Capacity limit alerts (warn when approaching threshold)
- Calculates time until space limit reached
Usage Examples:
dbbackup forecast mydb # Basic forecast
dbbackup forecast --all # All databases
dbbackup forecast mydb --days 180 # 6-month projection
dbbackup forecast mydb --limit 500GB # Set capacity limit
dbbackup forecast mydb --format json # JSON output
Key Metrics:
- Daily growth rate (bytes/day and percentage)
- Current utilization vs capacity limit
- Growth confidence (high/medium/low)
- Time to capacity limit (with critical/warning alerts)
Helps answer:
- When will we run out of space?
- How much storage to provision?
- Is growth accelerating?
- When to add capacity?
Quick Win #9: Space Forecast - 15 min implementation
Added interactive menu options for viewing backup schedules and chains:
- internal/tui/schedule.go: New TUI view for systemd timer schedules
* Parses systemctl output to show backup timers
* Displays status, next run time, and last run info
* Filters backup-related timers (dbbackup, backup)
- internal/tui/chain.go: New TUI view for backup chain relationships
* Shows full backups and their incremental children
* Detects incomplete chains (incrementals without full backup)
* Displays chain statistics (total size, count, timespan)
* Loads from catalog database
- internal/tui/menu.go: Added menu items #8 and #9
* View Backup Schedule (systemd timers)
* View Backup Chain (full → incremental)
* Connected handleSchedule() and handleChain() methods
This completes TUI integration for Quick Win features #1 and #8.
- Add 'dbbackup chain' command to visualize backup dependencies
- Display full backup → incremental backup relationships
- Show backup sequence and timeline
- Calculate total chain size and duration
- Detect incomplete chains (incrementals without full backup)
- Support --all flag to show all database chains
- Support --verbose for detailed metadata
- Support --format json for automation
- Provides restore guidance (which backups are needed)
- Warns about orphaned incremental backups
Quick Win #8 from TODO list
- Add 'dbbackup schedule' command to show backup schedules
- Query systemd timers for next run times
- Display last run time and duration
- Show time remaining until next backup
- Support --timer flag to show specific timer
- Support --all flag to show all system timers
- Support --format json for automation
- Automatically filters backup-related timers
- Works with dbbackup-databases, etc-backup, and custom timers
Quick Win #1 from TODO list - verified on production (mysql01)
- Add 'dbbackup catalog prune' subcommand
- Support --missing flag to remove entries for deleted backup files
- Support --older-than duration (90d, 6m, 1y) for retention cleanup
- Support --status flag to remove failed/corrupted entries
- Add --dry-run flag to preview changes without deleting
- Add --database filter for targeted pruning
- Display detailed results with space freed estimates
- Implement PruneAdvanced() in catalog package
- Add parseDuration() helper for flexible time parsing
Quick Win #2 from TODO list
