backupGlobals() used cmd.Output() which blocks until completion
even when context is cancelled. Changed to Start/Wait pattern
with proper context handling for immediate Ctrl+C response.
The actual bug: MkdirAll returns success on CIFS/NFS but the directory
isn't immediately visible for file operations.
Fix:
- Verify directory exists with os.Stat after MkdirAll
- Retry loop (5 attempts, 20ms delay) until directory is visible
- Add write retry loop with re-mkdir on failure
- Keep rename retry as fallback
On network filesystems (CIFS/SMB), atomic renames can fail with
'no such file or directory' due to stale directory caches.
Fix:
- Add MkdirAll before rename to refresh directory cache
- Retry rename up to 3 times with 10ms delay
- Re-ensure directory exists on each retry attempt
FIXES:
- Switch from mattn/go-sqlite3 (CGO) to modernc.org/sqlite (pure Go)
Binaries compiled with CGO_ENABLED=0 now work correctly
- Fix MySQL positional database argument being ignored
'dbbackup backup single gitea --db-type mysql' now uses 'gitea' correctly
Feature requested by DBA: Limit upload/download speed during business hours.
- New --bandwidth-limit flag for cloud operations (S3, GCS, Azure, MinIO, B2)
- Supports human-readable formats: 10MB/s, 50MiB/s, 100Mbps, unlimited
- Environment variable: DBBACKUP_BANDWIDTH_LIMIT
- Token-bucket style throttling with 100ms windows for smooth limiting
- Reduces multipart concurrency when throttled for better rate control
- Unit tests for parsing and throttle behavior
- tmpfs.go: Convert stat.Blocks/Bavail/Bfree to int64 for cross-platform math
- large_db_guard.go: Same fix for disk space calculation
- FreeBSD uses int64 for these fields, Linux uses uint64
- Remove ALL remaining exec.Command tar/gzip/gunzip calls from internal code
- diagnose.go: Replace 'tar -tzf' test with direct file open check
- large_restore_check.go: Replace 'gzip -t' and 'gzip -l' with in-process pgzip verification
- pitr/restore.go: Replace 'tar -xf' with in-process archive/tar extraction
- All backup/restore operations now 100% in-process using github.com/klauspost/pgzip
- Benefits: No external tool dependencies, 2-4x faster on multi-core, reliable error handling
- Note: Docker drill container commands still use gunzip for in-container ops (intentional)
- Add fs.NewParallelGzipWriter() for streaming compression
- Replace shell gzip with pgzip in executeMySQLWithCompression()
- Replace shell gzip with pgzip in executeMySQLWithProgressAndCompression()
- No external gzip binary dependency for MySQL backups
- 2-4x faster compression on multi-core systems
- Add fs.CreateTarGzParallel() using pgzip for archive creation
- Replace shell tar/pigz with in-process parallel compression
- 2-4x faster compression on multi-core systems
- No external process dependencies (tar, pigz not required)
- Matches parallel extraction already in place
- Both backup and restore now use pgzip for maximum performance
- Replace shell 'tar -xzf' with fs.ExtractTarGzParallel() in engine.go
- Replace shell 'tar -xzf' with fs.ExtractTarGzParallel() in diagnose.go
- All extraction now uses pgzip with runtime.NumCPU() cores
- 2-4x faster extraction on multi-core systems
- Includes path traversal protection and secure permissions
🔴 HIGH PRIORITY FIXES:
- Fix goroutine leak: semaphore acquisition now context-aware (prevents hang on cancel)
- Incremental lock boosting: 2048→4096→8192→16384→32768→65536 based on BLOB count
(no longer jumps straight to 65536 which uses too much shared memory)
🟡 MEDIUM PRIORITY:
- Resume capability: RestoreCheckpoint tracks completed/failed DBs for --resume
- Secure temp files: 0700 permissions prevent other users reading dump contents
- SecureMkdirTemp() and SecureWriteFile() utilities in fs package
🟢 LOW PRIORITY:
- PostgreSQL checkpoint tuning: checkpoint_timeout=30min, checkpoint_completion_target=0.9
- Added checkpoint_timeout and checkpoint_completion_target to RevertPostgresSettings()
Security improvements:
- Temp extraction directories now use 0700 (owner-only)
- Checkpoint files use 0600 permissions
- Created ProgressSnapshot struct without sync.RWMutex
- GetSnapshot() now returns ProgressSnapshot instead of UnifiedClusterProgress
- Fixes govet copylocks error
- Press 'w' cycles: SYSTEM → CONFIG → BACKUP → SYSTEM
- Clear labels: [SYS] SYSTEM TEMP, [CFG] CONFIG, [BKP] BACKUP DIR
- Shows actual path for each option
- Warning only shown when using /tmp (space issues)
- build_all.sh: reduced to 5 platforms (Linux/macOS only)
- Single unified progress tracker replaces 3 separate callbacks
- Phase-based weighting: Extract(20%), Globals(5%), Databases(70%), Verify(5%)
- Real-time ETA calculation based on completion rate
- Per-database progress with byte-level tracking
- Thread-safe with mutex protection
- FormatStatus() and FormatBar() for display
- GetSnapshot() for safe state copying
- Full test coverage including thread safety
Example output:
[67%] DB 12/18: orders_db (2.4 GB / 3.1 GB) | Elapsed: 34m12s ETA: 17m30s
[██████████████████████████████░░░░░░░░░░░░] 67%
Critical improvements:
- StreamCountBLOBs() - streams pg_restore -l output line by line
- StreamAnalyzeDump() - analyze dumps without loading into memory
- detectLargeObjects() now uses streaming (was: cmd.Output() into memory)
- TuneMySQLForRestore() - disable sync, constraints for fast restore
- RevertMySQLSettings() - restore safe defaults after restore
For 119GB restore: prevents OOM during dump analysis phase
- discoverTmpfsMounts() reads /proc/mounts for ALL tmpfs/devtmpfs
- No hardcoded /dev/shm, /tmp, /run paths
- Recommend any writable tmpfs with enough space
- Pick tmpfs with most free space
- Add TmpfsRecommendation to LargeDBGuard
- CheckTmpfsAvailable() scans /dev/shm, /run/shm, /tmp for writable tmpfs
- GetOptimalTempDir() returns best temp dir (tmpfs preferred)
- Add internal/fs/tmpfs.go with TmpfsManager utility
- All works without root - uses existing system tmpfs mounts
For 119GB restore on 32GB RAM:
- If /dev/shm has space, use it for faster temp files
- Falls back to disk if tmpfs too small
prepare_system.sh (run as root):
- Swap creation (auto-detects size)
- OOM killer protection
- Kernel tuning
prepare_postgres.sh (run as postgres user):
- PostgreSQL memory tuning
- Lock limit increase
- Disable parallel workers
No more connection issues - each script runs as the right user && git push origin main
New approach:
1. Find PostgreSQL data directory (checks common locations)
2. Write settings directly to postgresql.auto.conf file
3. Falls back to psql only if direct write fails
4. No environment variables, no passwords, no connection issues
Supports: RHEL/CentOS, Debian/Ubuntu, multiple PostgreSQL versions
- --swap auto now detects optimal size based on available disk
- --fix uses auto-detection instead of hardcoded 16G
- Reduces swap size automatically if disk space is limited
- Minimum 2GB buffer kept for system operations
- Works with as little as 3GB free disk space (creates 1GB swap)
- Add CheckSystemMemory() to LargeDBGuard for pre-restore memory analysis
- Add memory info parsing from /proc/meminfo
- Add TunePostgresForRestore() and RevertPostgresSettings() SQL helpers
- Integrate memory checking into restore engine with automatic low-memory mode
- Add --oom-protection and --low-memory flags to cluster restore command
- Add diagnose_restore_oom.sh emergency script for production OOM issues
For 119GB+ backups on 32GB RAM systems:
- Automatically detects insufficient memory and enables single-threaded mode
- Recommends swap creation when backup size exceeds available memory
- Provides PostgreSQL tuning recommendations (work_mem=64MB, disable parallel)
- Estimates restore time based on backup size
