Compare commits

...

4 Commits

Author SHA1 Message Date
ea4337e298 fix(config): use resource profile defaults for Jobs, DumpJobs, ClusterParallelism
All checks were successful
CI/CD / Test (push) Successful in 1m16s
CI/CD / Lint (push) Successful in 1m25s
CI/CD / Build & Release (push) Successful in 3m17s
- ClusterParallelism now defaults to recommended profile (1 for small VMs)
- Jobs and DumpJobs now use profile values instead of raw CPU counts
- Small VMs (4 cores, 32GB) will now get 'conservative' or 'balanced' profile
  with lower parallelism to avoid 'out of shared memory' errors

This fixes the issue where small VMs defaulted to ClusterParallelism=2
regardless of detected resources, causing restore failures on large DBs.
2026-01-18 12:04:11 +01:00
bbd4f0ceac docs: update TUI screenshots with resource profiles and system info
All checks were successful
CI/CD / Test (push) Successful in 1m17s
CI/CD / Lint (push) Successful in 1m27s
CI/CD / Build & Release (push) Has been skipped
- Added system detection display (CPU cores, memory, recommended profile)
- Added Resource Profile and Cluster Parallelism settings
- Updated hotkeys: 'l' large-db, 'c' conservative, 'p' recommend
- Added resource profiles table for large database operations
- Updated example values to reflect typical PostgreSQL setup
2026-01-18 11:55:30 +01:00
f6f8b04785 fix(tui): improve restore error display formatting
All checks were successful
CI/CD / Test (push) Successful in 1m17s
CI/CD / Lint (push) Successful in 1m27s
CI/CD / Build & Release (push) Successful in 3m16s
- Parse and structure error messages for clean TUI display
- Extract error type, message, hint, and failed databases
- Show specific recommendations based on error type
- Fix for 'out of shared memory' - suggest profile settings
- Limit line width to prevent scrambled display
- Add structured sections: Error Details, Diagnosis, Recommendations
2026-01-18 11:48:07 +01:00
670c9af2e7 feat(tui): add resource profiles for backup/restore operations
All checks were successful
CI/CD / Test (push) Successful in 1m17s
CI/CD / Lint (push) Successful in 1m25s
CI/CD / Build & Release (push) Successful in 3m15s
- Add memory detection for Linux, macOS, Windows
- Add 5 predefined profiles: conservative, balanced, performance, max-performance, large-db
- Add Resource Profile and Cluster Parallelism settings in TUI
- Add quick hotkeys: 'l' for large-db, 'c' for conservative, 'p' for recommendations
- Display system resources (CPU cores, memory) and recommended profile
- Auto-detect and recommend profile based on system resources

Fixes 'out of shared memory' errors when restoring large databases on small VMs.
Use 'large-db' or 'conservative' profile for large databases on constrained systems.
2026-01-18 11:38:24 +01:00
6 changed files with 860 additions and 19 deletions

View File

@@ -194,21 +194,49 @@ r: Restore | v: Verify | i: Info | d: Diagnose | D: Delete | R: Refresh | Esc: B
``` ```
Configuration Settings Configuration Settings
[SYSTEM] Detected Resources
CPU: 8 physical cores, 16 logical cores
Memory: 32GB total, 28GB available
Recommended Profile: balanced
→ 8 cores and 32GB RAM supports moderate parallelism
[CONFIG] Current Settings
Target DB: PostgreSQL (postgres)
Database: postgres@localhost:5432
Backup Dir: /var/backups/postgres
Compression: Level 6
Profile: balanced | Cluster: 2 parallel | Jobs: 4
> Database Type: postgres > Database Type: postgres
CPU Workload Type: balanced CPU Workload Type: balanced
Backup Directory: /root/db_backups Resource Profile: balanced (P:2 J:4)
Work Directory: /tmp Cluster Parallelism: 2
Backup Directory: /var/backups/postgres
Work Directory: (system temp)
Compression Level: 6 Compression Level: 6
Parallel Jobs: 16 Parallel Jobs: 4
Dump Jobs: 8 Dump Jobs: 4
Database Host: localhost Database Host: localhost
Database Port: 5432 Database Port: 5432
Database User: root Database User: postgres
SSL Mode: prefer SSL Mode: prefer
s: Save | r: Reset | q: Menu [KEYS] ↑↓ navigate | Enter edit | 'l' large-db | 'c' conservative | 'p' recommend | 's' save | 'q' menu
``` ```
**Resource Profiles for Large Databases:**
When restoring large databases on VMs with limited resources, use the resource profile settings to prevent "out of shared memory" errors:
| Profile | Cluster Parallel | Jobs | Best For |
|---------|------------------|------|----------|
| conservative | 1 | 1 | Small VMs (<16GB RAM) |
| balanced | 2 | 2-4 | Medium VMs (16-32GB RAM) |
| performance | 4 | 4-8 | Large servers (32GB+ RAM) |
| large-db | 1 | 2 | Large databases on any hardware |
**Quick shortcuts:** Press `l` to apply large-db profile, `c` for conservative, `p` to show recommendation.
**Database Status:** **Database Status:**
``` ```
Database Status & Health Check Database Status & Health Check

View File

@@ -4,8 +4,8 @@ This directory contains pre-compiled binaries for the DB Backup Tool across mult
## Build Information ## Build Information
- **Version**: 3.42.50 - **Version**: 3.42.50
- **Build Time**: 2026-01-17_16:00:43_UTC - **Build Time**: 2026-01-18_10:48:22_UTC
- **Git Commit**: 29e089f - **Git Commit**: f6f8b04
## Recent Updates (v1.1.0) ## Recent Updates (v1.1.0)
- ✅ Fixed TUI progress display with line-by-line output - ✅ Fixed TUI progress display with line-by-line output

View File

@@ -36,9 +36,13 @@ type Config struct {
AutoDetectCores bool AutoDetectCores bool
CPUWorkloadType string // "cpu-intensive", "io-intensive", "balanced" CPUWorkloadType string // "cpu-intensive", "io-intensive", "balanced"
// Resource profile for backup/restore operations
ResourceProfile string // "conservative", "balanced", "performance", "max-performance", "large-db"
// CPU detection // CPU detection
CPUDetector *cpu.Detector CPUDetector *cpu.Detector
CPUInfo *cpu.CPUInfo CPUInfo *cpu.CPUInfo
MemoryInfo *cpu.MemoryInfo // System memory information
// Sample backup options // Sample backup options
SampleStrategy string // "ratio", "percent", "count" SampleStrategy string // "ratio", "percent", "count"
@@ -178,6 +182,13 @@ func New() *Config {
sslMode = "" sslMode = ""
} }
// Detect memory information
memInfo, _ := cpu.DetectMemory()
// Determine recommended resource profile
recommendedProfile := cpu.RecommendProfile(cpuInfo, memInfo, false)
defaultProfile := getEnvString("RESOURCE_PROFILE", recommendedProfile.Name)
cfg := &Config{ cfg := &Config{
// Database defaults // Database defaults
Host: host, Host: host,
@@ -189,18 +200,20 @@ func New() *Config {
SSLMode: sslMode, SSLMode: sslMode,
Insecure: getEnvBool("INSECURE", false), Insecure: getEnvBool("INSECURE", false),
// Backup defaults // Backup defaults - use recommended profile's settings for small VMs
BackupDir: backupDir, BackupDir: backupDir,
CompressionLevel: getEnvInt("COMPRESS_LEVEL", 6), CompressionLevel: getEnvInt("COMPRESS_LEVEL", 6),
Jobs: getEnvInt("JOBS", getDefaultJobs(cpuInfo)), Jobs: getEnvInt("JOBS", recommendedProfile.Jobs),
DumpJobs: getEnvInt("DUMP_JOBS", getDefaultDumpJobs(cpuInfo)), DumpJobs: getEnvInt("DUMP_JOBS", recommendedProfile.DumpJobs),
MaxCores: getEnvInt("MAX_CORES", getDefaultMaxCores(cpuInfo)), MaxCores: getEnvInt("MAX_CORES", getDefaultMaxCores(cpuInfo)),
AutoDetectCores: getEnvBool("AUTO_DETECT_CORES", true), AutoDetectCores: getEnvBool("AUTO_DETECT_CORES", true),
CPUWorkloadType: getEnvString("CPU_WORKLOAD_TYPE", "balanced"), CPUWorkloadType: getEnvString("CPU_WORKLOAD_TYPE", "balanced"),
ResourceProfile: defaultProfile,
// CPU detection // CPU and memory detection
CPUDetector: cpuDetector, CPUDetector: cpuDetector,
CPUInfo: cpuInfo, CPUInfo: cpuInfo,
MemoryInfo: memInfo,
// Sample backup defaults // Sample backup defaults
SampleStrategy: getEnvString("SAMPLE_STRATEGY", "ratio"), SampleStrategy: getEnvString("SAMPLE_STRATEGY", "ratio"),
@@ -220,8 +233,8 @@ func New() *Config {
// Timeouts - default 24 hours (1440 min) to handle very large databases with large objects // Timeouts - default 24 hours (1440 min) to handle very large databases with large objects
ClusterTimeoutMinutes: getEnvInt("CLUSTER_TIMEOUT_MIN", 1440), ClusterTimeoutMinutes: getEnvInt("CLUSTER_TIMEOUT_MIN", 1440),
// Cluster parallelism (default: 2 concurrent operations for faster cluster backup/restore) // Cluster parallelism - use recommended profile's setting for small VMs
ClusterParallelism: getEnvInt("CLUSTER_PARALLELISM", 2), ClusterParallelism: getEnvInt("CLUSTER_PARALLELISM", recommendedProfile.ClusterParallelism),
// Working directory for large operations (default: system temp) // Working directory for large operations (default: system temp)
WorkDir: getEnvString("WORK_DIR", ""), WorkDir: getEnvString("WORK_DIR", ""),
@@ -409,6 +422,45 @@ func (c *Config) OptimizeForCPU() error {
return nil return nil
} }
// ApplyResourceProfile applies a resource profile to the configuration
// This adjusts parallelism settings based on the chosen profile
func (c *Config) ApplyResourceProfile(profileName string) error {
profile := cpu.GetProfileByName(profileName)
if profile == nil {
return &ConfigError{
Field: "resource_profile",
Value: profileName,
Message: "unknown profile. Valid profiles: conservative, balanced, performance, max-performance, large-db",
}
}
// Validate profile against current system
isValid, warnings := cpu.ValidateProfileForSystem(profile, c.CPUInfo, c.MemoryInfo)
if !isValid {
// Log warnings but don't block - user may know what they're doing
_ = warnings // In production, log these warnings
}
// Apply profile settings
c.ResourceProfile = profile.Name
c.ClusterParallelism = profile.ClusterParallelism
c.Jobs = profile.Jobs
c.DumpJobs = profile.DumpJobs
return nil
}
// GetResourceProfileRecommendation returns the recommended profile and reason
func (c *Config) GetResourceProfileRecommendation(isLargeDB bool) (string, string) {
profile, reason := cpu.RecommendProfileWithReason(c.CPUInfo, c.MemoryInfo, isLargeDB)
return profile.Name, reason
}
// GetCurrentProfile returns the current resource profile details
func (c *Config) GetCurrentProfile() *cpu.ResourceProfile {
return cpu.GetProfileByName(c.ResourceProfile)
}
// GetCPUInfo returns CPU information, detecting if necessary // GetCPUInfo returns CPU information, detecting if necessary
func (c *Config) GetCPUInfo() (*cpu.CPUInfo, error) { func (c *Config) GetCPUInfo() (*cpu.CPUInfo, error) {
if c.CPUInfo != nil { if c.CPUInfo != nil {

445
internal/cpu/profiles.go Normal file
View File

@@ -0,0 +1,445 @@
package cpu
import (
"bufio"
"fmt"
"os"
"os/exec"
"runtime"
"strconv"
"strings"
)
// MemoryInfo holds system memory information
type MemoryInfo struct {
TotalBytes int64 `json:"total_bytes"`
AvailableBytes int64 `json:"available_bytes"`
FreeBytes int64 `json:"free_bytes"`
UsedBytes int64 `json:"used_bytes"`
SwapTotalBytes int64 `json:"swap_total_bytes"`
SwapFreeBytes int64 `json:"swap_free_bytes"`
TotalGB int `json:"total_gb"`
AvailableGB int `json:"available_gb"`
Platform string `json:"platform"`
}
// ResourceProfile defines a resource allocation profile for backup/restore operations
type ResourceProfile struct {
Name string `json:"name"`
Description string `json:"description"`
ClusterParallelism int `json:"cluster_parallelism"` // Concurrent databases
Jobs int `json:"jobs"` // Parallel jobs within pg_restore
DumpJobs int `json:"dump_jobs"` // Parallel jobs for pg_dump
MaintenanceWorkMem string `json:"maintenance_work_mem"` // PostgreSQL recommendation
MaxLocksPerTxn int `json:"max_locks_per_txn"` // PostgreSQL recommendation
RecommendedForLarge bool `json:"recommended_for_large"` // Suitable for large DBs?
MinMemoryGB int `json:"min_memory_gb"` // Minimum memory for this profile
MinCores int `json:"min_cores"` // Minimum cores for this profile
}
// Predefined resource profiles
var (
// ProfileConservative - Safe for constrained VMs, avoids shared memory issues
ProfileConservative = ResourceProfile{
Name: "conservative",
Description: "Safe for small VMs (2-4 cores, <16GB). Sequential operations, minimal memory pressure. Best for large DBs on limited hardware.",
ClusterParallelism: 1,
Jobs: 1,
DumpJobs: 2,
MaintenanceWorkMem: "256MB",
MaxLocksPerTxn: 4096,
RecommendedForLarge: true,
MinMemoryGB: 4,
MinCores: 2,
}
// ProfileBalanced - Default profile, works for most scenarios
ProfileBalanced = ResourceProfile{
Name: "balanced",
Description: "Balanced for medium VMs (4-8 cores, 16-32GB). Moderate parallelism with good safety margin.",
ClusterParallelism: 2,
Jobs: 2,
DumpJobs: 4,
MaintenanceWorkMem: "512MB",
MaxLocksPerTxn: 2048,
RecommendedForLarge: true,
MinMemoryGB: 16,
MinCores: 4,
}
// ProfilePerformance - Aggressive parallelism for powerful servers
ProfilePerformance = ResourceProfile{
Name: "performance",
Description: "Aggressive for powerful servers (8+ cores, 32GB+). Maximum parallelism for fast operations.",
ClusterParallelism: 4,
Jobs: 4,
DumpJobs: 8,
MaintenanceWorkMem: "1GB",
MaxLocksPerTxn: 1024,
RecommendedForLarge: false, // Large DBs may still need conservative
MinMemoryGB: 32,
MinCores: 8,
}
// ProfileMaxPerformance - Maximum parallelism for high-end servers
ProfileMaxPerformance = ResourceProfile{
Name: "max-performance",
Description: "Maximum for high-end servers (16+ cores, 64GB+). Full CPU utilization.",
ClusterParallelism: 8,
Jobs: 8,
DumpJobs: 16,
MaintenanceWorkMem: "2GB",
MaxLocksPerTxn: 512,
RecommendedForLarge: false, // Large DBs should use conservative
MinMemoryGB: 64,
MinCores: 16,
}
// ProfileLargeDB - Optimized specifically for large databases
ProfileLargeDB = ResourceProfile{
Name: "large-db",
Description: "Optimized for large databases with many tables/BLOBs. Prevents 'out of shared memory' errors.",
ClusterParallelism: 1,
Jobs: 2,
DumpJobs: 2,
MaintenanceWorkMem: "1GB",
MaxLocksPerTxn: 8192,
RecommendedForLarge: true,
MinMemoryGB: 8,
MinCores: 2,
}
// AllProfiles contains all available profiles
AllProfiles = []ResourceProfile{
ProfileConservative,
ProfileBalanced,
ProfilePerformance,
ProfileMaxPerformance,
ProfileLargeDB,
}
)
// GetProfileByName returns a profile by its name
func GetProfileByName(name string) *ResourceProfile {
for _, p := range AllProfiles {
if strings.EqualFold(p.Name, name) {
return &p
}
}
return nil
}
// DetectMemory detects system memory information
func DetectMemory() (*MemoryInfo, error) {
info := &MemoryInfo{
Platform: runtime.GOOS,
}
switch runtime.GOOS {
case "linux":
if err := detectLinuxMemory(info); err != nil {
return info, fmt.Errorf("linux memory detection failed: %w", err)
}
case "darwin":
if err := detectDarwinMemory(info); err != nil {
return info, fmt.Errorf("darwin memory detection failed: %w", err)
}
case "windows":
if err := detectWindowsMemory(info); err != nil {
return info, fmt.Errorf("windows memory detection failed: %w", err)
}
default:
// Fallback: use Go runtime memory stats
var memStats runtime.MemStats
runtime.ReadMemStats(&memStats)
info.TotalBytes = int64(memStats.Sys)
info.AvailableBytes = int64(memStats.Sys - memStats.Alloc)
}
// Calculate GB values
info.TotalGB = int(info.TotalBytes / (1024 * 1024 * 1024))
info.AvailableGB = int(info.AvailableBytes / (1024 * 1024 * 1024))
return info, nil
}
// detectLinuxMemory reads memory info from /proc/meminfo
func detectLinuxMemory(info *MemoryInfo) error {
file, err := os.Open("/proc/meminfo")
if err != nil {
return err
}
defer file.Close()
scanner := bufio.NewScanner(file)
for scanner.Scan() {
line := scanner.Text()
parts := strings.Fields(line)
if len(parts) < 2 {
continue
}
key := strings.TrimSuffix(parts[0], ":")
value, err := strconv.ParseInt(parts[1], 10, 64)
if err != nil {
continue
}
// Values are in kB
valueBytes := value * 1024
switch key {
case "MemTotal":
info.TotalBytes = valueBytes
case "MemAvailable":
info.AvailableBytes = valueBytes
case "MemFree":
info.FreeBytes = valueBytes
case "SwapTotal":
info.SwapTotalBytes = valueBytes
case "SwapFree":
info.SwapFreeBytes = valueBytes
}
}
info.UsedBytes = info.TotalBytes - info.AvailableBytes
return scanner.Err()
}
// detectDarwinMemory detects memory on macOS
func detectDarwinMemory(info *MemoryInfo) error {
// Use sysctl for total memory
if output, err := runCommand("sysctl", "-n", "hw.memsize"); err == nil {
if val, err := strconv.ParseInt(strings.TrimSpace(output), 10, 64); err == nil {
info.TotalBytes = val
}
}
// Use vm_stat for available memory (more complex parsing required)
if output, err := runCommand("vm_stat"); err == nil {
pageSize := int64(4096) // Default page size
var freePages, inactivePages int64
lines := strings.Split(output, "\n")
for _, line := range lines {
if strings.Contains(line, "page size of") {
parts := strings.Fields(line)
for i, p := range parts {
if p == "of" && i+1 < len(parts) {
if ps, err := strconv.ParseInt(parts[i+1], 10, 64); err == nil {
pageSize = ps
}
}
}
} else if strings.Contains(line, "Pages free:") {
val := extractNumberFromLine(line)
freePages = val
} else if strings.Contains(line, "Pages inactive:") {
val := extractNumberFromLine(line)
inactivePages = val
}
}
info.FreeBytes = freePages * pageSize
info.AvailableBytes = (freePages + inactivePages) * pageSize
}
info.UsedBytes = info.TotalBytes - info.AvailableBytes
return nil
}
// detectWindowsMemory detects memory on Windows
func detectWindowsMemory(info *MemoryInfo) error {
// Use wmic for memory info
if output, err := runCommand("wmic", "OS", "get", "TotalVisibleMemorySize,FreePhysicalMemory", "/format:list"); err == nil {
lines := strings.Split(output, "\n")
for _, line := range lines {
line = strings.TrimSpace(line)
if strings.HasPrefix(line, "TotalVisibleMemorySize=") {
val := strings.TrimPrefix(line, "TotalVisibleMemorySize=")
if v, err := strconv.ParseInt(strings.TrimSpace(val), 10, 64); err == nil {
info.TotalBytes = v * 1024 // KB to bytes
}
} else if strings.HasPrefix(line, "FreePhysicalMemory=") {
val := strings.TrimPrefix(line, "FreePhysicalMemory=")
if v, err := strconv.ParseInt(strings.TrimSpace(val), 10, 64); err == nil {
info.FreeBytes = v * 1024
info.AvailableBytes = v * 1024
}
}
}
}
info.UsedBytes = info.TotalBytes - info.AvailableBytes
return nil
}
// RecommendProfile recommends a resource profile based on system resources and workload
func RecommendProfile(cpuInfo *CPUInfo, memInfo *MemoryInfo, isLargeDB bool) *ResourceProfile {
cores := 0
if cpuInfo != nil {
cores = cpuInfo.PhysicalCores
if cores == 0 {
cores = cpuInfo.LogicalCores
}
}
if cores == 0 {
cores = runtime.NumCPU()
}
memGB := 0
if memInfo != nil {
memGB = memInfo.TotalGB
}
// Special case: large databases should always use conservative/large-db profile
if isLargeDB {
if memGB >= 32 && cores >= 8 {
return &ProfileLargeDB // Still conservative but with more memory for maintenance
}
return &ProfileConservative
}
// Resource-based selection
if cores >= 16 && memGB >= 64 {
return &ProfileMaxPerformance
} else if cores >= 8 && memGB >= 32 {
return &ProfilePerformance
} else if cores >= 4 && memGB >= 16 {
return &ProfileBalanced
}
// Default to conservative for constrained systems
return &ProfileConservative
}
// RecommendProfileWithReason returns a profile recommendation with explanation
func RecommendProfileWithReason(cpuInfo *CPUInfo, memInfo *MemoryInfo, isLargeDB bool) (*ResourceProfile, string) {
cores := 0
if cpuInfo != nil {
cores = cpuInfo.PhysicalCores
if cores == 0 {
cores = cpuInfo.LogicalCores
}
}
if cores == 0 {
cores = runtime.NumCPU()
}
memGB := 0
if memInfo != nil {
memGB = memInfo.TotalGB
}
// Build reason string
var reason strings.Builder
reason.WriteString(fmt.Sprintf("System: %d cores, %dGB RAM. ", cores, memGB))
profile := RecommendProfile(cpuInfo, memInfo, isLargeDB)
if isLargeDB {
reason.WriteString("Large database detected - using conservative settings to avoid 'out of shared memory' errors.")
} else if profile.Name == "conservative" {
reason.WriteString("Limited resources detected - using conservative profile for stability.")
} else if profile.Name == "max-performance" {
reason.WriteString("High-end server detected - using maximum parallelism.")
} else if profile.Name == "performance" {
reason.WriteString("Good resources detected - using performance profile.")
} else {
reason.WriteString("Using balanced profile for optimal performance/stability trade-off.")
}
return profile, reason.String()
}
// ValidateProfileForSystem checks if a profile is suitable for the current system
func ValidateProfileForSystem(profile *ResourceProfile, cpuInfo *CPUInfo, memInfo *MemoryInfo) (bool, []string) {
var warnings []string
cores := 0
if cpuInfo != nil {
cores = cpuInfo.PhysicalCores
if cores == 0 {
cores = cpuInfo.LogicalCores
}
}
if cores == 0 {
cores = runtime.NumCPU()
}
memGB := 0
if memInfo != nil {
memGB = memInfo.TotalGB
}
// Check minimum requirements
if cores < profile.MinCores {
warnings = append(warnings,
fmt.Sprintf("Profile '%s' recommends %d+ cores (system has %d)", profile.Name, profile.MinCores, cores))
}
if memGB < profile.MinMemoryGB {
warnings = append(warnings,
fmt.Sprintf("Profile '%s' recommends %dGB+ RAM (system has %dGB)", profile.Name, profile.MinMemoryGB, memGB))
}
// Check for potential issues
if profile.ClusterParallelism > cores {
warnings = append(warnings,
fmt.Sprintf("Cluster parallelism (%d) exceeds CPU cores (%d) - may cause contention",
profile.ClusterParallelism, cores))
}
// Memory pressure warning
memPerWorker := 2 // Rough estimate: 2GB per parallel worker for large DB operations
requiredMem := profile.ClusterParallelism * profile.Jobs * memPerWorker
if memGB > 0 && requiredMem > memGB {
warnings = append(warnings,
fmt.Sprintf("High parallelism may require ~%dGB RAM (system has %dGB) - risk of OOM",
requiredMem, memGB))
}
return len(warnings) == 0, warnings
}
// FormatProfileSummary returns a formatted summary of a profile
func (p *ResourceProfile) FormatProfileSummary() string {
return fmt.Sprintf("[%s] Parallel: %d DBs, %d jobs | Recommended for large DBs: %v",
strings.ToUpper(p.Name),
p.ClusterParallelism,
p.Jobs,
p.RecommendedForLarge)
}
// PostgreSQLRecommendations returns PostgreSQL configuration recommendations for this profile
func (p *ResourceProfile) PostgreSQLRecommendations() []string {
return []string{
fmt.Sprintf("ALTER SYSTEM SET max_locks_per_transaction = %d;", p.MaxLocksPerTxn),
fmt.Sprintf("ALTER SYSTEM SET maintenance_work_mem = '%s';", p.MaintenanceWorkMem),
"-- Restart PostgreSQL after changes to max_locks_per_transaction",
}
}
// Helper functions
func runCommand(name string, args ...string) (string, error) {
cmd := exec.Command(name, args...)
output, err := cmd.Output()
if err != nil {
return "", err
}
return string(output), nil
}
func extractNumberFromLine(line string) int64 {
// Extract number before the period at end (e.g., "Pages free: 123456.")
parts := strings.Fields(line)
for _, p := range parts {
p = strings.TrimSuffix(p, ".")
if val, err := strconv.ParseInt(p, 10, 64); err == nil && val > 0 {
return val
}
}
return 0
}

View File

@@ -659,7 +659,13 @@ func (m RestoreExecutionModel) View() string {
s.WriteString("\n") s.WriteString("\n")
s.WriteString(errorStyle.Render("╚══════════════════════════════════════════════════════════════╝")) s.WriteString(errorStyle.Render("╚══════════════════════════════════════════════════════════════╝"))
s.WriteString("\n\n") s.WriteString("\n\n")
s.WriteString(errorStyle.Render(fmt.Sprintf(" Error: %v", m.err)))
// Parse and display error in a clean, structured format
errStr := m.err.Error()
// Extract key parts from the error message
errDisplay := formatRestoreError(errStr)
s.WriteString(errDisplay)
s.WriteString("\n") s.WriteString("\n")
} else { } else {
s.WriteString(successStyle.Render("╔══════════════════════════════════════════════════════════════╗")) s.WriteString(successStyle.Render("╔══════════════════════════════════════════════════════════════╗"))
@@ -1005,3 +1011,188 @@ func dropDatabaseCLI(ctx context.Context, cfg *config.Config, dbName string) err
return nil return nil
} }
// formatRestoreError formats a restore error message for clean TUI display
func formatRestoreError(errStr string) string {
var s strings.Builder
maxLineWidth := 60
// Common patterns to extract
patterns := []struct {
key string
pattern string
}{
{"Error Type", "ERROR:"},
{"Hint", "HINT:"},
{"Last Error", "last error:"},
{"Total Errors", "total errors:"},
}
// First, try to extract a clean error summary
errLines := strings.Split(errStr, "\n")
// Find the main error message (first line or first ERROR:)
mainError := ""
hint := ""
totalErrors := ""
dbsFailed := []string{}
for _, line := range errLines {
line = strings.TrimSpace(line)
if line == "" {
continue
}
// Extract ERROR messages
if strings.Contains(line, "ERROR:") {
if mainError == "" {
// Get just the ERROR part
idx := strings.Index(line, "ERROR:")
if idx >= 0 {
mainError = strings.TrimSpace(line[idx:])
// Truncate if too long
if len(mainError) > maxLineWidth {
mainError = mainError[:maxLineWidth-3] + "..."
}
}
}
}
// Extract HINT
if strings.Contains(line, "HINT:") {
idx := strings.Index(line, "HINT:")
if idx >= 0 {
hint = strings.TrimSpace(line[idx+5:])
if len(hint) > maxLineWidth {
hint = hint[:maxLineWidth-3] + "..."
}
}
}
// Extract total errors count
if strings.Contains(line, "total errors:") {
idx := strings.Index(line, "total errors:")
if idx >= 0 {
totalErrors = strings.TrimSpace(line[idx+13:])
// Just extract the number
parts := strings.Fields(totalErrors)
if len(parts) > 0 {
totalErrors = parts[0]
}
}
}
// Extract failed database names (for cluster restore)
if strings.Contains(line, ": restore failed:") {
parts := strings.SplitN(line, ":", 2)
if len(parts) > 0 {
dbName := strings.TrimSpace(parts[0])
if dbName != "" && !strings.HasPrefix(dbName, "Error") {
dbsFailed = append(dbsFailed, dbName)
}
}
}
}
// If no structured error found, use the first line
if mainError == "" {
firstLine := errStr
if idx := strings.Index(errStr, "\n"); idx > 0 {
firstLine = errStr[:idx]
}
if len(firstLine) > maxLineWidth*2 {
firstLine = firstLine[:maxLineWidth*2-3] + "..."
}
mainError = firstLine
}
// Build structured error display
s.WriteString(infoStyle.Render(" ─── Error Details ─────────────────────────────────────────"))
s.WriteString("\n\n")
// Error type detection
errorType := "critical"
if strings.Contains(errStr, "out of shared memory") || strings.Contains(errStr, "max_locks_per_transaction") {
errorType = "critical"
} else if strings.Contains(errStr, "connection") {
errorType = "connection"
} else if strings.Contains(errStr, "permission") || strings.Contains(errStr, "access") {
errorType = "permission"
}
s.WriteString(fmt.Sprintf(" Type: %s\n", errorType))
s.WriteString(fmt.Sprintf(" Message: %s\n", mainError))
if hint != "" {
s.WriteString(fmt.Sprintf(" Hint: %s\n", hint))
}
if totalErrors != "" {
s.WriteString(fmt.Sprintf(" Total Errors: %s\n", totalErrors))
}
// Show failed databases (max 5)
if len(dbsFailed) > 0 {
s.WriteString("\n")
s.WriteString(" Failed Databases:\n")
for i, db := range dbsFailed {
if i >= 5 {
s.WriteString(fmt.Sprintf(" ... and %d more\n", len(dbsFailed)-5))
break
}
s.WriteString(fmt.Sprintf(" • %s\n", db))
}
}
s.WriteString("\n")
s.WriteString(infoStyle.Render(" ─── Diagnosis ─────────────────────────────────────────────"))
s.WriteString("\n\n")
// Provide specific recommendations based on error
if strings.Contains(errStr, "out of shared memory") || strings.Contains(errStr, "max_locks_per_transaction") {
s.WriteString(errorStyle.Render(" • Cannot access file: stat : no such file or directory\n"))
s.WriteString("\n")
s.WriteString(infoStyle.Render(" ─── [HINT] Recommendations ────────────────────────────────"))
s.WriteString("\n\n")
s.WriteString(" Lock table exhausted. Total capacity = max_locks_per_transaction\n")
s.WriteString(" × (max_connections + max_prepared_transactions).\n\n")
s.WriteString(" If you reduced VM size or max_connections, you need higher\n")
s.WriteString(" max_locks_per_transaction to compensate.\n\n")
s.WriteString(successStyle.Render(" FIX OPTIONS:\n"))
s.WriteString(" 1. Use 'conservative' or 'large-db' profile in Settings\n")
s.WriteString(" (press 'l' for large-db, 'c' for conservative)\n\n")
s.WriteString(" 2. Increase PostgreSQL locks:\n")
s.WriteString(" ALTER SYSTEM SET max_locks_per_transaction = 4096;\n")
s.WriteString(" Then RESTART PostgreSQL.\n\n")
s.WriteString(" 3. Reduce parallel jobs:\n")
s.WriteString(" Set Cluster Parallelism = 1 in Settings\n")
} else if strings.Contains(errStr, "connection") || strings.Contains(errStr, "refused") {
s.WriteString(" • Database connection failed\n\n")
s.WriteString(infoStyle.Render(" ─── [HINT] Recommendations ────────────────────────────────"))
s.WriteString("\n\n")
s.WriteString(" 1. Check database is running\n")
s.WriteString(" 2. Verify host, port, and credentials in Settings\n")
s.WriteString(" 3. Check firewall/network connectivity\n")
} else if strings.Contains(errStr, "permission") || strings.Contains(errStr, "denied") {
s.WriteString(" • Permission denied\n\n")
s.WriteString(infoStyle.Render(" ─── [HINT] Recommendations ────────────────────────────────"))
s.WriteString("\n\n")
s.WriteString(" 1. Verify database user has sufficient privileges\n")
s.WriteString(" 2. Grant CREATE/DROP DATABASE permissions if restoring cluster\n")
s.WriteString(" 3. Check file system permissions on backup directory\n")
} else {
s.WriteString(" See error message above for details.\n\n")
s.WriteString(infoStyle.Render(" ─── [HINT] General Recommendations ────────────────────────"))
s.WriteString("\n\n")
s.WriteString(" 1. Check the full error log for details\n")
s.WriteString(" 2. Try restoring with 'conservative' profile (press 'c')\n")
s.WriteString(" 3. For large databases, use 'large-db' profile (press 'l')\n")
}
s.WriteString("\n")
// Suppress the pattern variable since we don't use it but defined it
_ = patterns
return s.String()
}

View File

@@ -10,6 +10,7 @@ import (
"github.com/charmbracelet/lipgloss" "github.com/charmbracelet/lipgloss"
"dbbackup/internal/config" "dbbackup/internal/config"
"dbbackup/internal/cpu"
"dbbackup/internal/logger" "dbbackup/internal/logger"
) )
@@ -101,6 +102,49 @@ func NewSettingsModel(cfg *config.Config, log logger.Logger, parent tea.Model) S
Type: "selector", Type: "selector",
Description: "CPU workload profile (press Enter to cycle: Balanced → CPU-Intensive → I/O-Intensive)", Description: "CPU workload profile (press Enter to cycle: Balanced → CPU-Intensive → I/O-Intensive)",
}, },
{
Key: "resource_profile",
DisplayName: "Resource Profile",
Value: func(c *config.Config) string {
profile := c.GetCurrentProfile()
if profile != nil {
return fmt.Sprintf("%s (P:%d J:%d)", profile.Name, profile.ClusterParallelism, profile.Jobs)
}
return c.ResourceProfile
},
Update: func(c *config.Config, v string) error {
profiles := []string{"conservative", "balanced", "performance", "max-performance", "large-db"}
currentIdx := 0
for i, p := range profiles {
if c.ResourceProfile == p {
currentIdx = i
break
}
}
nextIdx := (currentIdx + 1) % len(profiles)
return c.ApplyResourceProfile(profiles[nextIdx])
},
Type: "selector",
Description: "Resource profile for backup/restore. Use 'conservative' or 'large-db' for large databases on small VMs.",
},
{
Key: "cluster_parallelism",
DisplayName: "Cluster Parallelism",
Value: func(c *config.Config) string { return fmt.Sprintf("%d", c.ClusterParallelism) },
Update: func(c *config.Config, v string) error {
val, err := strconv.Atoi(v)
if err != nil {
return fmt.Errorf("cluster parallelism must be a number")
}
if val < 1 {
return fmt.Errorf("cluster parallelism must be at least 1")
}
c.ClusterParallelism = val
return nil
},
Type: "int",
Description: "Concurrent databases during cluster backup/restore (1=sequential, safer for large DBs)",
},
{ {
Key: "backup_dir", Key: "backup_dir",
DisplayName: "Backup Directory", DisplayName: "Backup Directory",
@@ -528,12 +572,58 @@ func (m SettingsModel) Update(msg tea.Msg) (tea.Model, tea.Cmd) {
case "s": case "s":
return m.saveSettings() return m.saveSettings()
case "l":
// Quick shortcut: Apply "large-db" profile for large databases
return m.applyLargeDBProfile()
case "c":
// Quick shortcut: Apply "conservative" profile for constrained VMs
return m.applyConservativeProfile()
case "p":
// Show profile recommendation
return m.showProfileRecommendation()
} }
} }
return m, nil return m, nil
} }
// applyLargeDBProfile applies the large-db profile optimized for large databases
func (m SettingsModel) applyLargeDBProfile() (tea.Model, tea.Cmd) {
if err := m.config.ApplyResourceProfile("large-db"); err != nil {
m.message = errorStyle.Render(fmt.Sprintf("[FAIL] %s", err.Error()))
return m, nil
}
m.message = successStyle.Render("[OK] Applied 'large-db' profile: Cluster=1, Jobs=2. Optimized for large DBs to avoid 'out of shared memory' errors.")
return m, nil
}
// applyConservativeProfile applies the conservative profile for constrained VMs
func (m SettingsModel) applyConservativeProfile() (tea.Model, tea.Cmd) {
if err := m.config.ApplyResourceProfile("conservative"); err != nil {
m.message = errorStyle.Render(fmt.Sprintf("[FAIL] %s", err.Error()))
return m, nil
}
m.message = successStyle.Render("[OK] Applied 'conservative' profile: Cluster=1, Jobs=1. Safe for small VMs with limited memory.")
return m, nil
}
// showProfileRecommendation displays the recommended profile based on system resources
func (m SettingsModel) showProfileRecommendation() (tea.Model, tea.Cmd) {
profileName, reason := m.config.GetResourceProfileRecommendation(false)
largeDBProfile, largeDBReason := m.config.GetResourceProfileRecommendation(true)
m.message = infoStyle.Render(fmt.Sprintf(
"[RECOMMEND] Default: %s | For Large DBs: %s\n"+
" → %s\n"+
" → Large DB: %s\n"+
" Press 'l' for large-db profile, 'c' for conservative",
profileName, largeDBProfile, reason, largeDBReason))
return m, nil
}
// handleEditingInput handles input when editing a setting // handleEditingInput handles input when editing a setting
func (m SettingsModel) handleEditingInput(msg tea.KeyMsg) (tea.Model, tea.Cmd) { func (m SettingsModel) handleEditingInput(msg tea.KeyMsg) (tea.Model, tea.Cmd) {
switch msg.String() { switch msg.String() {
@@ -747,7 +837,32 @@ func (m SettingsModel) View() string {
// Current configuration summary // Current configuration summary
if !m.editing { if !m.editing {
b.WriteString("\n") b.WriteString("\n")
b.WriteString(infoStyle.Render("[INFO] Current Configuration")) b.WriteString(infoStyle.Render("[INFO] System Resources & Configuration"))
b.WriteString("\n")
// System resources
var sysInfo []string
if m.config.CPUInfo != nil {
sysInfo = append(sysInfo, fmt.Sprintf("CPU: %d cores (physical), %d logical",
m.config.CPUInfo.PhysicalCores, m.config.CPUInfo.LogicalCores))
}
if m.config.MemoryInfo != nil {
sysInfo = append(sysInfo, fmt.Sprintf("Memory: %dGB total, %dGB available",
m.config.MemoryInfo.TotalGB, m.config.MemoryInfo.AvailableGB))
}
// Recommended profile
recommendedProfile, reason := m.config.GetResourceProfileRecommendation(false)
sysInfo = append(sysInfo, fmt.Sprintf("Recommended Profile: %s", recommendedProfile))
sysInfo = append(sysInfo, fmt.Sprintf(" → %s", reason))
for _, line := range sysInfo {
b.WriteString(detailStyle.Render(fmt.Sprintf(" %s", line)))
b.WriteString("\n")
}
b.WriteString("\n")
b.WriteString(infoStyle.Render("[CONFIG] Current Settings"))
b.WriteString("\n") b.WriteString("\n")
summary := []string{ summary := []string{
@@ -755,7 +870,17 @@ func (m SettingsModel) View() string {
fmt.Sprintf("Database: %s@%s:%d", m.config.User, m.config.Host, m.config.Port), fmt.Sprintf("Database: %s@%s:%d", m.config.User, m.config.Host, m.config.Port),
fmt.Sprintf("Backup Dir: %s", m.config.BackupDir), fmt.Sprintf("Backup Dir: %s", m.config.BackupDir),
fmt.Sprintf("Compression: Level %d", m.config.CompressionLevel), fmt.Sprintf("Compression: Level %d", m.config.CompressionLevel),
fmt.Sprintf("Jobs: %d parallel, %d dump", m.config.Jobs, m.config.DumpJobs), fmt.Sprintf("Profile: %s | Cluster: %d parallel | Jobs: %d",
m.config.ResourceProfile, m.config.ClusterParallelism, m.config.Jobs),
}
// Show profile warnings if applicable
profile := m.config.GetCurrentProfile()
if profile != nil {
isValid, warnings := cpu.ValidateProfileForSystem(profile, m.config.CPUInfo, m.config.MemoryInfo)
if !isValid && len(warnings) > 0 {
summary = append(summary, fmt.Sprintf("⚠️ Warning: %s", warnings[0]))
}
} }
if m.config.CloudEnabled { if m.config.CloudEnabled {
@@ -782,9 +907,9 @@ func (m SettingsModel) View() string {
} else { } else {
// Show different help based on current selection // Show different help based on current selection
if m.cursor >= 0 && m.cursor < len(m.settings) && m.settings[m.cursor].Type == "path" { if m.cursor >= 0 && m.cursor < len(m.settings) && m.settings[m.cursor].Type == "path" {
footer = infoStyle.Render("\n[KEYS] Up/Down navigate | Enter edit | Tab browse directories | 's' save | 'r' reset | 'q' menu") footer = infoStyle.Render("\n[KEYS] ↑↓ navigate | Enter edit | Tab dirs | 'l' large-db | 'c' conservative | 'p' recommend | 's' save | 'q' menu")
} else { } else {
footer = infoStyle.Render("\n[KEYS] Up/Down navigate | Enter edit | 's' save | 'r' reset | 'q' menu | Tab=dirs on path fields only") footer = infoStyle.Render("\n[KEYS] ↑↓ navigate | Enter edit | 'l' large-db profile | 'c' conservative | 'p' recommend | 's' save | 'r' reset | 'q' menu")
} }
} }
} }