docs: Update README and add CHANGELOG for v2.1.0

README.md updates: - Added Cloud Storage Integration section with quick start - Added cloud flags to Global Flags table - Added all 5 cloud providers (S3, MinIO, B2, Azure, GCS) - Updated Key Features to highlight cloud storage - Added Windows to cross-platform list CHANGELOG.md: - Complete v2.1.0 changelog with cloud storage features - Cross-platform support details (10/10 platforms) - TUI cloud integration documentation - Fixed issues from BSD/Windows build problems - v2.0.0 and earlier versions documented
2025-11-26 05:44:48 +00:00
parent b8d39cbbb0
commit 68df28f282
2 changed files with 230 additions and 1 deletions
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -0,0 +1,150 @@
+# Changelog
+
+All notable changes to dbbackup will be documented in this file.
+
+The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
+and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
+
+## [2.1.0] - 2025-11-26
+
+### Added - Cloud Storage Integration
+- **S3/MinIO/B2 Support**: Native S3-compatible storage backend with streaming uploads
+- **Azure Blob Storage**: Native Azure integration with block blob support for files >256MB
+- **Google Cloud Storage**: Native GCS integration with 16MB chunked uploads
+- **Cloud URI Syntax**: Direct backup/restore using `--cloud s3://bucket/path` URIs
+- **TUI Cloud Settings**: Configure cloud providers directly in interactive menu
+  - Cloud Storage Enabled toggle
+  - Provider selector (S3, MinIO, B2, Azure, GCS)
+  - Bucket/Container configuration
+  - Region configuration
+  - Credential management with masking
+  - Auto-upload toggle
+- **Multipart Uploads**: Automatic multipart uploads for files >100MB (S3/MinIO/B2)
+- **Streaming Transfers**: Memory-efficient streaming for all cloud operations
+- **Progress Tracking**: Real-time upload/download progress with ETA
+- **Metadata Sync**: Automatic .sha256 and .info file upload alongside backups
+- **Cloud Verification**: Verify backup integrity directly from cloud storage
+- **Cloud Cleanup**: Apply retention policies to cloud-stored backups
+
+### Added - Cross-Platform Support
+- **Windows Support**: Native binaries for Windows Intel (amd64) and ARM (arm64)
+- **NetBSD Support**: Full support for NetBSD amd64 (disk checks use safe defaults)
+- **Platform-Specific Implementations**: 
+  - `resources_unix.go` - Linux, macOS, FreeBSD, OpenBSD
+  - `resources_windows.go` - Windows stub implementation
+  - `disk_check_netbsd.go` - NetBSD disk space stub
+- **Build Tags**: Proper Go build constraints for platform-specific code
+- **All Platforms Building**: 10/10 platforms successfully compile
+  - ✅ Linux (amd64, arm64, armv7)
+  - ✅ macOS (Intel, Apple Silicon)
+  - ✅ Windows (Intel, ARM)
+  - ✅ FreeBSD amd64
+  - ✅ OpenBSD amd64
+  - ✅ NetBSD amd64
+
+### Changed
+- **Cloud Auto-Upload**: When `CloudEnabled=true` and `CloudAutoUpload=true`, backups automatically upload after creation
+- **Configuration**: Added cloud settings to TUI settings interface
+- **Backup Engine**: Integrated cloud upload into backup workflow with progress tracking
+
+### Fixed
+- **BSD Syscall Issues**: Fixed `syscall.Rlimit` type mismatches (int64 vs uint64) on BSD platforms
+- **OpenBSD RLIMIT_AS**: Made RLIMIT_AS check Linux-only (not available on OpenBSD)
+- **NetBSD Disk Checks**: Added safe default implementation for NetBSD (syscall.Statfs unavailable)
+- **Cross-Platform Builds**: Resolved Windows syscall.Rlimit undefined errors
+
+### Documentation
+- Updated README.md with Cloud Storage section and examples
+- Enhanced CLOUD.md with setup guides for all providers
+- Added testing scripts for Azure and GCS
+- Docker Compose files for Azurite and fake-gcs-server
+
+### Testing
+- Added `scripts/test_azure_storage.sh` - Azure Blob Storage integration tests
+- Added `scripts/test_gcs_storage.sh` - Google Cloud Storage integration tests
+- Docker Compose setups for local testing (Azurite, fake-gcs-server, MinIO)
+
+## [2.0.0] - 2025-11-25
+
+### Added - Production-Ready Release
+- **100% Test Coverage**: All 24 automated tests passing
+- **Zero Critical Issues**: Production-validated and deployment-ready
+- **Backup Verification**: SHA-256 checksum generation and validation
+- **JSON Metadata**: Structured .info files with backup metadata
+- **Retention Policy**: Automatic cleanup of old backups with configurable retention
+- **Configuration Management**:
+  - Auto-save/load settings to `.dbbackup.conf` in current directory
+  - Per-directory configuration for different projects
+  - CLI flags always take precedence over saved configuration
+  - Passwords excluded from saved configuration files
+
+### Added - Performance Optimizations
+- **Parallel Cluster Operations**: Worker pool pattern for concurrent database operations
+- **Memory Efficiency**: Streaming command output eliminates OOM errors
+- **Optimized Goroutines**: Ticker-based progress indicators reduce CPU overhead
+- **Configurable Concurrency**: `CLUSTER_PARALLELISM` environment variable
+
+### Added - Reliability Enhancements
+- **Context Cleanup**: Proper resource cleanup with `sync.Once` and `io.Closer` interface
+- **Process Management**: Thread-safe process tracking with automatic cleanup on exit
+- **Error Classification**: Regex-based error pattern matching for robust error handling
+- **Performance Caching**: Disk space checks cached with 30-second TTL
+- **Metrics Collection**: Structured logging with operation metrics
+
+### Fixed
+- **Configuration Bug**: CLI flags now correctly override config file values
+- **Memory Leaks**: Proper cleanup prevents resource leaks in long-running operations
+
+### Changed
+- **Streaming Architecture**: Constant ~1GB memory footprint regardless of database size
+- **Cross-Platform**: Native binaries for Linux (x64/ARM), macOS (x64/ARM), FreeBSD, OpenBSD
+
+## [1.2.0] - 2025-11-12
+
+### Added
+- **Interactive TUI**: Full terminal user interface with progress tracking
+- **Database Selector**: Interactive database selection for backup operations
+- **Archive Browser**: Browse and restore from backup archives
+- **Configuration Settings**: In-TUI configuration management
+- **CPU Detection**: Automatic CPU detection and optimization
+
+### Changed
+- Improved error handling and user feedback
+- Enhanced progress tracking with real-time updates
+
+## [1.1.0] - 2025-11-10
+
+### Added
+- **Multi-Database Support**: PostgreSQL, MySQL, MariaDB
+- **Cluster Operations**: Full cluster backup and restore for PostgreSQL
+- **Sample Backups**: Create reduced-size backups for testing
+- **Parallel Processing**: Automatic CPU detection and parallel jobs
+
+### Changed
+- Refactored command structure for better organization
+- Improved compression handling
+
+## [1.0.0] - 2025-11-08
+
+### Added
+- Initial release
+- Single database backup and restore
+- PostgreSQL support
+- Basic CLI interface
+- Streaming compression
+
+---
+
+## Version Numbering
+
+- **Major (X.0.0)**: Breaking changes, major feature additions
+- **Minor (0.X.0)**: New features, non-breaking changes
+- **Patch (0.0.X)**: Bug fixes, minor improvements
+
+## Upcoming Features
+
+See [ROADMAP.md](ROADMAP.md) for planned features:
+- Phase 3: Incremental Backups
+- Phase 4: Encryption (AES-256)
+- Phase 5: PITR (Point-in-Time Recovery)
+- Phase 6: Enterprise Features (Prometheus metrics, remote restore)
--- a/README.md
+++ b/README.md
@@ -8,11 +8,12 @@ Professional database backup and restore utility for PostgreSQL, MySQL, and Mari

 - Multi-database support: PostgreSQL, MySQL, MariaDB
 - Backup modes: Single database, cluster, sample data
+- **Cloud storage integration: S3, MinIO, B2, Azure Blob, Google Cloud Storage**
 - Restore operations with safety checks and validation
 - Automatic CPU detection and parallel processing
 - Streaming compression for large databases
 - Interactive terminal UI with progress tracking
- Cross-platform binaries (Linux, macOS, BSD)
+- Cross-platform binaries (Linux, macOS, BSD, Windows)

 ## Installation

@@ -214,6 +215,10 @@ Restore full cluster:
 | `--auto-detect-cores` | Auto-detect CPU cores | true |
 | `--no-config` | Skip loading .dbbackup.conf | false |
 | `--no-save-config` | Prevent saving configuration | false |
+| `--cloud` | Cloud storage URI (s3://, azure://, gcs://) | (empty) |
+| `--cloud-provider` | Cloud provider (s3, minio, b2, azure, gcs) | (empty) |
+| `--cloud-bucket` | Cloud bucket/container name | (empty) |
+| `--cloud-region` | Cloud region | (empty) |
 | `--debug` | Enable debug logging | false |
 | `--no-color` | Disable colored output | false |

@@ -571,6 +576,80 @@ Display version information:
 ./dbbackup version
 ```

+## Cloud Storage Integration
+
+dbbackup v2.0 includes native support for cloud storage providers. See [CLOUD.md](CLOUD.md) for complete documentation.
+
+### Quick Start - Cloud Backups
+
+**Configure cloud provider in TUI:**
+```bash
+# Launch interactive mode
+./dbbackup interactive
+
+# Navigate to: Configuration Settings
+# Set: Cloud Storage Enabled = true
+# Set: Cloud Provider = s3 (or azure, gcs, minio, b2)
+# Set: Cloud Bucket/Container = your-bucket-name
+# Set: Cloud Region = us-east-1 (if applicable)
+# Set: Cloud Auto-Upload = true
+```
+
+**Command-line cloud backup:**
+```bash
+# Backup directly to S3
+./dbbackup backup single mydb --cloud s3://my-bucket/backups/
+
+# Backup to Azure Blob Storage
+./dbbackup backup single mydb \
+  --cloud azure://my-container/backups/ \
+  --cloud-access-key myaccount \
+  --cloud-secret-key "account-key"
+
+# Backup to Google Cloud Storage
+./dbbackup backup single mydb \
+  --cloud gcs://my-bucket/backups/ \
+  --cloud-access-key /path/to/service-account.json
+
+# Restore from cloud
+./dbbackup restore single s3://my-bucket/backups/mydb_20251126.dump \
+  --target mydb_restored \
+  --confirm
+```
+
+**Supported Providers:**
+- **AWS S3** - `s3://bucket/path`
+- **MinIO** - `minio://bucket/path` (self-hosted S3-compatible)
+- **Backblaze B2** - `b2://bucket/path`
+- **Azure Blob Storage** - `azure://container/path` (native support)
+- **Google Cloud Storage** - `gcs://bucket/path` (native support)
+
+**Environment Variables:**
+```bash
+# AWS S3 / MinIO / B2
+export AWS_ACCESS_KEY_ID="your-key"
+export AWS_SECRET_ACCESS_KEY="your-secret"
+export AWS_REGION="us-east-1"
+
+# Azure Blob Storage
+export AZURE_STORAGE_ACCOUNT="myaccount"
+export AZURE_STORAGE_KEY="account-key"
+
+# Google Cloud Storage
+export GOOGLE_APPLICATION_CREDENTIALS="/path/to/service-account.json"
+```
+
+**Features:**
+- ✅ Streaming uploads (memory efficient)
+- ✅ Multipart upload for large files (>100MB)
+- ✅ Progress tracking
+- ✅ Automatic metadata sync (.sha256, .info files)
+- ✅ Restore directly from cloud URIs
+- ✅ Cloud backup verification
+- ✅ TUI integration for all cloud providers
+
+See [CLOUD.md](CLOUD.md) for detailed setup guides, testing with Docker, and advanced configuration.
+
 ## Configuration

 ### PostgreSQL Authentication