Renz
64f1458e9a
Implemented full native support for Azure Blob Storage and Google Cloud Storage: **Azure Blob Storage (internal/cloud/azure.go):** - Native Azure SDK integration (github.com/Azure/azure-sdk-for-go) - Block blob upload for large files (>256MB with 100MB blocks) - Azurite emulator support for local testing - Production Azure authentication (account name + key) - SHA-256 integrity verification with metadata - Streaming uploads with progress tracking **Google Cloud Storage (internal/cloud/gcs.go):** - Native GCS SDK integration (cloud.google.com/go/storage) - Chunked upload for large files (16MB chunks) - fake-gcs-server emulator support for local testing - Application Default Credentials support - Service account JSON key file support - SHA-256 integrity verification with metadata - Streaming uploads with progress tracking **Backend Integration:** - Updated NewBackend() factory to support azure/azblob and gs/gcs providers - Added Name() methods to both backends - Fixed ProgressReader usage across all backends - Updated Config comments to document Azure/GCS support **Testing Infrastructure:** - docker-compose.azurite.yml: Azurite + PostgreSQL + MySQL test environment - docker-compose.gcs.yml: fake-gcs-server + PostgreSQL + MySQL test environment - scripts/test_azure_storage.sh: 8 comprehensive Azure integration tests - scripts/test_gcs_storage.sh: 8 comprehensive GCS integration tests - Both test scripts validate upload/download/verify/cleanup/restore operations **Documentation:** - AZURE.md: Complete guide (600+ lines) covering setup, authentication, usage - GCS.md: Complete guide (600+ lines) covering setup, authentication, usage - Updated CLOUD.md with Azure and GCS sections - Updated internal/config/config.go with Azure/GCS field documentation **Test Coverage:** - Large file uploads (300MB for Azure, 200MB for GCS) - Block/chunked upload verification - Backup verification with SHA-256 checksums - Restore from cloud URIs - Cleanup and retention policies - Emulator support for both providers **Dependencies Added:** - Azure: github.com/Azure/azure-sdk-for-go/sdk/storage/azblob v1.6.3 - GCS: cloud.google.com/go/storage v1.57.2 - Plus transitive dependencies (~50+ packages) **Build:** - Compiles successfully: 68MB binary - All imports resolved - No compilation errors Sprint 4 closes the multi-cloud gap identified in Sprint 3 evaluation. Users can now use Azure and GCS URIs that were previously parsed but unsupported.
60 lines
1.4 KiB
YAML
60 lines
1.4 KiB
YAML
version: '3.8'
|
|
|
|
services:
|
|
# fake-gcs-server - Google Cloud Storage Emulator
|
|
gcs-emulator:
|
|
image: fsouza/fake-gcs-server:latest
|
|
container_name: dbbackup-gcs
|
|
ports:
|
|
- "4443:4443"
|
|
command: -scheme http -public-host localhost:4443 -external-url http://localhost:4443
|
|
healthcheck:
|
|
test: ["CMD", "wget", "--spider", "-q", "http://localhost:4443/storage/v1/b"]
|
|
interval: 5s
|
|
timeout: 3s
|
|
retries: 30
|
|
networks:
|
|
- dbbackup-net
|
|
|
|
# PostgreSQL 16 for testing
|
|
postgres:
|
|
image: postgres:16-alpine
|
|
container_name: dbbackup-postgres-gcs
|
|
environment:
|
|
POSTGRES_USER: testuser
|
|
POSTGRES_PASSWORD: testpass
|
|
POSTGRES_DB: testdb
|
|
ports:
|
|
- "5435:5432"
|
|
healthcheck:
|
|
test: ["CMD-SHELL", "pg_isready -U testuser -d testdb"]
|
|
interval: 5s
|
|
timeout: 3s
|
|
retries: 10
|
|
networks:
|
|
- dbbackup-net
|
|
|
|
# MySQL 8.0 for testing
|
|
mysql:
|
|
image: mysql:8.0
|
|
container_name: dbbackup-mysql-gcs
|
|
environment:
|
|
MYSQL_ROOT_PASSWORD: rootpass
|
|
MYSQL_DATABASE: testdb
|
|
MYSQL_USER: testuser
|
|
MYSQL_PASSWORD: testpass
|
|
ports:
|
|
- "3309:3306"
|
|
command: --default-authentication-plugin=mysql_native_password
|
|
healthcheck:
|
|
test: ["CMD", "mysqladmin", "ping", "-h", "localhost", "-u", "root", "-prootpass"]
|
|
interval: 5s
|
|
timeout: 3s
|
|
retries: 10
|
|
networks:
|
|
- dbbackup-net
|
|
|
|
networks:
|
|
dbbackup-net:
|
|
driver: bridge
|