Renz
64f1458e9a
Implemented full native support for Azure Blob Storage and Google Cloud Storage: **Azure Blob Storage (internal/cloud/azure.go):** - Native Azure SDK integration (github.com/Azure/azure-sdk-for-go) - Block blob upload for large files (>256MB with 100MB blocks) - Azurite emulator support for local testing - Production Azure authentication (account name + key) - SHA-256 integrity verification with metadata - Streaming uploads with progress tracking **Google Cloud Storage (internal/cloud/gcs.go):** - Native GCS SDK integration (cloud.google.com/go/storage) - Chunked upload for large files (16MB chunks) - fake-gcs-server emulator support for local testing - Application Default Credentials support - Service account JSON key file support - SHA-256 integrity verification with metadata - Streaming uploads with progress tracking **Backend Integration:** - Updated NewBackend() factory to support azure/azblob and gs/gcs providers - Added Name() methods to both backends - Fixed ProgressReader usage across all backends - Updated Config comments to document Azure/GCS support **Testing Infrastructure:** - docker-compose.azurite.yml: Azurite + PostgreSQL + MySQL test environment - docker-compose.gcs.yml: fake-gcs-server + PostgreSQL + MySQL test environment - scripts/test_azure_storage.sh: 8 comprehensive Azure integration tests - scripts/test_gcs_storage.sh: 8 comprehensive GCS integration tests - Both test scripts validate upload/download/verify/cleanup/restore operations **Documentation:** - AZURE.md: Complete guide (600+ lines) covering setup, authentication, usage - GCS.md: Complete guide (600+ lines) covering setup, authentication, usage - Updated CLOUD.md with Azure and GCS sections - Updated internal/config/config.go with Azure/GCS field documentation **Test Coverage:** - Large file uploads (300MB for Azure, 200MB for GCS) - Block/chunked upload verification - Backup verification with SHA-256 checksums - Restore from cloud URIs - Cleanup and retention policies - Emulator support for both providers **Dependencies Added:** - Azure: github.com/Azure/azure-sdk-for-go/sdk/storage/azblob v1.6.3 - GCS: cloud.google.com/go/storage v1.57.2 - Plus transitive dependencies (~50+ packages) **Build:** - Compiles successfully: 68MB binary - All imports resolved - No compilation errors Sprint 4 closes the multi-cloud gap identified in Sprint 3 evaluation. Users can now use Azure and GCS URIs that were previously parsed but unsupported.
67 lines
1.6 KiB
YAML
67 lines
1.6 KiB
YAML
version: '3.8'
|
|
|
|
services:
|
|
# Azurite - Azure Storage Emulator
|
|
azurite:
|
|
image: mcr.microsoft.com/azure-storage/azurite:latest
|
|
container_name: dbbackup-azurite
|
|
ports:
|
|
- "10000:10000" # Blob service
|
|
- "10001:10001" # Queue service
|
|
- "10002:10002" # Table service
|
|
volumes:
|
|
- azurite_data:/data
|
|
command: azurite --blobHost 0.0.0.0 --queueHost 0.0.0.0 --tableHost 0.0.0.0 --loose --skipApiVersionCheck
|
|
healthcheck:
|
|
test: ["CMD", "nc", "-z", "localhost", "10000"]
|
|
interval: 5s
|
|
timeout: 3s
|
|
retries: 30
|
|
networks:
|
|
- dbbackup-net
|
|
|
|
# PostgreSQL 16 for testing
|
|
postgres:
|
|
image: postgres:16-alpine
|
|
container_name: dbbackup-postgres-azure
|
|
environment:
|
|
POSTGRES_USER: testuser
|
|
POSTGRES_PASSWORD: testpass
|
|
POSTGRES_DB: testdb
|
|
ports:
|
|
- "5434:5432"
|
|
healthcheck:
|
|
test: ["CMD-SHELL", "pg_isready -U testuser -d testdb"]
|
|
interval: 5s
|
|
timeout: 3s
|
|
retries: 10
|
|
networks:
|
|
- dbbackup-net
|
|
|
|
# MySQL 8.0 for testing
|
|
mysql:
|
|
image: mysql:8.0
|
|
container_name: dbbackup-mysql-azure
|
|
environment:
|
|
MYSQL_ROOT_PASSWORD: rootpass
|
|
MYSQL_DATABASE: testdb
|
|
MYSQL_USER: testuser
|
|
MYSQL_PASSWORD: testpass
|
|
ports:
|
|
- "3308:3306"
|
|
command: --default-authentication-plugin=mysql_native_password
|
|
healthcheck:
|
|
test: ["CMD", "mysqladmin", "ping", "-h", "localhost", "-u", "root", "-prootpass"]
|
|
interval: 5s
|
|
timeout: 3s
|
|
retries: 10
|
|
networks:
|
|
- dbbackup-net
|
|
|
|
volumes:
|
|
azurite_data:
|
|
|
|
networks:
|
|
dbbackup-net:
|
|
driver: bridge
|