hmac-file-server/WIKI.MD

This documentation provides detailed information on configuring, setting up, and maintaining the HMAC File Server. Whether you're a developer, system administrator, or an enthusiast, this guide will help you navigate through the server's features and configurations effectively.

---

Table of Contents

1. Introduction
2. 3.2 "Tremora del Terra" Revolutionary Features
3. Configuration
   • Server Configuration
   • Deduplication Settings
   • ISO Settings
   • Timeout Settings
   • Security Settings
   • Versioning Settings
   • Uploads Settings
   • Downloads Settings
   • ClamAV Settings
   • Redis Settings
   • Worker Settings
4. Example Configuration
5. Setup Instructions
   • 1. HMAC File Server Installation
   • 2. Reverse Proxy Configuration
     • Apache2 Reverse Proxy
     • Nginx Reverse Proxy
   • 3. ejabberd Configuration
   • 4. Systemd Service Setup
6. Running with Docker & Docker Compose
7. Running with Podman
8. Building for Different Architectures
9. Network Resilience & Queue Optimization
10. Multi-Architecture Deployment
11. Additional Recommendations
12. Notes
13. Using HMAC File Server for CI/CD Build Artifacts
14. Monitoring

---

Introduction

The HMAC File Server 3.2 "Tremora del Terra" is a revolutionary secure and efficient file management solution designed to handle file uploads, downloads, deduplication, and more. This major release brings 93% configuration reduction, dramatically simplifying setup while maintaining enterprise-grade features.

Version 3.2 Revolutionary Features:

• 93% Configuration Reduction: Simplified setup with intelligent defaults
• Network Resilience: Advanced connection recovery and stability
• Queue Optimization: Enhanced dynamic worker scaling (40%/10% thresholds)
• Extended Timeouts: 4800s timeouts for seamless large file transfers
• Multi-Architecture Support: Native AMD64, ARM64, ARM32v7 builds
• XEP-0363 XMPP Integration: Full XMPP file sharing protocol support
• Prometheus Monitoring: Enterprise-grade metrics and observability

Built with a focus on security, scalability, and performance, it integrates seamlessly with various tools and services to provide a comprehensive file handling experience optimized for modern cloud environments.

---

3.2 "Tremora del Terra" Revolutionary Features

HMAC File Server 3.2 "Tremora del Terra" represents a revolutionary leap forward in file server technology, introducing breakthrough simplifications and advanced enterprise features:

🚀 93% Configuration Reduction

• Simplified Setup: Reduced configuration complexity by 93% through intelligent defaults
• Minimal Config Required: Essential settings only - server runs with just a few lines
• Smart Defaults: Automatically optimized settings for most use cases
• Zero-Touch Deployment: Ready for production with minimal configuration

🌐 Network Resilience System

• Connection Recovery: Automatic reconnection and retry mechanisms
• Timeout Optimization: Extended 4800s timeouts for seamless large file transfers
• Network Switching: Handles network changes gracefully without service interruption
• Connection Pooling: Intelligent connection management for high-load scenarios

⚡ Queue Optimization Engine

• Dynamic Worker Scaling: Optimized 40%/10% thresholds for perfect load balancing
• Queue Intelligence: Smart queue management preventing bottlenecks
• Load Prediction: Proactive scaling based on traffic patterns
• Memory Optimization: Reduced memory footprint while handling larger queues

🏗️ Multi-Architecture Excellence

• Native AMD64: Optimized performance for Intel/AMD processors
• ARM64 Support: Full native support for Apple Silicon and ARM servers
• ARM32v7 Compatibility: Raspberry Pi and IoT device support
• Cross-Platform: Consistent behavior across all architectures

📊 Enterprise Monitoring

• Prometheus Integration: Comprehensive metrics collection
• Real-time Dashboards: Advanced monitoring capabilities
• Performance Analytics: Detailed insights into server operations
• Alert Systems: Proactive issue detection and notification

🔗 XEP-0363 XMPP Integration

• Full Protocol Support: Complete XMPP file sharing implementation
• ejabberd Integration: Seamless integration with XMPP servers
• Secure File Sharing: HMAC-authenticated file sharing through XMPP
• Standard Compliance: Full XEP-0363 protocol compliance

---

Configuration

The HMAC File Server is configured using a config.toml file. Below are the detailed explanations of each configuration section and their respective options.

Server Configuration

# Server configuration
listen_address = ":8080"  # Listen address and port for incoming requests
storage_path = "/srv/hmac-file-server/uploads"  # Directory to store uploaded files
metrics_enabled = true   # Enable Prometheus metrics
metrics_path = "/metrics"    # Path for Prometheus metrics endpoint
pid_file = "/var/run/hmac-file-server.pid" # Path to PID file
max_upload_size = "10GB" # Maximum file upload size
max_header_bytes = 1048576 # Maximum header size (1MB)
cleanup_interval = "24h" # Interval for cleanup operations
max_file_age = "720h"    # Maximum age for files (30 days)
pre_cache = true         # Pre-cache file structures on startup
pre_cache_workers = 4    # Number of workers for pre-caching
pre_cache_interval = "1h" # Interval for pre-cache operations
global_extensions = [".txt", ".dat", ".iso", ".mp4", ".mkv", ".avi", ".mov"] # Global allowed extensions
deduplication_enabled = true # Enable file deduplication
min_free_bytes = "1GB"   # Minimum free disk space required
file_naming = "original" # File naming strategy: "original", "HMAC"
force_protocol = ""      # Force protocol: "http", "https" or empty for auto
enable_dynamic_workers = true # Enable dynamic worker scaling
worker_scale_up_thresh = 40   # Queue length % to scale up workers (40% optimized threshold)
worker_scale_down_thresh = 10 # Queue length % to scale down workers (10% stability threshold)

Configuration Options

• listen_address:

  • Type: String
  • Description: Specifies the address and port on which the server listens for incoming requests.
  • Default: ":8080"

• storage_path:

  • Type: String
  • Description: Defines the directory path where uploaded files are stored. Ensure this path exists and has appropriate permissions.
  • Default: "/srv/hmac-file-server/uploads"

• metrics_enabled:

  • Type: Boolean
  • Description: Enables or disables Prometheus metrics collection.
  • Default: true

• max_upload_size:

  • Type: String
  • Description: Maximum allowed file upload size. Supports B, KB, MB, GB, TB units.
  • Default: "10GB"

• deduplication_enabled:

  • Type: Boolean
  • Description: Enables file deduplication to save storage space by using hard links for identical files.
  • Default: true

• enable_dynamic_workers:

  • Type: Boolean
  • Description: Enables automatic scaling of worker threads based on upload queue length.
  • Default: true

• file_naming:

  • Type: String
  • Description: Strategy for naming uploaded files.
  • Options: "original" (preserve original names), "HMAC" (use HMAC-based names)
  • Default: "original"
  • Description: Enables or disables the Prometheus metrics endpoint.
  • Default: true

• metricsport:

  • Type: String
  • Description: Defines the port on which Prometheus metrics are exposed.
  • Default: "9090"

• deduplicationenabled:

  • Type: Boolean
  • Description: Enables or disables file deduplication to optimize storage usage.
  • Default: true

---

Deduplication Settings

# Deduplication settings
[deduplication]
enabled = true
directory = "/opt/hmac-file-server/data/dedup"  # Path to deduplication storage
maxsize = "1GB"  # Maximum file size for deduplication

Configuration Options

• enabled:

  • Type: Boolean
  • Description: Enables or disables the deduplication feature, which uses hard links to eliminate duplicate files and save storage space.
  • Default: true

• directory:

  • Type: String
  • Description: Specifies the directory path where deduplicated files are stored. Files are organized by their SHA256 hash.
  • Default: "/opt/hmac-file-server/data/dedup"

• maxsize:

  • Type: String
  • Description: Maximum file size eligible for deduplication. Larger files are not deduplicated to avoid performance impact.
  • Default: "1GB"

---

ISO Settings

# ISO settings
[iso]
enabled = false
size = "1TB"  # Maximum ISO size
mountpoint = "/path/to/hmac-file-server/iso/"  # ISO mount point
charset = "utf-8"  # Filesystem character set encoding

Configuration Options

• enabled:

  • Type: Boolean
  • Description: Enables or disables the mounting of an ISO-based filesystem for specialized storage needs.
  • Default: false

• size:

  • Type: String
  • Description: Defines the maximum allowed size for the ISO container.
  • Default: "1TB"

• mountpoint:

  • Type: String
  • Description: Specifies the directory path where the ISO is mounted. Ensure this path exists and has appropriate permissions.
  • Default: "/path/to/hmac-file-server/iso/"

• charset:

  • Type: String
  • Description: Sets the filesystem character set encoding for the ISO.
  • Default: "utf-8"

Note

: Ensure only one [iso] block is active in your config.toml to avoid configuration conflicts.

---

Timeout Settings

# Timeout settings
[timeouts]
readtimeout = "4800s"    # Maximum time to read a request (80 minutes for large files)
writetimeout = "4800s"   # Maximum time to write a response (80 minutes for large files)
idletimeout = "4800s"    # Maximum keep-alive time for idle connections

Configuration Options

• readtimeout:

  • Type: String
  • Description: Sets the maximum duration for reading the entire request, including the body. Extended timeout for large file uploads.
  • Format: Duration (e.g., "4800s" for 80 minutes)
  • Default: "4800s"

• writetimeout:

  • Type: String
  • Description: Defines the maximum duration before timing out writes of the response. Extended timeout for large file downloads.
  • Format: Duration (e.g., "4800s" for 80 minutes)
  • Default: "4800s"

• idletimeout:

  • Type: String
  • Description: Specifies the maximum amount of time to wait for the next request when keep-alives are enabled.
  • Format: Duration (e.g., "4800s" for 80 minutes)
  • Default: "4800s"

Note: These extended timeout values are specifically configured for handling large file uploads and downloads (GB-sized files). Ensure your reverse proxy configuration has matching timeout values.

---

Security Configuration

# Security settings
[security]
secret = "your-secure-secret-key"  # HMAC shared secret key (change to a secure value)
enablejwt = false                  # Enable JWT authentication
jwtsecret = "your-jwt-secret"      # JWT signing secret
jwtalgorithm = "HS256"             # JWT algorithm
jwtexpiration = "24h"              # JWT token expiration

Configuration Options

• secret:

  • Type: String
  • Description: The HMAC shared secret key used for signing requests and operations.
  • Default: "your-secure-secret-key"
  • Warning: Change this immediately to a unique, strong string in production environments to ensure the security of HMAC operations.

• enablejwt:

  • Type: Boolean
  • Description: Enables or disables JWT token authentication. When enabled, the server will accept JWT tokens for authentication.
  • Default: false

• jwtsecret:

  • Type: String
  • Description: The secret key used for signing and validating JWT tokens. Must be strong and secure.
  • Default: "your-jwt-secret"

• jwtalgorithm:

  • Type: String
  • Description: The algorithm used for JWT token signing.
  • Options: "HS256", "HS384", "HS512"
  • Default: "HS256"

• jwtexpiration:

  • Type: String
  • Description: The expiration time for JWT tokens.
  • Format: Duration (e.g., "24h" for 24 hours, "30m" for 30 minutes)
  • Default: "24h"

---

Versioning Settings

# Versioning settings
[versioning]
enableversioning = false
maxversions = 1  # Number of file versions to retain

Configuration Options

• enableversioning:

  • Type: Boolean
  • Description: Enables or disables the versioning feature, which maintains multiple versions of the same file.
  • Default: false

• maxversions:

  • Type: Integer
  • Description: Specifies the maximum number of versions to retain for each file.
  • Default: 1

---

Logging Configuration

# Logging settings
[logging]
level = "info"
file = "/var/log/hmac-file-server.log"
max_size = 100    # Maximum log file size in MB
max_backups = 7   # Number of backup log files to keep
max_age = 30      # Maximum age of log files in days
compress = true   # Compress old log files

Configuration Options

• level:

  • Type: String
  • Description: Sets the verbosity level of logs.
  • Options: "debug", "info", "warn", "error"
  • Default: "info"

• file:

  • Type: String
  • Description: Specifies the file path for logging. If left empty, logs are output to stdout.
  • Default: "/var/log/hmac-file-server.log"

• max_size:

  • Type: Integer
  • Description: Maximum size of log files before rotation (in MB).
  • Default: 100

• max_backups:

  • Type: Integer
  • Description: Number of backup log files to retain after rotation.
  • Default: 7

• max_age:

  • Type: Integer
  • Description: Maximum age of log files in days before deletion.
  • Default: 30

• compress:

  • Type: Boolean
  • Description: Whether to compress old log files with gzip.
  • Default: true

---

Uploads Configuration

# Upload settings
[uploads]
allowed_extensions = [".zip", ".rar", ".7z", ".tar.gz", ".tgz", ".gpg", ".enc", ".pgp"]
chunked_uploads_enabled = true
chunk_size = "10MB"  # Chunk size for uploads
resumable_uploads_enabled = true
max_resumable_age = "48h"  # Maximum age for resumable uploads

Configuration Options

• allowed_extensions:

  • Type: Array of Strings
  • Description: Lists the file extensions permitted for upload.
  • Default: [".zip", ".rar", ".7z", ".tar.gz", ".tgz", ".gpg", ".enc", ".pgp"]

• chunked_uploads_enabled:

  • Type: Boolean
  • Description: Enables or disables chunked file uploads for better performance with large files.
  • Default: true

• chunk_size:

  • Type: String
  • Description: Defines the size of each chunk in chunked uploads.
  • Format: Size (e.g., "10MB")
  • Default: "10MB"

• resumable_uploads_enabled:

  • Type: Boolean
  • Description: Enables or disables support for resumable uploads that can be continued after interruption.
  • Default: true

• max_resumable_age:

  • Type: String
  • Description: Maximum time a resumable upload session remains valid.
  • Format: Duration (e.g., "48h")
  • Default: "48h"

---

Downloads Configuration

# Downloads settings
[downloads]
resumable_downloads_enabled = true
chunked_downloads_enabled = true
chunk_size = "8192"  # Chunk size for downloads
allowed_extensions = [".txt", ".pdf", ".png", ".jpg", ".jpeg", ".gif", ".bmp", ".tiff", ".svg", ".webp"]

Configuration Options

• resumable_downloads_enabled:

  • Type: Boolean
  • Description: Enables or disables support for resumable downloads.
  • Default: true

• chunked_downloads_enabled:

  • Type: Boolean
  • Description: Enables or disables chunked downloads for better performance.
  • Default: true

• chunk_size:

  • Type: String
  • Description: Defines the size of each chunk in chunked downloads.
  • Format: Size (e.g., "8192" bytes)
  • Default: "8192"

• allowed_extensions:

  • Type: Array of Strings
  • Description: Lists the file extensions permitted for download.
  • Default: [".txt", ".pdf", ".png", ".jpg", ".jpeg", ".gif", ".bmp", ".tiff", ".svg", ".webp"]

---

ClamAV Settings

# ClamAV settings
[clamav]
clamavenabled = true
clamavsocket = "/var/run/clamav/clamd.ctl"  # Path to ClamAV socket
numscanworkers = 2  # Number of concurrent scan workers
# Only scan potentially dangerous file types, skip large media files
scanfileextensions = [".txt", ".pdf", ".doc", ".docx", ".xls", ".xlsx", ".exe", ".zip", ".rar", ".7z", ".tar", ".gz"]
# Skip scanning files larger than 200MB (ClamAV limit)
maxscansize = "200MB"

Configuration Options

• clamavenabled:

  • Type: Boolean
  • Description: Enables or disables ClamAV integration for virus scanning of uploaded files.
  • Default: true

• clamavsocket:

  • Type: String
  • Description: Specifies the file path to the ClamAV socket (.ctl file). Ensure ClamAV is installed and the socket path is correct.
  • Default: "/var/run/clamav/clamd.ctl"

• numscanworkers:

  • Type: Integer
  • Description: Sets the number of concurrent workers dedicated to scanning files with ClamAV.
  • Default: 2

• scanfileextensions:

  • Type: Array of Strings
  • Description: Lists the file extensions that should be scanned for viruses. Excludes large media files for performance.
  • Default: [".txt", ".pdf", ".doc", ".docx", ".xls", ".xlsx", ".exe", ".zip", ".rar", ".7z", ".tar", ".gz"]

• maxscansize:

  • Type: String
  • Description: Maximum file size for virus scanning. Files larger than this are skipped to avoid ClamAV limits.
  • Default: "200MB"

---

Redis Settings

# Redis settings
[redis]
redisenabled = true
redisdbindex = 0
redisaddr = "localhost:6379"  # Redis server address
redispassword = ""            # Redis password if required
redishealthcheckinterval = "120s"  # Interval for Redis health checks

Configuration Options

• redisenabled:

  • Type: Boolean
  • Description: Enables or disables Redis integration for caching or session tracking.
  • Default: true

• redisaddr:

  • Type: String
  • Description: Specifies the address of the Redis server (e.g., "localhost:6379").
  • Default: "localhost:6379"

• redispassword:

  • Type: String
  • Description: Sets the Redis authentication password, if required.
  • Default: ""

• redisdbindex:

  • Type: Integer
  • Description: Specifies the Redis database index to use.
  • Default: 0

• redishealthcheckinterval:

  • Type: String
  • Description: Defines the interval for performing health checks on the Redis connection.
  • Format: Duration (e.g., "120s" for two minutes)
  • Default: "120s"

---

Workers Configuration

# Workers settings
[workers]
numworkers = 4               # Number of worker threads
uploadqueuesize = 50         # Size of upload queue

Configuration Options

• numworkers:

  • Type: Integer
  • Description: Specifies the base number of worker threads to handle file operations. Works with dynamic worker scaling.
  • Default: 4

• uploadqueuesize:

  • Type: Integer
  • Description: Sets the size of the upload queue buffer. Used with dynamic scaling thresholds.
  • Default: 50

Note: When enable_dynamic_workers = true in the server section, the worker count automatically scales between 4-8 workers based on queue length (scale up at 50, scale down at 10).

---

Configuration Options

• maxfilesize:
  • Type: String
  • Description: Defines the maximum allowed file size for uploads.
  • Format: Size (e.g., "10GB")
  • Default: "10GB"

---

Configuration Validation

The HMAC File Server v3.2 includes a comprehensive configuration validation system with specialized command-line flags for different validation scenarios.

Available Validation Flags

Core Validation Commands

--validate-config

• Purpose: Full comprehensive validation of all configuration sections
• Usage: ./hmac-file-server --validate-config
• Output: Complete validation report with all errors and warnings

# Example
./hmac-file-server -config config.toml --validate-config

--test-config

• Purpose: Run predefined configuration test scenarios
• Usage: ./hmac-file-server --test-config
• Output: Test scenario results for configuration validation

Specialized Validation Modes

--check-security

• Purpose: Security-focused validation only
• Checks: Secret strength, default values, JWT algorithms, network exposure, file permissions
• Example: ./hmac-file-server -config config.toml --check-security

--check-performance

• Purpose: Performance-focused validation only
• Checks: Worker optimization, memory usage, timeout balance, large file handling
• Example: ./hmac-file-server -config config.toml --check-performance

--check-connectivity

• Purpose: Network connectivity validation only
• Checks: Redis connections, ClamAV sockets, address validation, DNS resolution
• Example: ./hmac-file-server -config config.toml --check-connectivity

Output Control Flags

--validate-quiet

• Purpose: Minimal output, returns only exit codes
• Usage: Perfect for automation and scripts

--validate-verbose

• Purpose: Detailed output with comprehensive analysis
• Usage: Best for troubleshooting and development

--check-fixable

• Purpose: Show only issues that can be automatically fixed
• Usage: Helps prioritize configuration improvements

Validation Categories

Security Checks (6 categories)

• Secret strength analysis
• Default value detection
• Algorithm recommendations
• Network exposure warnings
• File permission analysis
• Debug logging security

Performance Checks (5 categories)

• Resource optimization
• Memory usage analysis
• Timeout balancing
• Large file preparation
• Configuration efficiency

Connectivity Checks (4 categories)

• Service connectivity
• Socket accessibility
• Address validation
• DNS resolution

System Checks (5 categories)

• CPU availability
• Memory monitoring
• Disk space validation
• Permission testing
• Resource constraints

Integration Examples

Shell Script Integration

#!/bin/bash
CONFIG_FILE="/etc/hmac-file-server/config.toml"

echo "🔍 Validating HMAC File Server configuration..."

# Run validation
if ./hmac-file-server -config "$CONFIG_FILE" --validate-config; then
    echo "✅ Configuration validation passed"
    
    # Additional specific checks
    echo "🔐 Running security audit..."
    ./hmac-file-server -config "$CONFIG_FILE" --check-security
    
    echo "⚡ Checking performance settings..."
    ./hmac-file-server -config "$CONFIG_FILE" --check-performance
else
    echo "❌ Configuration validation failed"
    echo "💡 Try: ./hmac-file-server -config $CONFIG_FILE --check-fixable"
    exit 1
fi

Docker Integration

# Add validation step to Dockerfile
RUN ./hmac-file-server -config /etc/config.toml --validate-config && \
    ./hmac-file-server -config /etc/config.toml --check-security

Kubernetes Health Check

livenessProbe:
  exec:
    command:
    - /usr/local/bin/hmac-file-server
    - -config
    - /etc/config/config.toml
    - --validate-quiet
  initialDelaySeconds: 30
  periodSeconds: 60

The enhanced command-line validation system provides comprehensive coverage with 50+ validation checks across all configuration areas, making HMAC File Server v3.2 production-ready with enterprise-grade configuration management.

---

Example Configuration

Below is an example config.toml file with current settings:

# Example HMAC File Server configuration

[server]
listen_address = ":8080"
storage_path = "/srv/hmac-file-server/uploads"
metrics_enabled = true
metrics_path = "/metrics"
pid_file = "/var/run/hmac-file-server.pid"
max_upload_size = "10GB"
max_header_bytes = 1048576
cleanup_interval = "24h"
max_file_age = "720h"
pre_cache = true
pre_cache_workers = 4
pre_cache_interval = "1h"
global_extensions = [".txt", ".dat", ".iso", ".mp4", ".mkv", ".avi", ".mov", ".wmv", ".flv", ".webm", ".mpeg"]
deduplication_enabled = true
min_free_bytes = "1GB"
file_naming = "original"
force_protocol = ""
enable_dynamic_workers = true
worker_scale_up_thresh = 40   # 40% optimized threshold for 3.2
worker_scale_down_thresh = 10

[uploads]
allowed_extensions = [".zip", ".rar", ".7z", ".tar.gz", ".tgz", ".gpg", ".enc", ".pgp"]
chunked_uploads_enabled = true
chunk_size = "10MB"
resumable_uploads_enabled = true
max_resumable_age = "48h"

[downloads]
resumable_downloads_enabled = true
chunked_downloads_enabled = true
chunk_size = "8192"
allowed_extensions = [".txt", ".pdf", ".png", ".jpg", ".jpeg", ".gif", ".bmp", ".tiff", ".svg", ".webp"]

[security]
secret = "f6g4ldPvQM7O2UTFeBEUUj33VrXypDAcsDt0yqKrLiOr5oQW"
enablejwt = false
jwtsecret = "f6g4ldPvQM7O2UTFeBEUUj33VrXypDAcsDt0yqKrLiOr5oQW"
jwtalgorithm = "HS256"
jwtexpiration = "24h"

[logging]
level = "info"
file = "/var/log/hmac-file-server.log"
max_size = 100
max_backups = 7
max_age = 30
compress = true

[deduplication]
enabled = true
directory = "/opt/hmac-file-server/data/dedup"
maxsize = "1GB"

[iso]
enabled = true
size = "1GB"
mountpoint = "/mnt/iso"
charset = "utf-8"
containerfile = "/mnt/iso/container.iso"

[timeouts]
readtimeout = "4800s"
writetimeout = "4800s"
idletimeout = "4800s"

[versioning]
enableversioning = false
maxversions = 1

[clamav]
clamavenabled = true
clamavsocket = "/var/run/clamav/clamd.ctl"
numscanworkers = 2
scanfileextensions = [".txt", ".pdf", ".doc", ".docx", ".xls", ".xlsx", ".exe", ".zip", ".rar", ".7z", ".tar", ".gz"]
maxscansize = "200MB"

[redis]
redisenabled = true
redisdbindex = 0
redisaddr = "localhost:6379"
redispassword = ""
redishealthcheckinterval = "120s"

[workers]
numworkers = 4
uploadqueuesize = 50

[file]
# Add file-specific configurations here

[build]
version = "3.2"

---

Configuration Best Practices

Performance Optimizations

Large File Handling: The server is configured for efficient large file uploads:

• Extended timeouts (4800s) prevent disconnections during GB-sized transfers
• Dynamic worker scaling (enable_dynamic_workers = true) adjusts capacity automatically
• Chunked uploads (chunk_size = "10MB") improve reliability for large files

Storage Efficiency:

• Deduplication (enabled = true, maxsize = "1GB") uses hard links to save space
• Selective virus scanning (scanfileextensions) skips large media files for performance
• Automatic cleanup (cleanup_interval = "24h") maintains disk space

Network Configuration:

• Reverse proxy timeouts must match server settings (proxy_read_timeout 4800)
• Upload size limits should accommodate max_upload_size = "10GB"
• Queue settings (uploadqueuesize = 50) balance memory usage and throughput

Security Considerations

• Use strong HMAC secrets (64+ characters)
• Limit file extensions based on your use case
• Enable ClamAV for dangerous file types only
• Monitor logs for authentication failures

Monitoring

• Prometheus metrics available at /metrics endpoint
• Log rotation configured to prevent disk space issues
• Worker scaling and queue metrics help identify bottlenecks

---

Setup Instructions

1. HMAC File Server Installation

To install the HMAC File Server, follow these steps:

1. Clone the repository:

   git clone https://git.uuxo.net/uuxo/hmac-file-server.git
   cd hmac-file-server
   

2. Build the server:

   go build -o hmac-file-server ./cmd/server/main.go
   

3. Create the necessary directories:

   mkdir -p /path/to/hmac-file-server/data/
   mkdir -p /path/to/hmac-file-server/deduplication/
   mkdir -p /path/to/hmac-file-server/iso/
   

4. Copy the example configuration file:

   cp config.example.toml config.toml
   

5. Edit the config.toml file to match your environment and preferences.

6. Start the server:

   ./hmac-file-server -config config.toml
   

2. Reverse Proxy Configuration

To set up a reverse proxy for the HMAC File Server, you can use either Apache2 or Nginx. Below are the configuration examples for both.

Apache2 Reverse Proxy

1. Enable the necessary Apache2 modules:

   sudo a2enmod proxy
   sudo a2enmod proxy_http
   sudo a2enmod headers
   sudo a2enmod rewrite
   

2. Create a new virtual host configuration file:

   sudo nano /etc/apache2/sites-available/hmac-file-server.conf
   

3. Add the following configuration to the file:

   <VirtualHost *:80>
       ServerName your-domain.com
   
       ProxyPreserveHost On
       ProxyPass / http://localhost:8080/
       ProxyPassReverse / http://localhost:8080/
   
       <Location />
           Require all granted
           Header always set X-Content-Type-Options "nosniff"
           Header always set X-Frame-Options "DENY"
           Header always set X-XSS-Protection "1; mode=block"
       </Location>
   </VirtualHost>
   

4. Enable the new site and restart Apache2:

   sudo a2ensite hmac-file-server.conf
   sudo systemctl restart apache2
   

Nginx Reverse Proxy

1. Install Nginx if not already installed:

   sudo apt-get update
   sudo apt-get install nginx
   

2. Create a new server block configuration file:

   sudo nano /etc/nginx/sites-available/hmac-file-server
   

3. Add the following configuration to the file:

   server {
       listen 80;
       server_name your-domain.com;
   
       location / {
           proxy_pass http://localhost:8080;
           proxy_set_header Host $host;
           proxy_set_header X-Real-IP $remote_addr;
           proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
           proxy_set_header X-Forwarded-Proto $scheme;
           proxy_set_header X-Content-Type-Options "nosniff";
           proxy_set_header X-Frame-Options "DENY";
           proxy_set_header X-XSS-Protection "1; mode=block";
       }
   }
   

4. Enable the new site and restart Nginx:

   sudo ln -s /etc/nginx/sites-available/hmac-file-server /etc/nginx/sites-enabled/
   sudo systemctl restart nginx
   

---

Proxy Best Practices & Recommendations

For production deployments, consider the following reverse proxy best practices:

• Timeouts: Set extended timeouts (e.g., proxy_read_timeout 4800; in Nginx) for large file handling.
• Buffer Sizes: Increase buffer sizes for large file uploads/downloads if needed (e.g., client_max_body_size 2G; in Nginx).
• Headers: Always set security headers (X-Content-Type-Options, X-Frame-Options, X-XSS-Protection).
• Forwarded Headers: Ensure X-Forwarded-For and X-Forwarded-Proto are set for correct client IP and protocol logging.
• HTTP/2: Enable HTTP/2 for better performance if supported by your proxy and clients.
• SSL/TLS: Terminate SSL at the proxy and use strong ciphers. Redirect HTTP to HTTPS.
• Health Checks: Configure health checks for the backend server to enable automatic failover or alerting.
• Access Controls: Restrict access to the management endpoints (e.g., /metrics) to trusted IPs only.

See the official Nginx and Apache documentation for more advanced tuning options.

---

3. ejabberd Configuration

hosts:
  - "your-domain.com"

listen:
  -
    port: 5222
    module: ejabberd_c2s
    certfile: "/etc/ejabberd/ejabberd.pem"
    starttls: true
    starttls_required: true
    protocol_options:
      - "no_sslv3"
      - "no_tlsv1"
      - "no_tlsv1_1"
    ciphers: "HIGH:!aNULL:!eNULL:!3DES:@STRENGTH"
    dhfile: "/etc/ejabberd/dhparams.pem"
    max_stanza_size: 65536
    shaper: c2s_shaper
    access: c2s

  -
    port: 5269
    module: ejabberd_s2s_in
    certfile: "/etc/ejabberd/ejabberd.pem"
    starttls: true
    starttls_required: true
    protocol_options:
      - "no_sslv3"
      - "no_tlsv1"
      - "no_tlsv1_1"
    ciphers: "HIGH:!aNULL:!eNULL:!3DES:@STRENGTH"
    dhfile: "/etc/ejabberd/dhparams.pem"
    max_stanza_size: 131072
    shaper: s2s_shaper
    access: s2s

acl:
  local:
    user_regexp: ""

access_rules:
  local:
    allow: local

mod_http_upload:
    max_size: 1073741824
    thumbnail: true
    put_url: https://share.uuxo.net
    get_url: https://share.uuxo.net
    external_secret: "changeme"
    custom_headers:
      "Access-Control-Allow-Origin": "*"
      "Access-Control-Allow-Methods": "GET,HEAD,PUT,OPTIONS"
      "Access-Control-Allow-Headers": "Content-Type"

4. Restart ejabberd:

   sudo systemctl restart ejabberd
   

4. Systemd Service Setup

To set up the HMAC File Server as a systemd service, follow these steps:

1. Create a new systemd service file:

   sudo nano /etc/systemd/system/hmac-file-server.service
   

2. Add the following configuration to the file:

   [Unit]
   Description=HMAC File Server
   After=network.target
   
   [Service]
   ExecStart=/path/to/hmac-file-server -config /path/to/config.toml
   WorkingDirectory=/path/to/hmac-file-server
   Restart=always
   User=www-data
   Group=www-data
   
   [Install]
   WantedBy=multi-user.target
   

3. Reload systemd and enable the service:

   sudo systemctl daemon-reload
   sudo systemctl enable hmac-file-server
   sudo systemctl start hmac-file-server
   

---

Running with Docker & Docker Compose

You can run the HMAC File Server using Docker and Docker Compose for easy deployment and environment management.

Automated Docker Deployment (Recommended)

The easiest way to deploy with Docker is using the automated installer:

git clone https://git.uuxo.net/uuxo/hmac-file-server.git
cd hmac-file-server
sudo ./installer.sh

When prompted, select Option 2: Docker deployment (docker-compose). The installer will:

• Create a complete docker-compose.yml with Redis and ClamAV services
• Generate an optimized multi-stage Dockerfile
• Set up proper networking between services
• Create start/stop scripts (start.sh, stop.sh)
• Configure container-optimized configuration
• Provide an isolated deployment directory structure

After installation, manage your deployment with:

cd /path/to/your/deployment/directory
./start.sh    # Start all services
./stop.sh     # Stop all services

Manual Docker Compose Setup

Manual Docker Compose Setup

For manual Docker setup without the installer, use this docker-compose.yml:

version: '3.8'

services:
  hmac-file-server:
    image: ghcr.io/plusone/hmac-file-server:latest
    ports:
      - "8080:8080"
    volumes:
      - ./config:/etc/hmac-file-server
      - ./data/uploads:/opt/hmac-file-server/data/uploads
      - ./data/duplicates:/opt/hmac-file-server/data/duplicates
      - ./data/temp:/opt/hmac-file-server/data/temp
      - ./data/logs:/opt/hmac-file-server/data/logs
    environment:
      - CONFIG_PATH=/etc/hmac-file-server/config.toml
    restart: unless-stopped

Key paths:

• /etc/hmac-file-server/config.toml: Main config file (mount your config here)
• /opt/hmac-file-server/data/uploads: Upload storage
• /opt/hmac-file-server/data/duplicates: Deduplication data
• /opt/hmac-file-server/data/temp: Temporary files
• /opt/hmac-file-server/data/logs: Log files

---

Running with Podman

Podman is a daemonless container engine that's often preferred in enterprise environments for enhanced security and rootless capabilities. HMAC File Server 3.2 provides complete Podman support with optimized deployment scripts.

Why Choose Podman?

Feature	Docker	Podman
Daemon	Requires Docker daemon	Daemonless architecture
Root Access	Requires root for Docker daemon	Can run completely rootless
Security	Good, but daemon runs as root	Enhanced security, no privileged daemon
Systemd Integration	Via Docker service	Native systemd integration
Pod Support	Requires docker-compose or swarm	Native Kubernetes-style pods
Enterprise Use	Popular in startups/mid-size	Preferred in enterprise environments
SELinux	Basic support	Excellent SELinux integration

Quick Start with Podman

# Clone repository
git clone https://github.com/PlusOne/hmac-file-server.git
cd hmac-file-server/dockerenv/podman

# One-command deployment
./deploy-podman.sh

# Check status
./deploy-podman.sh status

# View logs
./deploy-podman.sh logs

Manual Directory & Permission Setup

Understanding Container Security Model

• Container User: appuser (UID: 1011, GID: 1011)
• Security Principle: Never run containers as root (UID 0)
• Compatibility: Works across different container runtimes and deployment modes

Step-by-Step Manual Setup

Step 1: Create Directory Structure

# Create organized directory layout
export HMAC_BASE="/opt/podman/hmac-file-server"
sudo mkdir -p ${HMAC_BASE}/{config,data,deduplication,logs}

# Create subdirectories for uploads and duplicates
sudo mkdir -p ${HMAC_BASE}/data/{uploads,temp}
sudo mkdir -p ${HMAC_BASE}/deduplication/store
sudo mkdir -p ${HMAC_BASE}/logs/{access,error,debug}

Step 2: Set Ownership (CRITICAL)

# For Podman Rootless (recommended)
podman unshare chown -R 1011:1011 ${HMAC_BASE}

# For Podman Rootful or Docker
sudo chown -R 1011:1011 ${HMAC_BASE}

# Alternative: Use numeric IDs to avoid user lookup issues
sudo chown -R 1011:1011 ${HMAC_BASE}

Step 3: Set Permissions

# Directory permissions (executable for traversal)
sudo chmod 755 ${HMAC_BASE}
sudo chmod 755 ${HMAC_BASE}/{config,data,deduplication,logs}
sudo chmod 755 ${HMAC_BASE}/data/{uploads,temp}
sudo chmod 755 ${HMAC_BASE}/deduplication/store
sudo chmod 755 ${HMAC_BASE}/logs/{access,error,debug}

# Configuration file (read-only)
sudo chmod 644 ${HMAC_BASE}/config/config.toml

Step 4: Verify Ownership

# Check ownership recursively
ls -laR ${HMAC_BASE}

# Expected output format:
# drwxr-xr-x 2 1011 1011 4096 Dec 20 10:30 data
# drwxr-xr-x 2 1011 1011 4096 Dec 20 10:30 deduplication  
# drwxr-xr-x 2 1011 1011 4096 Dec 20 10:30 logs
# -rw-r--r-- 1 1011 1011 1234 Dec 20 10:30 config/config.toml

Container Volume Mapping

Host Path	Container Mount	Access Mode	SELinux Label	Purpose
${HMAC_BASE}/config/config.toml	/app/config.toml	ro	:Z	Configuration file
${HMAC_BASE}/data/	/data/	rw	:Z	File uploads
${HMAC_BASE}/deduplication/	/deduplication/	rw	:Z	Dedup cache
${HMAC_BASE}/logs/	/logs/	rw	:Z	Application logs

Complete Manual Run Command

# Build container image
podman build -t localhost/hmac-file-server:latest -f dockerenv/podman/Dockerfile.podman .

# Run with proper volume mounts and SELinux labels
podman run -d --name hmac-file-server \
  --security-opt no-new-privileges \
  --cap-drop=ALL \
  --read-only \
  --tmpfs /tmp:rw,noexec,nosuid,size=100m \
  -p 8888:8888 \
  -p 9090:9090 \
  -v ${HMAC_BASE}/config/config.toml:/app/config.toml:ro,Z \
  -v ${HMAC_BASE}/data:/data:rw,Z \
  -v ${HMAC_BASE}/deduplication:/deduplication:rw,Z \
  -v ${HMAC_BASE}/logs:/logs:rw,Z \
  localhost/hmac-file-server:latest -config /app/config.toml

Troubleshooting Path & Permission Issues

Common Error: Permission Denied

# Error in logs
{"level":"error","msg":"failed to create directories: mkdir /data: permission denied"}

Root Cause: Incorrect ownership or missing directories

Solution:

# Check current ownership
ls -la ${HMAC_BASE}

# Fix ownership (adjust command based on your setup)
# For rootless Podman:
podman unshare chown -R 1011:1011 ${HMAC_BASE}

# For rootful Podman/Docker:
sudo chown -R 1011:1011 ${HMAC_BASE}

# Verify fix
sudo -u "#1011" touch ${HMAC_BASE}/data/test-write
rm ${HMAC_BASE}/data/test-write

Common Error: SELinux Denial

# Error in logs or journalctl
SELinux is preventing access to 'write' on the file /data/test.txt

Root Cause: SELinux context not set for container volumes

Solution:

# Option 1: Use :Z labels (recommended - private volumes)
-v ${HMAC_BASE}/data:/data:rw,Z

# Option 2: Use :z labels (shared volumes between containers)  
-v ${HMAC_BASE}/data:/data:rw,z

# Option 3: Set SELinux boolean (system-wide change)
sudo setsebool -P container_manage_cgroup on

# Option 4: Manual context setting
sudo semanage fcontext -a -t container_file_t "${HMAC_BASE}(/.*)?"
sudo restorecon -R ${HMAC_BASE}

Common Error: User Namespace Issues

# Error starting container
Error: can't stat ${HMAC_BASE}/data: permission denied

Root Cause: User namespace mapping issues in rootless mode

Solution:

# Check user namespace configuration
cat /etc/subuid /etc/subgid | grep $USER

# If missing, add mappings (requires root)
sudo usermod --add-subuids 1000-65536 $USER
sudo usermod --add-subgids 1000-65536 $USER

# Restart user services
systemctl --user restart podman

# Use podman unshare for ownership
podman unshare chown -R 1011:1011 ${HMAC_BASE}

Verification & Testing Commands

# Test 1: Verify container can access all paths
podman exec hmac-file-server sh -c '
  echo "=== Container User ==="
  id
  echo "=== Directory Access ==="
  ls -la /app /data /deduplication /logs
  echo "=== Write Test ==="
  touch /data/write-test && echo "✅ Data write: OK" || echo "❌ Data write: FAILED"
  touch /deduplication/dedup-test && echo "✅ Dedup write: OK" || echo "❌ Dedup write: FAILED"  
  touch /logs/log-test && echo "✅ Log write: OK" || echo "❌ Log write: FAILED"
  echo "=== Config Read Test ==="
  head -3 /app/config.toml && echo "✅ Config read: OK" || echo "❌ Config read: FAILED"
'

# Test 2: External health check
curl -f http://localhost:8888/health && echo "✅ HTTP Health: OK" || echo "❌ HTTP Health: FAILED"

# Test 3: Metrics endpoint
curl -s http://localhost:9090/metrics | grep -E "hmac_|go_|process_" | wc -l
# Should return > 0 if metrics are working

# Test 4: File upload simulation (requires auth)
curl -X POST http://localhost:8888/upload \
  -H "Authorization: Bearer your-token" \
  -F "file=@test-file.txt" && echo "✅ Upload: OK" || echo "❌ Upload: FAILED"

Advanced: Custom UID/GID Mapping

If you need different UIDs (e.g., for existing file ownership):

# Option 1: Rebuild container with custom UID
podman build -t localhost/hmac-file-server:custom \
  --build-arg USER_UID=1500 \
  --build-arg USER_GID=1500 \
  -f dockerenv/podman/Dockerfile.podman .

# Option 2: Use --user flag (may have limitations)
podman run --user 1500:1500 [other options] localhost/hmac-file-server:latest

# Option 3: Host ownership adjustment
sudo chown -R 1500:1500 ${HMAC_BASE}

Docker vs Podman Ownership Differences

Scenario	Docker	Podman Rootless	Podman Rootful
Host UID	1011:1011	1011:1011	1011:1011
Container UID	1011:1011	1011:1011	1011:1011
Volume Ownership	chown 1011:1011	podman unshare chown 1011:1011	chown 1011:1011
SELinux Labels	:Z or :z	:Z or :z	:Z or :z

Podman Deployment Script Features

The deploy-podman.sh script provides complete automation:

• ✅ Interactive deployment with colored output
• ✅ Auto-generates secure configuration with random HMAC/JWT secrets
• ✅ Security-hardened settings: --cap-drop=ALL, --read-only, --no-new-privileges
• ✅ Pod management for XMPP integration
• ✅ Health monitoring and status reporting
• ✅ Environment variable support for customization

Podman Commands Reference

# Build image
podman build -t localhost/hmac-file-server:latest -f dockerenv/podman/Dockerfile.podman .

# Run with basic settings
podman run -d --name hmac-file-server \
    -p 8888:8888 \
    -v ./config.toml:/app/config.toml:ro \
    -v ./data:/data:rw \
    localhost/hmac-file-server:latest -config /app/config.toml

# Create and manage pods for XMPP integration
podman pod create --name xmpp-services -p 8888:8888 -p 9090:9090
podman run -d --pod xmpp-services --name hmac-file-server localhost/hmac-file-server:latest

# View logs and status
podman logs hmac-file-server
podman ps -a
podman pod ps

# Health check
podman healthcheck run hmac-file-server

# Stop and cleanup
podman stop hmac-file-server
podman rm hmac-file-server

Podman Systemd Integration

User Service (Rootless - Recommended)

# Copy service file
cp dockerenv/podman/hmac-file-server.service ~/.config/systemd/user/

# Enable and start
systemctl --user daemon-reload
systemctl --user enable hmac-file-server.service
systemctl --user start hmac-file-server.service

# Check status
systemctl --user status hmac-file-server.service

System Service (Root)

# Copy service file
sudo cp dockerenv/podman/hmac-file-server.service /etc/systemd/system/

# Enable and start
sudo systemctl daemon-reload
sudo systemctl enable hmac-file-server.service
sudo systemctl start hmac-file-server.service

Podman with XMPP Integration

# Complete XMPP + HMAC File Server pod setup
podman pod create --name xmpp-pod \
    --publish 5222:5222 \
    --publish 5269:5269 \
    --publish 5443:5443 \
    --publish 8888:8888

# Run Prosody XMPP server
podman run -d --pod xmpp-pod --name prosody \
    -v ./prosody/config:/etc/prosody:ro \
    -v ./prosody/data:/var/lib/prosody:rw \
    docker.io/prosody/prosody:latest

# Run HMAC File Server
podman run -d --pod xmpp-pod --name hmac-file-server \
    -v ./hmac/config.toml:/app/config.toml:ro \
    -v ./hmac/data:/data:rw \
    localhost/hmac-file-server:latest -config /app/config.toml

# Check pod status
podman pod ps
podman ps --pod

Deployment Script Commands

# Management commands
./deploy-podman.sh deploy    # Full deployment (default)
./deploy-podman.sh start     # Start services
./deploy-podman.sh stop      # Stop all services
./deploy-podman.sh restart   # Restart services
./deploy-podman.sh status    # Show service status
./deploy-podman.sh logs      # Show container logs
./deploy-podman.sh config    # Show configuration
./deploy-podman.sh build     # Build container image only
./deploy-podman.sh pod       # Create pod only
./deploy-podman.sh clean     # Remove containers and image
./deploy-podman.sh help      # Show help

# Environment variables
export APP_DATA="/custom/path/hmac-file-server"
export LISTEN_PORT="9999"
export METRICS_PORT="9998"
./deploy-podman.sh

Podman Security Features

Container Security

• Rootless operation: Runs as non-root user (UID 1011)
• Capability dropping: --cap-drop=ALL
• No new privileges: --security-opt no-new-privileges
• Read-only filesystem: --read-only with tmpfs for /tmp
• SELinux labels: Volume mounts with :Z labels

Network Security

• Pod isolation: Containers run in isolated pods
• Port binding: Only necessary ports exposed
• Health monitoring: Built-in health checks

Troubleshooting Podman

Common Issues

Permission Errors:

# Fix SELinux contexts
restorecon -R /opt/podman/hmac-file-server

# Check volume permissions
podman unshare ls -la /opt/podman/hmac-file-server

Container Won't Start:

# Check image exists
podman images | grep hmac-file-server

# Validate configuration
./deploy-podman.sh config

# Debug with interactive container
podman run -it --rm localhost/hmac-file-server:latest /bin/sh

Network Issues:

# Check pod networking
podman pod ps
podman port hmac-file-server

# Test connectivity
nc -zv localhost 8888

---

Multi-Architecture Deployment

HMAC File Server 3.2 "Tremora del Terra" provides comprehensive multi-architecture support for modern deployment scenarios.

Supported Architectures

AMD64 (x86_64)

• Primary Platform: Optimized for Intel and AMD processors
• Performance: Maximum performance optimization
• Use Cases: Data centers, cloud instances, desktop deployments
• Binary: hmac-file-server-linux-amd64

ARM64 (aarch64)

• Modern ARM: Apple Silicon (M1/M2/M3), AWS Graviton, cloud ARM instances
• Performance: Native ARM64 optimizations
• Use Cases: Cloud-native deployments, Apple Silicon development
• Binary: hmac-file-server-linux-arm64

ARM32v7 (armhf)

• IoT & Edge: Raspberry Pi, embedded systems, edge computing
• Efficiency: Optimized for resource-constrained environments
• Use Cases: IoT gateways, edge file servers, embedded applications
• Binary: hmac-file-server-linux-arm32v7

Build Commands

# Build for all architectures
./build-multi-arch.sh

# Build specific architecture
GOOS=linux GOARCH=amd64 go build -o hmac-file-server-linux-amd64 ./cmd/server/main.go
GOOS=linux GOARCH=arm64 go build -o hmac-file-server-linux-arm64 ./cmd/server/main.go
GOOS=linux GOARCH=arm GOARM=7 go build -o hmac-file-server-linux-arm32v7 ./cmd/server/main.go

Docker Multi-Architecture

# Build multi-platform Docker images
docker buildx build --platform linux/amd64,linux/arm64,linux/arm/v7 -t hmac-file-server:3.2 .

# Run platform-specific image
docker run --platform linux/arm64 hmac-file-server:3.2

Architecture-Specific Optimizations

AMD64 Optimizations

• AVX2/SSE4 utilizations for hash calculations
• Memory prefetching optimizations
• Large file transfer optimizations

ARM64 Optimizations

• NEON SIMD instructions for crypto operations
• Apple Silicon memory management optimizations
• Energy-efficient processing patterns

ARM32v7 Optimizations

• Memory-constrained operation modes
• Reduced concurrent workers for stability
• Optimized for flash storage patterns

---

Network Resilience & Queue Optimization

HMAC File Server 3.2 introduces advanced network resilience and queue optimization systems designed for enterprise-grade reliability.

Network Resilience Features

Connection Recovery

• Automatic Reconnection: Seamless reconnection after network interruptions
• Retry Logic: Intelligent exponential backoff for failed operations
• Timeout Management: Extended 4800s timeouts prevent premature disconnections
• Circuit Breaker: Prevents cascade failures during network issues

Network Switching Support

• Interface Detection: Automatic detection of network interface changes
• IP Migration: Seamless handling of IP address changes
• Connection Pooling: Maintains connection pools across network changes
• Health Checks: Continuous connectivity monitoring

Queue Optimization Engine

Dynamic Worker Scaling

• Optimized Thresholds: 40% scale-up, 10% scale-down for perfect balance
• Load Prediction: Proactive scaling based on historical patterns
• Memory Management: Intelligent memory allocation for queue operations
• Priority Queuing: Critical operations get processing priority

Queue Intelligence

• Bottleneck Prevention: Automatic queue rebalancing
• Overflow Protection: Graceful handling of queue overflow scenarios
• Performance Analytics: Real-time queue performance metrics
• Auto-tuning: Self-optimizing queue parameters

# Network resilience configuration
[network]
enable_resilience = true
max_retries = 5
retry_delay = "2s"
connection_timeout = "30s"
keepalive_interval = "60s"

# Queue optimization settings
[queue]
enable_optimization = true
scale_up_threshold = 40    # Scale up at 40% queue capacity
scale_down_threshold = 10  # Scale down at 10% queue capacity
min_workers = 2
max_workers = 16
prediction_window = "5m"

Docker Build

The official Dockerfile supports multi-stage builds for minimal images:

# Stage 1: Build
FROM golang:1.24-alpine AS builder

WORKDIR /build
RUN apk add --no-cache git
COPY go.mod go.sum ./
RUN go mod download
COPY . .
RUN CGO_ENABLED=0 go build -o hmac-file-server ./cmd/server/main.go

# Stage 2: Runtime
FROM alpine:latest

RUN apk --no-cache add ca-certificates

RUN mkdir -p /opt/hmac-file-server/data/uploads \
    && mkdir -p /opt/hmac-file-server/data/duplicates \
    && mkdir -p /opt/hmac-file-server/data/temp \
    && mkdir -p /opt/hmac-file-server/data/logs

WORKDIR /opt/hmac-file-server

COPY --from=builder /build/hmac-file-server .

EXPOSE 8080

CMD ["./hmac-file-server", "--config", "/etc/hmac-file-server/config.toml"]

Example Docker Config

A sample config.toml for Docker deployments:

[server]
listen_address = ":8080"
storage_path = "/srv/hmac-file-server/uploads"
metrics_enabled = true
metrics_path = "/metrics"
pid_file = "/var/run/hmac-file-server.pid"
max_upload_size = "10GB"
deduplication_enabled = true
min_free_bytes = "1GB"
file_naming = "original"
enable_dynamic_workers = true
worker_scale_up_thresh = 40   # 40% optimized threshold for 3.2
worker_scale_down_thresh = 10

[uploads]
allowed_extensions = [".zip", ".rar", ".7z", ".tar.gz", ".tgz", ".gpg", ".enc", ".pgp"]
chunked_uploads_enabled = true
chunk_size = "10MB"
resumable_uploads_enabled = true
max_resumable_age = "48h"

[downloads]
resumable_downloads_enabled = true
chunked_downloads_enabled = true
chunk_size = "8192"
allowed_extensions = [".txt", ".pdf", ".png", ".jpg", ".jpeg", ".gif", ".bmp", ".tiff", ".svg", ".webp"]

[security]
secret = "f6g4ldPvQM7O2UTFeBEUUj33VrXypDAcsDt0yqKrLiOr5oQW"
enablejwt = false
jwtsecret = "f6g4ldPvQM7O2UTFeBEUUj33VrXypDAcsDt0yqKrLiOr5oQW"
jwtalgorithm = "HS256"
jwtexpiration = "24h"

[logging]
level = "info"
file = "/var/log/hmac-file-server.log"
max_size = 100
max_backups = 7
max_age = 30
compress = true

[deduplication]
enabled = true
directory = "./deduplication"
maxsize = "1GB"

[iso]
enabled = true
size = "1GB"
mountpoint = "/mnt/iso"
charset = "utf-8"
containerfile = "/mnt/iso/container.iso"

[timeouts]
readtimeout = "4800s"
writetimeout = "4800s"
idletimeout = "4800s"

[versioning]
enableversioning = false
maxversions = 1

[clamav]
clamavenabled = true
clamavsocket = "/var/run/clamav/clamd.ctl"
numscanworkers = 2
scanfileextensions = [".txt", ".pdf", ".doc", ".docx", ".xls", ".xlsx", ".exe", ".zip", ".rar", ".7z", ".tar", ".gz"]
maxscansize = "200MB"

[redis]
redisenabled = true
redisdbindex = 0
redisaddr = "localhost:6379"
redispassword = ""
redishealthcheckinterval = "120s"

[workers]
numworkers = 4
uploadqueuesize = 50

[file]
# Add file-specific configurations here

[build]
version = "3.2"

Quickstart with Docker Compose

1. Place your config.toml in the ./config directory.
2. Run:

docker compose up -d

3. The server will be available on http://localhost:8080.

---

Simplified Configuration Examples

HMAC File Server 3.2 "Tremora del Terra" achieves 93% configuration reduction through intelligent defaults. Here are minimal configurations for common scenarios:

Minimal Production Configuration (93% Simplified)

# Minimal config - just 4 lines for full production deployment!
[server]
listen_address = ":8080"
storage_path = "/srv/uploads"
hmac_secret = "your-secret-key-here"

This minimal configuration automatically provides:

• ✅ Dynamic worker scaling (40%/10% thresholds)
• ✅ Extended timeouts (4800s)
• ✅ File deduplication
• ✅ Prometheus metrics
• ✅ Network resilience
• ✅ Queue optimization
• ✅ Security hardening

Quick Development Setup

# Development - just 2 lines!
[server]
storage_path = "./uploads"

Enterprise Cloud Configuration

# Enterprise cloud deployment
[server]
listen_address = ":8080"
storage_path = "/data/uploads"
hmac_secret = "${HMAC_SECRET}"
max_upload_size = "50GB"

[monitoring]
prometheus_enabled = true
metrics_port = "9090"

XMPP Integration (XEP-0363)

# XMPP file sharing server
[server]
storage_path = "/srv/xmpp-uploads"
hmac_secret = "${HMAC_SECRET}"

[xmpp]
enabled = true
max_file_size = "10GB"

Previous versions required 100+ configuration lines - 3.2 "Tremora del Terra" does it with just a few!

---