HMAC File Server 3.2

Overview

The HMAC File Server ensures secure file uploads and downloads using HMAC authentication and JWT tokens. It incorporates comprehensive security features, file versioning, deduplication, ISO container support, virus scanning, and Unix socket support for enhanced flexibility. Redis integration provides efficient caching and session management. Prometheus metrics and graceful shutdown mechanisms ensure reliable and efficient file handling.

Special thanks to Thomas Leister for inspiration drawn from prosody-filer.

Features

• Multiple Authentication Methods: HMAC-based authentication and JWT token support
• Multiple Protocol Support: v1 (legacy), v2, v3 (mod_http_upload_external), and token-based uploads
• File Management: Deduplication, configurable TTL for automatic file cleanup
• Upload Methods: POST multipart uploads, PUT uploads for legacy protocols, v3 protocol support
• Security: Virus scanning via ClamAV, configurable file extensions validation
• Performance: Chunked uploads and downloads, worker pool management with auto-scaling
• Storage Options: Local storage, ISO container mounting for specialized needs
• Monitoring: Prometheus metrics integration with detailed system and operation metrics
• Network Support: IPv4/IPv6 dual-stack support with protocol forcing options
• Configuration: Hot-reloading of logging settings via SIGHUP signal

Table of Contents

1. Installation
2. Configuration
3. Authentication
4. API Endpoints
5. Usage Examples
6. Setup
   • Reverse Proxy
   • Systemd Service
7. Building
8. Docker Support
9. Changelog
10. License

---

Installation

Quick Installation for XMPP Operators

The easiest way to install HMAC File Server is using the automated installer:

git clone https://github.com/PlusOne/hmac-file-server.git
cd hmac-file-server
sudo ./installer.sh

The installer will:

• Install Go 1.24 (if needed)
• Create system user and directories
• Build and configure the server
• Set up systemd service
• Optionally install Redis and ClamAV

For detailed installation instructions, see INSTALL.MD.

Manual Installation

Tip: You can also run HMAC File Server using Docker Compose for easy deployment. See the Wiki for Docker setup instructions. The official image is available at ghcr.io/plusone/hmac-file-server:latest.

Prerequisites

• Go 1.24 or higher
• Redis server (optional, for caching)
• ClamAV (optional, for virus scanning)

Steps

1. Clone the repository:

   git clone https://github.com/PlusOne/hmac-file-server.git
   cd hmac-file-server
   

2. Build the server:

   go build -o hmac-file-server ./cmd/server/main.go
   

3. Generate example configuration:

   ./hmac-file-server -genconfig
   # or write to file:
   ./hmac-file-server -genconfig-path config.toml
   

4. Create necessary directories:

   mkdir -p /path/to/hmac-file-server/data/
   mkdir -p /path/to/hmac-file-server/deduplication/
   

5. Edit your config.toml file with appropriate settings.

6. Start the server:

   ./hmac-file-server -config config.toml
   

---

Configuration

The server uses a comprehensive config.toml file with the following main sections:

Key Configuration Sections

• [server]: Basic server settings (port, storage, metrics)
• [security]: HMAC secrets, JWT configuration
• [uploads/downloads]: File handling settings, allowed extensions
• [logging]: Log levels, file rotation settings
• [clamav]: Virus scanning configuration
• [redis]: Cache and session management
• [workers]: Thread pool and performance tuning
• [iso]: ISO container mounting (specialized storage)
• [timeouts]: HTTP timeout configurations

Example Configuration

[server]
bind_ip = "0.0.0.0"
listenport = "8080"
unixsocket = false
storagepath = "./uploads"
metricsenabled = true
metricsport = "9090"
deduplicationenabled = true
filenaming = "HMAC"  # Options: "HMAC", "original", "None"
forceprotocol = "auto"  # Options: "ipv4", "ipv6", "auto"

[security]
secret = "your-secure-hmac-secret"
enablejwt = false
jwtsecret = "your-jwt-secret"
jwtalgorithm = "HS256"
jwtexpiration = "24h"

[uploads]
allowedextensions = [".txt", ".pdf", ".jpg", ".png", ".zip"]
chunkeduploadsenabled = true
chunksize = "10MB"

For complete configuration details, see the Wiki.

---

Authentication

The server supports multiple authentication methods:

1. HMAC Authentication (Default)

• Legacy v1: Basic HMAC with path + content length
• v2: Enhanced HMAC with content type validation
• Token: Alternative HMAC parameter name
• v3: mod_http_upload_external compatible with expiration

2. JWT Authentication

When enablejwt = true:

• Bearer tokens in Authorization header
• Query parameter token as fallback
• Configurable expiration and algorithm

Authentication Examples

# HMAC v2 upload
curl -X PUT "http://localhost:8080/myfile.txt?v2=HMAC_SIGNATURE" -d @file.txt

# JWT upload
curl -X POST -H "Authorization: Bearer JWT_TOKEN" -F 'file=@myfile.txt' http://localhost:8080/upload

# v3 protocol (mod_http_upload_external)
curl -X PUT "http://localhost:8080/upload/file.txt?v3=SIGNATURE&expires=TIMESTAMP" -d @file.txt

---

API Endpoints

Upload Endpoints

• POST /upload: Multipart form uploads (modern clients)
• PUT /{filename}: Direct uploads with HMAC or JWT authentication
• PUT with v3 protocol: mod_http_upload_external compatible uploads

Download Endpoints

• GET /{filename}: Direct file downloads
• HEAD /{filename}: File metadata (size, type)
• GET /download/{filename}: Alternative download endpoint

Management Endpoints

• GET /health: Health check endpoint for monitoring
• GET /metrics: Prometheus metrics (if enabled)
• Various helper endpoints: Defined in router setup

---

Usage Examples

Upload Examples

Multipart POST Upload

curl -X POST -F 'file=@example.jpg' \
  -H "X-Signature: HMAC_OF_PATH" \
  http://localhost:8080/upload

Legacy PUT Upload (v2)

# Calculate HMAC of: filename + "\x00" + content_length + "\x00" + content_type
curl -X PUT "http://localhost:8080/example.jpg?v2=CALCULATED_HMAC" \
  --data-binary @example.jpg

v3 Protocol Upload (mod_http_upload_external)

# HMAC of: "PUT\n{timestamp}\n/path/to/file"
curl -X PUT "http://localhost:8080/upload/file.txt?v3=SIGNATURE&expires=1234567890" \
  --data-binary @file.txt

JWT Upload

curl -X POST \
  -H "Authorization: Bearer YOUR_JWT_TOKEN" \
  -F 'file=@example.jpg' \
  http://localhost:8080/upload

Download Examples

Direct Download

curl http://localhost:8080/example.jpg -o downloaded_file.jpg

Get File Info

curl -I http://localhost:8080/example.jpg

Health Check

curl http://localhost:8080/health

---

Setup

Reverse Proxy

Nginx Configuration

server {
    listen 80;
    server_name your-domain.com;
    client_max_body_size 10G;  # Important for large uploads

    location / {
        proxy_pass http://localhost:8080;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
        
        # Timeout settings for large uploads
        proxy_read_timeout 300;
        proxy_connect_timeout 60;
        proxy_send_timeout 300;
    }
}

Apache2 Configuration

<VirtualHost *:80>
    ServerName your-domain.com
    
    ProxyPreserveHost On
    ProxyPass / http://localhost:8080/
    ProxyPassReverse / http://localhost:8080/
    
    # Large upload support
    LimitRequestBody 10737418240  # 10GB
    ProxyTimeout 300
</VirtualHost>

Systemd Service

[Unit]
Description=HMAC File Server
After=network.target redis.service

[Service]
Type=simple
ExecStart=/path/to/hmac-file-server -config /path/to/config.toml
ExecReload=/bin/kill -SIGHUP $MAINPID
WorkingDirectory=/path/to/hmac-file-server
Restart=always
RestartSec=10
User=hmac-server
Group=hmac-server

# Security settings
NoNewPrivileges=true
PrivateTmp=true
ProtectSystem=strict
ReadWritePaths=/path/to/uploads /path/to/logs

[Install]
WantedBy=multi-user.target

Enable and start:

sudo systemctl daemon-reload
sudo systemctl enable hmac-file-server
sudo systemctl start hmac-file-server

# Reload configuration (logging settings)
sudo systemctl reload hmac-file-server

---

Building

Local Build

go build -o hmac-file-server ./cmd/server/main.go

Cross-Platform Builds

# Linux amd64
GOOS=linux GOARCH=amd64 go build -o hmac-file-server-linux-amd64 ./cmd/server/main.go

# Linux arm64
GOOS=linux GOARCH=arm64 go build -o hmac-file-server-linux-arm64 ./cmd/server/main.go

# Windows
GOOS=windows GOARCH=amd64 go build -o hmac-file-server-windows-amd64.exe ./cmd/server/main.go

---

Docker Support

Quick Start with Docker Compose

version: '3.8'
services:
  hmac-file-server:
    image: ghcr.io/plusone/hmac-file-server:latest
    ports:
      - "8080:8080"
      - "9090:9090"  # Metrics
    volumes:
      - ./config:/etc/hmac-file-server
      - ./uploads:/opt/hmac-file-server/data/uploads
    environment:
      - CONFIG_PATH=/etc/hmac-file-server/config.toml
    restart: unless-stopped

Docker Build

docker build -t hmac-file-server .
docker run -p 8080:8080 -v $(pwd)/config.toml:/etc/hmac-file-server/config.toml hmac-file-server

See the Wiki for detailed Docker setup instructions.

---

Changelog

Version 3.2 (Stable)

• Development Version: Active development branch with latest features
• Enhanced Documentation: Updated comprehensive documentation and protocol specifications
• Configuration Improvements: Better configuration validation and structure
• Authentication System: Full JWT and multi-protocol HMAC support

Version 3.1-Stable (2025-06-08)

• v3 Protocol Support: Added mod_http_upload_external compatibility
• Enhanced Authentication: Improved HMAC validation with multiple protocol support
• JWT Integration: Complete JWT authentication system
• File Naming Options: HMAC-based or original filename preservation
• Network Improvements: IPv4/IPv6 dual-stack with protocol forcing

Version 3.0-Stable (2025-06-07)

• Docker Support: Official Docker images and compose files
• Go 1.24 Requirement: Updated minimum Go version
• Configuration Improvements: Better validation and hot-reloading
• Performance Enhancements: Worker auto-scaling and memory optimization

Previous Versions

See CHANGELOG.MD for complete version history.

---

License

MIT License

Copyright (c) 2025 Alexander Renz

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.