uuxo/hmac-file-server
1
1
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 7 Wiki Activity

release: hmac-file-server 3.2

Browse Source
This commit is contained in:
Alexander Renz
2025-06-11 10:17:33 +02:00
parent 969cb0cfea
commit 3f8bb3bf69
23 changed files with 3729 additions and 2526 deletions
Download Patch File Download Diff File Expand all files Collapse all files

7
.gitignore vendored
View File

@ -1,7 +0,0 @@
.vscode/
.vscode/*
!.vscode/settings.json
!.vscode/tasks.json
!.vscode/launch.json
cmd/monitor/hmac-monitor-1.0_amd64
cmd/monitor/hmac-monitor-1.0_arm64

40
CHANGELOG.md → CHANGELOG.MD
View File

@ -1,22 +1,48 @@
# Changelog
> **Note:** This file is a technical changelog for developers and maintainers. For user-focused highlights, migration notes, and upgrade instructions, see [RELEASE-NOTES.MD](./RELEASE-NOTES.MD).
> **Note:** This file is a technical changelog for developers and maintainers. For user-focused highlights, migration notes, and upgrade instructions, see [README.MD](./README.MD).
All notable changes to this project will be documented in this file.
h
## [3.2] - Stable Release
### Added (3.2)
- **Enhanced Documentation**: Comprehensive update of all documentation files to match current codebase
- **Protocol Specification Updates**: Detailed protocol documentation with implementation examples
- **Configuration Validation**: Improved configuration structure validation and error reporting
- **Developer Resources**: Updated build instructions and development setup guides
### Changed (3.2)
- **Documentation Structure**: Reorganized documentation for better clarity and maintenance
- **Configuration Examples**: Updated all configuration examples to reflect current options
- **API Documentation**: Enhanced API endpoint documentation with comprehensive examples
### Completed (3.2)
- **Feature Development**: Active development of new features and improvements
- **Testing Enhancements**: Expanded test coverage and validation
- **Performance Optimizations**: Ongoing performance improvements and monitoring
---
## [3.1-Stable] - 2025-06-08
### Added (3.1)
- **v3 (mod_http_upload_external) Support**: Implemented secure file uploads using HMAC-SHA256 validation and expiration checks.
- **GET URL Construction**: Automatically generates GET URLs for uploaded files, making them accessible for download.
- **v3 (mod_http_upload_external) Support**: Implemented secure file uploads using HMAC-SHA256 validation and expiration checks, specifically designed for Prosody's mod_http_upload_external compatibility.
- **JWT Authentication**: Complete JWT token authentication system with configurable algorithms and expiration times.
- **Multiple Authentication Protocols**: Support for legacy v1, enhanced v2, token-based, and v3 HMAC protocols alongside JWT authentication.
- **File Naming Strategy**: Configurable file naming options including HMAC-based, original filename preservation, or no specific naming convention.
- **Advanced Configuration Structure**: Comprehensive configuration sections including server, security, uploads, downloads, logging, deduplication, ISO, timeouts, versioning, ClamAV, Redis, and workers.
### Changed (3.1)
- Enhanced HMAC validation logic to support v3 alongside existing fallback mechanisms (v2 and v).
- Improved error handling for invalid or expired signatures during file uploads.
- **Enhanced HMAC Validation**: Improved validation logic to support multiple protocol versions (v1, v2, token, v3) with proper fallback mechanisms.
- **Authentication Priority**: Implemented authentication priority system with JWT taking precedence when enabled, falling back to HMAC protocols.
- **Network Protocol Support**: Enhanced IPv4/IPv6 dual-stack support with protocol forcing options (ipv4, ipv6, auto).
- **Configuration Hot-Reloading**: Added support for reloading logging configuration via SIGHUP signal without full server restart.
### Fixed (3.1)
- Addressed minor issues in file upload handling to ensure compatibility with v3 clients.
- **Protocol Compatibility**: Addressed compatibility issues with different HMAC protocol versions and mod_http_upload_external clients.
- **Error Handling**: Improved error handling for invalid or expired signatures during file uploads.
- **Configuration Validation**: Enhanced configuration validation to prevent common misconfigurations.
---

263
INSTALL.MD Normal file
View File

@ -0,0 +1,263 @@
# HMAC File Server 3.2 Installation Guide
## Quick Installation for XMPP Operators
The HMAC File Server includes an automated installer script designed specifically for XMPP operators who want to quickly deploy a file sharing service for their chat servers.
### Prerequisites
- Linux system with systemd (Ubuntu 18.04+, CentOS 7+, Debian 9+, etc.)
- Root or sudo access
- At least 1GB free disk space
- Internet connection for downloading dependencies
### Installation
1. **Download or clone the repository:**
```bash
git clone https://github.com/PlusOne/hmac-file-server.git
cd hmac-file-server
```
2. **Run the installer:**
```bash
sudo ./installer.sh
```
**Alternative: Pre-set secrets via environment variables:**
```bash
# For automation or if interactive input doesn't work
HMAC_SECRET='your-super-secret-hmac-key-here-minimum-32-characters' sudo -E ./installer.sh
# With both HMAC and JWT secrets
HMAC_SECRET='your-hmac-secret-32-chars-minimum' \
JWT_SECRET='your-jwt-secret-also-32-chars-minimum' \
sudo -E ./installer.sh
```
3. **Follow the interactive prompts:**
- System user (default: `hmac-server`)
- Installation directories
- Server ports
- **HMAC secret**: Choose automatic generation (recommended) or enter manually
- **Optional features** (JWT, Redis, ClamAV, SSL/TLS)
- **JWT secret**: Also supports automatic generation if enabled
### Configuration Options
#### Core Settings
- **Server Port**: Default 8080 (HTTP file server)
- **Metrics Port**: Default 9090 (Prometheus metrics)
- **HMAC Secret**: Strong secret for authentication
- **Automatic generation** (recommended): Creates 48-character secure random key
- **Manual entry**: Minimum 32 characters required
- **Environment variable**: `HMAC_SECRET='your-secret'`
#### Optional Features
- **JWT Authentication**: Token-based auth for enhanced security
- **Automatic generation** available for JWT secrets
- Configurable expiration and algorithms
- **Redis Integration**: For session management and caching
- **ClamAV Scanning**: Real-time virus scanning of uploaded files
- **SSL/TLS**: Direct HTTPS support (or use reverse proxy)
### XMPP Server Integration
#### Prosody Configuration
Add to your Prosody configuration:
```lua
Component "upload.yourdomain.com" "http_file_share"
http_file_share_url = "http://localhost:8080"
```
#### Ejabberd Configuration
Add to your Ejabberd configuration:
```yaml
mod_http_file_share:
external_secret: "your-hmac-secret"
service_url: "http://localhost:8080"
```
### Post-Installation
1. **Start the service:**
```bash
sudo systemctl start hmac-file-server
```
2. **Check status:**
```bash
sudo systemctl status hmac-file-server
```
3. **View logs:**
```bash
sudo journalctl -u hmac-file-server -f
```
4. **Configure firewall (required):**
```bash
# Example for ufw (Ubuntu/Debian)
sudo ufw allow 8080/tcp comment "HMAC File Server"
sudo ufw allow 9090/tcp comment "HMAC File Server Metrics"
# Example for firewalld (CentOS/RHEL/Fedora)
sudo firewall-cmd --permanent --add-port=8080/tcp
sudo firewall-cmd --permanent --add-port=9090/tcp
sudo firewall-cmd --reload
# Example for iptables (manual)
sudo iptables -A INPUT -p tcp --dport 8080 -j ACCEPT
sudo iptables -A INPUT -p tcp --dport 9090 -j ACCEPT
```
5. **Configure reverse proxy (recommended):**
```nginx
server {
listen 443 ssl http2;
server_name upload.yourdomain.com;
ssl_certificate /path/to/cert.pem;
ssl_certificate_key /path/to/key.pem;
location / {
proxy_pass http://localhost:8080;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
# File upload settings
client_max_body_size 100M;
proxy_request_buffering off;
}
}
```
### File Locations
After installation:
- **Binary**: `/opt/hmac-file-server/hmac-file-server`
- **Configuration**: `/etc/hmac-file-server/config.toml`
- **Uploads**: `/var/lib/hmac-file-server/uploads/`
- **Logs**: `/var/log/hmac-file-server/hmac-file-server.log`
### Management Commands
```bash
# Service management
sudo systemctl start hmac-file-server
sudo systemctl stop hmac-file-server
sudo systemctl restart hmac-file-server
sudo systemctl reload hmac-file-server
# View logs
sudo journalctl -u hmac-file-server -f
sudo tail -f /var/log/hmac-file-server/hmac-file-server.log
# Edit configuration
sudo nano /etc/hmac-file-server/config.toml
sudo systemctl reload hmac-file-server # Apply changes
```
### Uninstallation
To completely remove HMAC File Server:
```bash
sudo ./installer.sh --uninstall
```
**What gets removed:**
- ✓ Systemd service (stopped and disabled)
- ✓ Installation directory (`/opt/hmac-file-server/`)
- ✓ Configuration files (`/etc/hmac-file-server/`)
- ✓ All uploaded files (`/var/lib/hmac-file-server/`)
- ✓ Log files (`/var/log/hmac-file-server/`)
- ✓ System user (`hmac-server`)
- ✓ Any remaining binaries
**⚠️ Warning**: This will permanently remove all uploaded files and configuration!
### Security Considerations
1. **Configure firewall properly** - Only allow necessary ports (8080, 9090) to authorized networks
2. **Use strong HMAC secrets** (minimum 32 characters, use random generators)
3. **Enable JWT authentication** for enhanced security
4. **Set up SSL/TLS** either directly or via reverse proxy
5. **Enable ClamAV** for virus scanning if handling untrusted files
6. **Regular backups** of configuration and uploaded files
7. **Monitor logs** for suspicious activity
8. **Restrict network access** - Consider limiting access to internal networks only
### Monitoring
The server provides Prometheus metrics at `/metrics` endpoint:
```bash
curl http://localhost:9090/metrics
```
Key metrics to monitor:
- `hmac_requests_total` - Total requests
- `hmac_upload_size_bytes` - Upload sizes
- `hmac_errors_total` - Error counts
- `hmac_active_connections` - Active connections
### Troubleshooting
#### Service won't start
1. Check logs: `sudo journalctl -u hmac-file-server -f`
2. Verify configuration: `sudo nano /etc/hmac-file-server/config.toml`
3. Check permissions on data directories
4. Ensure ports are not in use: `sudo netstat -tlnp | grep :8080`
#### High memory usage
1. Adjust worker settings in configuration
2. Enable Redis for session management
3. Check for large file uploads in progress
#### Files not uploading
1. Verify HMAC secret matches between XMPP server and file server
2. Check file size limits in configuration
3. Ensure sufficient disk space
4. Review ClamAV logs if virus scanning enabled
### Support
- **Documentation**: See `README.MD` and `WIKI.MD`
- **Protocol Details**: See `PROTOCOL_SPECIFICATIONS.MD`
- **Issues**: GitHub issue tracker
- **Configuration**: All options documented in `WIKI.MD`
### Example Production Setup
For a production XMPP server with 1000+ users:
```toml
[server]
listenport = "8080"
metricsenabled = true
deduplicationenabled = true
[security]
enablejwt = true
# Strong secrets here
[uploads]
maxfilesize = "50MB"
ttlenabled = true
ttl = "720h" # 30 days
[workers]
max = 200
autoscaling = true
[redis]
enabled = true
host = "localhost"
port = 6379
[clamav]
enabled = true
```
This setup provides robust file sharing with deduplication, automatic cleanup, virus scanning, and scalable worker management.

21
LICENSE
View File

@ -1,21 +0,0 @@
MIT License
Copyright (c) 2025 Alexander Renz
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.

358
PROTOCOL_SPECIFICATIONS.MD
View File

@ -1,123 +1,295 @@
# HMAC File Server Authentication Protocol Specifications
This document outlines the different versions of the HMAC-based authentication protocols used by the HMAC File Server for secure file uploads.
This document outlines the different authentication protocols supported by the HMAC File Server for secure file uploads and downloads. The server supports multiple authentication methods to ensure backward compatibility while providing enhanced security features.
## Introduction
## Overview
The server supports multiple versions of the HMAC authentication protocol to ensure backward compatibility while introducing enhanced security features in newer versions. The primary goal is to verify the authenticity and integrity of upload requests.
The HMAC File Server supports two primary authentication mechanisms:
1. **HMAC-based Authentication** (Multiple versions: v1, v2, token, v3)
2. **JWT Authentication** (Bearer tokens)
## Common Principles
- **HMAC Algorithm**: SHA256 is typically used unless specified otherwise.
- **Secret Key**: A shared secret key, configured on the server, is used to generate and verify HMAC signatures.
- **URL Parameters**: Authentication details are usually passed via URL query parameters.
All protocols use SHA256 hashing and require a shared secret key configured on the server.
---
## Protocol Versions
## HMAC Authentication Protocols
### Version 1 (Legacy - `v`)
**Overview:**
This is the initial version of the HMAC authentication.
**URL Parameters:**
- `v`: Specifies the version (e.g., `v=1` or simply `v`).
- `mac`: The HMAC signature.
- `expires`: Timestamp indicating when the signature expires.
**Signature Calculation:**
The message for HMAC calculation is constructed by concatenating the following, separated by a newline character (`\\n`):
1. The HTTP Method (e.g., "PUT", "POST")
2. The `expires` timestamp from the URL query.
3. The request URI (path and query, excluding the `mac` and `v` parameters themselves).
Example:
```
POST
1717804800
/upload/somefile.txt?expires=1717804800&user=john
```
**Verification Steps:**
1. Extract `v`, `mac`, and `expires` from the URL.
2. Check if `expires` is in the future.
3. Reconstruct the message string using request parameters.
4. Calculate the HMAC signature using the shared secret and the reconstructed message.
5. Compare the calculated signature with the provided `mac`.
### Common Elements
- **Algorithm**: HMAC-SHA256
- **Secret**: Shared secret key configured in `[security]` section
- **Transport**: URL query parameters for HMAC, headers for signatures
- **Encoding**: Hexadecimal encoding for HMAC values
---
### Version 2 (`v2`)
### Legacy v1 Protocol (`v` parameter)
**Overview:**
Version 2 enhances the signature by including the content SHA256 hash in the signed message, ensuring data integrity along with authenticity.
**Overview**: The original HMAC authentication protocol.
**URL Parameters:**
- `v2`: The HMAC-SHA256 signature for this version.
- `expires`: Timestamp indicating when the signature expires.
- `sha256`: The hex-encoded SHA256 hash of the file content being uploaded.
**Signature Calculation:**
The message for HMAC calculation is constructed by concatenating the following, separated by a newline character (`\\n`):
1. The HTTP Method (e.g., "PUT", "POST")
2. The `expires` timestamp from the URL query.
3. The `sha256` hash from the URL query.
4. The request URI (path and query, excluding the `v2` parameter itself).
Example:
**URL Format**:
```
POST
1717804800
a1b2c3d4...e5f6 (SHA256 hash of file content)
/upload/somefile.txt?expires=1717804800&sha256=a1b2c3d4...e5f6&user=jane
PUT /filename.ext?v=HMAC_SIGNATURE
```
**Verification Steps:**
1. Extract `v2` (signature), `expires`, and `sha256` from the URL.
2. Check if `expires` is in the future.
3. Reconstruct the message string according to v2 specification.
4. Calculate the HMAC-SHA256 signature using the shared secret.
5. Compare the calculated signature with the provided `v2` signature.
6. After successful HMAC verification, the server *must* also verify that the SHA256 hash of the received file content matches the `sha256` parameter from the URL.
**Message Construction**:
```
fileStorePath + "\x20" + contentLength
```
**Example**:
```bash
# For file "test.txt" with 1024 bytes
# Message: "test.txt\x201024"
curl -X PUT "http://localhost:8080/test.txt?v=a1b2c3d4..." --data-binary @test.txt
```
**Implementation Notes**:
- Uses space character (`\x20`) as separator
- Content-Length header must be accurate
- Simplest protocol, minimal metadata validation
---
### Version 3 (`v3` - mod_http_upload_external compatible)
### Enhanced v2 Protocol (`v2` parameter)
**Overview:**
Version 3 is specifically designed for compatibility with Prosody's `mod_http_upload_external`. It signs the HTTP method, the `expires` timestamp, and the full request path.
**Overview**: Enhanced version including content type validation.
**URL Parameters:**
- `v3`: The HMAC-SHA256 signature.
- `expires`: Timestamp (seconds since epoch) indicating when the signature expires.
**Signature Calculation (Example for `mod_http_upload_external`):**
The message to be signed is constructed by concatenating the following, separated by a newline character (`\\n`):
1. The HTTP Method (e.g., "PUT", "POST").
2. The `expires` timestamp from the URL query.
3. The request path (e.g., `/upload/myfile.dat`). Note: This is just the path, not the full URI with query parameters.
Example:
**URL Format**:
```
PUT
1717804800
/upload/myfile.dat
PUT /filename.ext?v2=HMAC_SIGNATURE
```
**Verification Steps:**
1. Extract `v3` (signature) and `expires` from the URL.
2. Verify that the `expires` timestamp is valid and not in the past.
3. Reconstruct the exact string that was signed on the client-side: `METHOD\\nEXPIRES_TIMESTAMP\\nREQUEST_PATH`.
4. Calculate the HMAC-SHA256 signature of the reconstructed string using the server's shared secret key.
5. Compare the calculated signature with the `v3` signature provided in the request. If they match, the request is considered authentic.
**Message Construction**:
```
fileStorePath + "\x00" + contentLength + "\x00" + contentType
```
**Example**:
```bash
# For file "document.pdf" with 2048 bytes
# Message: "document.pdf\x002048\x00application/pdf"
curl -X PUT "http://localhost:8080/document.pdf?v2=e5f6g7h8..." --data-binary @document.pdf
```
**Implementation Notes**:
- Uses null characters (`\x00`) as separators
- Content-Type automatically detected from file extension
- Fallback to "application/octet-stream" for unknown extensions
---
## Notes for Developers
### Token Protocol (`token` parameter)
- Ensure the client and server use the exact same method for constructing the message string to be HMACed.
- The `expires` parameter is crucial for preventing replay attacks.
- The shared secret key must be kept confidential.
**Overview**: Alternative parameter name for v2-style authentication.
*(This document will be updated as more specific details are extracted from the source code.)*
**URL Format**:
```
PUT /filename.ext?token=HMAC_SIGNATURE
```
**Message Construction**: Same as v2 protocol
```
fileStorePath + "\x00" + contentLength + "\x00" + contentType
```
**Example**:
```bash
curl -X PUT "http://localhost:8080/image.jpg?token=i9j0k1l2..." --data-binary @image.jpg
```
**Implementation Notes**:
- Identical to v2 protocol but uses `token` parameter
- Useful for clients that prefer different parameter naming
---
### v3 Protocol - mod_http_upload_external Compatible (`v3` parameter)
**Overview**: Specifically designed for Prosody's `mod_http_upload_external` compatibility with expiration support.
**URL Format**:
```
PUT /path/to/file.ext?v3=HMAC_SIGNATURE&expires=UNIX_TIMESTAMP
```
**Message Construction**:
```
METHOD + "\n" + expires + "\n" + requestPath
```
**Example**:
```bash
# Current timestamp: 1717804800
# Message: "PUT\n1717804800\n/upload/myfile.txt"
curl -X PUT "http://localhost:8080/upload/myfile.txt?v3=m3n4o5p6...&expires=1717804800" --data-binary @myfile.txt
```
**Verification Process**:
1. Extract `v3` signature and `expires` timestamp
2. Validate `expires` is in the future
3. Construct message: `"{METHOD}\n{expires}\n{path}"`
4. Calculate HMAC-SHA256 of message
5. Compare with provided signature
**Implementation Notes**:
- Includes expiration timestamp validation
- Prevents replay attacks through time-based validation
- Path-only signing (no query parameters in signed message)
- HTTP method is part of the signed message
---
## JWT Authentication
**Overview**: Token-based authentication using JSON Web Tokens.
**Configuration**:
```toml
[security]
enablejwt = true
jwtsecret = "your-256-bit-secret"
jwtalgorithm = "HS256"
jwtexpiration = "24h"
```
**Header Format**:
```
Authorization: Bearer JWT_TOKEN
```
**Fallback Query Parameter**:
```
GET /file.txt?token=JWT_TOKEN
```
**Example Usage**:
```bash
# Header-based JWT
curl -H "Authorization: Bearer eyJhbGciOiJIUzI1NiIs..." http://localhost:8080/file.txt
# Query parameter fallback
curl "http://localhost:8080/file.txt?token=eyJhbGciOiJIUzI1NiIs..."
```
**JWT Claims**: Standard JWT claims (exp, iat, iss, etc.) as configured.
---
## POST Upload Authentication
### X-Signature Header Method
**Overview**: For multipart form uploads via POST requests.
**Header Format**:
```
X-Signature: HMAC_OF_REQUEST_PATH
```
**Message Construction**:
```
requestPath (e.g., "/upload")
```
**Example**:
```bash
# HMAC of "/upload"
curl -X POST \
-H "X-Signature: CALCULATED_HMAC" \
-F 'file=@myfile.txt' \
http://localhost:8080/upload
```
---
## Authentication Priority and Fallbacks
The server checks authentication in the following order:
1. **JWT Authentication** (if `enablejwt = true`)
- Authorization header (Bearer token)
- Query parameter `token`
2. **HMAC Authentication** (if JWT disabled or not found)
- X-Signature header (for POST uploads)
- v3 protocol (with expires validation)
- v2 protocol
- token protocol
- v1 protocol (legacy)
---
## Security Considerations
### HMAC Protocols
- **Secret Management**: Use strong, randomly generated secrets
- **Time Validation**: v3 protocol includes expiration to prevent replay attacks
- **Content Validation**: v2/token protocols include content-type validation
- **Path Sanitization**: All protocols validate and sanitize file paths
### JWT Authentication
- **Token Expiration**: Configure appropriate expiration times
- **Secret Rotation**: Regularly rotate JWT signing keys
- **Algorithm Security**: Default HS256 is secure for most use cases
- **Transport Security**: Always use HTTPS in production
### General Security
- **HTTPS Only**: Use TLS encryption for all production deployments
- **Rate Limiting**: Implement reverse proxy rate limiting
- **File Validation**: Configure allowed file extensions
- **Virus Scanning**: Enable ClamAV integration for malware detection
- **Access Logs**: Monitor authentication failures and suspicious activity
---
## Migration Guide
### From v1 to v2
- Update HMAC calculation to include content type
- Change separator from `\x20` to `\x00`
- No breaking changes in URL structure
### From HMAC to JWT
- Set `enablejwt = true` in configuration
- Generate JWT tokens server-side or use external auth provider
- HMAC authentication remains available as fallback
### Adding v3 Support
- Implement expiration timestamp generation
- Update HMAC calculation to include HTTP method and expiration
- Useful for mod_http_upload_external compatibility
---
## Example Implementations
### Client-Side HMAC Generation (Python)
```python
import hmac
import hashlib
import time
def generate_v3_signature(secret, method, expires, path):
message = f"{method}\n{expires}\n{path}"
signature = hmac.new(
secret.encode(),
message.encode(),
hashlib.sha256
).hexdigest()
return signature
# Example usage
secret = "your-hmac-secret"
expires = int(time.time()) + 3600 # 1 hour from now
signature = generate_v3_signature(secret, "PUT", expires, "/upload/file.txt")
```
### Server-Side Validation (Reference)
See the main.go file for complete implementation details of all validation functions:
- `validateHMAC()`: Legacy v1, v2, and token protocols
- `validateV3HMAC()`: v3 protocol with expiration
- `validateJWTFromRequest()`: JWT validation
---
This specification ensures consistent implementation across clients and provides multiple authentication options for different use cases and security requirements.

407
README.MD
View File

@ -1,44 +1,70 @@
# HMAC File Server 3.1-Stable
## Overview
The **HMAC File Server** ensures secure file uploads and downloads using HMAC authentication. It incorporates rate limiting, CORS support, retries, file versioning, and Unix socket support for enhanced flexibility. Redis integration provides efficient caching and session management. Prometheus metrics and a graceful shutdown mechanism ensure reliable and efficient file handling.
# HMAC File Server 3.2
Special thanks to **Thomas Leister** for inspiration drawn from [[prosody-filer](https://github.com/ThomasLeister/prosody-filer)](https://github.com/ThomasLeister/prosody-filer).
## Overview
The **HMAC File Server** ensures secure file uploads and downloads using HMAC authentication and JWT tokens. It incorporates comprehensive security features, file versioning, deduplication, ISO container support, virus scanning, and Unix socket support for enhanced flexibility. Redis integration provides efficient caching and session management. Prometheus metrics and graceful shutdown mechanisms ensure reliable and efficient file handling.
Special thanks to **Thomas Leister** for inspiration drawn from [prosody-filer](https://github.com/ThomasLeister/prosody-filer).
## Features
- File deduplication
- Configurable TTL for automatic file cleanup
- Secure HMAC-based authentication
- Chunked uploads and downloads
- Virus scanning via ClamAV
- Prometheus metrics integration
- Customizable worker management
- Support ISO-based storage
- Support for mod_http_upload_external (v3) for secure file uploads with HMAC-SHA256 validation and expiration checks
- **Configuration Hot-Reloading**: Allows certain configuration changes (currently logging settings) to be applied without restarting the server by sending a `SIGHUP` signal.
- **Multiple Authentication Methods**: HMAC-based authentication and JWT token support
- **Multiple Protocol Support**: v1 (legacy), v2, v3 (mod_http_upload_external), and token-based uploads
- **File Management**: Deduplication, configurable TTL for automatic file cleanup
- **Upload Methods**: POST multipart uploads, PUT uploads for legacy protocols, v3 protocol support
- **Security**: Virus scanning via ClamAV, configurable file extensions validation
- **Performance**: Chunked uploads and downloads, worker pool management with auto-scaling
- **Storage Options**: Local storage, ISO container mounting for specialized needs
- **Monitoring**: Prometheus metrics integration with detailed system and operation metrics
- **Network Support**: IPv4/IPv6 dual-stack support with protocol forcing options
- **Configuration**: Hot-reloading of logging settings via SIGHUP signal
## Table of Contents
1. [Installation](#installation)
2. [Configuration](#configuration)
3. [Usage](#usage)
4. [Setup](#setup)
3. [Authentication](#authentication)
4. [API Endpoints](#api-endpoints)
5. [Usage Examples](#usage-examples)
6. [Setup](#setup)
- [Reverse Proxy](#reverse-proxy)
- [Systemd Service](#systemd-service)
5. [Building](#building)
6. [Changelog](#changelog)
7. [License](#license)
7. [Building](#building)
8. [Docker Support](#docker-support)
9. [Changelog](#changelog)
10. [License](#license)
---
## Installation
> **Tip:** You can also run HMAC File Server using Docker Compose for easy deployment. See `dockerenv/docker-compose.yml` for a ready-to-use example. The official image is available at `ghcr.io/plusone/hmac-file-server:latest`.
### Quick Installation for XMPP Operators
### Prerequisites
The easiest way to install HMAC File Server is using the automated installer:
```bash
git clone https://github.com/PlusOne/hmac-file-server.git
cd hmac-file-server
sudo ./installer.sh
```
The installer will:
- Install Go 1.24 (if needed)
- Create system user and directories
- Build and configure the server
- Set up systemd service
- Configure firewall rules
- Optionally install Redis and ClamAV
For detailed installation instructions, see [INSTALL.MD](INSTALL.MD).
### Manual Installation
> **Tip:** You can also run HMAC File Server using Docker Compose for easy deployment. See the Wiki for Docker setup instructions. The official image is available at `ghcr.io/plusone/hmac-file-server:latest`.
#### Prerequisites
- Go **1.24** or higher
- Redis server (optional, for caching)
- ClamAV (optional, for virus scanning)
### Steps
#### Steps
1. Clone the repository:
```bash
git clone https://github.com/PlusOne/hmac-file-server.git
@ -47,22 +73,25 @@ Special thanks to **Thomas Leister** for inspiration drawn from [[prosody-filer]
2. Build the server:
```bash
go build -o hmac-file-server
go build -o hmac-file-server ./cmd/server/main.go
```
3. Create necessary directories:
3. Generate example configuration:
```bash
./hmac-file-server -genconfig
# or write to file:
./hmac-file-server -genconfig-path config.toml
```
4. Create necessary directories:
```bash
mkdir -p /path/to/hmac-file-server/data/
mkdir -p /path/to/hmac-file-server/deduplication/
mkdir -p /path/to/hmac-file-server/iso/
```
4. Copy and edit the configuration file:
```bash
cp config.example.toml config.toml
```
5. Edit your `config.toml` file with appropriate settings.
5. Start the server:
6. Start the server:
```bash
./hmac-file-server -config config.toml
```
@ -70,38 +99,142 @@ Special thanks to **Thomas Leister** for inspiration drawn from [[prosody-filer]
---
## Configuration
The server is configured via a `config.toml` file. Key settings include:
- **Server Settings**: Port, logging, metrics
- **Security**: HMAC secret, TLS options
- **File Management**: TTL, deduplication, uploads, and downloads
- **ISO**: Generation and mounting settings
- **Workers**: Adjust thread management
The server uses a comprehensive `config.toml` file with the following main sections:
For detailed configuration options, refer to the [Wiki](./wiki.md).
### Key Configuration Sections
- **[server]**: Basic server settings (port, storage, metrics)
- **[security]**: HMAC secrets, JWT configuration
- **[uploads/downloads]**: File handling settings, allowed extensions
- **[logging]**: Log levels, file rotation settings
- **[clamav]**: Virus scanning configuration
- **[redis]**: Cache and session management
- **[workers]**: Thread pool and performance tuning
- **[iso]**: ISO container mounting (specialized storage)
- **[timeouts]**: HTTP timeout configurations
### Example Configuration
```toml
[server]
bind_ip = "0.0.0.0"
listenport = "8080"
unixsocket = false
storagepath = "./uploads"
metricsenabled = true
metricsport = "9090"
deduplicationenabled = true
filenaming = "HMAC" # Options: "HMAC", "original", "None"
forceprotocol = "auto" # Options: "ipv4", "ipv6", "auto"
[security]
secret = "your-secure-hmac-secret"
enablejwt = false
jwtsecret = "your-jwt-secret"
jwtalgorithm = "HS256"
jwtexpiration = "24h"
[uploads]
allowedextensions = [".txt", ".pdf", ".jpg", ".png", ".zip"]
chunkeduploadsenabled = true
chunksize = "10MB"
```
For complete configuration details, see the [Wiki](./WIKI.MD).
---
## Usage
Start the server and access it on the configured port. Use curl or a client library to interact with the API.
## Authentication
### Examples
The server supports multiple authentication methods:
### 1. HMAC Authentication (Default)
- **Legacy v1**: Basic HMAC with path + content length
- **v2**: Enhanced HMAC with content type validation
- **Token**: Alternative HMAC parameter name
- **v3**: mod_http_upload_external compatible with expiration
### 2. JWT Authentication
When `enablejwt = true`:
- Bearer tokens in Authorization header
- Query parameter `token` as fallback
- Configurable expiration and algorithm
### Authentication Examples
#### Upload a file (Multipart POST)
This example shows a basic file upload using a multipart POST request. This method might be suitable for simpler scenarios or internal use where HMAC protection on the URL itself is not strictly required for this specific endpoint.
```bash
curl -X POST -F 'file=@example.jpg' http://localhost:8080/upload
```
**Note:** For uploads to specific paths requiring HMAC signature validation (e.g., for `mod_http_upload_external` compatibility or general secure uploads), the server typically expects `PUT` requests with appropriate HMAC parameters in the URL query string. Consult the server's detailed API documentation or `config.toml` for specifics on enabling and using HMAC-protected uploads.
# HMAC v2 upload
curl -X PUT "http://localhost:8080/myfile.txt?v2=HMAC_SIGNATURE" -d @file.txt
#### Download a file
To download a file, assuming `example.jpg` exists at the root of the server's configured storage path:
```bash
curl http://localhost:8080/example.jpg -o downloaded_example.jpg
# JWT upload
curl -X POST -H "Authorization: Bearer JWT_TOKEN" -F 'file=@myfile.txt' http://localhost:8080/upload
# v3 protocol (mod_http_upload_external)
curl -X PUT "http://localhost:8080/upload/file.txt?v3=SIGNATURE&expires=TIMESTAMP" -d @file.txt
```
#### Get file information (HEAD request)
To retrieve metadata about a file (like `Content-Length` and `Content-Type`) without downloading its content, use a `HEAD` request:
---
## API Endpoints
### Upload Endpoints
- **POST /upload**: Multipart form uploads (modern clients)
- **PUT /{filename}**: Direct uploads with HMAC or JWT authentication
- **PUT with v3 protocol**: mod_http_upload_external compatible uploads
### Download Endpoints
- **GET /{filename}**: Direct file downloads
- **HEAD /{filename}**: File metadata (size, type)
- **GET /download/{filename}**: Alternative download endpoint
### Management Endpoints
- **GET /metrics**: Prometheus metrics (if enabled)
- **Various helper endpoints**: Defined in router setup
---
## Usage Examples
### Upload Examples
#### Multipart POST Upload
```bash
curl -X POST -F 'file=@example.jpg' \
-H "X-Signature: HMAC_OF_PATH" \
http://localhost:8080/upload
```
#### Legacy PUT Upload (v2)
```bash
# Calculate HMAC of: filename + "\x00" + content_length + "\x00" + content_type
curl -X PUT "http://localhost:8080/example.jpg?v2=CALCULATED_HMAC" \
--data-binary @example.jpg
```
#### v3 Protocol Upload (mod_http_upload_external)
```bash
# HMAC of: "PUT\n{timestamp}\n/path/to/file"
curl -X PUT "http://localhost:8080/upload/file.txt?v3=SIGNATURE&expires=1234567890" \
--data-binary @file.txt
```
#### JWT Upload
```bash
curl -X POST \
-H "Authorization: Bearer YOUR_JWT_TOKEN" \
-F 'file=@example.jpg' \
http://localhost:8080/upload
```
### Download Examples
#### Direct Download
```bash
curl http://localhost:8080/example.jpg -o downloaded_file.jpg
```
#### Get File Info
```bash
curl -I http://localhost:8080/example.jpg
```
@ -111,116 +244,156 @@ curl -I http://localhost:8080/example.jpg
## Setup
### Reverse Proxy
Set up a reverse proxy using Apache2 or Nginx to handle requests.
#### Apache2 Example
```apache
<VirtualHost *:80>
ServerName your-domain.com
ProxyPreserveHost On
ProxyPass / http://localhost:8080/
ProxyPassReverse / http://localhost:8080/
</VirtualHost>
```
#### Nginx Example
#### Nginx Configuration
```nginx
server {
listen 80;
server_name your-domain.com;
client_max_body_size 10G; # Important for large uploads
location / {
proxy_pass http://localhost:8080;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
# Timeout settings for large uploads
proxy_read_timeout 300;
proxy_connect_timeout 60;
proxy_send_timeout 300;
}
}
```
#### Apache2 Configuration
```apache
<VirtualHost *:80>
ServerName your-domain.com
ProxyPreserveHost On
ProxyPass / http://localhost:8080/
ProxyPassReverse / http://localhost:8080/
# Large upload support
LimitRequestBody 10737418240 # 10GB
ProxyTimeout 300
</VirtualHost>
```
### Systemd Service
Create a systemd service file for the HMAC File Server:
```ini
[Unit]
Description=HMAC File Server
After=network.target
After=network.target redis.service
[Service]
Type=simple
ExecStart=/path/to/hmac-file-server -config /path/to/config.toml
ExecReload=/bin/kill -SIGHUP $MAINPID
WorkingDirectory=/path/to/hmac-file-server
Restart=always
User=www-data
Group=www-data
RestartSec=10
User=hmac-server
Group=hmac-server
# Security settings
NoNewPrivileges=true
PrivateTmp=true
ProtectSystem=strict
ReadWritePaths=/path/to/uploads /path/to/logs
[Install]
WantedBy=multi-user.target
```
Enable and start the service:
Enable and start:
```bash
sudo systemctl daemon-reload
sudo systemctl enable hmac-file-server
sudo systemctl start hmac-file-server
# Reload configuration (logging settings)
sudo systemctl reload hmac-file-server
```
---
## Building
To build for different architectures:
- **Linux (amd64)**:
```bash
GOOS=linux GOARCH=amd64 go build -o hmac-file-server main.go
```
### Local Build
```bash
go build -o hmac-file-server ./cmd/server/main.go
```
- **Linux (arm64)**:
```bash
GOOS=linux GOARCH=arm64 go build -o hmac-file-server main.go
```
### Cross-Platform Builds
```bash
# Linux amd64
GOOS=linux GOARCH=amd64 go build -o hmac-file-server-linux-amd64 ./cmd/server/main.go
# Linux arm64
GOOS=linux GOARCH=arm64 go build -o hmac-file-server-linux-arm64 ./cmd/server/main.go
# Windows
GOOS=windows GOARCH=amd64 go build -o hmac-file-server-windows-amd64.exe ./cmd/server/main.go
```
---
## Docker Support
### Quick Start with Docker Compose
```yaml
version: '3.8'
services:
hmac-file-server:
image: ghcr.io/plusone/hmac-file-server:latest
ports:
- "8080:8080"
- "9090:9090" # Metrics
volumes:
- ./config:/etc/hmac-file-server
- ./uploads:/opt/hmac-file-server/data/uploads
environment:
- CONFIG_PATH=/etc/hmac-file-server/config.toml
restart: unless-stopped
```
### Docker Build
```bash
docker build -t hmac-file-server .
docker run -p 8080:8080 -v $(pwd)/config.toml:/etc/hmac-file-server/config.toml hmac-file-server
```
See the Wiki for detailed Docker setup instructions.
---
## Changelog
### Added
- **Deduplication Support:** Automatically remove duplicate files based on SHA256 hashing to save storage space.
- **ISO Container Management:** Create and mount ISO containers for specialized storage needs, enhancing flexibility in file management.
- **Prometheus Metrics Enhancements:** Added detailed metrics for deduplication and ISO container operations to improve monitoring and observability.
- **Redis Integration Improvements:** Enhanced caching mechanisms using Redis for faster access to file metadata and application states.
- **Precaching Feature:** Implemented precaching of file structures on startup to reduce access times for frequently used files.
- **Configuration Options:** Updated `config.toml` to include new settings for deduplication, ISO management, and worker scaling.
### Version 3.2 (Stable)
- **Development Version**: Active development branch with latest features
- **Enhanced Documentation**: Updated comprehensive documentation and protocol specifications
- **Configuration Improvements**: Better configuration validation and structure
- **Authentication System**: Full JWT and multi-protocol HMAC support
### Changed
- **Worker Pool Scaling:** Implemented dynamic adjustment of worker threads based on system resources to optimize performance.
- **Logging Enhancements:** Improved logging for file operations, including detailed information on file extensions and MIME types during uploads.
- **Temporary Path Configuration:** Replaced hardcoded temporary upload directories with a configurable `TempPath` parameter in `config.toml` for greater flexibility.
### Version 3.1-Stable (2025-06-08)
- **v3 Protocol Support**: Added mod_http_upload_external compatibility
- **Enhanced Authentication**: Improved HMAC validation with multiple protocol support
- **JWT Integration**: Complete JWT authentication system
- **File Naming Options**: HMAC-based or original filename preservation
- **Network Improvements**: IPv4/IPv6 dual-stack with protocol forcing
### Fixed
- **Temporary File Handling:** Resolved issues where temporary `.tmp` files caused "Unsupported file type" warnings by enhancing MIME type detection logic.
- **MIME Type Detection:** Improved MIME type detection to ensure better compatibility and accuracy during file uploads.
### Version 3.0-Stable (2025-06-07)
- **Docker Support**: Official Docker images and compose files
- **Go 1.24 Requirement**: Updated minimum Go version
- **Configuration Improvements**: Better validation and hot-reloading
- **Performance Enhancements**: Worker auto-scaling and memory optimization
### Deprecated
- **Thumbnail Support (Previous Implementation):** Dropped the previous thumbnail support mechanism. This feature will not return in future releases.
---
## Version 3.1 - 2025-06-08
### Features
- **v3 (mod_http_upload_external) Support**: Added support for secure file uploads using HMAC-SHA256 validation and expiration checks. This feature ensures that Prosody clients can upload files securely with signed URLs.
- **GET URL Construction**: Automatically generates GET URLs for uploaded files, making them accessible for download.
### Improvements
- Enhanced HMAC validation logic to support v3 alongside existing fallback mechanisms (v2 and v).
- Improved error handling for invalid or expired signatures during file uploads.
### Notes
- This release introduces significant improvements for secure file uploads, ensuring compatibility with mod_http_upload_external.
- Configuration Hot-Reloading: To reload the configuration (currently only logging settings), send a `SIGHUP` signal to the server process (e.g., `kill -SIGHUP <pid>`).
---
**Important Update:**
- The minimum Go version required is now **1.24**. Please ensure your environment meets this requirement for successful compilation.
### Previous Versions
See [CHANGELOG.MD](./CHANGELOG.MD) for complete version history.
---
@ -230,9 +403,7 @@ MIT License
Copyright (c) 2025 Alexander Renz
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

99
RELEASE-NOTES.MD
View File

@ -1,99 +0,0 @@
# Release Notes for HMAC File Server 3.1
> **Note:** This file summarizes user-facing changes, migration notes, and upgrade instructions. For a detailed technical changelog, see [CHANGELOG.md](./CHANGELOG.md).
## [3.1] - 2025-06-08
### Added
- **v3 (mod_http_upload_external) Support**: Implemented secure file uploads using HMAC-SHA256 validation and expiration checks. This feature enables Prosody clients to upload files securely with signed URLs.
- **GET URL Construction**: Automatically generates GET URLs for uploaded files, making them accessible for download.
### Changed
- Enhanced HMAC validation logic to support v3 alongside existing fallback mechanisms (v2 and v).
- Improved error handling for invalid or expired signatures during file uploads.
- Updated version strings from `2.9-Stable` to `3.1-Stable` in the codebase.
### Fixed
- Addressed minor issues in file upload handling to ensure compatibility with v3 clients.
---
# Release Notes for HMAC File Server 3.0
> **Note:** This file summarizes user-facing changes, migration notes, and upgrade instructions. For a detailed technical changelog, see [CHANGELOG.md](./CHANGELOG.md).
## [3.0] - 2025-06-05
### Added
- `--genconfig` and `--genconfig-path` flags to generate example configurations on demand.
- Robust dual-stack HTTP client with idle connection timeouts and keep-alive settings for seamless IPv4/IPv6 support.
- Automated end-to-end tests for `--genconfig` generation and IPv4/IPv6 startup behavior.
- Proxy best practices and recommendations (timeouts, buffer sizes, security headers) in documentation.
- Enhanced startup logging of `BindIP`, `FileNaming`, and `ForceProtocol` settings.
### Changed
- Consolidated all configuration structs into a single `Config` type; removed duplicated and unused fields.
- Improved `parseSize` and `parseTTL` functions with strict unit parsing and error handling.
- Fixed Markdown lint issues in `CHANGELOG.md` and `wiki.md` for consistent headings and formatting.
- Refined PID file write/remove logic and added graceful shutdown logging enhancements.
### Fixed
- Ensured seamless IPv4/IPv6 dual-stack handling with appropriate timeouts and error control.
- Validated `--config` flag usage with clear error messages and fallback behavior.
- Resolved edge cases in network protocol initialization preventing unexpected errors.
---
# Release Notes for HMAC File Server 2.7-Stable
## Summary
Version 2.6-Stable focuses on improving the overall stability and performance of the HMAC File Server. Significant changes have been made to prioritize reliability and scalability for production environments.
## Key Changes
### Breaking Changes
- **Thumbnail Generation Dropped**: Support for automatic thumbnail generation has been removed in this release. This decision was made to enhance system stability and reduce resource consumption. Users requiring thumbnails are encouraged to use external tools.
### New Features
- **ISO-Based Storage Support**: Introduced support for ISO-based storage to accommodate specialized use cases.
- **Enhanced ClamAV Integration**: Improved ClamAV scanning with concurrent workers, providing better performance for large-scale deployments.
- **Timeout Configuration**: Added granular timeout settings for read, write, and idle connections, improving connection management.
- **FileNaming Configuration**: Added support for a "None" option in the `FileNaming` configuration. When set to "None", the filename remains unchanged.
- **Example Configuration Generation**: If no configuration file is found, the server will output an example configuration for the user to copy and paste.
- **Prometheus Metrics**: Enhanced Prometheus metrics for better monitoring and performance tracking. New metrics include upload and download durations, error counts, memory and CPU usage, and more.
### Improvements
- **Worker Management**: Auto-scaling worker threads based on system load for optimal performance.
- **Logging Enhancements**: Improved log verbosity control, making debugging and monitoring easier.
### Bug Fixes
- Resolved minor issues affecting deduplication and file upload performance.
- Fixed a rare crash scenario during high-concurrency file uploads.
## Migration Notes
1. **Thumbnail Settings**: Remove `[thumbnails]` configuration blocks from your `config.toml` file to avoid errors.
2. **Updated Configuration**: Review new timeout settings in `[timeouts]` and adjust as needed.
3. **ISO Integration**: Configure the new `[iso]` block for environments utilizing ISO-based storage.
4. **FileNaming Configuration**: Update the `FileNaming` setting in `[server]` to use the new "None" option if you want filenames to remain unchanged.
[server]
# FileNaming options: "HMAC", "None"
FileNaming = "HMAC"
## Recommendations
- **Security**: Ensure that the HMAC secret key in `config.toml` is updated to a strong, unique value.
- **Backups**: Regularly back up your `config.toml` and important data directories.
- **Monitoring**: Leverage Prometheus metrics for real-time monitoring of server performance.
For a detailed guide on setting up and configuring the HMAC File Server, refer to the [README.md](./README.md).
---
Thank you for using HMAC File Server! If you encounter any issues, feel free to report them on our GitHub repository.
## Version 2.7
- Refinements in worker scaling logic
- Removed obsolete parameters for sorting
- Further improvements to ISO-based storage handling
- Fixed dual stack for upload (IPv4/IPv6)

124
wiki.md → WIKI.MD
View File

@ -249,12 +249,16 @@ idletimeout = "3600s" # Maximum keep-alive time for idle connections
---
### Security Settings
### Security Configuration
```toml
# Security settings
[security]
secret = "your-secure-secret-key" # HMAC shared secret key (change to a secure value)
enablejwt = false # Enable JWT authentication
jwtsecret = "your-jwt-secret" # JWT signing secret
jwtalgorithm = "HS256" # JWT algorithm
jwtexpiration = "24h" # JWT token expiration
```
#### Configuration Options
@ -265,6 +269,28 @@ secret = "your-secure-secret-key" # HMAC shared secret key (change to a secure
- *Default*: `"your-secure-secret-key"`
- *Warning*: **Change this immediately** to a unique, strong string in production environments to ensure the security of HMAC operations.
- **enablejwt**:
- *Type*: `Boolean`
- *Description*: Enables or disables JWT token authentication. When enabled, the server will accept JWT tokens for authentication.
- *Default*: `false`
- **jwtsecret**:
- *Type*: `String`
- *Description*: The secret key used for signing and validating JWT tokens. Must be strong and secure.
- *Default*: `"your-jwt-secret"`
- **jwtalgorithm**:
- *Type*: `String`
- *Description*: The algorithm used for JWT token signing.
- *Options*: `"HS256"`, `"HS384"`, `"HS512"`
- *Default*: `"HS256"`
- **jwtexpiration**:
- *Type*: `String`
- *Description*: The expiration time for JWT tokens.
- *Format*: Duration (e.g., `"24h"` for 24 hours, `"30m"` for 30 minutes)
- *Default*: `"24h"`
---
### Versioning Settings
@ -290,7 +316,55 @@ maxversions = 1 # Number of file versions to retain
---
### Uploads Settings
### Logging Configuration
```toml
# Logging settings
[logging]
level = "debug"
file = "/path/to/hmac-file-server.log"
max_size = 100 # Maximum log file size in MB
max_backups = 7 # Number of backup log files to keep
max_age = 30 # Maximum age of log files in days
compress = true # Compress old log files
```
#### Configuration Options
- **level**:
- *Type*: `String`
- *Description*: Sets the verbosity level of logs.
- *Options*: `"debug"`, `"info"`, `"warn"`, `"error"`
- *Default*: `"debug"`
- **file**:
- *Type*: `String`
- *Description*: Specifies the file path for logging. If left empty, logs are output to `stdout`.
- *Default*: `"/path/to/hmac-file-server.log"`
- **max_size**:
- *Type*: `Integer`
- *Description*: Maximum size of log files before rotation (in MB).
- *Default*: `100`
- **max_backups**:
- *Type*: `Integer`
- *Description*: Number of backup log files to retain after rotation.
- *Default*: `7`
- **max_age**:
- *Type*: `Integer`
- *Description*: Maximum age of log files in days before deletion.
- *Default*: `30`
- **compress**:
- *Type*: `Boolean`
- *Description*: Whether to compress old log files with gzip.
- *Default*: `true`
---
### Uploads Configuration
```toml
# Upload settings
@ -300,8 +374,6 @@ chunkeduploadsenabled = true
chunksize = "32MB" # Chunk size for uploads
allowedextensions = [
".txt", ".pdf", ".png", ".jpg", ".jpeg", ".gif",
".bmp", ".tiff", ".svg", ".webp", ".wav", ".mp4",
".avi", ".mkv", ".mov", ".wmv", ".flv", ".webm",
".mpeg", ".mpg", ".m4v", ".3gp", ".3g2", ".mp3", ".ogg"
]
```
@ -331,15 +403,13 @@ allowedextensions = [
```toml
allowedextensions = [
".txt", ".pdf", ".png", ".jpg", ".jpeg", ".gif",
".bmp", ".tiff", ".svg", ".webp", ".wav", ".mp4",
".avi", ".mkv", ".mov", ".wmv", ".flv", ".webm",
".mpeg", ".mpg", ".m4v", ".3gp", ".3g2", ".mp3", ".ogg"
]
```
---
### Downloads Settings
### Downloads Configuration
```toml
# Downloads settings
@ -367,7 +437,7 @@ chunksize = "32MB"
- *Format*: Size (e.g., `"32MB"`)
- *Default*: `"32MB"`
> **Note**: The `allowedextensions` key is **not** part of the `[downloads]` configuration based on the provided code. Ensure that it is omitted to prevent configuration errors.
> **Note**: Downloads inherit allowed extensions from the uploads configuration. There is no separate `allowedextensions` setting for downloads.
---
@ -457,12 +527,13 @@ redishealthcheckinterval = "120s" # Interval for Redis health checks
---
### Worker Settings
### Workers Configuration
```toml
# Worker settings
[worker]
numworkers = 10 # Number of worker threads
# Workers settings
[workers]
numworkers = 10 # Number of worker threads
uploadqueuesize = 5000 # Size of upload queue
```
#### Configuration Options
@ -472,6 +543,11 @@ numworkers = 10 # Number of worker threads
- *Description*: Specifies the number of worker threads to handle file operations.
- *Default*: `10`
- **uploadqueuesize**:
- *Type*: `Integer`
- *Description*: Sets the size of the upload queue buffer.
- *Default*: `5000`
---
#### Configuration Options
@ -493,10 +569,9 @@ Below is an example `config.toml` file with default settings:
# Server configuration
listenport = "8080"
bind_ip = "0.0.0.0"
unixsocket = false
storagepath = "/path/to/hmac-file-server/data/"
loglevel = "debug"
logfile = "/path/to/hmac-file-server.log"
metricsenabled = true
metricsport = "9090"
deduplicationenabled = true
@ -507,8 +582,18 @@ autoadjustworkers = true
networkevents = false
pidfilepath = "./hmac-file-server.pid"
precaching = true
filenaming = "HMAC"
forceprotocol = "auto"
# Logging settings
[logging]
level = "debug"
file = "/path/to/hmac-file-server.log"
max_size = 100
max_backups = 7
max_age = 30
compress = true
# Deduplication settings
[deduplication]
enabled = true
@ -530,6 +615,10 @@ idletimeout = "3600s"
# Security settings
[security]
secret = "your-secure-secret-key"
enablejwt = false
jwtsecret = "your-jwt-secret"
jwtalgorithm = "HS256"
jwtexpiration = "24h"
# Versioning settings
[versioning]
@ -572,9 +661,10 @@ redisaddr = "localhost:6379"
redispassword = ""
redishealthcheckinterval = "120s"
# Worker settings
[worker]
# Workers settings
[workers]
numworkers = 10
uploadqueuesize = 5000
```
---
@ -593,7 +683,7 @@ To install the HMAC File Server, follow these steps:
2. Build the server:
```sh
go build -o hmac-file-server
go build -o hmac-file-server ./cmd/server/main.go
```
3. Create the necessary directories:

0
dockerbuild.sh → builddocker.sh
View File

80
buildgo.sh Executable file
View File

@ -0,0 +1,80 @@
#!/bin/bash
# HMAC File Server - Build Script
set -e
# Colors
GREEN='\033[0;32m'
BLUE='\033[0;34m'
YELLOW='\033[1;33m'
RED='\033[0;31m'
NC='\033[0m'
print_status() {
echo -e "${GREEN}[BUILD]${NC} $1"
}
print_info() {
echo -e "${BLUE}[INFO]${NC} $1"
}
print_error() {
echo -e "${RED}[ERROR]${NC} $1"
}
# Check if Go is installed
if ! command -v go &> /dev/null; then
print_error "Go is not installed or not in PATH"
exit 1
fi
# Build the application
print_status "Building HMAC File Server..."
go build -o hmac-file-server cmd/server/main.go cmd/server/helpers.go
if [ $? -eq 0 ]; then
print_status "Build successful! Binary created: ./hmac-file-server"
# Check binary size
SIZE=$(du -h hmac-file-server | cut -f1)
print_info "Binary size: $SIZE"
# Show help to verify it works
print_info "Testing binary functionality..."
./hmac-file-server --help > /dev/null 2>&1
if [ $? -eq 0 ]; then
print_status "Binary is functional!"
else
print_error "Binary test failed"
exit 1
fi
else
print_error "Build failed!"
exit 1
fi
# Create test file for manual testing
print_info "Creating test file..."
echo "Hello, HMAC File Server! $(date)" > test_upload.txt
# Generate HMAC signature for manual testing
print_info "HMAC signature generation for testing:"
SECRET="hmac-file-server-is-the-win"
MESSAGE="/upload"
# Check if openssl is available
if command -v openssl &> /dev/null; then
SIGNATURE=$(echo -n "$MESSAGE" | openssl dgst -sha256 -hmac "$SECRET" | cut -d' ' -f2)
echo "Secret: $SECRET"
echo "Message: $MESSAGE"
echo "Signature: $SIGNATURE"
echo ""
echo "Test with curl (requires server running on localhost:8080):"
echo "curl -v -X POST -H \"X-Signature: $SIGNATURE\" -F \"file=@test_upload.txt\" http://localhost:8080/upload"
else
print_info "OpenSSL not found. You can generate HMAC manually or use the Go tests."
echo "To start server: ./hmac-file-server"
echo "For testing, check the test/ directory for Go test files."
fi
print_status "Build complete! Ready to run: ./hmac-file-server"

13
cmd/server/config.toml
View File

@ -1,18 +1,19 @@
[server]
#bind_ip = "127.0.0.1"
listenport = "8080"
listenport = "8081"
unixsocket = false
storagepath = "./uploads/"
metricsenabled = true
metricsport = "9090"
metricsport = "9092"
deduplicationenabled = true
minfreebytes = "5GB"
filettl = "2y"
filettlenabled = false
autoadjustworkers = true
networkevents = false
pidfilepath = "./hmac-file-server.pid"
pidfilepath = "/tmp/hmac-file-server.pid"
precaching = false
force_protocol = "auto"
#globalextensions = ["*"]
[deduplication]
@ -46,7 +47,11 @@ writetimeout = "3600s"
idletimeout = "3600s"
[security]
secret = "hmac-file-server-is-the-win"
secret = "matrix-does-not-solve-thinking"
enablejwt = false
jwtsecret = "jwt-secret-key-for-testing"
jwtalgorithm = "HS256"
jwtexpiration = "24h"
[versioning]
enableversioning = false

1
cmd/server/deduplication/43320c1b705f95e3e824a804e8c95e0885c0b019509e8037652f853878c73d62/test_upload.txt Normal file
View File

@ -0,0 +1 @@
Hello, HMAC File Server! Mo 9. Jun 20:09:03 CEST 2025

713
cmd/server/helpers.go Normal file
View File

@ -0,0 +1,713 @@
package main
import (
"context"
"crypto/sha256"
"encoding/hex"
"fmt"
"io"
"net"
"net/http"
"os"
"os/signal"
"path/filepath"
"runtime"
"strings"
"syscall"
"time"
"github.com/dutchcoders/go-clamd"
"github.com/go-redis/redis/v8"
"github.com/prometheus/client_golang/prometheus"
"github.com/prometheus/client_golang/prometheus/promhttp"
"github.com/shirou/gopsutil/cpu"
"github.com/shirou/gopsutil/mem"
"gopkg.in/natefinch/lumberjack.v2"
)
// WorkerPool represents a pool of workers
type WorkerPool struct {
workers int
taskQueue chan UploadTask
scanQueue chan ScanTask
ctx context.Context
cancel context.CancelFunc
}
// NewWorkerPool creates a new worker pool
func NewWorkerPool(workers int, queueSize int) *WorkerPool {
ctx, cancel := context.WithCancel(context.Background())
return &WorkerPool{
workers: workers,
taskQueue: make(chan UploadTask, queueSize),
scanQueue: make(chan ScanTask, queueSize),
ctx: ctx,
cancel: cancel,
}
}
// Start starts the worker pool
func (wp *WorkerPool) Start() {
for i := 0; i < wp.workers; i++ {
go wp.worker()
}
}
// Stop stops the worker pool
func (wp *WorkerPool) Stop() {
wp.cancel()
close(wp.taskQueue)
close(wp.scanQueue)
}
// worker is the worker function
func (wp *WorkerPool) worker() {
for {
select {
case <-wp.ctx.Done():
return
case task := <-wp.taskQueue:
if task.Result != nil {
task.Result <- nil // Simple implementation
}
case scanTask := <-wp.scanQueue:
err := processScan(scanTask)
if scanTask.Result != nil {
scanTask.Result <- err
}
}
}
}
// Stub for precacheStoragePath
func precacheStoragePath(storagePath string) error {
// TODO: Implement actual pre-caching logic
// This would typically involve walking the storagePath
// and loading file information into a cache.
log.Infof("Pre-caching for storage path '%s' is a stub and not yet implemented.", storagePath)
return nil
}
func checkFreeSpaceWithRetry(path string, retries int, delay time.Duration) error {
for i := 0; i < retries; i++ {
minFreeBytes, err := parseSize(conf.Server.MinFreeBytes)
if err != nil {
log.Fatalf("Invalid MinFreeBytes: %v", err)
}
if err := checkStorageSpace(path, minFreeBytes); err != nil {
log.Warnf("Free space check failed (attempt %d/%d): %v", i+1, retries, err)
time.Sleep(delay)
continue
}
return nil
}
return fmt.Errorf("insufficient free space after %d attempts", retries)
}
func handleFileCleanup(conf *Config) {
if !conf.Server.FileTTLEnabled {
log.Println("File TTL is disabled.")
return
}
ttlDuration, err := parseTTL(conf.Server.FileTTL)
if err != nil {
log.Fatalf("Invalid TTL configuration: %v", err)
}
log.Printf("TTL cleanup enabled. Files older than %v will be deleted.", ttlDuration)
ticker := time.NewTicker(24 * time.Hour)
defer ticker.Stop()
for range ticker.C {
deleteOldFiles(conf, ttlDuration)
}
}
func computeSHA256(ctx context.Context, filePath string) (string, error) {
if filePath == "" {
return "", fmt.Errorf("file path is empty")
}
file, err := os.Open(filePath)
if err != nil {
return "", fmt.Errorf("failed to open file %s: %w", filePath, err)
}
defer file.Close()
hasher := sha256.New()
if _, err := io.Copy(hasher, file); err != nil {
return "", fmt.Errorf("failed to hash file: %w", err)
}
return hex.EncodeToString(hasher.Sum(nil)), nil
}
func handleDeduplication(ctx context.Context, absFilename string) error {
checksum, err := computeSHA256(ctx, absFilename)
if err != nil {
return err
}
dedupDir := conf.Deduplication.Directory
if dedupDir == "" {
return fmt.Errorf("deduplication directory not configured")
}
dedupPath := filepath.Join(dedupDir, checksum)
if err := os.MkdirAll(dedupPath, os.ModePerm); err != nil {
return err
}
existingPath := filepath.Join(dedupPath, filepath.Base(absFilename))
if _, err := os.Stat(existingPath); err == nil {
return os.Link(existingPath, absFilename)
}
if err := os.Rename(absFilename, existingPath); err != nil {
return err
}
return os.Link(existingPath, absFilename)
}
func handleISOContainer(absFilename string) error {
isoPath := filepath.Join(conf.ISO.MountPoint, "container.iso")
if err := CreateISOContainer([]string{absFilename}, isoPath, conf.ISO.Size, conf.ISO.Charset); err != nil {
return err
}
if err := MountISOContainer(isoPath, conf.ISO.MountPoint); err != nil {
return err
}
return UnmountISOContainer(conf.ISO.MountPoint)
}
func sanitizeFilePath(baseDir, filePath string) (string, error) {
absBaseDir, err := filepath.Abs(baseDir)
if err != nil {
return "", err
}
absFilePath, err := filepath.Abs(filepath.Join(absBaseDir, filePath))
if err != nil {
return "", err
}
if !strings.HasPrefix(absFilePath, absBaseDir) {
return "", fmt.Errorf("invalid file path: %s", filePath)
}
return absFilePath, nil
}
// Stub for formatBytes
func formatBytes(bytes int64) string {
const unit = 1024
if bytes < unit {
return fmt.Sprintf("%d B", bytes)
}
div, exp := int64(unit), 0
for n := bytes / unit; n >= unit; n /= unit {
div *= unit
exp++
}
return fmt.Sprintf("%.1f %ciB", float64(bytes)/float64(div), "KMGTPE"[exp])
}
// Stub for deleteOldFiles
func deleteOldFiles(conf *Config, ttlDuration time.Duration) {
// TODO: Implement actual file deletion logic based on TTL
log.Infof("deleteOldFiles is a stub and not yet implemented. It would check for files older than %v.", ttlDuration)
}
// Stub for CreateISOContainer
func CreateISOContainer(files []string, isoPath, size, charset string) error {
// TODO: Implement actual ISO container creation logic
log.Infof("CreateISOContainer is a stub and not yet implemented. It would create an ISO at %s.", isoPath)
return nil
}
// Stub for MountISOContainer
func MountISOContainer(isoPath, mountPoint string) error {
// TODO: Implement actual ISO container mounting logic
log.Infof("MountISOContainer is a stub and not yet implemented. It would mount %s to %s.", isoPath, mountPoint)
return nil
}
// Stub for UnmountISOContainer
func UnmountISOContainer(mountPoint string) error {
// TODO: Implement actual ISO container unmounting logic
log.Infof("UnmountISOContainer is a stub and not yet implemented. It would unmount %s.", mountPoint)
return nil
}
func checkStorageSpace(storagePath string, minFreeBytes int64) error {
var stat syscall.Statfs_t
if err := syscall.Statfs(storagePath, &stat); err != nil {
return err
}
availableBytes := stat.Bavail * uint64(stat.Bsize)
if int64(availableBytes) < minFreeBytes {
return fmt.Errorf("not enough space: available %d < required %d", availableBytes, minFreeBytes)
}
return nil
}
// setupLogging initializes logging configuration
func setupLogging() {
log.Infof("DEBUG: Starting setupLogging function")
if conf.Logging.File != "" {
log.Infof("DEBUG: Setting up file logging to: %s", conf.Logging.File)
log.SetOutput(&lumberjack.Logger{
Filename: conf.Logging.File,
MaxSize: conf.Logging.MaxSize,
MaxBackups: conf.Logging.MaxBackups,
MaxAge: conf.Logging.MaxAge,
Compress: conf.Logging.Compress,
})
log.Infof("Logging configured to file: %s", conf.Logging.File)
}
log.Infof("DEBUG: setupLogging function completed")
}
// logSystemInfo logs system information
func logSystemInfo() {
memStats, err := mem.VirtualMemory()
if err != nil {
log.Warnf("Failed to get memory stats: %v", err)
} else {
log.Infof("System Memory: Total=%s, Available=%s, Used=%.1f%%",
formatBytes(int64(memStats.Total)),
formatBytes(int64(memStats.Available)),
memStats.UsedPercent)
}
cpuStats, err := cpu.Info()
if err != nil {
log.Warnf("Failed to get CPU stats: %v", err)
} else if len(cpuStats) > 0 {
log.Infof("CPU: %s, Cores=%d", cpuStats[0].ModelName, len(cpuStats))
}
log.Infof("Go Runtime: Version=%s, NumCPU=%d, NumGoroutine=%d",
runtime.Version(), runtime.NumCPU(), runtime.NumGoroutine())
}
// initMetrics initializes Prometheus metrics
func initMetrics() {
uploadDuration = prometheus.NewHistogram(prometheus.HistogramOpts{
Name: "upload_duration_seconds",
Help: "Duration of upload operations in seconds",
})
uploadErrorsTotal = prometheus.NewCounter(prometheus.CounterOpts{
Name: "upload_errors_total",
Help: "Total number of upload errors",
})
uploadsTotal = prometheus.NewCounter(prometheus.CounterOpts{
Name: "uploads_total",
Help: "Total number of uploads",
})
downloadDuration = prometheus.NewHistogram(prometheus.HistogramOpts{
Name: "download_duration_seconds",
Help: "Duration of download operations in seconds",
})
downloadsTotal = prometheus.NewCounter(prometheus.CounterOpts{
Name: "downloads_total",
Help: "Total number of downloads",
})
downloadErrorsTotal = prometheus.NewCounter(prometheus.CounterOpts{
Name: "download_errors_total",
Help: "Total number of download errors",
})
memoryUsage = prometheus.NewGauge(prometheus.GaugeOpts{
Name: "memory_usage_percent",
Help: "Current memory usage percentage",
})
cpuUsage = prometheus.NewGauge(prometheus.GaugeOpts{
Name: "cpu_usage_percent",
Help: "Current CPU usage percentage",
})
activeConnections = prometheus.NewGauge(prometheus.GaugeOpts{
Name: "active_connections_total",
Help: "Number of active connections",
})
requestsTotal = prometheus.NewCounterVec(prometheus.CounterOpts{
Name: "requests_total",
Help: "Total number of requests",
}, []string{"method", "status"})
goroutines = prometheus.NewGauge(prometheus.GaugeOpts{
Name: "goroutines_total",
Help: "Number of goroutines",
})
uploadSizeBytes = prometheus.NewHistogram(prometheus.HistogramOpts{
Name: "upload_size_bytes",
Help: "Size of uploaded files in bytes",
})
downloadSizeBytes = prometheus.NewHistogram(prometheus.HistogramOpts{
Name: "download_size_bytes",
Help: "Size of downloaded files in bytes",
})
filesDeduplicatedTotal = prometheus.NewCounter(prometheus.CounterOpts{
Name: "files_deduplicated_total",
Help: "Total number of deduplicated files",
})
deduplicationErrorsTotal = prometheus.NewCounter(prometheus.CounterOpts{
Name: "deduplication_errors_total",
Help: "Total number of deduplication errors",
})
isoContainersCreatedTotal = prometheus.NewCounter(prometheus.CounterOpts{
Name: "iso_containers_created_total",
Help: "Total number of ISO containers created",
})
isoCreationErrorsTotal = prometheus.NewCounter(prometheus.CounterOpts{
Name: "iso_creation_errors_total",
Help: "Total number of ISO creation errors",
})
isoContainersMountedTotal = prometheus.NewCounter(prometheus.CounterOpts{
Name: "iso_containers_mounted_total",
Help: "Total number of ISO containers mounted",
})
isoMountErrorsTotal = prometheus.NewCounter(prometheus.CounterOpts{
Name: "iso_mount_errors_total",
Help: "Total number of ISO mount errors",
})
workerAdjustmentsTotal = prometheus.NewCounter(prometheus.CounterOpts{
Name: "worker_adjustments_total",
Help: "Total number of worker adjustments",
})
workerReAdjustmentsTotal = prometheus.NewCounter(prometheus.CounterOpts{
Name: "worker_readjustments_total",
Help: "Total number of worker readjustments",
})
// Register all metrics
prometheus.MustRegister(
uploadDuration, uploadErrorsTotal, uploadsTotal,
downloadDuration, downloadsTotal, downloadErrorsTotal,
memoryUsage, cpuUsage, activeConnections, requestsTotal,
goroutines, uploadSizeBytes, downloadSizeBytes,
filesDeduplicatedTotal, deduplicationErrorsTotal,
isoContainersCreatedTotal, isoCreationErrorsTotal,
isoContainersMountedTotal, isoMountErrorsTotal,
workerAdjustmentsTotal, workerReAdjustmentsTotal,
)
log.Info("Prometheus metrics initialized successfully")
}
// scanFileWithClamAV scans a file using ClamAV
func scanFileWithClamAV(filename string) error {
if clamClient == nil {
return fmt.Errorf("ClamAV client not initialized")
}
result, err := clamClient.ScanFile(filename)
if err != nil {
return fmt.Errorf("ClamAV scan failed: %w", err)
}
// Handle the result channel
if result != nil {
select {
case scanResult := <-result:
if scanResult != nil && scanResult.Status != "OK" {
return fmt.Errorf("virus detected in %s: %s", filename, scanResult.Status)
}
case <-time.After(30 * time.Second):
return fmt.Errorf("ClamAV scan timeout for file: %s", filename)
}
}
log.Debugf("File %s passed ClamAV scan", filename)
return nil
}
// initClamAV initializes ClamAV client
func initClamAV(socketPath string) (*clamd.Clamd, error) {
if socketPath == "" {
socketPath = "/var/run/clamav/clamd.ctl"
}
client := clamd.NewClamd(socketPath)
// Test connection
err := client.Ping()
if err != nil {
return nil, fmt.Errorf("failed to ping ClamAV daemon: %w", err)
}
log.Infof("ClamAV client initialized with socket: %s", socketPath)
return client, nil
}
// initRedis initializes Redis client
func initRedis() {
redisClient = redis.NewClient(&redis.Options{
Addr: conf.Redis.RedisAddr,
Password: conf.Redis.RedisPassword,
DB: conf.Redis.RedisDBIndex,
})
ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
defer cancel()
_, err := redisClient.Ping(ctx).Result()
if err != nil {
log.Warnf("Failed to connect to Redis: %v", err)
redisConnected = false
} else {
log.Info("Redis client initialized successfully")
redisConnected = true
}
}
// monitorNetwork monitors network events
func monitorNetwork(ctx context.Context) {
log.Info("Starting network monitoring")
ticker := time.NewTicker(30 * time.Second)
defer ticker.Stop()
for {
select {
case <-ctx.Done():
log.Info("Network monitoring stopped")
return
case <-ticker.C:
// Simple network monitoring - check interface status
interfaces, err := net.Interfaces()
if err != nil {
log.Warnf("Failed to get network interfaces: %v", err)
continue
}
for _, iface := range interfaces {
if iface.Flags&net.FlagUp != 0 && iface.Flags&net.FlagLoopback == 0 {
select {
case networkEvents <- NetworkEvent{
Type: "interface_up",
Details: fmt.Sprintf("Interface %s is up", iface.Name),
}:
default:
// Channel full, skip
}
}
}
}
}
}
// handleNetworkEvents handles network events
func handleNetworkEvents(ctx context.Context) {
log.Info("Starting network event handler")
for {
select {
case <-ctx.Done():
log.Info("Network event handler stopped")
return
case event := <-networkEvents:
log.Debugf("Network event: %s - %s", event.Type, event.Details)
}
}
}
// updateSystemMetrics updates system metrics
func updateSystemMetrics(ctx context.Context) {
ticker := time.NewTicker(15 * time.Second)
defer ticker.Stop()
for {
select {
case <-ctx.Done():
return
case <-ticker.C:
// Update memory metrics
if memStats, err := mem.VirtualMemory(); err == nil {
memoryUsage.Set(memStats.UsedPercent)
}
// Update CPU metrics
if cpuPercents, err := cpu.Percent(time.Second, false); err == nil && len(cpuPercents) > 0 {
cpuUsage.Set(cpuPercents[0])
}
// Update goroutine count
goroutines.Set(float64(runtime.NumGoroutine()))
}
}
}
// setupRouter sets up HTTP routes
func setupRouter() *http.ServeMux {
mux := http.NewServeMux()
mux.HandleFunc("/upload", handleUpload)
mux.HandleFunc("/download/", handleDownload)
mux.HandleFunc("/health", func(w http.ResponseWriter, r *http.Request) {
w.WriteHeader(http.StatusOK)
w.Write([]byte("OK"))
})
if conf.Server.MetricsEnabled {
mux.Handle("/metrics", promhttp.Handler())
}
// Catch-all handler for all upload protocols (v, v2, token, v3)
// This must be added last as it matches all paths
mux.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
// Handle PUT requests for all upload protocols
if r.Method == http.MethodPut {
query := r.URL.Query()
// Check if this is a v3 request (mod_http_upload_external)
if query.Get("v3") != "" && query.Get("expires") != "" {
handleV3Upload(w, r)
return
}
// Check if this is a legacy protocol request (v, v2, token)
if query.Get("v") != "" || query.Get("v2") != "" || query.Get("token") != "" {
handleLegacyUpload(w, r)
return
}
}
// Handle GET/HEAD requests for downloads
if r.Method == http.MethodGet || r.Method == http.MethodHead {
// Only handle download requests if the path looks like a file
path := strings.TrimPrefix(r.URL.Path, "/")
if path != "" && !strings.HasSuffix(path, "/") {
handleLegacyDownload(w, r)
return
}
}
// For all other requests, return 404
http.NotFound(w, r)
})
log.Info("HTTP router configured successfully with full protocol support (v, v2, token, v3)")
return mux
}
// setupGracefulShutdown sets up graceful shutdown
func setupGracefulShutdown(server *http.Server, cancel context.CancelFunc) {
sigChan := make(chan os.Signal, 1)
signal.Notify(sigChan, syscall.SIGINT, syscall.SIGTERM)
go func() {
<-sigChan
log.Info("Received shutdown signal, initiating graceful shutdown...")
// Cancel context
cancel()
// Shutdown server with timeout
ctx, shutdownCancel := context.WithTimeout(context.Background(), 30*time.Second)
defer shutdownCancel()
if err := server.Shutdown(ctx); err != nil {
log.Errorf("Server shutdown error: %v", err)
} else {
log.Info("Server shutdown completed")
}
// Clean up PID file
if conf.Server.CleanUponExit {
removePIDFile(conf.Server.PIDFilePath)
}
// Stop worker pool if it exists
if workerPool != nil {
workerPool.Stop()
log.Info("Worker pool stopped")
}
os.Exit(0)
}()
}
// ProgressWriter wraps an io.Writer to provide upload progress reporting
type ProgressWriter struct {
dst io.Writer
total int64
written int64
filename string
onProgress func(written, total int64, filename string)
lastReport time.Time
}
// NewProgressWriter creates a new ProgressWriter
func NewProgressWriter(dst io.Writer, total int64, filename string) *ProgressWriter {
return &ProgressWriter{
dst: dst,
total: total,
filename: filename,
onProgress: func(written, total int64, filename string) {
if total > 0 {
percentage := float64(written) / float64(total) * 100
sizeMiB := float64(written) / (1024 * 1024)
totalMiB := float64(total) / (1024 * 1024)
log.Infof("Upload progress for %s: %.1f%% (%.1f/%.1f MiB)",
filepath.Base(filename), percentage, sizeMiB, totalMiB)
}
},
lastReport: time.Now(),
}
}
// Write implements io.Writer interface with progress reporting
func (pw *ProgressWriter) Write(p []byte) (int, error) {
n, err := pw.dst.Write(p)
if err != nil {
return n, err
}
pw.written += int64(n)
// Report progress every 30 seconds or every 50MB for large files
now := time.Now()
shouldReport := false
if pw.total > 100*1024*1024 { // Files larger than 100MB
shouldReport = now.Sub(pw.lastReport) > 30*time.Second ||
(pw.written%(50*1024*1024) == 0 && pw.written > 0)
} else if pw.total > 10*1024*1024 { // Files larger than 10MB
shouldReport = now.Sub(pw.lastReport) > 10*time.Second ||
(pw.written%(10*1024*1024) == 0 && pw.written > 0)
}
if shouldReport && pw.onProgress != nil {
pw.onProgress(pw.written, pw.total, pw.filename)
pw.lastReport = now
}
return n, err
}
// copyWithProgress copies data from src to dst with progress reporting
func copyWithProgress(dst io.Writer, src io.Reader, total int64, filename string) (int64, error) {
progressWriter := NewProgressWriter(dst, total, filename)
// Use a pooled buffer for efficient copying
bufPtr := bufferPool.Get().(*[]byte)
defer bufferPool.Put(bufPtr)
buf := *bufPtr
return io.CopyBuffer(progressWriter, src, buf)
}

BIN
cmd/server/hmac-file-server Executable file
View File

Binary file not shown.

3044
cmd/server/main.go
View File

File diff suppressed because it is too large Load Diff

1
cmd/server/uploads/test_upload.txt Normal file
View File

@ -0,0 +1 @@
Hello, HMAC File Server! Mo 9. Jun 20:09:03 CEST 2025

136
config-example-xmpp.toml Normal file
View File

@ -0,0 +1,136 @@
# HMAC File Server 3.2 Example Configuration for XMPP Operators
# This configuration is optimized for production XMPP deployments
[server]
# Server binding configuration
bind_ip = "127.0.0.1" # Bind to localhost only (use reverse proxy for public access)
listenport = "8080" # HTTP port for file operations
unixsocket = false # Use TCP instead of Unix socket
# Storage configuration
storagepath = "/var/lib/hmac-file-server/uploads"
deduplicationenabled = true # Save space by detecting duplicate files
deduplicationpath = "/var/lib/hmac-file-server/deduplication"
# File naming strategy
filenaming = "HMAC" # Use HMAC-based filenames for security
# Protocol enforcement (auto-detect by default)
forceprotocol = "auto"
# Metrics for monitoring
metricsenabled = true
metricsport = "9090"
# SSL/TLS configuration (optional - use reverse proxy recommended)
sslenabled = false
# sslcert = "/path/to/certificate.pem"
# sslkey = "/path/to/private-key.pem"
[security]
# HMAC secret - MUST match your XMPP server configuration
# Generate with: openssl rand -base64 32
secret = "CHANGE_THIS_TO_A_STRONG_SECRET_AT_LEAST_32_CHARS_LONG"
# JWT authentication (recommended for enhanced security)
enablejwt = true
jwtsecret = "CHANGE_THIS_TO_A_DIFFERENT_STRONG_JWT_SECRET"
jwtalgorithm = "HS256" # Supported: HS256, HS384, HS512
jwtexpiration = "24h" # Token validity period
[uploads]
# Allowed file extensions (customize based on your needs)
allowedextensions = [
# Documents
".txt", ".pdf", ".doc", ".docx", ".odt",
".xls", ".xlsx", ".ods", ".ppt", ".pptx", ".odp",
# Images
".jpg", ".jpeg", ".png", ".gif", ".webp", ".bmp", ".svg",
# Audio
".mp3", ".wav", ".ogg", ".flac", ".m4a", ".aac",
# Video
".mp4", ".webm", ".avi", ".mkv", ".mov", ".wmv",
# Archives
".zip", ".tar", ".gz", ".7z", ".rar", ".bz2"
]
# File size limits
maxfilesize = "50MB" # Maximum file size per upload
# Chunked uploads for large files
chunkeduploadsenabled = true
chunksize = "10MB" # Size of each chunk
# Automatic file cleanup (TTL)
ttlenabled = true # Enable automatic deletion of old files
ttl = "720h" # Delete files after 30 days (720 hours)
[downloads]
# Chunked downloads for large files
chunkeddownloadsenabled = true
chunksize = "10MB"
[logging]
# Log level: DEBUG, INFO, WARN, ERROR
loglevel = "INFO"
# File logging
logtofile = true
logfilepath = "/var/log/hmac-file-server/hmac-file-server.log"
# Log rotation
logrotation = true
maxlogsize = "100MB" # Rotate when log file exceeds this size
maxlogfiles = 5 # Keep this many rotated log files
maxage = 30 # Delete log files older than this (days)
compress = true # Compress rotated log files
[workers]
# Worker pool configuration for handling concurrent requests
maxworkers = 100 # Maximum number of worker goroutines
minworkers = 10 # Minimum number of worker goroutines
autoscaling = true # Automatically scale workers based on load
[timeouts]
# Network timeouts
readtimeout = "60s" # Maximum time to read request
writetimeout = "60s" # Maximum time to write response
idletimeout = "120s" # Maximum time for idle connections
[redis]
# Redis integration for session management and caching
enabled = false # Set to true if you have Redis available
host = "localhost"
port = 6379
database = 0
password = "" # Set if Redis requires authentication
timeout = "5s"
# Connection pooling
maxidle = 10
maxactive = 100
idletimeout = "240s"
[clamav]
# Virus scanning integration
enabled = false # Set to true to enable virus scanning
socket = "/var/run/clamav/clamd.ctl" # ClamAV socket path
timeout = "30s" # Maximum time to wait for scan
# XMPP Server Integration Examples:
#
# Prosody Configuration:
# Component "upload.yourdomain.com" "http_file_share"
# http_file_share_url = "https://upload.yourdomain.com"
# http_file_share_secret = "SAME_SECRET_AS_ABOVE"
#
# Ejabberd Configuration:
# mod_http_file_share:
# external_secret: "SAME_SECRET_AS_ABOVE"
# service_url: "https://upload.yourdomain.com"
#
# Note: Always use HTTPS in production with proper SSL certificates

2
dockerenv/docker-compose.yml
View File

@ -3,7 +3,7 @@
services:
hmac-file-server:
container_name: hmac-file-server
image: ghcr.io/plusone/hmac-file-server:latest
image: hmac-file-server:latest
ports:
- "8080:8080"
volumes:

2
dockerenv/dockerbuild/Dockerfile
View File

@ -6,7 +6,7 @@ RUN apk add --no-cache git
COPY go.mod go.sum ./
RUN go mod download
COPY . .
RUN CGO_ENABLED=0 go build -o hmac-file-server ./cmd/server/main.go
RUN CGO_ENABLED=0 go build -o hmac-file-server cmd/server/main.go cmd/server/helpers.go
# Stage 2: Runtime
FROM alpine:latest

32
docs/updates.md
View File

@ -1,32 +0,0 @@
**Important Update Regarding HMAC File Server Version 2.6-Stable**
We've identified and resolved critical issues with the deduplication processes in **Version 2.6-Stable** of the HMAC File Server. This update brings significant improvements to ensure reliable deduplication and overall server performance.
**Impact:**
- **Resolved Deduplication Issues:** The deduplication feature in **Version 2.6-Stable** has been thoroughly tested and stabilized, ensuring consistent file integrity and optimal server performance.
- **Enhanced Stability:** Addressed previous unexpected behaviors, including lost links to files and incorrect file handling operations.
**Recommended Actions:**
- **Upgrade to Version 2.6-Stable:** If you are using any earlier version, we strongly recommend upgrading to **Version 2.6-Stable** to benefit from the latest fixes and improvements.
- **Rollback Not Required:** Users on older releases can upgrade without the need to rollback, as **Version 2.6-Stable** resolves all critical issues present in prior versions.
**What's New in Version 2.6-Stable:**
- **Deduplication Enhancements:**
- **Reliable Deduplication:** Improved algorithms for accurate and efficient processing.
- **Performance Optimizations:** Reduced resource consumption for faster deduplication.
- **Enhanced Features:**
- **Robust Malware Scanning:** Integrated advanced scanning to ensure file safety.
- **Enhanced Redis Support:** Improved performance with Redis for better caching.
- **Better Configuration Options:** More flexible settings for customization.
- **Security Improvements:**
- **Strengthened Authentication Mechanisms:** Safeguards against unauthorized access.
- **Improved Data Encryption:** Advanced encryption standards for data at rest and in transit.
**Stay Updated:**
For detailed information on addressed issues, fixes, and enhancements introduced in **Version 2.6-Stable**, please visit our repositories:
- [Git uuxo.net](https://git.uuxo.net/uuxo/hmac-file-server)
- [GitHub](https://github.com/PlusOne/hmac-file-server)
We appreciate your patience and support as we continue improving HMAC File Server. **Version 2.6-Stable** reflects our commitment to delivering a reliable and efficient file server solution. Thank you for choosing HMAC File Server!

1
go.mod
View File

@ -17,6 +17,7 @@ require (
github.com/dgryski/go-rendezvous v0.0.0-20200823014737-9f7001d12a5f // indirect
github.com/fsnotify/fsnotify v1.7.0 // indirect
github.com/gdamore/encoding v1.0.0 // indirect
github.com/golang-jwt/jwt/v5 v5.2.2
github.com/hashicorp/hcl v1.0.0 // indirect
github.com/lucasb-eyer/go-colorful v1.2.0 // indirect
github.com/lufia/plan9stats v0.0.0-20211012122336-39d0f177ccd0 // indirect

2
go.sum
View File

@ -22,6 +22,8 @@ github.com/go-ole/go-ole v1.3.0 h1:Dt6ye7+vXGIKZ7Xtk4s6/xVdGDQynvom7xCFEdWr6uE=
github.com/go-ole/go-ole v1.3.0/go.mod h1:5LS6F96DhAwUc7C+1HLexzMXY1xGRSryjyPPKW6zv78=
github.com/go-redis/redis/v8 v8.11.5 h1:AcZZR7igkdvfVmQTPnu9WE37LRrO/YrBH5zWyjDC0oI=
github.com/go-redis/redis/v8 v8.11.5/go.mod h1:gREzHqY1hg6oD9ngVRbLStwAWKhA0FEgq8Jd4h5lpwo=
github.com/golang-jwt/jwt/v5 v5.2.2 h1:Rl4B7itRWVtYIHFrSNd7vhTiz9UpLdi6gZhZ3wEeDy8=
github.com/golang-jwt/jwt/v5 v5.2.2/go.mod h1:pqrtFR0X4osieyHYxtmOUWsAWrfe1Q5UVIyoH402zdk=
github.com/google/go-cmp v0.5.6/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
github.com/google/go-cmp v0.6.0 h1:ofyhxvXcZhMsU5ulbFiLKl/XBFqE1GSq7atu8tAmTRI=
github.com/google/go-cmp v0.6.0/go.mod h1:17dUlkBOakJ0+DkrSSNjCkIjxS6bF9zb3elmeNGIjoY=

909
installer.sh Executable file
View File

@ -0,0 +1,909 @@
#!/bin/bash
# HMAC File Server Installer Script
# Version: 3.2
# Compatible with systemd Linux distributions
set -e
# Trap to handle script errors
trap 'handle_error $? $LINENO' ERR
# Error handling function
handle_error() {
local exit_code=$1
local line_number=$2
echo -e "${RED}Error occurred in script at line $line_number with exit code $exit_code${NC}"
echo -e "${YELLOW}Installation failed. Please check the error message above.${NC}"
exit $exit_code
}
# Colors for output
RED='\033[0;31m'
GREEN='\033[0;32m'
YELLOW='\033[1;33m'
BLUE='\033[0;34m'
NC='\033[0m' # No Color
# Default values
DEFAULT_USER="hmac-server"
DEFAULT_INSTALL_DIR="/opt/hmac-file-server"
DEFAULT_CONFIG_DIR="/etc/hmac-file-server"
DEFAULT_DATA_DIR="/var/lib/hmac-file-server"
DEFAULT_LOG_DIR="/var/log/hmac-file-server"
DEFAULT_PORT="8080"
DEFAULT_METRICS_PORT="9090"
# Help function
show_help() {
echo -e "${BLUE}HMAC File Server 3.2 Installer${NC}"
echo ""
echo "Usage: $0 [OPTION]"
echo ""
echo "Options:"
echo " --help Show this help message"
echo " --uninstall Uninstall HMAC File Server completely"
echo ""
echo "Environment Variables (optional):"
echo " HMAC_SECRET Pre-set HMAC secret (minimum 32 characters)"
echo " JWT_SECRET Pre-set JWT secret (minimum 32 characters)"
echo ""
echo "Example:"
echo " HMAC_SECRET='your-super-secret-hmac-key-here-32chars' sudo -E $0"
echo ""
echo "This installer will:"
echo " • Install Go 1.24 (if not present)"
echo " • Create system user and directories"
echo " • Build and install HMAC File Server"
echo " • Configure systemd service"
echo " • Install Redis and/or ClamAV (optional)"
echo ""
echo "For XMPP operators: This installer is optimized for easy integration"
echo "with Prosody, Ejabberd, and other XMPP servers."
echo ""
}
# Check for help flag first (before root check)
if [[ "$1" == "--help" || "$1" == "-h" ]]; then
show_help
exit 0
fi
# Professional installer header with branding
echo ""
echo -e "${BLUE} __ _____ __ ${NC}"
echo -e "${BLUE} / /_ ____ ___ ____ ______ / __(_) /__ ________ ______ _____ _____${NC}"
echo -e "${BLUE} / __ \\/ __ \`__ \\/ __ \`/ ___/_____/ /_/ / / _ \\______/ ___/ _ \\/ ___/ | / / _ \\/ ___/${NC}"
echo -e "${BLUE} / / / / / / / / / /_/ / /__/_____/ __/ / / __/_____(__ ) __/ / | |/ / __/ / ${NC}"
echo -e "${BLUE}/_/ /_/_/ /_/ /_/\\__,_/\\___/ /_/ /_/_/\\___/ /____/\\___/_/ |___/\\___/_/ ${NC}"
echo ""
echo -e "${BLUE} █ HMAC File Server 3.2 Installer █${NC}"
echo -e "${BLUE} Professional XMPP Integration${NC}"
echo ""
echo -e "${YELLOW}────────────────────────────────────────────────────────────────────────────────${NC}"
echo -e "${GREEN} ✓ Secure File Uploads & Downloads ✓ JWT & HMAC Authentication${NC}"
echo -e "${GREEN} ✓ Prometheus Metrics Integration ✓ ClamAV Virus Scanning${NC}"
echo -e "${GREEN} ✓ Redis Cache & Session Management ✓ Chunked Upload/Download Support${NC}"
echo -e "${YELLOW}────────────────────────────────────────────────────────────────────────────────${NC}"
echo ""
# Check if running as root
if [[ $EUID -ne 0 ]]; then
echo -e "${RED}This script must be run as root (use sudo)${NC}"
exit 1
fi
# Check for systemd
if ! command -v systemctl &> /dev/null; then
echo -e "${RED}Error: systemctl not found. This installer requires a systemd-based Linux distribution.${NC}"
exit 1
fi
# Pre-installation checks
pre_installation_checks() {
echo -e "${YELLOW}Running pre-installation checks...${NC}"
# Check if service already exists
if systemctl is-enabled hmac-file-server.service &>/dev/null; then
echo -e "${YELLOW}Warning: HMAC File Server service already exists${NC}"
read -p "Do you want to continue and overwrite the existing installation? (y/N): " OVERWRITE
if [[ ! $OVERWRITE =~ ^[Yy]$ ]]; then
echo -e "${YELLOW}Installation cancelled${NC}"
exit 0
fi
# Stop existing service
echo -e "${YELLOW}Stopping existing service...${NC}"
systemctl stop hmac-file-server.service || true
fi
# Check available disk space (minimum 1GB)
AVAILABLE_SPACE=$(df / | awk 'NR==2 {print $4}')
if [[ $AVAILABLE_SPACE -lt 1048576 ]]; then
echo -e "${RED}Error: Insufficient disk space. At least 1GB required${NC}"
exit 1
fi
# Check if we're in the correct directory (should contain go.mod)
if [[ ! -f "go.mod" ]]; then
echo -e "${RED}Error: go.mod not found. Please run this installer from the HMAC File Server source directory${NC}"
exit 1
fi
echo -e "${GREEN}✅ Pre-installation checks passed${NC}"
echo ""
}
# Check for Go installation
check_go() {
if ! command -v go &> /dev/null; then
echo -e "${YELLOW}Go is not installed. Installing Go 1.24...${NC}"
# Detect architecture
ARCH=$(uname -m)
case $ARCH in
x86_64) GO_ARCH="amd64" ;;
aarch64|arm64) GO_ARCH="arm64" ;;
armv7l) GO_ARCH="armv6l" ;;
*) echo -e "${RED}Unsupported architecture: $ARCH${NC}"; exit 1 ;;
esac
# Download and install Go
cd /tmp
wget -q "https://go.dev/dl/go1.24.linux-${GO_ARCH}.tar.gz"
tar -C /usr/local -xzf "go1.24.linux-${GO_ARCH}.tar.gz"
# Add Go to PATH
echo 'export PATH=$PATH:/usr/local/go/bin' >> /etc/profile
export PATH=$PATH:/usr/local/go/bin
echo -e "${GREEN}Go 1.24 installed successfully${NC}"
else
GO_VERSION=$(go version | awk '{print $3}' | sed 's/go//')
echo -e "${GREEN}Go $GO_VERSION is already installed${NC}"
fi
}
# User input function
get_user_input() {
echo -e "${BLUE}Configuration Setup${NC}"
echo "Please provide the following information (or press Enter for defaults):"
echo ""
# System user
read -p "System user for HMAC File Server [$DEFAULT_USER]: " HMAC_USER
HMAC_USER=${HMAC_USER:-$DEFAULT_USER}
# Installation directory
read -p "Installation directory [$DEFAULT_INSTALL_DIR]: " INSTALL_DIR
INSTALL_DIR=${INSTALL_DIR:-$DEFAULT_INSTALL_DIR}
# Data directory
read -p "Data directory (uploads) [$DEFAULT_DATA_DIR]: " DATA_DIR
DATA_DIR=${DATA_DIR:-$DEFAULT_DATA_DIR}
# Server port
read -p "Server port [$DEFAULT_PORT]: " SERVER_PORT
SERVER_PORT=${SERVER_PORT:-$DEFAULT_PORT}
# Metrics port
read -p "Metrics port [$DEFAULT_METRICS_PORT]: " METRICS_PORT
METRICS_PORT=${METRICS_PORT:-$DEFAULT_METRICS_PORT}
# HMAC secret
if [[ -n "$HMAC_SECRET" ]]; then
# Use environment variable if provided
if [[ ${#HMAC_SECRET} -ge 32 ]]; then
echo -e "${GREEN}✅ Using HMAC secret from environment variable${NC}"
else
echo -e "${RED}Error: HMAC_SECRET environment variable must be at least 32 characters long${NC}"
echo -e "${YELLOW}Current length: ${#HMAC_SECRET}${NC}"
exit 1
fi
else
# Interactive input with auto-generation option
echo ""
echo -e "${BLUE}HMAC Secret Configuration${NC}"
echo "Choose how to set the HMAC secret:"
echo " 1) Generate automatically (recommended)"
echo " 2) Enter manually"
echo ""
while true; do
read -p "Choice [1]: " hmac_choice
hmac_choice=${hmac_choice:-1}
case $hmac_choice in
1)
echo -e "${YELLOW}Generating secure HMAC secret...${NC}"
HMAC_SECRET=$(generate_random_key 48)
echo -e "${GREEN}✅ Generated 48-character HMAC secret${NC}"
echo -e "${BLUE}Secret preview: ${HMAC_SECRET:0:8}...${HMAC_SECRET: -8}${NC}"
break
;;
2)
while true; do
echo -n "HMAC secret (minimum 32 characters): "
# Use bash built-in silent read if available
if read -s -r HMAC_SECRET 2>/dev/null; then
echo ""
else
# Fallback: use regular read with warning
echo ""
echo -e "${YELLOW}⚠️ Note: Input will be visible (your terminal doesn't support hidden input)${NC}"
echo -n "HMAC secret (minimum 32 characters): "
read -r HMAC_SECRET
fi
if [[ ${#HMAC_SECRET} -ge 32 ]]; then
echo -e "${GREEN}✅ HMAC secret accepted (${#HMAC_SECRET} characters)${NC}"
break 2
else
echo -e "${RED}HMAC secret must be at least 32 characters long (you entered ${#HMAC_SECRET} characters)${NC}"
echo -e "${YELLOW}Tip: Choose option 1 for automatic generation${NC}"
fi
done
;;
*)
echo -e "${RED}Please enter 1 or 2${NC}"
;;
esac
done
fi
# JWT settings
echo ""
read -p "Enable JWT authentication? (y/N): " ENABLE_JWT
if [[ $ENABLE_JWT =~ ^[Yy]$ ]]; then
ENABLE_JWT="true"
# JWT secret
if [[ -n "$JWT_SECRET" ]]; then
# Use environment variable if provided
if [[ ${#JWT_SECRET} -ge 32 ]]; then
echo -e "${GREEN}✅ Using JWT secret from environment variable${NC}"
else
echo -e "${RED}Error: JWT_SECRET environment variable must be at least 32 characters long${NC}"
echo -e "${YELLOW}Current length: ${#JWT_SECRET}${NC}"
exit 1
fi
else
# Interactive input with auto-generation option
echo ""
echo -e "${BLUE}JWT Secret Configuration${NC}"
echo "Choose how to set the JWT secret:"
echo " 1) Generate automatically (recommended)"
echo " 2) Enter manually"
echo ""
while true; do
read -p "Choice [1]: " jwt_choice
jwt_choice=${jwt_choice:-1}
case $jwt_choice in
1)
echo -e "${YELLOW}Generating secure JWT secret...${NC}"
JWT_SECRET=$(generate_random_key 48)
echo -e "${GREEN}✅ Generated 48-character JWT secret${NC}"
echo -e "${BLUE}Secret preview: ${JWT_SECRET:0:8}...${JWT_SECRET: -8}${NC}"
break
;;
2)
while true; do
echo -n "JWT secret (minimum 32 characters): "
# Use bash built-in silent read if available
if read -s -r JWT_SECRET 2>/dev/null; then
echo ""
else
# Fallback: use regular read with warning
echo ""
echo -e "${YELLOW}⚠️ Note: Input will be visible (your terminal doesn't support hidden input)${NC}"
echo -n "JWT secret (minimum 32 characters): "
read -r JWT_SECRET
fi
if [[ ${#JWT_SECRET} -ge 32 ]]; then
echo -e "${GREEN}✅ JWT secret accepted (${#JWT_SECRET} characters)${NC}"
break 2
else
echo -e "${RED}JWT secret must be at least 32 characters long (you entered ${#JWT_SECRET} characters)${NC}"
echo -e "${YELLOW}Tip: Choose option 1 for automatic generation${NC}"
fi
done
;;
*)
echo -e "${RED}Please enter 1 or 2${NC}"
;;
esac
done
fi
# JWT expiration
read -p "JWT token expiration [24h]: " JWT_EXPIRATION
JWT_EXPIRATION=${JWT_EXPIRATION:-"24h"}
# JWT algorithm
read -p "JWT algorithm (HS256/HS384/HS512) [HS256]: " JWT_ALGORITHM
JWT_ALGORITHM=${JWT_ALGORITHM:-"HS256"}
else
ENABLE_JWT="false"
JWT_SECRET=""
JWT_EXPIRATION="24h"
JWT_ALGORITHM="HS256"
fi
# Redis settings
echo ""
read -p "Enable Redis integration? (y/N): " ENABLE_REDIS
if [[ $ENABLE_REDIS =~ ^[Yy]$ ]]; then
ENABLE_REDIS="true"
read -p "Redis host [localhost]: " REDIS_HOST
REDIS_HOST=${REDIS_HOST:-"localhost"}
read -p "Redis port [6379]: " REDIS_PORT
REDIS_PORT=${REDIS_PORT:-"6379"}
read -p "Redis database [0]: " REDIS_DB
REDIS_DB=${REDIS_DB:-"0"}
read -s -p "Redis password (optional): " REDIS_PASSWORD
echo ""
else
ENABLE_REDIS="false"
fi
# ClamAV settings
echo ""
read -p "Enable ClamAV virus scanning? (y/N): " ENABLE_CLAMAV
if [[ $ENABLE_CLAMAV =~ ^[Yy]$ ]]; then
ENABLE_CLAMAV="true"
else
ENABLE_CLAMAV="false"
fi
# SSL/TLS settings
echo ""
read -p "Enable SSL/TLS? (y/N): " ENABLE_TLS
if [[ $ENABLE_TLS =~ ^[Yy]$ ]]; then
ENABLE_TLS="true"
read -p "SSL certificate path: " SSL_CERT
read -p "SSL private key path: " SSL_KEY
else
ENABLE_TLS="false"
fi
# Show configuration summary
# Professional configuration summary
echo ""
echo -e "${BLUE} █ Configuration Summary █${NC}"
echo -e "${YELLOW}────────────────────────────────────────────────────────────────${NC}"
echo -e "${YELLOW}System User:${NC} $HMAC_USER"
echo -e "${YELLOW}Install Dir:${NC} $INSTALL_DIR"
echo -e "${YELLOW}Data Dir:${NC} $DATA_DIR"
echo -e "${YELLOW}Server Port:${NC} $SERVER_PORT"
echo -e "${YELLOW}Metrics Port:${NC} $METRICS_PORT"
echo -e "${YELLOW}JWT Auth:${NC} $([[ "$ENABLE_JWT" == "true" ]] && echo "✅ Enabled" || echo "❌ Disabled")"
echo -e "${YELLOW}Redis:${NC} $([[ "$ENABLE_REDIS" == "true" ]] && echo "✅ Enabled ($REDIS_HOST:$REDIS_PORT)" || echo "❌ Disabled")"
echo -e "${YELLOW}ClamAV:${NC} $([[ "$ENABLE_CLAMAV" == "true" ]] && echo "✅ Enabled" || echo "❌ Disabled")"
echo -e "${YELLOW}SSL/TLS:${NC} $([[ "$ENABLE_TLS" == "true" ]] && echo "✅ Enabled" || echo "❌ Disabled")"
echo -e "${YELLOW}────────────────────────────────────────────────────────────────${NC}"
echo ""
read -p "Continue with installation? (y/N): " CONFIRM_INSTALL
if [[ ! $CONFIRM_INSTALL =~ ^[Yy]$ ]]; then
echo -e "${YELLOW}Installation cancelled by user${NC}"
exit 0
fi
}
# Create system user
create_user() {
if ! id "$HMAC_USER" &>/dev/null; then
echo -e "${YELLOW}Creating system user: $HMAC_USER${NC}"
useradd --system --home-dir "$INSTALL_DIR" --shell /bin/false --comment "HMAC File Server" "$HMAC_USER"
else
echo -e "${GREEN}User $HMAC_USER already exists${NC}"
fi
}
# Create directories
create_directories() {
echo -e "${YELLOW}Creating directories...${NC}"
mkdir -p "$INSTALL_DIR"
mkdir -p "$DEFAULT_CONFIG_DIR"
mkdir -p "$DATA_DIR/uploads"
mkdir -p "$DATA_DIR/deduplication"
mkdir -p "$DEFAULT_LOG_DIR"
# Set ownership
chown -R "$HMAC_USER:$HMAC_USER" "$INSTALL_DIR"
chown -R "$HMAC_USER:$HMAC_USER" "$DATA_DIR"
chown -R "$HMAC_USER:$HMAC_USER" "$DEFAULT_LOG_DIR"
# Set permissions
chmod 755 "$INSTALL_DIR"
chmod 755 "$DATA_DIR"
chmod 750 "$DEFAULT_LOG_DIR"
}
# Build HMAC File Server
build_server() {
echo -e "${YELLOW}Building HMAC File Server...${NC}"
# Build the server
cd "$(dirname "$0")"
go build -o "$INSTALL_DIR/hmac-file-server" cmd/server/main.go cmd/server/helpers.go
# Set ownership and permissions
chown "$HMAC_USER:$HMAC_USER" "$INSTALL_DIR/hmac-file-server"
chmod 755 "$INSTALL_DIR/hmac-file-server"
echo -e "${GREEN}HMAC File Server built successfully${NC}"
}
# Generate configuration file
generate_config() {
echo -e "${YELLOW}Generating configuration file...${NC}"
cat > "$DEFAULT_CONFIG_DIR/config.toml" << EOF
# HMAC File Server Configuration
# Generated by installer on $(date)
[server]
bind_ip = "0.0.0.0"
listenport = "$SERVER_PORT"
unixsocket = false
storagepath = "$DATA_DIR/uploads"
metricsenabled = true
metricsport = "$METRICS_PORT"
deduplicationenabled = true
deduplicationpath = "$DATA_DIR/deduplication"
filenaming = "HMAC"
forceprotocol = "auto"
EOF
if [[ $ENABLE_TLS == "true" ]]; then
cat >> "$DEFAULT_CONFIG_DIR/config.toml" << EOF
sslenabled = true
sslcert = "$SSL_CERT"
sslkey = "$SSL_KEY"
EOF
else
cat >> "$DEFAULT_CONFIG_DIR/config.toml" << EOF
sslenabled = false
EOF
fi
cat >> "$DEFAULT_CONFIG_DIR/config.toml" << EOF
[security]
secret = "$HMAC_SECRET"
enablejwt = $ENABLE_JWT
EOF
if [[ $ENABLE_JWT == "true" ]]; then
cat >> "$DEFAULT_CONFIG_DIR/config.toml" << EOF
jwtsecret = "$JWT_SECRET"
jwtalgorithm = "$JWT_ALGORITHM"
jwtexpiration = "$JWT_EXPIRATION"
EOF
fi
cat >> "$DEFAULT_CONFIG_DIR/config.toml" << EOF
[uploads]
allowedextensions = [".txt", ".pdf", ".jpg", ".jpeg", ".png", ".gif", ".webp", ".zip", ".tar", ".gz", ".7z", ".mp4", ".webm", ".ogg", ".mp3", ".wav", ".flac", ".doc", ".docx", ".xls", ".xlsx", ".ppt", ".pptx", ".odt", ".ods", ".odp"]
maxfilesize = "100MB"
chunkeduploadsenabled = true
chunksize = "10MB"
ttlenabled = false
ttl = "168h"
[downloads]
chunkeddownloadsenabled = true
chunksize = "10MB"
[logging]
loglevel = "INFO"
logfile = "$DEFAULT_LOG_DIR/hmac-file-server.log"
enablerotation = true
maxsize = 100
maxbackups = 3
maxage = 30
compress = true
[workers]
initial = 10
max = 100
idletimeout = "60s"
autoscaling = true
[timeouts]
read = "30s"
write = "30s"
idle = "120s"
shutdown = "30s"
EOF
if [[ $ENABLE_CLAMAV == "true" ]]; then
cat >> "$DEFAULT_CONFIG_DIR/config.toml" << EOF
[clamav]
enabled = true
socket = "/var/run/clamav/clamd.ctl"
timeout = "30s"
EOF
else
cat >> "$DEFAULT_CONFIG_DIR/config.toml" << EOF
[clamav]
enabled = false
EOF
fi
if [[ $ENABLE_REDIS == "true" ]]; then
cat >> "$DEFAULT_CONFIG_DIR/config.toml" << EOF
[redis]
enabled = true
host = "$REDIS_HOST"
port = $REDIS_PORT
database = $REDIS_DB
password = "$REDIS_PASSWORD"
timeout = "5s"
EOF
else
cat >> "$DEFAULT_CONFIG_DIR/config.toml" << EOF
[redis]
enabled = false
EOF
fi
# Set ownership and permissions
chown "$HMAC_USER:$HMAC_USER" "$DEFAULT_CONFIG_DIR/config.toml"
chmod 640 "$DEFAULT_CONFIG_DIR/config.toml"
echo -e "${GREEN}Configuration file created: $DEFAULT_CONFIG_DIR/config.toml${NC}"
}
# Create systemd service
create_systemd_service() {
echo -e "${YELLOW}Creating systemd service...${NC}"
cat > /etc/systemd/system/hmac-file-server.service << EOF
[Unit]
Description=HMAC File Server 3.2
Documentation=https://github.com/PlusOne/hmac-file-server
After=network.target
Wants=network-online.target
EOF
if [[ $ENABLE_REDIS == "true" ]]; then
echo "After=redis.service" >> /etc/systemd/system/hmac-file-server.service
fi
if [[ $ENABLE_CLAMAV == "true" ]]; then
echo "After=clamav-daemon.service" >> /etc/systemd/system/hmac-file-server.service
fi
cat >> /etc/systemd/system/hmac-file-server.service << EOF
[Service]
Type=simple
User=$HMAC_USER
Group=$HMAC_USER
ExecStart=$INSTALL_DIR/hmac-file-server -config $DEFAULT_CONFIG_DIR/config.toml
ExecReload=/bin/kill -SIGHUP \$MAINPID
WorkingDirectory=$INSTALL_DIR
Restart=always
RestartSec=10
StandardOutput=journal
StandardError=journal
SyslogIdentifier=hmac-file-server
# Security settings
NoNewPrivileges=true
PrivateTmp=true
ProtectSystem=strict
ProtectHome=true
ReadWritePaths=$DATA_DIR $DEFAULT_LOG_DIR
CapabilityBoundingSet=CAP_NET_BIND_SERVICE
AmbientCapabilities=CAP_NET_BIND_SERVICE
# Resource limits
LimitNOFILE=65536
LimitNPROC=4096
[Install]
WantedBy=multi-user.target
EOF
# Reload systemd and enable service
systemctl daemon-reload
systemctl enable hmac-file-server.service
echo -e "${GREEN}Systemd service created and enabled${NC}"
}
# Install dependencies
install_dependencies() {
echo -e "${YELLOW}Installing dependencies...${NC}"
# Detect package manager and install dependencies
if command -v apt-get &> /dev/null; then
apt-get update
if [[ $ENABLE_REDIS == "true" ]]; then
apt-get install -y redis-server
systemctl enable redis-server
fi
if [[ $ENABLE_CLAMAV == "true" ]]; then
apt-get install -y clamav clamav-daemon
systemctl enable clamav-daemon
# Update virus definitions
freshclam || true
fi
elif command -v yum &> /dev/null; then
if [[ $ENABLE_REDIS == "true" ]]; then
yum install -y redis
systemctl enable redis
fi
if [[ $ENABLE_CLAMAV == "true" ]]; then
yum install -y clamav clamav-update clamd
systemctl enable clamd
freshclam || true
fi
elif command -v dnf &> /dev/null; then
if [[ $ENABLE_REDIS == "true" ]]; then
dnf install -y redis
systemctl enable redis
fi
if [[ $ENABLE_CLAMAV == "true" ]]; then
dnf install -y clamav clamav-update clamd
systemctl enable clamd
freshclam || true
fi
else
echo -e "${YELLOW}Unknown package manager. Please install Redis and/or ClamAV manually if needed.${NC}"
fi
}
# Generate secure random key
generate_random_key() {
local length=${1:-48} # Default 48 characters for extra security
local key=""
# Try different methods in order of preference
if command -v openssl &> /dev/null; then
# Method 1: OpenSSL (most common and secure)
key=$(openssl rand -base64 $((length * 3 / 4 + 1)) | tr -d "=+/\n" | cut -c1-$length)
elif command -v head &> /dev/null && [[ -r /dev/urandom ]]; then
# Method 2: /dev/urandom with head (Linux/Unix)
key=$(head -c $((length * 3 / 4 + 1)) /dev/urandom | base64 | tr -d "=+/\n" | cut -c1-$length)
elif command -v dd &> /dev/null && [[ -r /dev/urandom ]]; then
# Method 3: dd with /dev/urandom
key=$(dd if=/dev/urandom bs=$((length * 3 / 4 + 1)) count=1 2>/dev/null | base64 | tr -d "=+/\n" | cut -c1-$length)
elif command -v date &> /dev/null; then
# Method 4: Fallback using date and process info (less secure but works)
local timestamp=$(date +%s%N)
local random_data="${timestamp}${RANDOM}${$}$(hostname)"
key=$(echo -n "$random_data" | sha256sum | cut -c1-$length)
else
# Method 5: Last resort - basic fallback
echo -e "${YELLOW}Warning: Using basic key generation (consider installing openssl)${NC}" >&2
key="hmac-file-server-$(date +%s)-$(hostname | cut -c1-16)"
key=$(echo -n "$key" | sha256sum | cut -c1-$length)
fi
# Ensure exact length
key=$(echo -n "$key" | cut -c1-$length)
# If still too short, pad with additional random data
while [[ ${#key} -lt $length ]]; do
local padding=$(date +%s | sha256sum | cut -c1-$((length - ${#key})))
key="${key}${padding}"
key=$(echo -n "$key" | cut -c1-$length)
done
echo "$key"
}
# Main installation function
main() {
echo -e "${BLUE}Starting HMAC File Server installation...${NC}"
echo ""
# Run pre-installation checks
pre_installation_checks
# Get user input
get_user_input
echo ""
echo -e "${BLUE}Installation Summary:${NC}"
echo "User: $HMAC_USER"
echo "Install Directory: $INSTALL_DIR"
echo "Data Directory: $DATA_DIR"
echo "Config Directory: $DEFAULT_CONFIG_DIR"
echo "Server Port: $SERVER_PORT"
echo "Metrics Port: $METRICS_PORT"
echo "JWT Enabled: $ENABLE_JWT"
echo "Redis Enabled: $ENABLE_REDIS"
echo "ClamAV Enabled: $ENABLE_CLAMAV"
echo "TLS Enabled: $ENABLE_TLS"
echo ""
read -p "Continue with installation? (y/N): " CONFIRM
if [[ ! $CONFIRM =~ ^[Yy]$ ]]; then
echo -e "${YELLOW}Installation cancelled.${NC}"
exit 0
fi
echo ""
echo -e "${BLUE}Installing...${NC}"
# Installation steps
check_go
create_user
create_directories
install_dependencies
build_server
generate_config
create_systemd_service
# Ask if user wants to start the service now
echo ""
read -p "Start HMAC File Server service now? (Y/n): " START_SERVICE
START_SERVICE=${START_SERVICE:-Y}
if [[ $START_SERVICE =~ ^[Yy]$ ]]; then
echo -e "${YELLOW}Starting HMAC File Server service...${NC}"
systemctl start hmac-file-server.service
# Wait a moment and check status
sleep 3
if systemctl is-active --quiet hmac-file-server.service; then
echo -e "${GREEN}✅ Service started successfully${NC}"
else
echo -e "${RED}❌ Service failed to start. Check logs with: journalctl -u hmac-file-server.service${NC}"
fi
fi
print_completion_info
}
# Function to print completion information
print_completion_info() {
echo ""
echo -e "${GREEN} █ Installation Complete! █${NC}"
echo -e "${GREEN}────────────────────────────────────────────────────────────────${NC}"
echo -e "${GREEN} HMAC File Server 3.2 Successfully Deployed! ${NC}"
echo -e "${GREEN}────────────────────────────────────────────────────────────────${NC}"
echo ""
echo -e "${BLUE}🚀 Service Information:${NC}"
echo -e " Status: ${YELLOW}sudo systemctl status hmac-file-server${NC}"
echo -e " Logs: ${YELLOW}sudo journalctl -u hmac-file-server -f${NC}"
echo -e " Config: ${YELLOW}sudo nano $DEFAULT_CONFIG_DIR/config.toml${NC}"
echo -e " Reload: ${YELLOW}sudo systemctl reload hmac-file-server${NC}"
echo ""
echo -e "${BLUE}🌐 Service Endpoints:${NC}"
if [[ $ENABLE_TLS == "true" ]]; then
echo -e " Server: ${YELLOW}https://$(hostname -I | awk '{print $1}'):$SERVER_PORT${NC}"
else
echo -e " Server: ${YELLOW}http://$(hostname -I | awk '{print $1}'):$SERVER_PORT${NC}"
fi
echo -e " Metrics: ${YELLOW}http://$(hostname -I | awk '{print $1}'):$METRICS_PORT/metrics${NC}"
echo ""
echo -e "${BLUE}📁 File Locations:${NC}"
echo -e " Binary: ${YELLOW}$INSTALL_DIR/hmac-file-server${NC}"
echo -e " Config: ${YELLOW}$DEFAULT_CONFIG_DIR/config.toml${NC}"
echo -e " Uploads: ${YELLOW}$DATA_DIR/uploads${NC}"
echo -e " Logs: ${YELLOW}$DEFAULT_LOG_DIR/hmac-file-server.log${NC}"
echo ""
echo -e "${BLUE}⚡ Quick Commands:${NC}"
echo -e " Start: ${YELLOW}sudo systemctl start hmac-file-server${NC}"
echo -e " Stop: ${YELLOW}sudo systemctl stop hmac-file-server${NC}"
echo -e " Restart: ${YELLOW}sudo systemctl restart hmac-file-server${NC}"
echo -e " Status: ${YELLOW}sudo systemctl status hmac-file-server${NC}"
echo ""
echo -e "${BLUE}🔧 Next Steps for XMPP Integration:${NC}"
echo "1. ${YELLOW}Configure firewall${NC} to allow ports $SERVER_PORT (server) and $METRICS_PORT (metrics)"
echo "2. Configure your reverse proxy (nginx/apache) with SSL"
echo "3. Update your Prosody/Ejabberd configuration:"
echo -e " ${YELLOW}http_file_share = \"http://localhost:$SERVER_PORT\"${NC}"
echo "4. Set up monitoring and log rotation"
echo "5. Test file uploads with your XMPP client"
echo ""
echo -e "${BLUE}📚 Documentation & Support:${NC}"
echo " README: https://github.com/PlusOne/hmac-file-server/blob/main/README.MD"
echo " Wiki: https://github.com/PlusOne/hmac-file-server/blob/main/WIKI.MD"
echo " Issues: https://github.com/PlusOne/hmac-file-server/issues"
echo ""
echo -e "${GREEN}────────────────────────────────────────────────────────────────${NC}"
echo -e "${GREEN} Thank you for choosing HMAC File Server for your XMPP setup! ${NC}"
echo -e "${GREEN}────────────────────────────────────────────────────────────────${NC}"
}
# Uninstaller function (can be called with ./installer.sh --uninstall)
uninstall() {
echo ""
echo -e "${RED} █ HMAC File Server Uninstaller █${NC}"
echo -e "${RED}────────────────────────────────────────────────────────────────${NC}"
echo -e "${RED} Warning: This will remove all server data! ${NC}"
echo -e "${RED}────────────────────────────────────────────────────────────────${NC}"
echo ""
read -p "Are you sure you want to uninstall HMAC File Server? This will remove all data! (y/N): " CONFIRM_UNINSTALL
if [[ ! $CONFIRM_UNINSTALL =~ ^[Yy]$ ]]; then
echo -e "${YELLOW}Uninstall cancelled${NC}"
exit 0
fi
echo -e "${YELLOW}Stopping and disabling service...${NC}"
if systemctl is-active --quiet hmac-file-server.service; then
systemctl stop hmac-file-server.service || true
echo " ✓ Service stopped"
else
echo " ⚠ Service was not running"
fi
if systemctl is-enabled --quiet hmac-file-server.service 2>/dev/null; then
systemctl disable hmac-file-server.service || true
echo " ✓ Service disabled"
else
echo " ⚠ Service was not enabled"
fi
if [[ -f /etc/systemd/system/hmac-file-server.service ]]; then
rm -f /etc/systemd/system/hmac-file-server.service
echo " ✓ Service file removed"
else
echo " ⚠ Service file not found"
fi
systemctl daemon-reload
echo " ✓ Systemd reloaded"
echo -e "${YELLOW}Removing installation directories...${NC}"
for dir in "$DEFAULT_INSTALL_DIR" "$DEFAULT_CONFIG_DIR" "$DEFAULT_DATA_DIR" "$DEFAULT_LOG_DIR"; do
if [[ -d "$dir" ]]; then
rm -rf "$dir"
echo " ✓ Removed $dir"
else
echo " ⚠ Directory $dir not found"
fi
done
echo -e "${YELLOW}Removing system user...${NC}"
if id "$DEFAULT_USER" &>/dev/null; then
userdel "$DEFAULT_USER" || true
echo " ✓ User $DEFAULT_USER removed"
else
echo " ⚠ User $DEFAULT_USER not found"
fi
# Remove any remaining binary in common locations
echo -e "${YELLOW}Cleaning up any remaining files...${NC}"
for location in "/usr/local/bin/hmac-file-server" "/usr/bin/hmac-file-server"; do
if [[ -f "$location" ]]; then
rm -f "$location"
echo " ✓ Removed $location"
fi
done
echo ""
echo -e "${GREEN}✅ HMAC File Server uninstalled successfully${NC}"
echo -e "${BLUE}All files, services, and user accounts have been removed.${NC}"
}
# Check for help flag
if [[ "$1" == "--help" || "$1" == "-h" ]]; then
show_help
exit 0
fi
# Check for uninstall flag
if [[ "$1" == "--uninstall" ]]; then
uninstall
exit 0
fi
# Run main function
main "$@"