uuxo/hmac-file-server
1
1
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 2 Wiki Activity

2.4-stable release

Browse Source
This commit is contained in:
Alexander Renz 2024-12-31 08:34:16 +01:00
parent e8af2d127b
commit aab6b07316
8 changed files with 919 additions and 55 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

4
README.MD
View File

@ -439,4 +439,8 @@ This project is licensed under the [MIT License](LICENSE).
--- ---
## Changelog
For a detailed list of changes, please refer to the [Changelog](./Changelog.md).
By following this updated `README.md`, you can ensure that users and contributors have a clear understanding of the **HMAC File Server**'s capabilities, configuration options, and best practices for deployment and maintenance. By following this updated `README.md`, you can ensure that users and contributors have a clear understanding of the **HMAC File Server**'s capabilities, configuration options, and best practices for deployment and maintenance.

66
RELEASE-NOTES.MD
View File

@ -253,48 +253,12 @@ curl -X PUT -H "Authorization: Bearer <HMAC-TOKEN>" -F "file=@example.txt" http:
``` ```
Replace `<HMAC-TOKEN>` with a valid HMAC signature Replace `<HMAC-TOKEN>` with a valid HMAC signature
````
## [2.3.0] - 2024-12-28
## [2.4.1] - 2025-03-10
### Changed ### Changed
- **Server:** Replaced the hardcoded temporary upload directory `/tmp/uploads` with a configurable `TempPath` parameter in `config.toml`. Ensure to set `tempPath` in your configuration file accordingly. - **Configuration:** Updated `globalextensions` in `config.toml` to `["*"]`, allowing all file types globally for uploads. This change simplifies the configuration by removing the need to specify individual file extensions.
#### Version 2.2.1 - 2024-12-27
- **Enhancements:**
- Added detailed logging for file extensions and MIME types during file uploads to assist in diagnosing unsupported file type issues.
- **Configuration:**
- Updated `config.toml` to ensure necessary file extensions are allowed for uploads.
#### Version 2.2.2 - 2024-12-27
- **Bug Fixes:**
- Resolved issue where temporary `.tmp` files caused "Unsupported file type" warnings by adjusting MIME type detection to use the final file extension.
- **Enhancements:**
- Improved logging for file extension and MIME type during uploads.
### Summary of Changes
1. **Added Log Message Limitation:**
- Implemented `logMessages = logMessages[1:]` to remove the oldest log message once the `logMessages` slice exceeds 1000 entries, preventing unbounded memory growth.
---
### Additional Files
2024-12-30 endpoint /thumbnails
No changes are required for the other files (`RELEASE-NOTES.MD`, `README.MD`, `config.toml`).
## [2.3.1] - 2025-01-15
- Added metrics for deduplication and ISO container operations to improve observability.
- Additional detail: Thumbnails now support a "concurrency" configuration parameter for parallel image processing.
## [2.4.0] - 2025-02-20 ## [2.4.0] - 2025-02-20
### Added ### Added
- **Pre-Caching Support:** Introduced pre-caching of storage paths to improve access speeds. - **Pre-Caching Support:** Introduced pre-caching of storage paths to improve access speeds.
- **ISO Container Management:** Added functionality to create and mount ISO containers for specialized storage needs. - **ISO Container Management:** Added functionality to create and mount ISO containers for specialized storage needs.
@ -307,12 +271,24 @@ No changes are required for the other files (`RELEASE-NOTES.MD`, `README.MD`, `c
### Fixed ### Fixed
- **Bug Fixes:** Resolved minor issues related to file versioning and deduplication processes. - **Bug Fixes:** Resolved minor issues related to file versioning and deduplication processes.
### Summary of Changes ## [2.3.1] - 2025-01-15
1. **Pre-Caching:** Improved performance by allowing the server to pre-cache frequently accessed storage paths.
2. **ISO Management:** Enabled the creation and mounting of ISO containers for better filesystem management.
3. **Thumbnail Generation:** Added a concurrency setting to allow parallel processing of image thumbnails, speeding up the generation process.
## [2.4.1] - 2025-03-10
### Changed ### Changed
- **Configuration:** Updated `globalextensions` in `config.toml` to `["*"]`, allowing all file types globally for uploads. This change simplifies the configuration by removing the need to specify individual file extensions. - **Configuration:** Updated `globalextensions` in `config.toml` to `["*"]`, allowing all file types globally for uploads. This change simplifies the configuration by removing the need to specify individual file extensions.
## [2.3.0] - 2024-12-28
### Changed
- **Server:** Replaced the hardcoded temporary upload directory `/tmp/uploads` with a configurable `TempPath` parameter in `config.toml`. Ensure to set `tempPath` in your configuration file accordingly.
## [2.2.2] - 2024-12-27
### Bug Fixes
- Resolved issue where temporary `.tmp` files caused "Unsupported file type" warnings by adjusting MIME type detection to use the final file extension.
### Enhancements
- Improved logging for file extension and MIME type during uploads.
## [2.2.1] - 2024-12-27
### Enhancements
- Added detailed logging for file extensions and MIME types during file uploads to assist in diagnosing unsupported file type issues.
### Configuration
- Updated `config.toml` to ensure necessary file extensions are allowed for uploads.

892
WiKi.md Normal file
View File

@ -0,0 +1,892 @@
This documentation provides detailed information on configuring, setting up, and maintaining the HMAC File Server. Whether you're a developer, system administrator, or an enthusiast, this guide will help you navigate through the server's features and configurations effectively.
---
## Table of Contents
1. [Introduction](#introduction)
2. [Configuration](#configuration)
- [Server Configuration](#server-configuration)
- [Deduplication Settings](#deduplication-settings)
- [Thumbnails Settings](#thumbnails-settings)
- [ISO Settings](#iso-settings)
- [Timeout Settings](#timeout-settings)
- [Security Settings](#security-settings)
- [Versioning Settings](#versioning-settings)
- [Uploads Settings](#uploads-settings)
- [Downloads Settings](#downloads-settings)
- [ClamAV Settings](#clamav-settings)
- [Redis Settings](#redis-settings)
- [Worker Settings](#worker-settings)
- [File Settings](#file-settings)
3. [Example Configuration](#example-configuration)
4. [Setup Instructions](#setup-instructions)
- [1. HMAC File Server Installation](#1-hmac-file-server-installation)
- [2. Reverse Proxy Configuration](#2-reverse-proxy-configuration)
- [Apache2 Reverse Proxy](#apache2-reverse-proxy)
- [Nginx Reverse Proxy](#nginx-reverse-proxy)
- [3. ejabberd Configuration](#3-ejabberd-configuration)
- [4. Systemd Service Setup](#4-systemd-service-setup)
5. [Building for Different Architectures](#building-for-different-architectures)
6. [Additional Recommendations](#additional-recommendations)
7. [Notes](#notes)
---
## Introduction
The **HMAC File Server** is a secure and efficient file management solution designed to handle file uploads, downloads, deduplication, and more. Built with a focus on security, scalability, and performance, it integrates seamlessly with various tools and services to provide a comprehensive file handling experience.
---
## Configuration
The HMAC File Server is configured using a `config.toml` file. Below are the detailed explanations of each configuration section and their respective options.
### Server Configuration
```toml
# Server configuration
listenport = "8080" # TCP port for incoming requests
unixsocket = false # Use Unix domain socket instead of TCP
storagepath = "/path/to/hmac-file-server/data/" # Directory to store uploaded files
loglevel = "debug" # Logging level: "debug", "info", "warn", "error"
logfile = "/path/to/hmac-file-server.log" # Path to log file; leave empty to use stdout
metricsenabled = true # Enable Prometheus metrics
metricsport = "9090" # Port for Prometheus metrics
deduplicationenabled = true
minfreebytes = "5GB" # Minimum free disk space required
filettl = "2Y" # Time-to-live for files
filettlenabled = false # Enable TTL checks and cleanup
autoadjustworkers = true # Automatically adjust worker threads based on load
networkevents = false # Enable detailed network event logging
pidfilepath = "./hmac-file-server.pid" # Path to PID file
precaching = true # Pre-cache file structures on startup
```
#### Configuration Options
- **listenport**:
- *Type*: `String`
- *Description*: Specifies the TCP port on which the server listens for incoming requests.
- *Default*: `"8080"`
- **unixsocket**:
- *Type*: `Boolean`
- *Description*: Determines whether to use a Unix domain socket instead of a TCP port for communication.
- *Default*: `false`
- **storagepath**:
- *Type*: `String`
- *Description*: Defines the directory path where uploaded files are stored. Ensure this path exists and has appropriate permissions.
- *Default*: `"/path/to/hmac-file-server/data/"`
- **loglevel**:
- *Type*: `String`
- *Description*: Sets the verbosity level of logs.
- *Options*: `"debug"`, `"info"`, `"warn"`, `"error"`
- *Default*: `"debug"`
- **logfile**:
- *Type*: `String`
- *Description*: Specifies the file path for logging. If left empty, logs are output to `stdout`.
- *Default*: `"/path/to/hmac-file-server.log"`
- **metricsenabled**:
- *Type*: `Boolean`
- *Description*: Enables or disables the Prometheus metrics endpoint.
- *Default*: `true`
- **metricsport**:
- *Type*: `String`
- *Description*: Defines the port on which Prometheus metrics are exposed.
- *Default*: `"9090"`
- **deduplicationenabled**:
- *Type*: `Boolean`
- *Description*: Enables or disables file deduplication to optimize storage usage.
- *Default*: `true`
- **minfreebytes**:
- *Type*: `String`
- *Description*: Specifies the minimum free disk space required for the server to operate effectively.
- *Default*: `"5GB"`
- **filettl**:
- *Type*: `String`
- *Description*: Sets the default Time-to-Live (TTL) for files, determining how long files are retained before deletion.
- *Format*: Duration (e.g., `"2Y"` for two years)
- *Default*: `"2Y"`
- **filettlenabled**:
- *Type*: `Boolean`
- *Description*: Enables or disables TTL checks and automatic file cleanup based on the `filettl` value.
- *Default*: `false`
- **autoadjustworkers**:
- *Type*: `Boolean`
- *Description*: Automatically adjusts the number of worker threads based on server load and system resources.
- *Default*: `true`
- **networkevents**:
- *Type*: `Boolean`
- *Description*: Enables detailed logging of network events, which can be useful for debugging but may increase log verbosity.
- *Default*: `false`
- **pidfilepath**:
- *Type*: `String`
- *Description*: Specifies the file path where the server writes its Process ID (PID) file. This is useful for managing the server process.
- *Default*: `"./hmac-file-server.pid"`
- **precaching**:
- *Type*: `Boolean`
- *Description*: Enables pre-caching of file structures on startup to improve access speed and performance.
- *Default*: `true`
---
### Deduplication Settings
```toml
# Deduplication settings
[deduplication]
enabled = true
directory = "/path/to/hmac-file-server/deduplication/" # Path to deduplication metadata store
```
#### Configuration Options
- **enabled**:
- *Type*: `Boolean`
- *Description*: Enables or disables the deduplication feature, which helps in eliminating duplicate files to save storage space.
- *Default*: `true`
- **directory**:
- *Type*: `String`
- *Description*: Specifies the directory path where deduplication metadata is stored. Ensure this directory exists and has appropriate permissions.
- *Default*: `"/path/to/hmac-file-server/deduplication/"`
---
### Thumbnails Settings
```toml
# Thumbnails settings
[thumbnails]
enabled = true
directory = "/path/to/hmac-file-server/thumbnails/" # Directory for storing thumbnails
size = "200x200" # Thumbnail dimensions
thumbnailintervalscan = "1h" # Interval for scheduled thumbnail generation
```
#### Configuration Options
- **enabled**:
- *Type*: `Boolean`
- *Description*: Enables or disables the automatic generation of thumbnails for uploaded images.
- *Default*: `true`
- **directory**:
- *Type*: `String`
- *Description*: Defines the directory where generated thumbnails are stored. Ensure this directory exists and has appropriate permissions.
- *Default*: `"/path/to/hmac-file-server/thumbnails/"`
- **size**:
- *Type*: `String`
- *Description*: Specifies the dimensions to which thumbnails are resized. The format should be `"widthxheight"` (e.g., `"200x200"`).
- *Default*: `"200x200"`
- **thumbnailintervalscan**:
- *Type*: `String`
- *Description*: Sets the frequency at which the server scans for new or updated images to generate thumbnails.
- *Format*: Duration (e.g., `"1h"` for one hour)
- *Default*: `"1h"`
---
### ISO Settings
```toml
# ISO settings
[iso]
enabled = false
size = "1TB" # Maximum ISO size
mountpoint = "/path/to/hmac-file-server/iso/" # ISO mount point
charset = "utf-8" # Filesystem character set encoding
```
#### Configuration Options
- **enabled**:
- *Type*: `Boolean`
- *Description*: Enables or disables the mounting of an ISO-based filesystem for specialized storage needs.
- *Default*: `false`
- **size**:
- *Type*: `String`
- *Description*: Defines the maximum allowed size for the ISO container.
- *Default*: `"1TB"`
- **mountpoint**:
- *Type*: `String`
- *Description*: Specifies the directory path where the ISO is mounted. Ensure this path exists and has appropriate permissions.
- *Default*: `"/path/to/hmac-file-server/iso/"`
- **charset**:
- *Type*: `String`
- *Description*: Sets the filesystem character set encoding for the ISO.
- *Default*: `"utf-8"`
> **Note**: Ensure only one `[iso]` block is active in your `config.toml` to avoid configuration conflicts.
---
### Timeout Settings
```toml
# Timeout settings
[timeouts]
readtimeout = "3600s" # Maximum time to read a request
writetimeout = "3600s" # Maximum time to write a response
idletimeout = "3600s" # Maximum keep-alive time for idle connections
```
#### Configuration Options
- **readtimeout**:
- *Type*: `String`
- *Description*: Sets the maximum duration for reading the entire request, including the body.
- *Format*: Duration (e.g., `"3600s"` for one hour)
- *Default*: `"3600s"`
- **writetimeout**:
- *Type*: `String`
- *Description*: Defines the maximum duration before timing out writes of the response.
- *Format*: Duration (e.g., `"3600s"` for one hour)
- *Default*: `"3600s"`
- **idletimeout**:
- *Type*: `String`
- *Description*: Specifies the maximum amount of time to wait for the next request when keep-alives are enabled.
- *Format*: Duration (e.g., `"3600s"` for one hour)
- *Default*: `"3600s"`
---
### Security Settings
```toml
# Security settings
[security]
secret = "your-secure-secret-key" # HMAC shared secret key (change to a secure value)
```
#### Configuration Options
- **secret**:
- *Type*: `String`
- *Description*: The HMAC shared secret key used for signing requests and operations.
- *Default*: `"your-secure-secret-key"`
- *Warning*: **Change this immediately** to a unique, strong string in production environments to ensure the security of HMAC operations.
---
### Versioning Settings
```toml
# Versioning settings
[versioning]
enableversioning = false
maxversions = 1 # Number of file versions to retain
```
#### Configuration Options
- **enableversioning**:
- *Type*: `Boolean`
- *Description*: Enables or disables the versioning feature, which maintains multiple versions of the same file.
- *Default*: `false`
- **maxversions**:
- *Type*: `Integer`
- *Description*: Specifies the maximum number of versions to retain for each file.
- *Default*: `1`
---
### Uploads Settings
```toml
# Upload settings
[uploads]
resumableuploadsenabled = false
chunkeduploadsenabled = true
chunksize = "32MB" # Chunk size for uploads
allowedextensions = [
".txt", ".pdf", ".png", ".jpg", ".jpeg", ".gif",
".bmp", ".tiff", ".svg", ".webp", ".wav", ".mp4",
".avi", ".mkv", ".mov", ".wmv", ".flv", ".webm",
".mpeg", ".mpg", ".m4v", ".3gp", ".3g2", ".mp3", ".ogg"
]
```
#### Configuration Options
- **resumableuploadsenabled**:
- *Type*: `Boolean`
- *Description*: Enables or disables support for resumable (chunked) file uploads.
- *Default*: `false`
- **chunkeduploadsenabled**:
- *Type*: `Boolean`
- *Description*: Specifically enables or disables chunked uploads.
- *Default*: `true`
- **chunksize**:
- *Type*: `String`
- *Description*: Defines the size of each chunk in chunked uploads.
- *Format*: Size (e.g., `"32MB"`)
- *Default*: `"32MB"`
- **allowedextensions**:
- *Type*: `Array of Strings`
- *Description*: Lists the file extensions permitted for upload.
- *Default*:
```toml
allowedextensions = [
".txt", ".pdf", ".png", ".jpg", ".jpeg", ".gif",
".bmp", ".tiff", ".svg", ".webp", ".wav", ".mp4",
".avi", ".mkv", ".mov", ".wmv", ".flv", ".webm",
".mpeg", ".mpg", ".m4v", ".3gp", ".3g2", ".mp3", ".ogg"
]
```
---
### Downloads Settings
```toml
# Downloads settings
[downloads]
resumabledownloadsenabled = false
chunkeddownloadsenabled = true
chunksize = "32MB"
```
#### Configuration Options
- **resumabledownloadsenabled**:
- *Type*: `Boolean`
- *Description*: Enables or disables support for resumable (chunked) downloads.
- *Default*: `false`
- **chunkeddownloadsenabled**:
- *Type*: `Boolean`
- *Description*: Specifically enables or disables chunked downloads.
- *Default*: `true`
- **chunksize**:
- *Type*: `String`
- *Description*: Defines the size of each chunk in chunked downloads.
- *Format*: Size (e.g., `"32MB"`)
- *Default*: `"32MB"`
> **Note**: The `allowedextensions` key is **not** part of the `[downloads]` configuration based on the provided code. Ensure that it is omitted to prevent configuration errors.
---
### ClamAV Settings
```toml
# ClamAV settings
[clamav]
clamavenabled = true
clamavsocket = "/path/to/clamav/clamd.ctl" # Path to ClamAV socket
numscanworkers = 4 # Number of concurrent scan workers
scanfileextensions = [
".exe", ".dll", ".bin", ".com", ".bat",
".sh", ".php", ".js"
]
```
#### Configuration Options
- **clamavenabled**:
- *Type*: `Boolean`
- *Description*: Enables or disables ClamAV integration for virus scanning of uploaded files.
- *Default*: `true`
- **clamavsocket**:
- *Type*: `String`
- *Description*: Specifies the file path to the ClamAV socket (`.ctl` file). Ensure ClamAV is installed and the socket path is correct.
- *Default*: `"/path/to/clamav/clamd.ctl"`
- **numscanworkers**:
- *Type*: `Integer`
- *Description*: Sets the number of concurrent workers dedicated to scanning files with ClamAV.
- *Default*: `4`
- **scanfileextensions**:
- *Type*: `Array of Strings`
- *Description*: Lists the file extensions that should be scanned for viruses.
- *Default*:
```toml
scanfileextensions = [
".exe", ".dll", ".bin", ".com", ".bat",
".sh", ".php", ".js"
]
```
---
### Redis Settings
```toml
# Redis settings
[redis]
redisenabled = true
redisdbindex = 0
redisaddr = "localhost:6379" # Redis server address
redispassword = "" # Redis password if required
redishealthcheckinterval = "120s" # Interval for Redis health checks
```
#### Configuration Options
- **redisenabled**:
- *Type*: `Boolean`
- *Description*: Enables or disables Redis integration for caching or session tracking.
- *Default*: `true`
- **redisaddr**:
- *Type*: `String`
- *Description*: Specifies the address of the Redis server (e.g., `"localhost:6379"`).
- *Default*: `"localhost:6379"`
- **redispassword**:
- *Type*: `String`
- *Description*: Sets the Redis authentication password, if required.
- *Default*: `""`
- **redisdbindex**:
- *Type*: `Integer`
- *Description*: Specifies the Redis database index to use.
- *Default*: `0`
- **redishealthcheckinterval**:
- *Type*: `String`
- *Description*: Defines the interval for performing health checks on the Redis connection.
- *Format*: Duration (e.g., `"120s"` for two minutes)
- *Default*: `"120s"`
---
### Worker Settings
```toml
# Worker settings
[worker]
numworkers = 10 # Number of worker threads
```
#### Configuration Options
- **numworkers**:
- *Type*: `Integer`
- *Description*: Specifies the number of worker threads to handle file operations.
- *Default*: `10`
---
### File Settings
```toml
# File settings
[file]
maxfilesize = "10GB" # Maximum file size for uploads
```
#### Configuration Options
- **maxfilesize**:
- *Type*: `String`
- *Description*: Defines the maximum allowed file size for uploads.
- *Format*: Size (e.g., `"10GB"`)
- *Default*: `"10GB"`
---
## Example Configuration
Below is an example `config.toml` file with default settings:
```toml
# Example HMAC File Server configuration
# Server configuration
listenport = "8080"
unixsocket = false
storagepath = "/path/to/hmac-file-server/data/"
loglevel = "debug"
logfile = "/path/to/hmac-file-server.log"
metricsenabled = true
metricsport = "9090"
deduplicationenabled = true
minfreebytes = "5GB"
filettl = "2Y"
filettlenabled = false
autoadjustworkers = true
networkevents = false
pidfilepath = "./hmac-file-server.pid"
precaching = true
# Deduplication settings
[deduplication]
enabled = true
directory = "/path/to/hmac-file-server/deduplication/"
# Thumbnails settings
[thumbnails]
enabled = true
directory = "/path/to/hmac-file-server/thumbnails/"
size = "200x200"
thumbnailintervalscan = "1h"
# ISO settings
[iso]
enabled = false
size = "1TB"
mountpoint = "/path/to/hmac-file-server/iso/"
charset = "utf-8"
# Timeout settings
[timeouts]
readtimeout = "3600s"
writetimeout = "3600s"
idletimeout = "3600s"
# Security settings
[security]
secret = "your-secure-secret-key"
# Versioning settings
[versioning]
enableversioning = false
maxversions = 1
# Upload settings
[uploads]
resumableuploadsenabled = false
chunkeduploadsenabled = true
chunksize = "32MB"
allowedextensions = [
".txt", ".pdf", ".png", ".jpg", ".jpeg", ".gif",
".bmp", ".tiff", ".svg", ".webp", ".wav", ".mp4",
".avi", ".mkv", ".mov", ".wmv", ".flv", ".webm",
".mpeg", ".mpg", ".m4v", ".3gp", ".3g2", ".mp3", ".ogg"
]
# Download settings
[downloads]
resumabledownloadsenabled = false
chunkeddownloadsenabled = true
chunksize = "32MB"
# ClamAV settings
[clamav]
clamavenabled = true
clamavsocket = "/path/to/clamav/clamd.ctl"
numscanworkers = 4
scanfileextensions = [
".exe", ".dll", ".bin", ".com", ".bat",
".sh", ".php", ".js"
]
# Redis settings
[redis]
redisenabled = true
redisdbindex = 0
redisaddr = "localhost:6379"
redispassword = ""
redishealthcheckinterval = "120s"
# Worker settings
[worker]
numworkers = 10
# File settings
[file]
maxfilesize = "10GB"
```
---
## Setup Instructions
### 1. HMAC File Server Installation
To install the HMAC File Server, follow these steps:
1. Clone the repository:
```sh
git clone https://github.com/your-repo/hmac-file-server.git
cd hmac-file-server
```
2. Build the server:
```sh
go build -o hmac-file-server
```
3. Create the necessary directories:
```sh
mkdir -p /path/to/hmac-file-server/data/
mkdir -p /path/to/hmac-file-server/deduplication/
mkdir -p /path/to/hmac-file-server/thumbnails/
mkdir -p /path/to/hmac-file-server/iso/
```
4. Copy the example configuration file:
```sh
cp config.example.toml config.toml
```
5. Edit the `config.toml` file to match your environment and preferences.
6. Start the server:
```sh
./hmac-file-server -config config.toml
```
### 2. Reverse Proxy Configuration
To set up a reverse proxy for the HMAC File Server, you can use either Apache2 or Nginx. Below are the configuration examples for both.
#### Apache2 Reverse Proxy
1. Enable the necessary Apache2 modules:
```sh
sudo a2enmod proxy
sudo a2enmod proxy_http
sudo a2enmod headers
sudo a2enmod rewrite
```
2. Create a new virtual host configuration file:
```sh
sudo nano /etc/apache2/sites-available/hmac-file-server.conf
```
3. Add the following configuration to the file:
```apache
<VirtualHost *:80>
ServerName your-domain.com
ProxyPreserveHost On
ProxyPass / http://localhost:8080/
ProxyPassReverse / http://localhost:8080/
<Location />
Require all granted
Header always set X-Content-Type-Options "nosniff"
Header always set X-Frame-Options "DENY"
Header always set X-XSS-Protection "1; mode=block"
</Location>
</VirtualHost>
```
4. Enable the new site and restart Apache2:
```sh
sudo a2ensite hmac-file-server.conf
sudo systemctl restart apache2
```
#### Nginx Reverse Proxy
1. Install Nginx if not already installed:
```sh
sudo apt-get update
sudo apt-get install nginx
```
2. Create a new server block configuration file:
```sh
sudo nano /etc/nginx/sites-available/hmac-file-server
```
3. Add the following configuration to the file:
```nginx
server {
listen 80;
server_name your-domain.com;
location / {
proxy_pass http://localhost:8080;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header X-Content-Type-Options "nosniff";
proxy_set_header X-Frame-Options "DENY";
proxy_set_header X-XSS-Protection "1; mode=block";
}
}
```
4. Enable the new site and restart Nginx:
```sh
sudo ln -s /etc/nginx/sites-available/hmac-file-server /etc/nginx/sites-enabled/
sudo systemctl restart nginx
```
### 3. ejabberd Configuration
To configure ejabberd for use with the HMAC File Server, follow these steps:
1. Install ejabberd:
```sh
sudo apt-get update
sudo apt-get install ejabberd
```
2. Edit the ejabberd configuration file:
```sh
sudo nano /etc/ejabberd/ejabberd.yml
```
3. Add the following configuration to the file:
```yaml
hosts:
- "your-domain.com"
listen:
-
port: 5222
module: ejabberd_c2s
certfile: "/etc/ejabberd/ejabberd.pem"
starttls: true
starttls_required: true
protocol_options:
- "no_sslv3"
- "no_tlsv1"
- "no_tlsv1_1"
ciphers: "HIGH:!aNULL:!eNULL:!3DES:@STRENGTH"
dhfile: "/etc/ejabberd/dhparams.pem"
max_stanza_size: 65536
shaper: c2s_shaper
access: c2s
-
port: 5269
module: ejabberd_s2s_in
certfile: "/etc/ejabberd/ejabberd.pem"
starttls: true
starttls_required: true
protocol_options:
- "no_sslv3"
- "no_tlsv1"
- "no_tlsv1_1"
ciphers: "HIGH:!aNULL:!eNULL:!3DES:@STRENGTH"
dhfile: "/etc/ejabberd/dhparams.pem"
max_stanza_size: 131072
shaper: s2s_shaper
access: s2s
acl:
local:
user_regexp: ""
access_rules:
local:
allow: local
```
4. Restart ejabberd:
```sh
sudo systemctl restart ejabberd
```
### 4. Systemd Service Setup
To set up the HMAC File Server as a systemd service, follow these steps:
1. Create a new systemd service file:
```sh
sudo nano /etc/systemd/system/hmac-file-server.service
```
2. Add the following configuration to the file:
```ini
[Unit]
Description=HMAC File Server
After=network.target
[Service]
ExecStart=/path/to/hmac-file-server -config /path/to/config.toml
WorkingDirectory=/path/to/hmac-file-server
Restart=always
User=www-data
Group=www-data
[Install]
WantedBy=multi-user.target
```
3. Reload systemd and enable the service:
```sh
sudo systemctl daemon-reload
sudo systemctl enable hmac-file-server
sudo systemctl start hmac-file-server
```
---
## Building for Different Architectures
To build the HMAC File Server for different architectures, you can use the following commands:
### Building for Linux (x86_64)
```sh
GOOS=linux GOARCH=amd64 go build -o hmac-file-server-linux-amd64
```
### Building for macOS (x86_64)
```sh
GOOS=darwin GOARCH=amd64 go build -o hmac-file-server-darwin-amd64
```
### Building for Windows (x86_64)
```sh
GOOS=windows GOARCH=amd64 go build -o hmac-file-server-windows-amd64.exe
```
### Building for ARM (32-bit)
```sh
GOOS=linux GOARCH=arm GOARM=7 go build -o hmac-file-server-linux-arm
```
### Building for ARM (64-bit)
```sh
GOOS=linux GOARCH=arm64 go build -o hmac-file-server-linux-arm64
```
---
## Additional Recommendations
- **Security**: Ensure that the `secret` key in the `config.toml` file is changed to a unique, strong value to secure HMAC operations.
- **Backups**: Regularly back up the `config.toml` file and any important data stored by the HMAC File Server.
- **Monitoring**: Use monitoring tools like Prometheus and Grafana to keep track of server performance and metrics.
---
## Notes
- The HMAC File Server is designed to be flexible and configurable. Adjust the settings in the `config.toml` file to match your specific requirements and environment.
- For any issues or questions, refer to the project's GitHub repository and documentation.
````

2
changelog
View File

@ -44,4 +44,4 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/).
- Added detailed logging for file extensions and MIME types during file uploads to assist in diagnosing unsupported file type issues. - Added detailed logging for file extensions and MIME types during file uploads to assist in diagnosing unsupported file type issues.
### Configuration ### Configuration
- Updated `config.toml` to ensure necessary file extensions are allowed for uploads. - Updated `config.toml` to ensure necessary file extensions are allowed for uploads.

BIN
cmd/server/hmac-file-server Executable file
View File

Binary file not shown.

8
cmd/server/main.go
View File

@ -1508,14 +1508,6 @@ func processUpload(task UploadTask) error {
} }
} }
// Generate thumbnail after deduplication
thumbnailDir := conf.Thumbnails.Directory
err = generateThumbnail(task.AbsFilename, thumbnailDir, conf.Thumbnails.Size)
if err != nil {
log.Errorf("Thumbnail generation failed for %s: %v", task.AbsFilename, err)
return err // Return early to avoid logging processing completion
}
log.WithFields(logrus.Fields{"file": absFilename}).Info("File uploaded and processed successfully") log.WithFields(logrus.Fields{"file": absFilename}).Info("File uploaded and processed successfully")
uploadDuration.Observe(time.Since(startTime).Seconds()) uploadDuration.Observe(time.Since(startTime).Seconds())

0
cmd/server/uploads/test Normal file
View File

2
test/hmac_test.go
View File

@ -16,7 +16,7 @@ import (
const ( const (
serverURL = "http://[::1]:8080" // Replace with your actual server URL serverURL = "http://[::1]:8080" // Replace with your actual server URL
secret = "hmac-file-server-is-the-win" // Replace with your HMAC secret key secret = "a-orc-and-a-humans-is-drinking-ale" // Replace with your HMAC secret key
uploadPath = "hmac_icon.png" // Test file to upload uploadPath = "hmac_icon.png" // Test file to upload
protocolType = "v2" // Use v2, v, or token as needed protocolType = "v2" // Use v2, v, or token as needed
) )