diff --git a/README.MD b/README.MD index 4f41ea1..94bf715 100644 --- a/README.MD +++ b/README.MD @@ -439,4 +439,8 @@ This project is licensed under the [MIT License](LICENSE). --- +## Changelog + +For a detailed list of changes, please refer to the [Changelog](./Changelog.md). + By following this updated `README.md`, you can ensure that users and contributors have a clear understanding of the **HMAC File Server**'s capabilities, configuration options, and best practices for deployment and maintenance. \ No newline at end of file diff --git a/RELEASE-NOTES.MD b/RELEASE-NOTES.MD index 046dde5..462a496 100644 --- a/RELEASE-NOTES.MD +++ b/RELEASE-NOTES.MD @@ -253,48 +253,12 @@ curl -X PUT -H "Authorization: Bearer " -F "file=@example.txt" http: ``` Replace `` with a valid HMAC signature -```` - -## [2.3.0] - 2024-12-28 +## [2.4.1] - 2025-03-10 ### Changed -- **Server:** Replaced the hardcoded temporary upload directory `/tmp/uploads` with a configurable `TempPath` parameter in `config.toml`. Ensure to set `tempPath` in your configuration file accordingly. - -#### Version 2.2.1 - 2024-12-27 - -- **Enhancements:** - - Added detailed logging for file extensions and MIME types during file uploads to assist in diagnosing unsupported file type issues. - -- **Configuration:** - - Updated `config.toml` to ensure necessary file extensions are allowed for uploads. - -#### Version 2.2.2 - 2024-12-27 - -- **Bug Fixes:** - - Resolved issue where temporary `.tmp` files caused "Unsupported file type" warnings by adjusting MIME type detection to use the final file extension. - -- **Enhancements:** - - Improved logging for file extension and MIME type during uploads. - -### Summary of Changes - -1. **Added Log Message Limitation:** - - Implemented `logMessages = logMessages[1:]` to remove the oldest log message once the `logMessages` slice exceeds 1000 entries, preventing unbounded memory growth. - ---- - -### Additional Files - -2024-12-30 endpoint /thumbnails - -No changes are required for the other files (`RELEASE-NOTES.MD`, `README.MD`, `config.toml`). - -## [2.3.1] - 2025-01-15 -- Added metrics for deduplication and ISO container operations to improve observability. -- Additional detail: Thumbnails now support a "concurrency" configuration parameter for parallel image processing. +- **Configuration:** Updated `globalextensions` in `config.toml` to `["*"]`, allowing all file types globally for uploads. This change simplifies the configuration by removing the need to specify individual file extensions. ## [2.4.0] - 2025-02-20 - ### Added - **Pre-Caching Support:** Introduced pre-caching of storage paths to improve access speeds. - **ISO Container Management:** Added functionality to create and mount ISO containers for specialized storage needs. @@ -307,12 +271,24 @@ No changes are required for the other files (`RELEASE-NOTES.MD`, `README.MD`, `c ### Fixed - **Bug Fixes:** Resolved minor issues related to file versioning and deduplication processes. -### Summary of Changes -1. **Pre-Caching:** Improved performance by allowing the server to pre-cache frequently accessed storage paths. -2. **ISO Management:** Enabled the creation and mounting of ISO containers for better filesystem management. -3. **Thumbnail Generation:** Added a concurrency setting to allow parallel processing of image thumbnails, speeding up the generation process. - -## [2.4.1] - 2025-03-10 - +## [2.3.1] - 2025-01-15 ### Changed - **Configuration:** Updated `globalextensions` in `config.toml` to `["*"]`, allowing all file types globally for uploads. This change simplifies the configuration by removing the need to specify individual file extensions. + +## [2.3.0] - 2024-12-28 +### Changed +- **Server:** Replaced the hardcoded temporary upload directory `/tmp/uploads` with a configurable `TempPath` parameter in `config.toml`. Ensure to set `tempPath` in your configuration file accordingly. + +## [2.2.2] - 2024-12-27 +### Bug Fixes +- Resolved issue where temporary `.tmp` files caused "Unsupported file type" warnings by adjusting MIME type detection to use the final file extension. + +### Enhancements +- Improved logging for file extension and MIME type during uploads. + +## [2.2.1] - 2024-12-27 +### Enhancements +- Added detailed logging for file extensions and MIME types during file uploads to assist in diagnosing unsupported file type issues. + +### Configuration +- Updated `config.toml` to ensure necessary file extensions are allowed for uploads. diff --git a/WiKi.md b/WiKi.md new file mode 100644 index 0000000..f1a15a6 --- /dev/null +++ b/WiKi.md @@ -0,0 +1,892 @@ +This documentation provides detailed information on configuring, setting up, and maintaining the HMAC File Server. Whether you're a developer, system administrator, or an enthusiast, this guide will help you navigate through the server's features and configurations effectively. + +--- + +## Table of Contents + +1. [Introduction](#introduction) +2. [Configuration](#configuration) + - [Server Configuration](#server-configuration) + - [Deduplication Settings](#deduplication-settings) + - [Thumbnails Settings](#thumbnails-settings) + - [ISO Settings](#iso-settings) + - [Timeout Settings](#timeout-settings) + - [Security Settings](#security-settings) + - [Versioning Settings](#versioning-settings) + - [Uploads Settings](#uploads-settings) + - [Downloads Settings](#downloads-settings) + - [ClamAV Settings](#clamav-settings) + - [Redis Settings](#redis-settings) + - [Worker Settings](#worker-settings) + - [File Settings](#file-settings) +3. [Example Configuration](#example-configuration) +4. [Setup Instructions](#setup-instructions) + - [1. HMAC File Server Installation](#1-hmac-file-server-installation) + - [2. Reverse Proxy Configuration](#2-reverse-proxy-configuration) + - [Apache2 Reverse Proxy](#apache2-reverse-proxy) + - [Nginx Reverse Proxy](#nginx-reverse-proxy) + - [3. ejabberd Configuration](#3-ejabberd-configuration) + - [4. Systemd Service Setup](#4-systemd-service-setup) +5. [Building for Different Architectures](#building-for-different-architectures) +6. [Additional Recommendations](#additional-recommendations) +7. [Notes](#notes) + +--- + +## Introduction + +The **HMAC File Server** is a secure and efficient file management solution designed to handle file uploads, downloads, deduplication, and more. Built with a focus on security, scalability, and performance, it integrates seamlessly with various tools and services to provide a comprehensive file handling experience. + +--- + +## Configuration + +The HMAC File Server is configured using a `config.toml` file. Below are the detailed explanations of each configuration section and their respective options. + +### Server Configuration + +```toml +# Server configuration +listenport = "8080" # TCP port for incoming requests +unixsocket = false # Use Unix domain socket instead of TCP +storagepath = "/path/to/hmac-file-server/data/" # Directory to store uploaded files +loglevel = "debug" # Logging level: "debug", "info", "warn", "error" +logfile = "/path/to/hmac-file-server.log" # Path to log file; leave empty to use stdout +metricsenabled = true # Enable Prometheus metrics +metricsport = "9090" # Port for Prometheus metrics +deduplicationenabled = true +minfreebytes = "5GB" # Minimum free disk space required +filettl = "2Y" # Time-to-live for files +filettlenabled = false # Enable TTL checks and cleanup +autoadjustworkers = true # Automatically adjust worker threads based on load +networkevents = false # Enable detailed network event logging +pidfilepath = "./hmac-file-server.pid" # Path to PID file +precaching = true # Pre-cache file structures on startup +``` + +#### Configuration Options + +- **listenport**: + - *Type*: `String` + - *Description*: Specifies the TCP port on which the server listens for incoming requests. + - *Default*: `"8080"` + +- **unixsocket**: + - *Type*: `Boolean` + - *Description*: Determines whether to use a Unix domain socket instead of a TCP port for communication. + - *Default*: `false` + +- **storagepath**: + - *Type*: `String` + - *Description*: Defines the directory path where uploaded files are stored. Ensure this path exists and has appropriate permissions. + - *Default*: `"/path/to/hmac-file-server/data/"` + +- **loglevel**: + - *Type*: `String` + - *Description*: Sets the verbosity level of logs. + - *Options*: `"debug"`, `"info"`, `"warn"`, `"error"` + - *Default*: `"debug"` + +- **logfile**: + - *Type*: `String` + - *Description*: Specifies the file path for logging. If left empty, logs are output to `stdout`. + - *Default*: `"/path/to/hmac-file-server.log"` + +- **metricsenabled**: + - *Type*: `Boolean` + - *Description*: Enables or disables the Prometheus metrics endpoint. + - *Default*: `true` + +- **metricsport**: + - *Type*: `String` + - *Description*: Defines the port on which Prometheus metrics are exposed. + - *Default*: `"9090"` + +- **deduplicationenabled**: + - *Type*: `Boolean` + - *Description*: Enables or disables file deduplication to optimize storage usage. + - *Default*: `true` + +- **minfreebytes**: + - *Type*: `String` + - *Description*: Specifies the minimum free disk space required for the server to operate effectively. + - *Default*: `"5GB"` + +- **filettl**: + - *Type*: `String` + - *Description*: Sets the default Time-to-Live (TTL) for files, determining how long files are retained before deletion. + - *Format*: Duration (e.g., `"2Y"` for two years) + - *Default*: `"2Y"` + +- **filettlenabled**: + - *Type*: `Boolean` + - *Description*: Enables or disables TTL checks and automatic file cleanup based on the `filettl` value. + - *Default*: `false` + +- **autoadjustworkers**: + - *Type*: `Boolean` + - *Description*: Automatically adjusts the number of worker threads based on server load and system resources. + - *Default*: `true` + +- **networkevents**: + - *Type*: `Boolean` + - *Description*: Enables detailed logging of network events, which can be useful for debugging but may increase log verbosity. + - *Default*: `false` + +- **pidfilepath**: + - *Type*: `String` + - *Description*: Specifies the file path where the server writes its Process ID (PID) file. This is useful for managing the server process. + - *Default*: `"./hmac-file-server.pid"` + +- **precaching**: + - *Type*: `Boolean` + - *Description*: Enables pre-caching of file structures on startup to improve access speed and performance. + - *Default*: `true` + +--- + +### Deduplication Settings + +```toml +# Deduplication settings +[deduplication] +enabled = true +directory = "/path/to/hmac-file-server/deduplication/" # Path to deduplication metadata store +``` + +#### Configuration Options + +- **enabled**: + - *Type*: `Boolean` + - *Description*: Enables or disables the deduplication feature, which helps in eliminating duplicate files to save storage space. + - *Default*: `true` + +- **directory**: + - *Type*: `String` + - *Description*: Specifies the directory path where deduplication metadata is stored. Ensure this directory exists and has appropriate permissions. + - *Default*: `"/path/to/hmac-file-server/deduplication/"` + +--- + +### Thumbnails Settings + +```toml +# Thumbnails settings +[thumbnails] +enabled = true +directory = "/path/to/hmac-file-server/thumbnails/" # Directory for storing thumbnails +size = "200x200" # Thumbnail dimensions +thumbnailintervalscan = "1h" # Interval for scheduled thumbnail generation +``` + +#### Configuration Options + +- **enabled**: + - *Type*: `Boolean` + - *Description*: Enables or disables the automatic generation of thumbnails for uploaded images. + - *Default*: `true` + +- **directory**: + - *Type*: `String` + - *Description*: Defines the directory where generated thumbnails are stored. Ensure this directory exists and has appropriate permissions. + - *Default*: `"/path/to/hmac-file-server/thumbnails/"` + +- **size**: + - *Type*: `String` + - *Description*: Specifies the dimensions to which thumbnails are resized. The format should be `"widthxheight"` (e.g., `"200x200"`). + - *Default*: `"200x200"` + +- **thumbnailintervalscan**: + - *Type*: `String` + - *Description*: Sets the frequency at which the server scans for new or updated images to generate thumbnails. + - *Format*: Duration (e.g., `"1h"` for one hour) + - *Default*: `"1h"` + +--- + +### ISO Settings + +```toml +# ISO settings +[iso] +enabled = false +size = "1TB" # Maximum ISO size +mountpoint = "/path/to/hmac-file-server/iso/" # ISO mount point +charset = "utf-8" # Filesystem character set encoding +``` + +#### Configuration Options + +- **enabled**: + - *Type*: `Boolean` + - *Description*: Enables or disables the mounting of an ISO-based filesystem for specialized storage needs. + - *Default*: `false` + +- **size**: + - *Type*: `String` + - *Description*: Defines the maximum allowed size for the ISO container. + - *Default*: `"1TB"` + +- **mountpoint**: + - *Type*: `String` + - *Description*: Specifies the directory path where the ISO is mounted. Ensure this path exists and has appropriate permissions. + - *Default*: `"/path/to/hmac-file-server/iso/"` + +- **charset**: + - *Type*: `String` + - *Description*: Sets the filesystem character set encoding for the ISO. + - *Default*: `"utf-8"` + +> **Note**: Ensure only one `[iso]` block is active in your `config.toml` to avoid configuration conflicts. + +--- + +### Timeout Settings + +```toml +# Timeout settings +[timeouts] +readtimeout = "3600s" # Maximum time to read a request +writetimeout = "3600s" # Maximum time to write a response +idletimeout = "3600s" # Maximum keep-alive time for idle connections +``` + +#### Configuration Options + +- **readtimeout**: + - *Type*: `String` + - *Description*: Sets the maximum duration for reading the entire request, including the body. + - *Format*: Duration (e.g., `"3600s"` for one hour) + - *Default*: `"3600s"` + +- **writetimeout**: + - *Type*: `String` + - *Description*: Defines the maximum duration before timing out writes of the response. + - *Format*: Duration (e.g., `"3600s"` for one hour) + - *Default*: `"3600s"` + +- **idletimeout**: + - *Type*: `String` + - *Description*: Specifies the maximum amount of time to wait for the next request when keep-alives are enabled. + - *Format*: Duration (e.g., `"3600s"` for one hour) + - *Default*: `"3600s"` + +--- + +### Security Settings + +```toml +# Security settings +[security] +secret = "your-secure-secret-key" # HMAC shared secret key (change to a secure value) +``` + +#### Configuration Options + +- **secret**: + - *Type*: `String` + - *Description*: The HMAC shared secret key used for signing requests and operations. + - *Default*: `"your-secure-secret-key"` + - *Warning*: **Change this immediately** to a unique, strong string in production environments to ensure the security of HMAC operations. + +--- + +### Versioning Settings + +```toml +# Versioning settings +[versioning] +enableversioning = false +maxversions = 1 # Number of file versions to retain +``` + +#### Configuration Options + +- **enableversioning**: + - *Type*: `Boolean` + - *Description*: Enables or disables the versioning feature, which maintains multiple versions of the same file. + - *Default*: `false` + +- **maxversions**: + - *Type*: `Integer` + - *Description*: Specifies the maximum number of versions to retain for each file. + - *Default*: `1` + +--- + +### Uploads Settings + +```toml +# Upload settings +[uploads] +resumableuploadsenabled = false +chunkeduploadsenabled = true +chunksize = "32MB" # Chunk size for uploads +allowedextensions = [ + ".txt", ".pdf", ".png", ".jpg", ".jpeg", ".gif", + ".bmp", ".tiff", ".svg", ".webp", ".wav", ".mp4", + ".avi", ".mkv", ".mov", ".wmv", ".flv", ".webm", + ".mpeg", ".mpg", ".m4v", ".3gp", ".3g2", ".mp3", ".ogg" +] +``` + +#### Configuration Options + +- **resumableuploadsenabled**: + - *Type*: `Boolean` + - *Description*: Enables or disables support for resumable (chunked) file uploads. + - *Default*: `false` + +- **chunkeduploadsenabled**: + - *Type*: `Boolean` + - *Description*: Specifically enables or disables chunked uploads. + - *Default*: `true` + +- **chunksize**: + - *Type*: `String` + - *Description*: Defines the size of each chunk in chunked uploads. + - *Format*: Size (e.g., `"32MB"`) + - *Default*: `"32MB"` + +- **allowedextensions**: + - *Type*: `Array of Strings` + - *Description*: Lists the file extensions permitted for upload. + - *Default*: + ```toml + allowedextensions = [ + ".txt", ".pdf", ".png", ".jpg", ".jpeg", ".gif", + ".bmp", ".tiff", ".svg", ".webp", ".wav", ".mp4", + ".avi", ".mkv", ".mov", ".wmv", ".flv", ".webm", + ".mpeg", ".mpg", ".m4v", ".3gp", ".3g2", ".mp3", ".ogg" + ] + ``` + +--- + +### Downloads Settings + +```toml +# Downloads settings +[downloads] +resumabledownloadsenabled = false +chunkeddownloadsenabled = true +chunksize = "32MB" +``` + +#### Configuration Options + +- **resumabledownloadsenabled**: + - *Type*: `Boolean` + - *Description*: Enables or disables support for resumable (chunked) downloads. + - *Default*: `false` + +- **chunkeddownloadsenabled**: + - *Type*: `Boolean` + - *Description*: Specifically enables or disables chunked downloads. + - *Default*: `true` + +- **chunksize**: + - *Type*: `String` + - *Description*: Defines the size of each chunk in chunked downloads. + - *Format*: Size (e.g., `"32MB"`) + - *Default*: `"32MB"` + +> **Note**: The `allowedextensions` key is **not** part of the `[downloads]` configuration based on the provided code. Ensure that it is omitted to prevent configuration errors. + +--- + +### ClamAV Settings + +```toml +# ClamAV settings +[clamav] +clamavenabled = true +clamavsocket = "/path/to/clamav/clamd.ctl" # Path to ClamAV socket +numscanworkers = 4 # Number of concurrent scan workers +scanfileextensions = [ + ".exe", ".dll", ".bin", ".com", ".bat", + ".sh", ".php", ".js" +] +``` + +#### Configuration Options + +- **clamavenabled**: + - *Type*: `Boolean` + - *Description*: Enables or disables ClamAV integration for virus scanning of uploaded files. + - *Default*: `true` + +- **clamavsocket**: + - *Type*: `String` + - *Description*: Specifies the file path to the ClamAV socket (`.ctl` file). Ensure ClamAV is installed and the socket path is correct. + - *Default*: `"/path/to/clamav/clamd.ctl"` + +- **numscanworkers**: + - *Type*: `Integer` + - *Description*: Sets the number of concurrent workers dedicated to scanning files with ClamAV. + - *Default*: `4` + +- **scanfileextensions**: + - *Type*: `Array of Strings` + - *Description*: Lists the file extensions that should be scanned for viruses. + - *Default*: + ```toml + scanfileextensions = [ + ".exe", ".dll", ".bin", ".com", ".bat", + ".sh", ".php", ".js" + ] + ``` + +--- + +### Redis Settings + +```toml +# Redis settings +[redis] +redisenabled = true +redisdbindex = 0 +redisaddr = "localhost:6379" # Redis server address +redispassword = "" # Redis password if required +redishealthcheckinterval = "120s" # Interval for Redis health checks +``` + +#### Configuration Options + +- **redisenabled**: + - *Type*: `Boolean` + - *Description*: Enables or disables Redis integration for caching or session tracking. + - *Default*: `true` + +- **redisaddr**: + - *Type*: `String` + - *Description*: Specifies the address of the Redis server (e.g., `"localhost:6379"`). + - *Default*: `"localhost:6379"` + +- **redispassword**: + - *Type*: `String` + - *Description*: Sets the Redis authentication password, if required. + - *Default*: `""` + +- **redisdbindex**: + - *Type*: `Integer` + - *Description*: Specifies the Redis database index to use. + - *Default*: `0` + +- **redishealthcheckinterval**: + - *Type*: `String` + - *Description*: Defines the interval for performing health checks on the Redis connection. + - *Format*: Duration (e.g., `"120s"` for two minutes) + - *Default*: `"120s"` + +--- + +### Worker Settings + +```toml +# Worker settings +[worker] +numworkers = 10 # Number of worker threads +``` + +#### Configuration Options + +- **numworkers**: + - *Type*: `Integer` + - *Description*: Specifies the number of worker threads to handle file operations. + - *Default*: `10` + +--- + +### File Settings + +```toml +# File settings +[file] +maxfilesize = "10GB" # Maximum file size for uploads +``` + +#### Configuration Options + +- **maxfilesize**: + - *Type*: `String` + - *Description*: Defines the maximum allowed file size for uploads. + - *Format*: Size (e.g., `"10GB"`) + - *Default*: `"10GB"` + +--- + +## Example Configuration + +Below is an example `config.toml` file with default settings: + +```toml +# Example HMAC File Server configuration + +# Server configuration +listenport = "8080" +unixsocket = false +storagepath = "/path/to/hmac-file-server/data/" +loglevel = "debug" +logfile = "/path/to/hmac-file-server.log" +metricsenabled = true +metricsport = "9090" +deduplicationenabled = true +minfreebytes = "5GB" +filettl = "2Y" +filettlenabled = false +autoadjustworkers = true +networkevents = false +pidfilepath = "./hmac-file-server.pid" +precaching = true + +# Deduplication settings +[deduplication] +enabled = true +directory = "/path/to/hmac-file-server/deduplication/" + +# Thumbnails settings +[thumbnails] +enabled = true +directory = "/path/to/hmac-file-server/thumbnails/" +size = "200x200" +thumbnailintervalscan = "1h" + +# ISO settings +[iso] +enabled = false +size = "1TB" +mountpoint = "/path/to/hmac-file-server/iso/" +charset = "utf-8" + +# Timeout settings +[timeouts] +readtimeout = "3600s" +writetimeout = "3600s" +idletimeout = "3600s" + +# Security settings +[security] +secret = "your-secure-secret-key" + +# Versioning settings +[versioning] +enableversioning = false +maxversions = 1 + +# Upload settings +[uploads] +resumableuploadsenabled = false +chunkeduploadsenabled = true +chunksize = "32MB" +allowedextensions = [ + ".txt", ".pdf", ".png", ".jpg", ".jpeg", ".gif", + ".bmp", ".tiff", ".svg", ".webp", ".wav", ".mp4", + ".avi", ".mkv", ".mov", ".wmv", ".flv", ".webm", + ".mpeg", ".mpg", ".m4v", ".3gp", ".3g2", ".mp3", ".ogg" +] + +# Download settings +[downloads] +resumabledownloadsenabled = false +chunkeddownloadsenabled = true +chunksize = "32MB" + +# ClamAV settings +[clamav] +clamavenabled = true +clamavsocket = "/path/to/clamav/clamd.ctl" +numscanworkers = 4 +scanfileextensions = [ + ".exe", ".dll", ".bin", ".com", ".bat", + ".sh", ".php", ".js" +] + +# Redis settings +[redis] +redisenabled = true +redisdbindex = 0 +redisaddr = "localhost:6379" +redispassword = "" +redishealthcheckinterval = "120s" + +# Worker settings +[worker] +numworkers = 10 + +# File settings +[file] +maxfilesize = "10GB" +``` + +--- + +## Setup Instructions + +### 1. HMAC File Server Installation + +To install the HMAC File Server, follow these steps: + +1. Clone the repository: + ```sh + git clone https://github.com/your-repo/hmac-file-server.git + cd hmac-file-server + ``` + +2. Build the server: + ```sh + go build -o hmac-file-server + ``` + +3. Create the necessary directories: + ```sh + mkdir -p /path/to/hmac-file-server/data/ + mkdir -p /path/to/hmac-file-server/deduplication/ + mkdir -p /path/to/hmac-file-server/thumbnails/ + mkdir -p /path/to/hmac-file-server/iso/ + ``` + +4. Copy the example configuration file: + ```sh + cp config.example.toml config.toml + ``` + +5. Edit the `config.toml` file to match your environment and preferences. + +6. Start the server: + ```sh + ./hmac-file-server -config config.toml + ``` + +### 2. Reverse Proxy Configuration + +To set up a reverse proxy for the HMAC File Server, you can use either Apache2 or Nginx. Below are the configuration examples for both. + +#### Apache2 Reverse Proxy + +1. Enable the necessary Apache2 modules: + ```sh + sudo a2enmod proxy + sudo a2enmod proxy_http + sudo a2enmod headers + sudo a2enmod rewrite + ``` + +2. Create a new virtual host configuration file: + ```sh + sudo nano /etc/apache2/sites-available/hmac-file-server.conf + ``` + +3. Add the following configuration to the file: + ```apache + + ServerName your-domain.com + + ProxyPreserveHost On + ProxyPass / http://localhost:8080/ + ProxyPassReverse / http://localhost:8080/ + + + Require all granted + Header always set X-Content-Type-Options "nosniff" + Header always set X-Frame-Options "DENY" + Header always set X-XSS-Protection "1; mode=block" + + + ``` + +4. Enable the new site and restart Apache2: + ```sh + sudo a2ensite hmac-file-server.conf + sudo systemctl restart apache2 + ``` + +#### Nginx Reverse Proxy + +1. Install Nginx if not already installed: + ```sh + sudo apt-get update + sudo apt-get install nginx + ``` + +2. Create a new server block configuration file: + ```sh + sudo nano /etc/nginx/sites-available/hmac-file-server + ``` + +3. Add the following configuration to the file: + ```nginx + server { + listen 80; + server_name your-domain.com; + + location / { + proxy_pass http://localhost:8080; + proxy_set_header Host $host; + proxy_set_header X-Real-IP $remote_addr; + proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; + proxy_set_header X-Forwarded-Proto $scheme; + proxy_set_header X-Content-Type-Options "nosniff"; + proxy_set_header X-Frame-Options "DENY"; + proxy_set_header X-XSS-Protection "1; mode=block"; + } + } + ``` + +4. Enable the new site and restart Nginx: + ```sh + sudo ln -s /etc/nginx/sites-available/hmac-file-server /etc/nginx/sites-enabled/ + sudo systemctl restart nginx + ``` + +### 3. ejabberd Configuration + +To configure ejabberd for use with the HMAC File Server, follow these steps: + +1. Install ejabberd: + ```sh + sudo apt-get update + sudo apt-get install ejabberd + ``` + +2. Edit the ejabberd configuration file: + ```sh + sudo nano /etc/ejabberd/ejabberd.yml + ``` + +3. Add the following configuration to the file: + ```yaml + hosts: + - "your-domain.com" + + listen: + - + port: 5222 + module: ejabberd_c2s + certfile: "/etc/ejabberd/ejabberd.pem" + starttls: true + starttls_required: true + protocol_options: + - "no_sslv3" + - "no_tlsv1" + - "no_tlsv1_1" + ciphers: "HIGH:!aNULL:!eNULL:!3DES:@STRENGTH" + dhfile: "/etc/ejabberd/dhparams.pem" + max_stanza_size: 65536 + shaper: c2s_shaper + access: c2s + + - + port: 5269 + module: ejabberd_s2s_in + certfile: "/etc/ejabberd/ejabberd.pem" + starttls: true + starttls_required: true + protocol_options: + - "no_sslv3" + - "no_tlsv1" + - "no_tlsv1_1" + ciphers: "HIGH:!aNULL:!eNULL:!3DES:@STRENGTH" + dhfile: "/etc/ejabberd/dhparams.pem" + max_stanza_size: 131072 + shaper: s2s_shaper + access: s2s + + acl: + local: + user_regexp: "" + + access_rules: + local: + allow: local + ``` + +4. Restart ejabberd: + ```sh + sudo systemctl restart ejabberd + ``` + +### 4. Systemd Service Setup + +To set up the HMAC File Server as a systemd service, follow these steps: + +1. Create a new systemd service file: + ```sh + sudo nano /etc/systemd/system/hmac-file-server.service + ``` + +2. Add the following configuration to the file: + ```ini + [Unit] + Description=HMAC File Server + After=network.target + + [Service] + ExecStart=/path/to/hmac-file-server -config /path/to/config.toml + WorkingDirectory=/path/to/hmac-file-server + Restart=always + User=www-data + Group=www-data + + [Install] + WantedBy=multi-user.target + ``` + +3. Reload systemd and enable the service: + ```sh + sudo systemctl daemon-reload + sudo systemctl enable hmac-file-server + sudo systemctl start hmac-file-server + ``` + +--- + +## Building for Different Architectures + +To build the HMAC File Server for different architectures, you can use the following commands: + +### Building for Linux (x86_64) + +```sh +GOOS=linux GOARCH=amd64 go build -o hmac-file-server-linux-amd64 +``` + +### Building for macOS (x86_64) + +```sh +GOOS=darwin GOARCH=amd64 go build -o hmac-file-server-darwin-amd64 +``` + +### Building for Windows (x86_64) + +```sh +GOOS=windows GOARCH=amd64 go build -o hmac-file-server-windows-amd64.exe +``` + +### Building for ARM (32-bit) + +```sh +GOOS=linux GOARCH=arm GOARM=7 go build -o hmac-file-server-linux-arm +``` + +### Building for ARM (64-bit) + +```sh +GOOS=linux GOARCH=arm64 go build -o hmac-file-server-linux-arm64 +``` + +--- + +## Additional Recommendations + +- **Security**: Ensure that the `secret` key in the `config.toml` file is changed to a unique, strong value to secure HMAC operations. +- **Backups**: Regularly back up the `config.toml` file and any important data stored by the HMAC File Server. +- **Monitoring**: Use monitoring tools like Prometheus and Grafana to keep track of server performance and metrics. + +--- + +## Notes + +- The HMAC File Server is designed to be flexible and configurable. Adjust the settings in the `config.toml` file to match your specific requirements and environment. +- For any issues or questions, refer to the project's GitHub repository and documentation. +```` diff --git a/changelog b/changelog index 121957c..da5250b 100644 --- a/changelog +++ b/changelog @@ -44,4 +44,4 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/). - Added detailed logging for file extensions and MIME types during file uploads to assist in diagnosing unsupported file type issues. ### Configuration -- Updated `config.toml` to ensure necessary file extensions are allowed for uploads. +- Updated `config.toml` to ensure necessary file extensions are allowed for uploads. \ No newline at end of file diff --git a/cmd/server/hmac-file-server b/cmd/server/hmac-file-server new file mode 100755 index 0000000..c298a74 Binary files /dev/null and b/cmd/server/hmac-file-server differ diff --git a/cmd/server/main.go b/cmd/server/main.go index b65f032..6b9bb55 100644 --- a/cmd/server/main.go +++ b/cmd/server/main.go @@ -1508,14 +1508,6 @@ func processUpload(task UploadTask) error { } } - // Generate thumbnail after deduplication - thumbnailDir := conf.Thumbnails.Directory - err = generateThumbnail(task.AbsFilename, thumbnailDir, conf.Thumbnails.Size) - if err != nil { - log.Errorf("Thumbnail generation failed for %s: %v", task.AbsFilename, err) - return err // Return early to avoid logging processing completion - } - log.WithFields(logrus.Fields{"file": absFilename}).Info("File uploaded and processed successfully") uploadDuration.Observe(time.Since(startTime).Seconds()) diff --git a/cmd/server/uploads/test b/cmd/server/uploads/test new file mode 100644 index 0000000..e69de29 diff --git a/test/hmac_test.go b/test/hmac_test.go index f9bbf37..6600f6f 100644 --- a/test/hmac_test.go +++ b/test/hmac_test.go @@ -16,7 +16,7 @@ import ( const ( serverURL = "http://[::1]:8080" // Replace with your actual server URL - secret = "hmac-file-server-is-the-win" // Replace with your HMAC secret key + secret = "a-orc-and-a-humans-is-drinking-ale" // Replace with your HMAC secret key uploadPath = "hmac_icon.png" // Test file to upload protocolType = "v2" // Use v2, v, or token as needed )