uuxo/hmac-file-server
1
1
Fork 0
Code Issues 2 Pull Requests Actions Packages Projects Releases 4 Wiki Activity

4 Releases 8 Tags

RSS Feed
  • 2.2-stable 0933482f64

    2.2-stable

    renz released this 2024-12-24 11:33:17 +01:00 | 51 commits to main since this release

    Release Notes - hmac-file-server v2.2-stable

    Release Date: December 24, 2024 (Happy Holidays!)

    Overview

    We are thrilled to announce the release of hmac-file-server v2.2-stable. This version builds upon our previous releases with significant enhancements, new features, and crucial bug fixes aimed at boosting the performance, security, and usability of the HMAC File Server.

    Attention:

    Due to a critical deduplication issue identified in v2.1-stable, which may impact file handling under specific configurations, we recommend users skip v2.1-stable and upgrade directly to v2.2-stable from earlier stable versions.

    New Features

    1. Thumbnail Creation

    • Automated Image Thumbnails: Generate image thumbnails for uploaded files to provide quick previews and enhance user experience.
    • Scheduled Generation: Thumbnails are created at configurable intervals to optimize server performance.
    • Configuration: 
      [thumbnails]
enabled = true
directory = "/path/to/hmac-file-server/thumbnails/"
size = "200x200"
thumbnailintervalscan = "1h"  # Interval for scheduled thumbnail generation

    2. ClamAV Integration

    • Robust Malware Scanning: Integrated ClamAV for scanning uploaded files, ensuring file integrity and security.
    • Configurable Scan Workers: Optimize performance with a configurable number of scan workers.
    • Configuration: 
      [clamav]
clamavenabled = true
clamavsocket = "/path/to/clamav/clamd.ctl"  # Path to ClamAV socket
numscanworkers = 4  # Number of concurrent scan workers
scanfileextensions = [
    ".exe", ".dll", ".bin", ".com", ".bat",
    ".sh", ".php", ".js"
]

    3. Redis Support

    • Enhanced Caching and Session Management: Utilize Redis for efficient caching of file metadata and managing application states.
    • High-Load Scalability: Supports high-load environments with optimized Redis configurations.
    • Configuration: 
      [redis]
redisenabled = true
redisdbindex = 0
redisaddr = "localhost:6379"  # Redis server address
redispassword = ""            # Redis password if required
redishealthcheckinterval = "120s"  # Interval for Redis health checks

    4. Enhanced Configuration Management

    • Expanded Customization Options: More configuration options for server behavior, including auto-adjusting worker pools and network event logging.
    • Improved Configuration Structure: Streamlined configuration keys for better clarity and maintainability.
    • Configuration: 
      [server]
autoadjustworkers = true  # Automatically adjust worker threads based on load
networkevents = false     # Enable detailed network event logging
precaching = true         # Pre-cache file structures on startup for faster access

    5. Improved Logging

    • JSON-Formatted Logs: Optional support for JSON-formatted logs for better integration with log management systems.
    • Configurable Log Output: Direct logs to files with log rotation and compression.
    • Configuration: 
      [server]
loglevel = "debug"  # Logging level: "debug", "info", "warn", "error"
logfile = "/path/to/hmac-file-server.log"  # Path to log file; leave empty to use stdout

[server]
loggingjson = false  # Set to true for JSON-formatted logs

    Enhancements

    • Graceful Shutdown: Ensures data integrity during server restarts by handling OS signals and performing necessary cleanup.
    • Auto-Adjusting Worker Pools: Optimizes the number of worker threads dynamically based on system resources to maintain optimal performance.
    • Extended Timeouts: Customizable connection timeouts to handle long-running requests and prevent resource exhaustion.
    • Configuration: 
      [timeouts]
readtimeout = "3600s"    # Maximum time to read a request (1 hour)
writetimeout = "3600s"   # Maximum time to write a response (1 hour)
idletimeout = "3600s"    # Maximum keep-alive time for idle connections (1 hour)

    Bug Fixes

    • Deduplication Issue Resolved: Fixed the critical deduplication bug from v2.1-stable to ensure reliable file handling under all configurations.
    • Syntax Cleanup: Streamlined codebase by removing unused parameters and functions to enhance code maintainability.
    • Improved Error Handling: Enhanced startup checks for ClamAV and Redis to provide clearer error messages and prevent server crashes.
    • Configuration Validation: Added comprehensive configuration validation to catch misconfigurations early during server startup.

    Performance Improvements

    • Optimized File Handling: Enhanced processing speeds for uploads and downloads, reducing latency and improving user experience.
    • Enhanced Caching Mechanisms: Improved caching strategies using Redis and in-memory caches to reduce load times and server strain.
    • Thumbnail Generation Optimization: Scheduled thumbnail generation minimizes performance hits during peak usage times.

    Security Enhancements

    • ClamAV and Redis Secure Integration: Implemented secure connections and enhanced virus scanning protocols to safeguard against malicious files.
    • HMAC Secret Management: Enforced the use of strong, unique HMAC secrets to ensure secure authentication and request signing.

    Configuration Changes

    • Removed Duplicate [iso] Block: Consolidated ISO configuration into a single [iso] section to prevent configuration conflicts.
    • Updated Configuration Keys: Renamed scanInterval to thumbnailintervalscan in the [thumbnails] section to align with the codebase.
    • Deprecated Options Removed: Streamlined configuration by removing outdated or unnecessary options, simplifying the setup process.
    • Example Configuration Updated: Provided an updated config.toml example reflecting the latest configuration structure and keys.

    Known Issues

    • Auto-Adjusting Delays: Potential delays in worker pool adjustments under extreme load conditions. Monitoring is recommended to ensure optimal performance.
    • JSON Logs Integration: JSON-formatted logs may require additional tools or configurations for seamless integration with existing log management systems.
    • Thumbnail Generation Overhead: In environments with high upload rates of large images, thumbnail generation may introduce additional CPU load. Adjust thumbnailintervalscan as needed.

    Upgrade Instructions

    1. Backup Current Configuration:

      cp ./config.toml ./config.toml.backup

    2. Pull the Latest Version:

      git pull origin v2.2-stable

    3. Update Dependencies:

      go mod tidy

    4. Review and Merge Configuration Changes:

      • Compare your existing config.toml with the example provided in the Example config.toml section.
      • Update configuration keys and values as necessary, especially ensuring that thumbnailintervalscan is used instead of scanInterval.

    5. Build the Server:

      go build -o hmac-file-server main.go

    6. Restart the Server:

      systemctl restart hmac-file-server
      • OR
      ./hmac-file-server -config ./config.toml

    Acknowledgments

    Thank you to our contributors and the community for their invaluable feedback and support. Special thanks to the developers of Thomas Leister's prosody-filer for inspiring many of our core features.

    Support

    For support, please open an issue on our GitHub repository.

    Enjoy the enhanced features and stability of hmac-file-server v2.2-stable, and thank you for choosing our solution!

    Downloads