Merge branch 'main' of git.uuxo.net:uuxo/hmac-file-server

.drone.yml

@@ -0,0 +1,10 @@
+kind: pipeline
+name: build
+
+steps:
+  - name: build docker image
+    image: docker
+    commands:
+      - docker build -t my-hmac-file-server .
+      - docker run -d -p 8080:8080 my-hmac-file-server
+      - docker ps  # Optional: To verify the container is running

CHANGELOG.md

@@ -0,0 +1,110 @@
+# Changelog
+
+All notable changes to this project will be documented in this file.
+
+## [Unreleased]
+
+- Initial creation of a comprehensive changelog.
+
+---
+
+## [2.8-Stable] - 2026-05-01
+
+### Added
+
+- Version check history for improved tracking.
+- Enhanced ClamAV scanning with concurrent workers.
+
+### Changed
+
+- Improved ISO-based storage for specialized use cases.
+- Auto-scaling workers for optimized performance.
+
+### Fixed
+
+- Minor issues in worker thread adjustments under high load.
+
+---
+
+## [2.7] - 2026-02-10
+
+### Added
+
+- Concurrency improvements and auto-scaling worker enhancements
+- Cleanup and removal of unused parameters in sorting functions
+
+### Changed
+
+- Additional logging for file scanning operations
+
+### Fixed
+
+- Minor stability issues related to ISO container mounting
+- Fixed dual stack for upload (IPv4/IPv6)
+
+---
+
+## [2.6-Stable] - 2025-12-01
+
+### Added
+
+- Deduplication support (removes duplicate files).
+- ISO Container management.
+- Dynamic worker scaling based on CPU & memory.
+- PreCaching feature for faster file access.
+
+### Changed
+
+- Worker pool scaling strategies for better performance.
+- Enhanced logging with rotating logs using lumberjack.
+
+### Fixed
+
+- Temporary file handling issues causing "Unsupported file type" warnings.
+- MIME type checks for file extension mismatches.
+
+---
+
+## [2.5] - 2025-09-15
+
+### Added
+
+- Redis caching integration for file metadata.
+- ClamAV scanning for virus detection before finalizing uploads.
+
+### Changed
+
+- Extended the default chunk size for chunked uploads.
+- Updated official documentation links.
+
+### Fixed
+
+- Edge case with versioning causing file rename conflicts.
+
+---
+
+## [2.0] - 2025-06-01
+
+### Added
+
+- Chunked file uploads and downloads.
+- Resumable upload support with partial file retention.
+
+### Changed
+
+- Moved configuration management to Viper.
+- Default Prometheus metrics for tracking memory & CPU usage.
+
+### Fixed
+
+- Race conditions in file locking under heavy concurrency.
+
+---
+
+## [1.0] - 2025-01-01
+
+### Added
+
+- Initial release with HMAC-based authentication.
+- Basic file upload/download endpoints.
+- Logging and fundamental configuration using .toml files.

COMBINED_RELEASE_NOTES.md

@@ -0,0 +1,48 @@
+## HMAC File Server – Release v2.7
+
+### What’s New
+1. **Configurable Filenaming**  
+   - Added support for `filenaming=None` to **skip** the default HMAC-based renaming.  
+   - Allows users to keep the original filename instead of hashing, while preserving all HMAC authentication for security.
+
+2. **Enhanced Logging**  
+   - Improved log level usage (`info`, `warn`, `error`, `debug`) across the **login flow** and **file handling** operations.  
+   - Added more structured fields (e.g., `method`, `remote`, `url`) for easier log analysis.  
+   - Better security defaults to avoid exposing sensitive data in logs.
+
+3. **Prometheus Metrics Adjustments**  
+   - Refined counters, gauges, and histograms to cover **all** critical events (upload/download, dedup, ClamAV scanning).  
+   - Ensured consistent increments for success/failure paths.  
+   - Simplified registration to avoid double-registration issues.
+
+4. **Deduplication Improvements**  
+   - Confirmed that after moving a file to the dedup directory, a **hard link** is consistently created back to the original location.  
+   - Logs now clearly indicate successful dedup steps and any errors.
+
+5. **Worker Pool Enhancements**  
+   - Better dynamic scaling logs (e.g., “Added worker. Total workers: X”).  
+   - Ensures no duplicate or redundant worker creation.  
+   - Additional metrics for worker adjustments and re-adjustments.
+
+### Bug Fixes
+- **Resolved “File Not Found” During GET**  
+  - Clarified that when `filenaming=None`, the server does **not** rename files to HMAC paths, preventing mismatches between upload and download URLs.  
+  - Fixed potential race conditions in dedup moving vs. linking.
+
+- **Reduced Log Noise**  
+  - Eliminated repetitive or misleading error messages around networking events.  
+  - Improved clarity in ClamAV scanning logs to better distinguish scan failures vs. actual malware detections.
+
+- Fixed dual stack IPv4 and IPv6 upload for improved reliability.
+
+### Upgrade Notes
+1. **Config File**:  
+   - Check your `[server]` section for `filenaming`. If you previously relied on HMAC-based filenames, confirm whether you want to set `filenaming="HMAC"` explicitly.  
+2. **Metrics**:  
+   - If you track Prometheus data, your dashboards may need to be updated for any renamed metrics or new labels.  
+3. **Logging**:  
+   - Logging defaults remain at `info` level. Increase to `debug` only for troubleshooting to avoid excessive detail in production logs.
+
+---
+
+**Thank you** to everyone who contributed feedback and testing for this release! As always, please report any issues, and we welcome suggestions to further improve the HMAC File Server. Enjoy the streamlined filenames, more comprehensive logging, and robust Prometheus metrics!

LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2025 Alexander Renz
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

README.MD

@@ -1,216 +1,204 @@
-# HMAC File Server Release Notes
+# HMAC File Server 2.9-Stable
+## Overview
+The **HMAC File Server** ensures secure file uploads and downloads using HMAC authentication. It incorporates rate limiting, CORS support, retries, file versioning, and Unix socket support for enhanced flexibility. Redis integration provides efficient caching and session management. Prometheus metrics and a graceful shutdown mechanism ensure reliable and efficient file handling.
 
-**HMAC File Server** is a secure, scalable, and feature-rich file server with advanced capabilities like HMAC authentication, resumable uploads, chunked uploads, file versioning, and optional ClamAV scanning for file integrity and security. This server is built with extensibility and operational monitoring in mind, including Prometheus metrics support and Redis integration.
+Special thanks to **Thomas Leister** for inspiration drawn from [[prosody-filer](https://github.com/ThomasLeister/prosody-filer)](https://github.com/ThomasLeister/prosody-filer).
 
 ## Features
+- File deduplication
+- Configurable TTL for automatic file cleanup
+- Secure HMAC-based authentication
+- Chunked uploads and downloads
+- Virus scanning via ClamAV
+- Prometheus metrics integration
+- Customizable worker management
+- Support ISO-based storage
 
-- **HMAC Authentication:** Secure file uploads and downloads with HMAC tokens.
-- **File Versioning:** Enable versioning for uploaded files with configurable retention.
-- **Chunked and Resumable Uploads:** Handle large files efficiently with support for resumable and chunked uploads.
-- **ClamAV Scanning:** Optional virus scanning for uploaded files.
-- **Prometheus Metrics:** Monitor system and application-level metrics.
-- **Redis Integration:** Use Redis for caching or storing application states.
-- **File Expiration:** Automatically delete files after a specified TTL.
-- **Graceful Shutdown:** Handles signals and ensures proper cleanup.
-- **Deduplication:** Remove duplicate files based on hashing for storage efficiency.
+## Table of Contents
+1. [Installation](#installation)
+2. [Configuration](#configuration)
+3. [Usage](#usage)
+4. [Setup](#setup)
+   - [Reverse Proxy](#reverse-proxy)
+   - [Systemd Service](#systemd-service)
+5. [Building](#building)
+6. [Changelog](#changelog)
+7. [License](#license)
 
 ---
 
 ## Installation
 
 ### Prerequisites
+- Go **1.20** or higher
+- Redis server (optional, for caching)
+- ClamAV (optional, for virus scanning)
 
-- Go 1.20+
-- Redis (optional, if Redis integration is enabled)
-- ClamAV (optional, if file scanning is enabled)
+### Steps
+1. Clone the repository:
+   ```bash
+   git clone https://github.com/PlusOne/hmac-file-server.git
+   cd hmac-file-server
+   ```
 
-### Clone and Build
+2. Build the server:
+   ```bash
+   go build -o hmac-file-server
+   ```
 
-```bash
-git clone https://github.com/your-repo/hmac-file-server.git
-cd hmac-file-server
-go build -o hmac-file-server main.go
-```
+3. Create necessary directories:
+   ```bash
+   mkdir -p /path/to/hmac-file-server/data/
+   mkdir -p /path/to/hmac-file-server/deduplication/
+   mkdir -p /path/to/hmac-file-server/iso/
+   ```
+
+4. Copy and edit the configuration file:
+   ```bash
+   cp config.example.toml config.toml
+   ```
+
+5. Start the server:
+   ```bash
+   ./hmac-file-server -config config.toml
+   ```
 
 ---
 
 ## Configuration
+The server is configured via a `config.toml` file. Key settings include:
 
-The server configuration is managed through a `config.toml` file. Below are the supported configuration options:
+- **Server Settings**: Port, logging, metrics
+- **Security**: HMAC secret, TLS options
+- **File Management**: TTL, deduplication, uploads, and downloads
+- **ISO**: Generation and mounting settings
+- **Workers**: Adjust thread management
 
-### **Server Configuration**
-
-| Key                    | Description                                         | Example                         |
-|------------------------|-----------------------------------------------------|---------------------------------|
-| `ListenPort`           | Port or Unix socket to listen on                    | `":8080"`                      |
-| `UnixSocket`           | Use a Unix socket (`true`/`false`)                  | `false`                        |
-| `Secret`               | Secret key for HMAC authentication                 | `"your-secret-key"`            |
-| `StoragePath`          | Directory to store uploaded files                  | `"/mnt/storage/hmac-file-server"` |
-| `LogLevel`             | Logging level (`info`, `debug`, etc.)               | `"info"`                       |
-| `LogFile`              | Log file path (optional)                           | `"/var/log/hmac-file-server.log"` |
-| `MetricsEnabled`       | Enable Prometheus metrics (`true`/`false`)          | `true`                         |
-| `MetricsPort`          | Prometheus metrics server port                     | `"9090"`                       |
-| `FileTTL`              | File Time-to-Live duration                         | `"168h0m0s"`                   |
-| `DeduplicationEnabled` | Enable file deduplication based on hashing          | `true`                         |
-| `MinFreeBytes`         | Minimum free space required on storage path (in bytes) | `104857600`                    |
-
-### **Uploads**
-
-| Key                        | Description                                   | Example     |
-|----------------------------|-----------------------------------------------|-------------|
-| `ResumableUploadsEnabled`  | Enable resumable uploads                     | `true`      |
-| `ChunkedUploadsEnabled`    | Enable chunked uploads                       | `true`      |
-| `ChunkSize`                | Chunk size for chunked uploads (bytes)       | `1048576`   |
-| `AllowedExtensions`        | Allowed file extensions for uploads          | `[".png", ".jpg"]` |
-
-### **Time Settings**
-
-| Key             | Description                    | Example  |
-|------------------|--------------------------------|----------|
-| `ReadTimeout`    | HTTP server read timeout      | `"2h"`   |
-| `WriteTimeout`   | HTTP server write timeout     | `"2h"`   |
-| `IdleTimeout`    | HTTP server idle timeout      | `"2h"`   |
-
-### **ClamAV Configuration**
-
-| Key                | Description                               | Example                          |
-|--------------------|-------------------------------------------|----------------------------------|
-| `ClamAVEnabled`    | Enable ClamAV virus scanning (`true`)     | `true`                          |
-| `ClamAVSocket`     | Path to ClamAV Unix socket               | `"/var/run/clamav/clamd.ctl"`   |
-| `NumScanWorkers`   | Number of workers for file scanning       | `2`                             |
-
-### **Redis Configuration**
-
-| Key                        | Description                      | Example           |
-|----------------------------|----------------------------------|-------------------|
-| `RedisEnabled`             | Enable Redis integration         | `true`            |
-| `RedisDBIndex`             | Redis database index             | `0`               |
-| `RedisAddr`                | Redis server address             | `"localhost:6379"`|
-| `RedisPassword`            | Password for Redis authentication| `""`              |
-| `RedisHealthCheckInterval` | Health check interval for Redis  | `"30s"`           |
-
-### **Workers and Connections**
-
-| Key                    | Description                        | Example           |
-|------------------------|------------------------------------|-------------------|
-| `NumWorkers`           | Number of upload workers           | `2`               |
-| `UploadQueueSize`      | Size of the upload queue           | `50`              |
+For detailed configuration options, refer to the [Wiki](./wiki.md).
 
 ---
 
-## Running the Server
-
-### Basic Usage
-
-Run the server with a configuration file:
+## Usage
+Start the server and access it on the configured port. Use curl or a client library to interact with the API.
 
+### Example
+Upload a file:
 ```bash
-./hmac-file-server -config ./config.toml
+curl -X POST -F 'file=@example.jpg' http://localhost:8080/upload
 ```
 
-### Metrics Server
-
-If `MetricsEnabled` is `true`, the Prometheus metrics server will run on the port specified in `MetricsPort` (default: `9090`).
-
 ---
 
-## Development Notes
+## Setup
 
-- **Versioning:** Enabled via `EnableVersioning`. Ensure `MaxVersions` is set appropriately to prevent storage issues.
-- **File Cleaner:** The file cleaner runs hourly and deletes files older than the configured `FileTTL`.
-- **Redis Health Check:** Automatically monitors Redis connectivity and logs warnings on failure.
+### Reverse Proxy
+Set up a reverse proxy using Apache2 or Nginx to handle requests.
 
----
+#### Apache2 Example
+```apache
+<VirtualHost *:80>
+    ServerName your-domain.com
 
-## Testing
+    ProxyPreserveHost On
+    ProxyPass / http://localhost:8080/
+    ProxyPassReverse / http://localhost:8080/
+</VirtualHost>
+```
 
-To run the server locally for development:
+#### Nginx Example
+```nginx
+server {
+    listen 80;
+    server_name your-domain.com;
 
+    location / {
+        proxy_pass http://localhost:8080;
+        proxy_set_header Host $host;
+        proxy_set_header X-Real-IP $remote_addr;
+    }
+}
+```
+
+### Systemd Service
+Create a systemd service file for the HMAC File Server:
+```ini
+[Unit]
+Description=HMAC File Server
+After=network.target
+
+[Service]
+ExecStart=/path/to/hmac-file-server -config /path/to/config.toml
+WorkingDirectory=/path/to/hmac-file-server
+Restart=always
+User=www-data
+Group=www-data
+
+[Install]
+WantedBy=multi-user.target
+```
+Enable and start the service:
 ```bash
-go run main.go -config ./config.toml
+sudo systemctl daemon-reload
+sudo systemctl enable hmac-file-server
+sudo systemctl start hmac-file-server
 ```
 
-Use tools like **cURL** or **Postman** to test file uploads and downloads.
-
-### Example File Upload with HMAC Token
-
-```bash
-curl -X PUT -H "Authorization: Bearer <HMAC-TOKEN>" -F "file=@example.txt" http://localhost:8080/uploads/example.txt
-```
-
-Replace `<HMAC-TOKEN>` with a valid HMAC signature generated using the configured `Secret`.
-
 ---
 
-## Monitoring
+## Building
+To build for different architectures:
 
-Prometheus metrics include:
-- File upload/download durations
-- Memory usage
-- CPU usage
-- Active connections
-- HTTP requests metrics (total, method, path)
+- **Linux (amd64)**:
+  ```bash
+  GOOS=linux GOARCH=amd64 go build -o hmac-file-server main.go
+  ```
+
+- **Linux (arm64)**:
+  ```bash
+  GOOS=linux GOARCH=arm64 go build -o hmac-file-server main.go
+  ```
 
 ---
 
-## Example `config.toml`
+## Changelog
 
-```toml
-[server]
-listenport = "8080"
-unixsocket = false
-storagepath = "/mnt/storage/"
-loglevel = "info"
-logfile = "/var/log/file-server.log"
-metricsenabled = true
-metricsport = "9090"
-DeduplicationEnabled = true
-filettl = "336h" # 14 days
-minfreebytes = 104857600 # 100 MB in bytes
+### Added
+- **Deduplication Support:** Automatically remove duplicate files based on SHA256 hashing to save storage space.
+- **ISO Container Management:** Create and mount ISO containers for specialized storage needs, enhancing flexibility in file management.
+- **Prometheus Metrics Enhancements:** Added detailed metrics for deduplication and ISO container operations to improve monitoring and observability.
+- **Redis Integration Improvements:** Enhanced caching mechanisms using Redis for faster access to file metadata and application states.
+- **Precaching Feature:** Implemented precaching of file structures on startup to reduce access times for frequently used files.
+- **Configuration Options:** Updated `config.toml` to include new settings for deduplication, ISO management, and worker scaling.
 
-[timeouts]
-readtimeout = "4800s"
-writetimeout = "4800s"
-idletimeout = "24h"
+### Changed
+- **Worker Pool Scaling:** Implemented dynamic adjustment of worker threads based on system resources to optimize performance.
+- **Logging Enhancements:** Improved logging for file operations, including detailed information on file extensions and MIME types during uploads.
+- **Temporary Path Configuration:** Replaced hardcoded temporary upload directories with a configurable `TempPath` parameter in `config.toml` for greater flexibility.
 
-[security]
-secret = "example-secret-key"
+### Fixed
+- **Temporary File Handling:** Resolved issues where temporary `.tmp` files caused "Unsupported file type" warnings by enhancing MIME type detection logic.
+- **MIME Type Detection:** Improved MIME type detection to ensure better compatibility and accuracy during file uploads.
 
-[versioning]
-enableversioning = false
-maxversions = 1
+### Deprecated
+- **Thumbnail Support (Previous Implementation):** Dropped the previous thumbnail support mechanism. This feature will not return in future releases.
 
-[uploads]
-resumableuploadsenabled = true
-chunkeduploadsenabled = true
-chunksize = 8192
-allowedextensions = [".txt", ".pdf", ".png", ".jpg", ".jpeg", ".gif", ".bmp", ".tiff", ".svg", ".webp", ".wav", ".mp4", ".avi", ".mkv", ".mov", ".wmv", ".flv", ".webm", ".mpeg", ".mpg", ".m4v", ".3gp", ".3g2", ".mp3", ".ogg"]
+---
 
-[clamav]
-clamavenabled = true
-clamavsocket = "/var/run/clamav/clamd.ctl"
-numscanworkers = 2
+**Important Update:**
+- The minimum Go version required is now **1.20**. Please ensure your environment meets this requirement for successful compilation.
 
-[redis]
-redisenabled = true
-redisdbindex = 0
-redisaddr = "localhost:6379"
-redispassword = ""
-redishealthcheckinterval = "120s"
+---
 
-[workers]
-numworkers = 2
-uploadqueuesize = 50
-```
+## License
 
-This configuration file is set up with essential features like Prometheus integration, ClamAV scanning, and file handling with deduplication and versioning options. Adjust the settings according to your infrastructure needs.
+MIT License
 
-### Additional Features
+Copyright (c) 2025 Alexander Renz
 
-- **Deduplication**: Automatically remove duplicate files based on hashing.
-- **Versioning**: Store multiple versions of files and keep a maximum of `MaxVersions` versions.
-- **ClamAV Integration**: Scan uploaded files for viruses using ClamAV.
-- **Redis Caching**: Utilize Redis for caching file metadata for faster access.
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
 
-This release ensures an efficient and secure file management system, suited for environments requiring high levels of data security and availability.
-```
+The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

RELEASE-NOTES.MD

@@ -0,0 +1,53 @@
+# Release Notes for HMAC File Server 2.7-Stable
+
+## Summary
+Version 2.6-Stable focuses on improving the overall stability and performance of the HMAC File Server. Significant changes have been made to prioritize reliability and scalability for production environments.
+
+## Key Changes
+
+### Breaking Changes
+- **Thumbnail Generation Dropped**: Support for automatic thumbnail generation has been removed in this release. This decision was made to enhance system stability and reduce resource consumption. Users requiring thumbnails are encouraged to use external tools.
+
+### New Features
+- **ISO-Based Storage Support**: Introduced support for ISO-based storage to accommodate specialized use cases.
+- **Enhanced ClamAV Integration**: Improved ClamAV scanning with concurrent workers, providing better performance for large-scale deployments.
+- **Timeout Configuration**: Added granular timeout settings for read, write, and idle connections, improving connection management.
+- **FileNaming Configuration**: Added support for a "None" option in the `FileNaming` configuration. When set to "None", the filename remains unchanged.
+- **Example Configuration Generation**: If no configuration file is found, the server will output an example configuration for the user to copy and paste.
+- **Prometheus Metrics**: Enhanced Prometheus metrics for better monitoring and performance tracking. New metrics include upload and download durations, error counts, memory and CPU usage, and more.
+
+### Improvements
+- **Worker Management**: Auto-scaling worker threads based on system load for optimal performance.
+- **Logging Enhancements**: Improved log verbosity control, making debugging and monitoring easier.
+
+### Bug Fixes
+- Resolved minor issues affecting deduplication and file upload performance.
+- Fixed a rare crash scenario during high-concurrency file uploads.
+
+## Migration Notes
+1. **Thumbnail Settings**: Remove `[thumbnails]` configuration blocks from your `config.toml` file to avoid errors.
+2. **Updated Configuration**: Review new timeout settings in `[timeouts]` and adjust as needed.
+3. **ISO Integration**: Configure the new `[iso]` block for environments utilizing ISO-based storage.
+4. **FileNaming Configuration**: Update the `FileNaming` setting in `[server]` to use the new "None" option if you want filenames to remain unchanged.
+
+[server]
+# FileNaming options: "HMAC", "None"
+FileNaming = "HMAC"
+
+## Recommendations
+- **Security**: Ensure that the HMAC secret key in `config.toml` is updated to a strong, unique value.
+- **Backups**: Regularly back up your `config.toml` and important data directories.
+- **Monitoring**: Leverage Prometheus metrics for real-time monitoring of server performance.
+
+For a detailed guide on setting up and configuring the HMAC File Server, refer to the [README.md](./README.md).
+
+---
+
+Thank you for using HMAC File Server! If you encounter any issues, feel free to report them on our GitHub repository.
+
+## Version 2.7
+
+- Refinements in worker scaling logic
+- Removed obsolete parameters for sorting
+- Further improvements to ISO-based storage handling
+- Fixed dual stack for upload (IPv4/IPv6)

SHORT_RELEASE_NOTE.md

@@ -0,0 +1,11 @@
+# Short Release Note
+
+Key Highlights from 2.8-Stable:
+- Version check history added for improved tracking.
+- Improved ISO-based storage for specialized use cases.
+- Enhanced ClamAV scanning with concurrent workers.
+- Auto-scaling workers for optimized performance.
+
+Go 1.24.0 is required.
+
+For more details, see the README and CHANGELOG.

changelog

@@ -0,0 +1,47 @@
+
+# Changelog
+
+All notable changes to this project will be documented in this file.
+
+The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/).
+
+## [Unreleased]
+
+## [2.4.1] - 2025-03-10
+### Changed
+- **Configuration:** Updated `globalextensions` in `config.toml` to `["*"]`, allowing all file types globally for uploads. This change simplifies the configuration by removing the need to specify individual file extensions.
+
+## [2.4.0] - 2025-02-20
+### Added
+- **Pre-Caching Support:** Introduced pre-caching of storage paths to improve access speeds.
+- **ISO Container Management:** Added functionality to create and mount ISO containers for specialized storage needs.
+- **Thumbnail Concurrency Parameter:** Users can now set the level of concurrency for thumbnail generation to optimize performance.
+
+### Changed
+- **Configuration Options:** Updated `config.toml` to include new settings for pre-caching and ISO management.
+- **Documentation:** Enhanced `README.MD` with detailed instructions on new features and best practices.
+
+### Fixed
+- **Bug Fixes:** Resolved minor issues related to file versioning and deduplication processes.
+
+## [2.3.1] - 2025-01-15
+### Changed
+- **Configuration:** Updated `globalextensions` in `config.toml` to `["*"]`, allowing all file types globally for uploads. This change simplifies the configuration by removing the need to specify individual file extensions.
+
+## [2.3.0] - 2024-12-28
+### Changed
+- **Server:** Replaced the hardcoded temporary upload directory `/tmp/uploads` with a configurable `TempPath` parameter in `config.toml`. Ensure to set `tempPath` in your configuration file accordingly.
+
+## [2.2.2] - 2024-12-27
+### Bug Fixes
+- Resolved issue where temporary `.tmp` files caused "Unsupported file type" warnings by adjusting MIME type detection to use the final file extension.
+
+### Enhancements
+- Improved logging for file extension and MIME type during uploads.
+
+## [2.2.1] - 2024-12-27
+### Enhancements
+- Added detailed logging for file extensions and MIME types during file uploads to assist in diagnosing unsupported file type issues.
+
+### Configuration
+- Updated `config.toml` to ensure necessary file extensions are allowed for uploads.

cmd/monitor/monitor.go

@@ -0,0 +1,892 @@
+package main
+
+import (
+	"fmt"
+	"log"
+	"net/http"
+	"os"
+	"sort"
+	"strconv"
+	"strings"
+	"time"
+	"context"
+	"io"
+	"sync"
+	"bufio"
+
+	"github.com/gdamore/tcell/v2"
+	"github.com/pelletier/go-toml"
+	"github.com/prometheus/common/expfmt"
+	"github.com/rivo/tview"
+	"github.com/shirou/gopsutil/v3/cpu"
+	"github.com/shirou/gopsutil/v3/mem"
+	"github.com/shirou/gopsutil/v3/process"
+)
+
+var (
+	prometheusURL   string
+	configFilePath  string // Pfad der gefundenen Konfiguration
+	logFilePath     string // Pfad der Logdatei aus der Konfiguration
+	metricsEnabled  bool   // Neue Variable für die Aktivierung von Metriken
+	bindIP          string // Neue Variable für die gebundene IP-Adresse
+)
+
+func init() {
+	configPaths := []string{
+		"/etc/hmac-file-server/config.toml",
+		"../config.toml",
+		"./config.toml",
+	}
+
+	var config *toml.Tree
+	var err error
+
+	// Lade die config.toml aus den definierten Pfaden
+	for _, path := range configPaths {
+		config, err = toml.LoadFile(path)
+		if err == nil {
+			configFilePath = path
+			log.Printf("Using config file: %s", configFilePath)
+			break
+		}
+	}
+
+	if err != nil {
+		log.Fatalf("Error loading config file: %v\nPlease create a config.toml in one of the following locations:\n%v", err, configPaths)
+	}
+
+	// Metricsport auslesen
+	portValue := config.Get("server.metricsport")
+	if portValue == nil {
+		log.Println("Warning: 'server.metricsport' is missing in the configuration, using default port 9090")
+		portValue = int64(9090)
+	}
+
+	var port int64
+	switch v := portValue.(type) {
+	case int64:
+		port = v
+	case string:
+		parsedPort, err := strconv.ParseInt(v, 10, 64)
+		if err != nil {
+			log.Fatalf("Error parsing 'server.metricsport' as int64: %v", err)
+		}
+		port = parsedPort
+	default:
+		log.Fatalf("Error: 'server.metricsport' is not of type int64 or string, got %T", v)
+	}
+
+	// Lesen von 'metricsenabled' aus der Konfiguration
+	metricsEnabledValue := config.Get("server.metricsenabled")
+	if metricsEnabledValue == nil {
+		log.Println("Warning: 'server.metricsenabled' ist in der Konfiguration nicht gesetzt. Standardmäßig deaktiviert.")
+		metricsEnabled = false
+	} else {
+		var ok bool
+		metricsEnabled, ok = metricsEnabledValue.(bool)
+		if !ok {
+			log.Fatalf("Konfigurationsfehler: 'server.metricsenabled' sollte ein boolescher Wert sein, aber %T wurde gefunden.", metricsEnabledValue)
+		}
+	}
+
+	// Lesen von 'bind_ip' aus der Konfiguration
+	bindIPValue := config.Get("server.bind_ip")
+	if bindIPValue == nil {
+		log.Println("Warning: 'server.bind_ip' ist in der Konfiguration nicht gesetzt. Standardmäßig auf 'localhost' gesetzt.")
+		bindIP = "localhost"
+	} else {
+		var ok bool
+		bindIP, ok = bindIPValue.(string)
+		if !ok {
+			log.Fatalf("Konfigurationsfehler: 'server.bind_ip' sollte ein String sein, aber %T wurde gefunden.", bindIPValue)
+		}
+	}
+
+	// Konstruktion der prometheusURL basierend auf 'bind_ip' und 'metricsport'
+	prometheusURL = fmt.Sprintf("http://%s:%d/metrics", bindIP, port)
+	log.Printf("Metrics URL gesetzt auf: %s", prometheusURL)
+
+	// Log-Datei auslesen über server.logfile
+	logFileValue := config.Get("server.logfile")
+	if logFileValue == nil {
+		log.Println("Warning: 'server.logfile' is missing, using default '/var/log/hmac-file-server.log'")
+		logFilePath = "/var/log/hmac-file-server.log"
+	} else {
+		lf, ok := logFileValue.(string)
+		if !ok {
+			log.Fatalf("Error: 'server.logfile' is not of type string, got %T", logFileValue)
+		}
+		logFilePath = lf
+	}
+}
+
+// Thresholds for color coding
+const (
+	HighUsage   = 80.0
+	MediumUsage = 50.0
+)
+
+// ProcessInfo holds information about a process
+type ProcessInfo struct {
+	PID                   int32
+	Name                  string
+	CPUPercent            float64
+	MemPercent            float32
+	CommandLine           string
+	Uptime                string // Neues Feld für die Uptime
+	Status                string // Neues Feld für den Status
+	ErrorCount            int    // Neues Feld für die Anzahl der Fehler
+	TotalRequests         int64  // Neues Feld für die Gesamtanzahl der Anfragen
+	ActiveConnections     int    // Neues Feld für aktive Verbindungen
+	AverageResponseTime   float64 // Neues Feld für die durchschnittliche Antwortzeit in Millisekunden
+}
+
+// Function to fetch and parse Prometheus metrics
+func fetchMetrics() (map[string]float64, error) {
+	resp, err := http.Get(prometheusURL)
+	if err != nil {
+		return nil, fmt.Errorf("failed to fetch metrics: %w", err)
+	}
+	defer resp.Body.Close()
+
+	parser := &expfmt.TextParser{}
+	metricFamilies, err := parser.TextToMetricFamilies(resp.Body)
+	if err != nil {
+		return nil, fmt.Errorf("failed to parse metrics: %w", err)
+	}
+
+	metrics := make(map[string]float64)
+	for name, mf := range metricFamilies {
+		// Filter the metrics you're interested in
+		if strings.HasPrefix(name, "hmac_file_server_") ||
+			name == "memory_usage_bytes" ||
+			name == "cpu_usage_percent" ||
+			name == "active_connections_total" ||
+			name == "goroutines_count" ||
+			name == "total_requests" ||
+			name == "average_response_time_ms" {
+
+			for _, m := range mf.GetMetric() {
+				var value float64
+				if m.GetGauge() != nil {
+					value = m.GetGauge().GetValue()
+				} else if m.GetCounter() != nil {
+					value = m.GetCounter().GetValue()
+				} else if m.GetUntyped() != nil {
+					value = m.GetUntyped().GetValue()
+				} else {
+					// If the metric type is not handled, skip it
+					continue
+				}
+
+				// Handle metrics with labels
+				if len(m.GetLabel()) > 0 {
+					labels := make([]string, 0)
+					for _, label := range m.GetLabel() {
+						labels = append(labels, fmt.Sprintf("%s=\"%s\"", label.GetName(), label.GetValue()))
+					}
+					metricKey := fmt.Sprintf("%s{%s}", name, strings.Join(labels, ","))
+					metrics[metricKey] = value
+				} else {
+					metrics[name] = value
+				}
+			}
+		}
+	}
+
+	return metrics, nil
+}
+
+// Function to fetch system data
+func fetchSystemData() (float64, float64, int, error) {
+	v, err := mem.VirtualMemory()
+	if err != nil {
+		return 0, 0, 0, fmt.Errorf("failed to fetch memory data: %w", err)
+	}
+
+	c, err := cpu.Percent(0, false)
+	if err != nil {
+		return 0, 0, 0, fmt.Errorf("failed to fetch CPU data: %w", err)
+	}
+
+	cores, err := cpu.Counts(true)
+	if err != nil {
+		return 0, 0, 0, fmt.Errorf("failed to fetch CPU cores: %w", err)
+	}
+
+	cpuUsage := 0.0
+	if len(c) > 0 {
+		cpuUsage = c[0]
+	}
+
+	return v.UsedPercent, cpuUsage, cores, nil
+}
+
+// Funktion zum Abrufen der Prozessliste mit paralleler Verarbeitung
+func fetchProcessList() ([]ProcessInfo, error) {
+	processes, err := process.Processes()
+	if err != nil {
+		return nil, fmt.Errorf("failed to fetch processes: %w", err)
+	}
+
+	var processList []ProcessInfo
+	var mu sync.Mutex
+	var wg sync.WaitGroup
+
+	// Begrenzung der gleichzeitigen Goroutinen auf 10
+	sem := make(chan struct{}, 10)
+
+	for _, p := range processes {
+		wg.Add(1)
+		sem <- struct{}{} // Eintritt in semaphor
+
+		go func(p *process.Process) {
+			defer wg.Done()
+			defer func() { <-sem }() // Austritt aus semaphor
+
+			cpuPercent, err := p.CPUPercent()
+			if err != nil {
+				return
+			}
+
+			memPercent, err := p.MemoryPercent()
+			if err != nil {
+				return
+			}
+
+			name, err := p.Name()
+			if err != nil {
+				return
+			}
+
+			cmdline, err := p.Cmdline()
+			if err != nil {
+				cmdline = ""
+			}
+
+			info := ProcessInfo{
+				PID:         p.Pid,
+				Name:        name,
+				CPUPercent:  cpuPercent,
+				MemPercent:  memPercent,
+				CommandLine: cmdline,
+			}
+
+			mu.Lock()
+			processList = append(processList, info)
+			mu.Unlock()
+		}(p)
+	}
+
+	wg.Wait()
+	return processList, nil
+}
+
+// Function to fetch detailed information about hmac-file-server
+func fetchHmacFileServerInfo() (*ProcessInfo, error) {
+	processes, err := process.Processes()
+	if err != nil {
+		return nil, fmt.Errorf("failed to fetch processes: %w", err)
+	}
+
+	for _, p := range processes {
+		name, err := p.Name()
+		if err != nil {
+			continue
+		}
+
+		if name == "hmac-file-server" {
+			cpuPercent, err := p.CPUPercent()
+			if err != nil {
+				cpuPercent = 0.0
+			}
+
+			memPercent, err := p.MemoryPercent()
+			if err != nil {
+				memPercent = 0.0
+			}
+
+			cmdline, err := p.Cmdline()
+			if err != nil {
+				cmdline = ""
+			}
+
+			createTime, err := p.CreateTime()
+			if err != nil {
+				return nil, fmt.Errorf("failed to get process start time: %w", err)
+			}
+			uptime := time.Since(time.Unix(0, createTime*int64(time.Millisecond)))
+
+			status := "Running" // Standardstatus
+
+			// Überprüfung, ob der Prozess aktiv ist
+			isRunning, err := p.IsRunning()
+			if err != nil || !isRunning {
+				status = "Stopped"
+			}
+
+			errorCount, err := countHmacErrors()
+			if err != nil {
+				errorCount = 0
+			}
+
+			metrics, err := fetchMetrics()
+			if err != nil {
+				return nil, fmt.Errorf("failed to fetch metrics: %w", err)
+			}
+
+			totalRequests, ok := metrics["total_requests"]
+			if !ok {
+				totalRequests = 0
+			}
+
+			activeConnections, ok := metrics["active_connections_total"]
+			if !ok {
+				activeConnections = 0
+			}
+
+			averageResponseTime, ok := metrics["average_response_time_ms"]
+			if !ok {
+				averageResponseTime = 0.0
+			}
+
+			return &ProcessInfo{
+				PID:                 p.Pid,
+				Name:                name,
+				CPUPercent:          cpuPercent,
+				MemPercent:          memPercent,
+				CommandLine:         cmdline,
+				Uptime:              uptime.String(),
+				Status:              status,
+				ErrorCount:          errorCount,
+				TotalRequests:       int64(totalRequests),
+				ActiveConnections:   int(activeConnections),
+				AverageResponseTime: averageResponseTime,
+			}, nil
+		}
+	}
+
+	return nil, fmt.Errorf("hmac-file-server process not found")
+}
+
+// Neue Funktion zur Zählung der Fehler in den Logs
+func countHmacErrors() (int, error) {
+	logFilePath := "/var/log/hmac-file-server.log" // Pfad zur Logdatei
+	file, err := os.Open(logFilePath)
+	if err != nil {
+		return 0, err
+	}
+	defer file.Close()
+
+	scanner := bufio.NewScanner(file)
+	errorCount := 0
+	for scanner.Scan() {
+		line := scanner.Text()
+		if strings.Contains(line, "level=error") {
+			errorCount++
+		}
+	}
+
+	if err := scanner.Err(); err != nil {
+		return 0, err
+	}
+
+	return errorCount, nil
+}
+
+// Funktion zur Aktualisierung der UI mit paralleler Datenbeschaffung
+func updateUI(ctx context.Context, app *tview.Application, pages *tview.Pages, sysPage, hmacPage tview.Primitive) {
+	ticker := time.NewTicker(2 * time.Second)
+	defer ticker.Stop()
+
+	// Einführung von Channels für verschiedene Daten
+	systemDataCh := make(chan struct {
+		memUsage float64
+		cpuUsage float64
+		cores    int
+		err      error
+	})
+	var metricsCh chan struct {
+		metrics map[string]float64
+		err     error
+	}
+	if metricsEnabled {
+		metricsCh = make(chan struct {
+			metrics map[string]float64
+			err     error
+		})
+	}
+	processListCh := make(chan struct {
+		processes []ProcessInfo
+		err       error
+	})
+	hmacInfoCh := make(chan struct {
+		info    *ProcessInfo
+		metrics map[string]float64
+		err     error
+	})
+
+	// Goroutine zur Datenbeschaffung
+	go func() {
+		for {
+			select {
+			case <-ctx.Done():
+				close(systemDataCh)
+				if metricsEnabled {
+					close(metricsCh)
+				}
+				close(processListCh)
+				close(hmacInfoCh)
+				return
+			case <-ticker.C:
+				// Systemdaten abrufen asynchron
+				go func() {
+					memUsage, cpuUsage, cores, err := fetchSystemData()
+					systemDataCh <- struct {
+						memUsage float64
+						cpuUsage float64
+						cores    int
+						err      error
+					}{memUsage, cpuUsage, cores, err}
+				}()
+
+				if metricsEnabled {
+					// Metriken abrufen asynchron
+					go func() {
+						metrics, err := fetchMetrics()
+						metricsCh <- struct {
+							metrics map[string]float64
+							err     error
+						}{metrics, err}
+					}()
+				}
+
+				// Prozessliste abrufen asynchron
+				go func() {
+					processes, err := fetchProcessList()
+					processListCh <- struct {
+						processes []ProcessInfo
+						err       error
+					}{processes, err}
+				}()
+
+				// hmac-file-server Informationen abrufen asynchron
+				go func() {
+					hmacInfo, err := fetchHmacFileServerInfo()
+					var metrics map[string]float64
+					if metricsEnabled {
+						metrics, err = fetchMetrics()
+					}
+					hmacInfoCh <- struct {
+						info    *ProcessInfo
+						metrics map[string]float64
+						err     error
+					}{hmacInfo, metrics, err}
+				}()
+			}
+		}
+	}()
+
+	for {
+		select {
+		case <-ctx.Done():
+			return
+		case data, ok := <-systemDataCh:
+			if !ok {
+				systemDataCh = nil
+				continue
+			}
+			if data.err != nil {
+				log.Printf("Fehler beim Abrufen der Systemdaten: %v\n", data.err)
+				continue
+			}
+			// UI aktualisieren mit Systemdaten
+			app.QueueUpdateDraw(func() {
+				if currentPage, _ := pages.GetFrontPage(); currentPage == "system" {
+					sysFlex := sysPage.(*tview.Flex)
+					sysTable := sysFlex.GetItem(0).(*tview.Table)
+					updateSystemTable(sysTable, data.memUsage, data.cpuUsage, data.cores)
+				}
+			})
+		case data, ok := <-metricsCh:
+			if !ok {
+				metricsCh = nil
+				continue
+			}
+			if data.err != nil {
+				log.Printf("Fehler beim Abrufen der Metriken: %v\n", data.err)
+				continue
+			}
+			// UI aktualisieren mit Metriken
+			app.QueueUpdateDraw(func() {
+				if currentPage, _ := pages.GetFrontPage(); currentPage == "system" {
+					sysFlex := sysPage.(*tview.Flex)
+					metricsTable := sysFlex.GetItem(1).(*tview.Table)
+					updateMetricsTable(metricsTable, data.metrics)
+				}
+			})
+		case data, ok := <-processListCh:
+			if !ok {
+				processListCh = nil
+				continue
+			}
+			if data.err != nil {
+				log.Printf("Fehler beim Abrufen der Prozessliste: %v\n", data.err)
+				continue
+			}
+			// UI aktualisieren mit Prozessliste
+			app.QueueUpdateDraw(func() {
+				if currentPage, _ := pages.GetFrontPage(); currentPage == "system" {
+					sysFlex := sysPage.(*tview.Flex)
+					processTable := sysFlex.GetItem(2).(*tview.Table)
+					updateProcessTable(processTable, data.processes)
+				}
+			})
+		case data, ok := <-hmacInfoCh:
+			if !ok {
+				hmacInfoCh = nil
+				continue
+			}
+			if data.err != nil {
+				log.Printf("Fehler beim Abrufen der hmac-file-server Informationen: %v\n", data.err)
+				continue
+			}
+			// UI aktualisieren mit hmac-file-server Informationen
+			app.QueueUpdateDraw(func() {
+				if currentPage, _ := pages.GetFrontPage(); currentPage == "hmac" && data.info != nil {
+					hmacFlex := hmacPage.(*tview.Flex)
+					hmacTable := hmacFlex.GetItem(0).(*tview.Table)
+					updateHmacTable(hmacTable, data.info, data.metrics)
+				}
+			})
+		}
+
+		// Abbruchbedingung, wenn alle Channels geschlossen sind
+		if systemDataCh == nil && (!metricsEnabled || metricsCh == nil) && processListCh == nil && hmacInfoCh == nil {
+			break
+		}
+	}
+}
+
+// Helper function to update system data table
+func updateSystemTable(sysTable *tview.Table, memUsage, cpuUsage float64, cores int) {
+	sysTable.Clear()
+	sysTable.SetCell(0, 0, tview.NewTableCell("Metric").SetAttributes(tcell.AttrBold))
+	sysTable.SetCell(0, 1, tview.NewTableCell("Value").SetAttributes(tcell.AttrBold))
+
+	// CPU Usage Row
+	cpuUsageCell := tview.NewTableCell(fmt.Sprintf("%.2f%%", cpuUsage))
+	if cpuUsage > HighUsage {
+		cpuUsageCell.SetTextColor(tcell.ColorRed)
+	} else if cpuUsage > MediumUsage {
+		cpuUsageCell.SetTextColor(tcell.ColorYellow)
+	} else {
+		cpuUsageCell.SetTextColor(tcell.ColorGreen)
+	}
+	sysTable.SetCell(1, 0, tview.NewTableCell("CPU Usage"))
+	sysTable.SetCell(1, 1, cpuUsageCell)
+
+	// Memory Usage Row
+	memUsageCell := tview.NewTableCell(fmt.Sprintf("%.2f%%", memUsage))
+	if memUsage > HighUsage {
+		memUsageCell.SetTextColor(tcell.ColorRed)
+	} else if memUsage > MediumUsage {
+		memUsageCell.SetTextColor(tcell.ColorYellow)
+	} else {
+		memUsageCell.SetTextColor(tcell.ColorGreen)
+	}
+	sysTable.SetCell(2, 0, tview.NewTableCell("Memory Usage"))
+	sysTable.SetCell(2, 1, memUsageCell)
+
+	// CPU Cores Row
+	sysTable.SetCell(3, 0, tview.NewTableCell("CPU Cores"))
+	sysTable.SetCell(3, 1, tview.NewTableCell(fmt.Sprintf("%d", cores)))
+}
+
+// Helper function to update metrics table
+func updateMetricsTable(metricsTable *tview.Table, metrics map[string]float64) {
+	metricsTable.Clear()
+	metricsTable.SetCell(0, 0, tview.NewTableCell("Metric").SetAttributes(tcell.AttrBold))
+	metricsTable.SetCell(0, 1, tview.NewTableCell("Value").SetAttributes(tcell.AttrBold))
+
+	row := 1
+	for key, value := range metrics {
+		metricsTable.SetCell(row, 0, tview.NewTableCell(key))
+		metricsTable.SetCell(row, 1, tview.NewTableCell(fmt.Sprintf("%.2f", value)))
+		row++
+	}
+}
+
+// Helper function to update process table
+func updateProcessTable(processTable *tview.Table, processes []ProcessInfo) {
+	processTable.Clear()
+	processTable.SetCell(0, 0, tview.NewTableCell("PID").SetAttributes(tcell.AttrBold))
+	processTable.SetCell(0, 1, tview.NewTableCell("Name").SetAttributes(tcell.AttrBold))
+	processTable.SetCell(0, 2, tview.NewTableCell("CPU%").SetAttributes(tcell.AttrBold))
+	processTable.SetCell(0, 3, tview.NewTableCell("Mem%").SetAttributes(tcell.AttrBold))
+	processTable.SetCell(0, 4, tview.NewTableCell("Command").SetAttributes(tcell.AttrBold))
+
+	// Sort processes by CPU usage
+	sort.Slice(processes, func(i, j int) bool {
+		return processes[i].CPUPercent > processes[j].CPUPercent
+	})
+
+	// Limit to top 20 processes
+	maxRows := 20
+	if len(processes) < maxRows {
+		maxRows = len(processes)
+	}
+
+	for i := 0; i < maxRows; i++ {
+		p := processes[i]
+		processTable.SetCell(i+1, 0, tview.NewTableCell(fmt.Sprintf("%d", p.PID)))
+		processTable.SetCell(i+1, 1, tview.NewTableCell(p.Name))
+		processTable.SetCell(i+1, 2, tview.NewTableCell(fmt.Sprintf("%.2f", p.CPUPercent)))
+		processTable.SetCell(i+1, 3, tview.NewTableCell(fmt.Sprintf("%.2f", p.MemPercent)))
+		processTable.SetCell(i+1, 4, tview.NewTableCell(p.CommandLine))
+	}
+}
+
+// Helper function to update hmac-table
+func updateHmacTable(hmacTable *tview.Table, hmacInfo *ProcessInfo, metrics map[string]float64) {
+	hmacTable.Clear()
+	hmacTable.SetCell(0, 0, tview.NewTableCell("Property").SetAttributes(tcell.AttrBold))
+	hmacTable.SetCell(0, 1, tview.NewTableCell("Value").SetAttributes(tcell.AttrBold))
+
+	// Process information
+	hmacTable.SetCell(1, 0, tview.NewTableCell("PID"))
+	hmacTable.SetCell(1, 1, tview.NewTableCell(fmt.Sprintf("%d", hmacInfo.PID)))
+
+	hmacTable.SetCell(2, 0, tview.NewTableCell("CPU%"))
+	hmacTable.SetCell(2, 1, tview.NewTableCell(fmt.Sprintf("%.2f", hmacInfo.CPUPercent)))
+
+	hmacTable.SetCell(3, 0, tview.NewTableCell("Mem%"))
+	hmacTable.SetCell(3, 1, tview.NewTableCell(fmt.Sprintf("%.2f", hmacInfo.MemPercent)))
+
+	hmacTable.SetCell(4, 0, tview.NewTableCell("Command"))
+	hmacTable.SetCell(4, 1, tview.NewTableCell(hmacInfo.CommandLine))
+	
+	hmacTable.SetCell(5, 0, tview.NewTableCell("Uptime"))
+	hmacTable.SetCell(5, 1, tview.NewTableCell(hmacInfo.Uptime)) // Neue Zeile für Uptime
+	
+	hmacTable.SetCell(6, 0, tview.NewTableCell("Status"))
+	hmacTable.SetCell(6, 1, tview.NewTableCell(hmacInfo.Status)) // Neue Zeile für Status
+	
+	hmacTable.SetCell(7, 0, tview.NewTableCell("Error Count"))
+	hmacTable.SetCell(7, 1, tview.NewTableCell(fmt.Sprintf("%d", hmacInfo.ErrorCount))) // Neue Zeile für Error Count
+
+	hmacTable.SetCell(8, 0, tview.NewTableCell("Total Requests"))
+	hmacTable.SetCell(8, 1, tview.NewTableCell(fmt.Sprintf("%d", hmacInfo.TotalRequests))) // Neue Zeile für Total Requests
+
+	hmacTable.SetCell(9, 0, tview.NewTableCell("Active Connections"))
+	hmacTable.SetCell(9, 1, tview.NewTableCell(fmt.Sprintf("%d", hmacInfo.ActiveConnections))) // Neue Zeile für Active Connections
+
+	hmacTable.SetCell(10, 0, tview.NewTableCell("Avg. Response Time (ms)"))
+	hmacTable.SetCell(10, 1, tview.NewTableCell(fmt.Sprintf("%.2f", hmacInfo.AverageResponseTime))) // Neue Zeile für Average Response Time
+
+	// Metrics related to hmac-file-server
+	row := 12
+	hmacTable.SetCell(row, 0, tview.NewTableCell("Metric").SetAttributes(tcell.AttrBold))
+	hmacTable.SetCell(row, 1, tview.NewTableCell("Value").SetAttributes(tcell.AttrBold))
+	row++
+
+	for key, value := range metrics {
+		if strings.Contains(key, "hmac_file_server_") {
+			hmacTable.SetCell(row, 0, tview.NewTableCell(key))
+			hmacTable.SetCell(row, 1, tview.NewTableCell(fmt.Sprintf("%.2f", value)))
+			row++
+		}
+	}
+}
+
+func createSystemPage() tview.Primitive {
+	// Create system data table
+	sysTable := tview.NewTable().SetBorders(false)
+	sysTable.SetTitle(" [::b]System Data ").SetBorder(true)
+
+	// Create Prometheus metrics table
+	metricsTable := tview.NewTable().SetBorders(false)
+	metricsTable.SetTitle(" [::b]Prometheus Metrics ").SetBorder(true)
+
+	// Create process list table
+	processTable := tview.NewTable().SetBorders(false)
+	processTable.SetTitle(" [::b]Process List ").SetBorder(true)
+
+	// Create a flex layout to hold the tables
+	sysFlex := tview.NewFlex().
+		SetDirection(tview.FlexRow).
+		AddItem(sysTable, 7, 0, false).
+		AddItem(metricsTable, 0, 1, false).
+		AddItem(processTable, 0, 2, false)
+
+	return sysFlex
+}
+
+func createHmacPage() tview.Primitive {
+	hmacTable := tview.NewTable().SetBorders(false)
+	hmacTable.SetTitle(" [::b]hmac-file-server Details ").SetBorder(true)
+
+	hmacFlex := tview.NewFlex().
+		SetDirection(tview.FlexRow).
+		AddItem(hmacTable, 0, 1, false)
+
+	return hmacFlex
+}
+
+func createLogsPage(ctx context.Context, app *tview.Application, logFilePath string) tview.Primitive {
+	logsTextView := tview.NewTextView().
+		SetDynamicColors(true).
+		SetRegions(true).
+		SetWordWrap(true)
+	logsTextView.SetTitle(" [::b]Logs ").SetBorder(true)
+
+	const numLines = 100 // Number of lines to read from the end of the log file
+
+	// Read logs periodically
+	go func() {
+		for {
+			select {
+			case <-ctx.Done():
+				return
+			default:
+				content, err := readLastNLines(logFilePath, numLines)
+				if err != nil {
+					app.QueueUpdateDraw(func() {
+						logsTextView.SetText(fmt.Sprintf("[red]Error reading log file: %v[white]", err))
+					})
+				} else {
+					// Process the log content to add colors
+					lines := strings.Split(content, "\n")
+					var coloredLines []string
+					for _, line := range lines {
+						if strings.Contains(line, "level=info") {
+							coloredLines = append(coloredLines, "[green]"+line+"[white]")
+						} else if strings.Contains(line, "level=warn") {
+							coloredLines = append(coloredLines, "[yellow]"+line+"[white]")
+						} else if strings.Contains(line, "level=error") {
+							coloredLines = append(coloredLines, "[red]"+line+"[white]")
+						} else {
+							// Default color
+							coloredLines = append(coloredLines, line)
+						}
+					}
+					app.QueueUpdateDraw(func() {
+						logsTextView.SetText(strings.Join(coloredLines, "\n"))
+					})
+				}
+				time.Sleep(2 * time.Second) // Refresh interval for logs
+			}
+		}
+	}()
+
+	return logsTextView
+}
+
+// Optimized readLastNLines to handle large files efficiently
+func readLastNLines(filePath string, n int) (string, error) {
+	file, err := os.Open(filePath)
+	if err != nil {
+		return "", err
+	}
+	defer file.Close()
+
+	const bufferSize = 1024
+	buffer := make([]byte, bufferSize)
+	var content []byte
+	var fileSize int64
+
+	fileInfo, err := file.Stat()
+	if err != nil {
+		return "", err
+	}
+	fileSize = fileInfo.Size()
+
+	var offset int64 = 0
+	for {
+		if fileSize-offset < bufferSize {
+			offset = fileSize
+		} else {
+			offset += bufferSize
+		}
+
+		_, err := file.Seek(-offset, io.SeekEnd)
+		if err != nil {
+			return "", err
+		}
+
+		bytesRead, err := file.Read(buffer)
+		if err != nil && err != io.EOF {
+			return "", err
+		}
+
+		content = append(buffer[:bytesRead], content...)
+
+		if bytesRead < bufferSize || len(strings.Split(string(content), "\n")) > n+1 {
+			break
+		}
+
+		if offset >= fileSize {
+			break
+		}
+	}
+
+	lines := strings.Split(string(content), "\n")
+	if len(lines) > n {
+		lines = lines[len(lines)-n:]
+	}
+	return strings.Join(lines, "\n"), nil
+}
+
+func main() {
+	app := tview.NewApplication()
+
+	// Create a cancellable context
+	ctx, cancel := context.WithCancel(context.Background())
+	defer cancel()
+
+	// Create pages
+	pages := tview.NewPages()
+
+	// System page
+	sysPage := createSystemPage()
+	pages.AddPage("system", sysPage, true, true)
+
+	// hmac-file-server page
+	hmacPage := createHmacPage()
+	pages.AddPage("hmac", hmacPage, true, false)
+
+	// Logs page mit dem gelesenen logFilePath
+	logsPage := createLogsPage(ctx, app, logFilePath)
+	pages.AddPage("logs", logsPage, true, false)
+
+	// Add key binding to switch views and handle exit
+	app.SetInputCapture(func(event *tcell.EventKey) *tcell.EventKey {
+		if event.Key() == tcell.KeyRune {
+			switch event.Rune() {
+			case 'q', 'Q':
+				cancel()
+				app.Stop()
+				return nil
+			case 's', 'S':
+				// Switch to system page
+				pages.SwitchToPage("system")
+			case 'h', 'H':
+				// Switch to hmac-file-server page
+				pages.SwitchToPage("hmac")
+			case 'l', 'L':
+				// Switch to logs page
+				pages.SwitchToPage("logs")
+			}
+		}
+		return event
+	})
+
+	// Start the UI update loop in a separate goroutine
+	go updateUI(ctx, app, pages, sysPage, hmacPage)
+
+	// Set the root and run the application
+	if err := app.SetRoot(pages, true).EnableMouse(true).Run(); err != nil {
+		log.Fatalf("Error running application: %v", err)
+		log.Fatalf("Error running application: %v", err)
+	}
+}

cmd/server/config.toml

@@ -1,67 +1,97 @@
-# Server Settings
 [server]
-ListenPort = "8080"
-UnixSocket = false
-StoreDir = "./testupload"
-LogLevel = "info"
-LogFile = "./hmac-file-server.log"
-MetricsEnabled = true
-MetricsPort = "9090"
-FileTTL = "8760h"
+#bind_ip = "127.0.0.1"
+listenport = "8080"
+unixsocket = false
+storagepath = "./uploads/"
+metricsenabled = true
+metricsport = "9090"
+deduplicationenabled = true
+minfreebytes = "5GB"
+filettl = "2y"
+filettlenabled = false
+autoadjustworkers = true
+networkevents = false
+pidfilepath = "./hmac-file-server.pid"
+precaching = false
+#globalextensions = ["*"]
 
-# Workers and Connections
-[workers]
-NumWorkers = 2
-UploadQueueSize = 500
+[deduplication]
+enabled = true
+directory = "./deduplication/"
+
+[logging]
+level = "debug"
+file = "./hmac-file-server.log"
+max_size = 100
+max_backups = 7
+max_age = 30
+compress = true
+
+[thumbnails]
+enabled = false
+directory = "./thumbnails/"
+size = "200x200"
+thumbnailintervalscan = "1h"
+concurrency = 5
+
+[iso]
+enabled = false
+size = "1TB"
+mountpoint = "/mnt/nfs_vol01/hmac-file-server/iso/"
+charset = "utf-8"
 
-# Timeout Settings
 [timeouts]
-ReadTimeout = "600s"
-WriteTimeout = "600s"
-IdleTimeout = "600s"
+readtimeout = "3600s"
+writetimeout = "3600s"
+idletimeout = "3600s"
 
-# Security Settings
 [security]
-Secret = "a-orc-and-a-humans-is-drinking-ale"
+secret = "hmac-file-server-is-the-win"
 
-# Versioning Settings
 [versioning]
-EnableVersioning = false
-MaxVersions = 1
+enableversioning = false
+maxversions = 1
 
-# Upload/Download Settings
 [uploads]
-ResumableUploadsEnabled = true
-ChunkedUploadsEnabled = true
-ChunkSize = 16777216
-AllowedExtensions = [
-    # Document formats
-    ".txt", ".pdf",
-
-    # Image formats
-    ".png", ".jpg", ".jpeg", ".gif", ".bmp", ".tiff", ".svg", ".webp",
-
-    # Video formats
-    ".wav", ".mp4", ".avi", ".mkv", ".mov", ".wmv", ".flv", ".webm", ".mpeg", ".mpg", ".m4v", ".3gp", ".3g2",
-
-    # Audio formats
-    ".mp3", ".ogg"
+resumableuploadsenabled = false
+chunkeduploadsenabled = true
+chunksize = "32MB"
+allowedextensions = [
+    ".txt", ".pdf", ".png", ".jpg", ".jpeg", ".gif",
+    ".bmp", ".tiff", ".svg", ".webp", ".wav", ".mp4",
+    ".avi", ".mkv", ".mov", ".wmv", ".flv", ".webm",
+    ".mpeg", ".mpg", ".m4v", ".3gp", ".3g2", ".mp3", ".ogg"
+]
+
+[downloads]
+chunkeddownloadsenabled = false
+chunksize = "32MB"
+allowedextensions = [
+    ".txt", ".pdf", ".png", ".jpg", ".jpeg", ".gif",
+    ".bmp", ".tiff", ".svg", ".webp", ".wav", ".mp4",
+    ".avi", ".mkv", ".mov", ".wmv", ".flv", ".webm",
+    ".mpeg", ".mpg", ".m4v", ".3gp", ".3g2", ".mp3", ".ogg"
 ]
 
-# ClamAV Settings
 [clamav]
-ClamAVEnabled = false
-ClamAVSocket = "/var/run/clamav/clamd.ctl"
-NumScanWorkers = 4
+clamavenabled = false
+clamavsocket = "/var/run/clamav/clamd.ctl"
+numscanworkers = 4
+scanfileextensions = [
+    ".exe", ".dll", ".bin", ".com", ".bat",
+    ".sh", ".php", ".js"
+]
 
-# Redis Settings
 [redis]
-RedisEnabled = false
-RedisAddr = "localhost:6379"
-RedisPassword = ""
-RedisDBIndex = 0
-RedisHealthCheckInterval = "120s"
+redisenabled = false
+redisdbindex = 0
+redisaddr = "localhost:6379"
+redispassword = ""
+redishealthcheckinterval = "120s"
 
-# Deduplication
-[deduplication]
-enabled = false
+[workers]
+numworkers = 4
+uploadqueuesize = 5000
+
+[file]
+filerevision = 1

config.example.toml

@@ -0,0 +1,86 @@
+[server]
+bind_ip = "0.0.0.0"
+listenport = "8080"
+unixsocket = false
+storagepath = "./uploads"
+logfile = "/var/log/hmac-file-server.log"
+metricsenabled = true
+metricsport = "9090"
+minfreebytes = "100MB"
+filettl = "8760h"
+filettlenabled = true
+autoadjustworkers = true
+networkevents = true
+pidfilepath = "/var/run/hmacfileserver.pid"
+cleanuponexit = true
+precaching = true
+deduplicationenabled = true
+globalextensions = [".txt", ".pdf", ".png", ".jpg", ".jpeg", ".gif", ".bmp", ".tiff", ".svg", ".webp"]
+# FileNaming options: "HMAC", "None"
+filenaming = "HMAC"
+
+[logging]
+level = "info"
+file = "/var/log/hmac-file-server.log"
+max_size = 100
+max_backups = 7
+max_age = 30
+compress = true
+
+[deduplication]
+enabled = true
+directory = "./deduplication"
+
+[iso]
+enabled = true
+size = "1GB"
+mountpoint = "/mnt/iso"
+charset = "utf-8"
+containerfile = "/mnt/iso/container.iso"
+
+[timeouts]
+readtimeout = "4800s"
+writetimeout = "4800s"
+idletimeout = "4800s"
+
+[security]
+secret = "changeme"
+
+[versioning]
+enableversioning = false
+maxversions = 1
+
+[uploads]
+resumableuploadsenabled = true
+chunkeduploadsenabled = true
+chunksize = "8192"
+allowedextensions = [".txt", ".pdf", ".png", ".jpg", ".jpeg", ".gif", ".bmp", ".tiff", ".svg", ".webp"]
+
+[downloads]
+resumabledownloadsenabled = true
+chunkeddownloadsenabled = true
+chunksize = "8192"
+allowedextensions = [".txt", ".pdf", ".png", ".jpg", ".jpeg", ".gif", ".bmp", ".tiff", ".svg", ".webp"]
+
+[clamav]
+clamavenabled = true
+clamavsocket = "/var/run/clamav/clamd.ctl"
+numscanworkers = 2
+scanfileextensions = [".txt", ".pdf", ".png", ".jpg", ".jpeg", ".gif", ".bmp", ".tiff", ".svg", ".webp"]
+
+[redis]
+redisenabled = true
+redisdbindex = 0
+redisaddr = "localhost:6379"
+redispassword = ""
+redishealthcheckinterval = "120s"
+
+[workers]
+numworkers = 4
+uploadqueuesize = 50
+
+[file]
+# Add file-specific configurations here
+
+[build]
+version = "2.6-Stable"

config.toml

@@ -0,0 +1,19 @@
+# Server configuration
+listenport = "8080"  # TCP port for incoming requests
+unixsocket = false   # Use Unix domain socket instead of TCP
+storagepath = "/path/to/hmac-file-server/data/"  # Directory to store uploaded files
+loglevel = "debug"   # Logging level: "debug", "info", "warn", "error"
+logfile = "/path/to/hmac-file-server.log"  # Path to log file; leave empty to use stdout
+metricsenabled = true   # Enable Prometheus metrics
+metricsport = "9090"    # Port for Prometheus metrics
+deduplicationenabled = true
+minfreebytes = "5GB"     # Minimum free disk space required
+filettl = "2Y"           # Time-to-live for files
+filettlenabled = false   # Enable TTL checks and cleanup
+autoadjustworkers = true # Automatically adjust worker threads based on load
+networkevents = false    # Enable detailed network event logging
+pidfilepath = "./hmac-file-server.pid" # Path to PID file
+precaching = true        # Pre-cache file structures on startup
+
+# New option to force network protocol
+forceprotocol = "auto"  # Options: "ipv4", "ipv6", "auto"

config.toml.example

@@ -0,0 +1,89 @@
+### Issue
+In `config.toml`, comments are using `//`, which is invalid in TOML. TOML uses `#` for comments. This causes a syntax error.
+
+### Corrected `config.toml`
+
+```toml
+# filepath: /home/renz/source/hmac-file-server/config.toml
+
+[server]
+listenport = "8080"
+unixsocket = false
+storagepath = "./uploads"
+metricsenabled = true
+metricsport = "9090"
+filettl = "8760h"
+minfreebytes = "100MB"
+autoadjustworkers = true
+networkevents = true
+temppath = "/tmp/hmac-file-server"
+loggingjson = false
+pidfilepath = "/var/run/hmacfileserver.pid"
+cleanuponexit = true
+precaching = true
+filettlenabled = true
+globalextensions = ["*"] # Allows all file types globally
+bind_ip = "0.0.0.0" # Specify the IP address to bind to (IPv4 or IPv6)
+
+[logging]
+level = "info"
+file = "/var/log/hmac-file-server.log"
+max_size = 100
+max_backups = 7
+max_age = 30
+compress = true
+
+[deduplication]
+enabled = true
+directory = "./deduplication"
+
+[iso]
+enabled = true
+size = "1GB"
+mountpoint = "/mnt/iso"
+charset = "utf-8"
+containerfile = "/path/to/iso/container.iso"
+
+[timeouts]
+readtimeout = "4800s"
+writetimeout = "4800s"
+idletimeout = "4800s"
+
+[security]
+secret = "changeme"
+
+[versioning]
+enableversioning = false
+maxversions = 5
+
+[uploads]
+resumableuploadsenabled = false
+chunkeduploadsenabled = true
+chunksize = "64mb"
+allowedextensions = ["*"] # Use ["*"] to allow all or specify extensions
+
+[downloads]
+resumabledownloadsenabled = false
+chunkeddownloadsenabled = true
+chunksize = "64mb"
+allowedextensions = [".jpg", ".png"] # Restricts downloads to specific types
+
+[clamav]
+clamavenabled = false
+clamavsocket = "/var/run/clamav/clamd.ctl"
+numscanworkers = 2
+scanfileextensions = [".exe", ".dll", ".pdf"]
+
+[redis]
+redisenabled = false
+redisaddr = "localhost:6379"
+redispassword = ""
+redisdbindex = 0
+redishealthcheckinterval = "120s"
+
+[workers]
+numworkers = 4
+uploadqueuesize = 5000
+
+[build]
+version = "v2.5"

dashboard/dashboard.json

@@ -27,8 +27,8 @@
         "overrides": []
       },
       "gridPos": {
-        "h": 6,
-        "w": 24,
+        "h": 7,
+        "w": 3,
         "x": 0,
         "y": 0
       },
@@ -39,12 +39,11 @@
           "showLineNumbers": false,
           "showMiniMap": false
         },
-        "content": "<div style=\"text-align: center; background-color: #111217; padding: 20px;\">\n    <h3 style=\"color: white; font-family: 'Arial', sans-serif; font-weight: bold;\">HMAC Dashboard</h3>\n    <img src=\"https://block.uuxo.net/hmac_icon.png\" alt=\"HMAC Icon\" style=\"width: 50px; height: 50px; display: block; margin: 10px auto;\">\n    <p style=\"font-family: 'Verdana', sans-serif; color: white;\">\n        This dashboard monitors <strong style=\"color: #FF5733;\">key metrics</strong> for the \n        <span style=\"font-style: italic; color: #007BFF;\">HMAC File Server</span>.\n    </p>\n</div>\n",
+        "content": "<div style=\"text-align: center; background-color: transparent; padding: 20px;\">\n    <h3 style=\"color: white; font-family: 'Arial', sans-serif; font-weight: bold;\">HMAC Dashboard</h3>\n    <img src=\"https://git.uuxo.net/uuxo/hmac-file-server/raw/branch/main/dashboard/hmac_icon.png\" alt=\"HMAC Icon\" style=\"width: 50px; height: 50px; display: block; margin: 10px auto;\">\n    <p style=\"font-family: 'Verdana', sans-serif; color: white;\">\n        This dashboard monitors <strong style=\"color: #FF5733;\">key metrics</strong> for the \n        <span style=\"font-style: italic; color: #007BFF;\">HMAC File Server</span>.\n    </p>\n</div>\n",
         "mode": "html"
       },
-      "pluginVersion": "11.3.1",
+      "pluginVersion": "11.4.0",
       "title": "HMAC Dashboard",
-      "transparent": true,
       "type": "text"
     },
     {
@@ -76,10 +75,10 @@
         "overrides": []
       },
       "gridPos": {
-        "h": 5,
+        "h": 7,
         "w": 6,
-        "x": 0,
-        "y": 6
+        "x": 3,
+        "y": 0
       },
       "id": 14,
       "options": {
@@ -106,7 +105,7 @@
         "sizing": "auto",
         "valueMode": "color"
       },
-      "pluginVersion": "11.3.1",
+      "pluginVersion": "11.4.0",
       "targets": [
         {
           "editorMode": "code",
@@ -142,10 +141,10 @@
         "overrides": []
       },
       "gridPos": {
-        "h": 5,
+        "h": 7,
         "w": 6,
-        "x": 6,
-        "y": 6
+        "x": 9,
+        "y": 0
       },
       "id": 18,
       "options": {
@@ -165,7 +164,7 @@
         "textMode": "auto",
         "wideLayout": true
       },
-      "pluginVersion": "11.3.1",
+      "pluginVersion": "11.4.0",
       "targets": [
         {
           "editorMode": "code",
@@ -192,6 +191,10 @@
               {
                 "color": "green",
                 "value": null
+              },
+              {
+                "color": "red",
+                "value": 80
               }
             ]
           }
@@ -199,10 +202,68 @@
         "overrides": []
       },
       "gridPos": {
-        "h": 5,
-        "w": 6,
-        "x": 12,
-        "y": 6
+        "h": 7,
+        "w": 5,
+        "x": 15,
+        "y": 0
+      },
+      "id": 10,
+      "options": {
+        "colorMode": "value",
+        "graphMode": "none",
+        "justifyMode": "auto",
+        "orientation": "auto",
+        "percentChangeColorMode": "standard",
+        "reduceOptions": {
+          "calcs": [
+            "lastNotNull"
+          ],
+          "fields": "",
+          "values": false
+        },
+        "showPercentChange": false,
+        "textMode": "value",
+        "wideLayout": true
+      },
+      "pluginVersion": "11.4.0",
+      "targets": [
+        {
+          "editorMode": "code",
+          "expr": "go_threads",
+          "format": "table",
+          "legendFormat": "{{hmac-file-server}}",
+          "range": true,
+          "refId": "A"
+        }
+      ],
+      "title": "HMAC GO Threads",
+      "type": "stat"
+    },
+    {
+      "datasource": {
+        "default": true,
+        "type": "prometheus"
+      },
+      "fieldConfig": {
+        "defaults": {
+          "mappings": [],
+          "thresholds": {
+            "mode": "absolute",
+            "steps": [
+              {
+                "color": "green",
+                "value": null
+              }
+            ]
+          }
+        },
+        "overrides": []
+      },
+      "gridPos": {
+        "h": 7,
+        "w": 4,
+        "x": 20,
+        "y": 0
       },
       "id": 17,
       "options": {
@@ -222,7 +283,7 @@
         "textMode": "auto",
         "wideLayout": true
       },
-      "pluginVersion": "11.3.1",
+      "pluginVersion": "11.4.0",
       "targets": [
         {
           "editorMode": "code",
@@ -262,10 +323,10 @@
         "overrides": []
       },
       "gridPos": {
-        "h": 5,
-        "w": 3,
-        "x": 18,
-        "y": 6
+        "h": 7,
+        "w": 5,
+        "x": 0,
+        "y": 7
       },
       "id": 11,
       "options": {
@@ -285,11 +346,11 @@
         "textMode": "value",
         "wideLayout": true
       },
-      "pluginVersion": "11.3.1",
+      "pluginVersion": "11.4.0",
       "targets": [
         {
           "editorMode": "code",
-          "expr": "hmac_file_server_uploads_total",
+          "expr": "increase(hmac_file_server_uploads_total[1h])",
           "format": "table",
           "legendFormat": "Uploads",
           "range": true,
@@ -325,10 +386,10 @@
         "overrides": []
       },
       "gridPos": {
-        "h": 5,
-        "w": 3,
-        "x": 21,
-        "y": 6
+        "h": 7,
+        "w": 5,
+        "x": 5,
+        "y": 7
       },
       "id": 12,
       "options": {
@@ -348,7 +409,7 @@
         "textMode": "value",
         "wideLayout": true
       },
-      "pluginVersion": "11.3.1",
+      "pluginVersion": "11.4.0",
       "targets": [
         {
           "editorMode": "code",
@@ -390,10 +451,10 @@
         "overrides": []
       },
       "gridPos": {
-        "h": 5,
+        "h": 7,
         "w": 3,
-        "x": 0,
-        "y": 11
+        "x": 10,
+        "y": 7
       },
       "id": 15,
       "options": {
@@ -413,7 +474,7 @@
         "textMode": "auto",
         "wideLayout": true
       },
-      "pluginVersion": "11.3.1",
+      "pluginVersion": "11.4.0",
       "targets": [
         {
           "datasource": {
@@ -457,201 +518,12 @@
         "overrides": []
       },
       "gridPos": {
-        "h": 5,
+        "h": 7,
         "w": 3,
-        "x": 3,
-        "y": 11
+        "x": 13,
+        "y": 7
       },
-      "id": 10,
-      "options": {
-        "colorMode": "value",
-        "graphMode": "none",
-        "justifyMode": "auto",
-        "orientation": "auto",
-        "percentChangeColorMode": "standard",
-        "reduceOptions": {
-          "calcs": [
-            "lastNotNull"
-          ],
-          "fields": "",
-          "values": false
-        },
-        "showPercentChange": false,
-        "textMode": "value",
-        "wideLayout": true
-      },
-      "pluginVersion": "11.3.1",
-      "targets": [
-        {
-          "editorMode": "code",
-          "expr": "go_threads",
-          "format": "table",
-          "legendFormat": "{{hmac-file-server}}",
-          "range": true,
-          "refId": "A"
-        }
-      ],
-      "title": "HMAC GO Threads",
-      "type": "stat"
-    },
-    {
-      "datasource": {
-        "default": true,
-        "type": "prometheus"
-      },
-      "fieldConfig": {
-        "defaults": {
-          "mappings": [],
-          "thresholds": {
-            "mode": "absolute",
-            "steps": [
-              {
-                "color": "green",
-                "value": null
-              },
-              {
-                "color": "red",
-                "value": 80
-              }
-            ]
-          }
-        },
-        "overrides": []
-      },
-      "gridPos": {
-        "h": 5,
-        "w": 3,
-        "x": 6,
-        "y": 11
-      },
-      "id": 21,
-      "options": {
-        "colorMode": "value",
-        "graphMode": "none",
-        "justifyMode": "auto",
-        "orientation": "auto",
-        "percentChangeColorMode": "standard",
-        "reduceOptions": {
-          "calcs": [
-            "lastNotNull"
-          ],
-          "fields": "",
-          "values": false
-        },
-        "showPercentChange": false,
-        "textMode": "value",
-        "wideLayout": true
-      },
-      "pluginVersion": "11.3.1",
-      "targets": [
-        {
-          "editorMode": "code",
-          "expr": "hmac_file_deletions_total",
-          "format": "table",
-          "legendFormat": "{{hmac-file-server}}",
-          "range": true,
-          "refId": "A"
-        }
-      ],
-      "title": "HMAC FileTTL Deletion(s)",
-      "type": "stat"
-    },
-    {
-      "datasource": {
-        "default": true,
-        "type": "prometheus"
-      },
-      "fieldConfig": {
-        "defaults": {
-          "mappings": [],
-          "thresholds": {
-            "mode": "absolute",
-            "steps": [
-              {
-                "color": "green",
-                "value": null
-              },
-              {
-                "color": "red",
-                "value": 80
-              }
-            ]
-          }
-        },
-        "overrides": []
-      },
-      "gridPos": {
-        "h": 5,
-        "w": 3,
-        "x": 9,
-        "y": 11
-      },
-      "id": 20,
-      "options": {
-        "colorMode": "value",
-        "graphMode": "none",
-        "justifyMode": "auto",
-        "orientation": "auto",
-        "percentChangeColorMode": "standard",
-        "reduceOptions": {
-          "calcs": [
-            "lastNotNull"
-          ],
-          "fields": "",
-          "values": false
-        },
-        "showPercentChange": false,
-        "textMode": "value",
-        "wideLayout": true
-      },
-      "pluginVersion": "11.3.1",
-      "targets": [
-        {
-          "editorMode": "code",
-          "expr": "hmac_cache_misses_total",
-          "format": "table",
-          "legendFormat": "{{hmac-file-server}}",
-          "range": true,
-          "refId": "A"
-        }
-      ],
-      "title": "HMAC Cache Misses",
-      "type": "stat"
-    },
-    {
-      "datasource": {
-        "default": true,
-        "type": "prometheus"
-      },
-      "fieldConfig": {
-        "defaults": {
-          "color": {
-            "mode": "thresholds"
-          },
-          "mappings": [],
-          "thresholds": {
-            "mode": "absolute",
-            "steps": [
-              {
-                "color": "green",
-                "value": null
-              },
-              {
-                "color": "red",
-                "value": 80
-              }
-            ]
-          }
-        },
-        "overrides": []
-      },
-      "gridPos": {
-        "h": 5,
-        "w": 3,
-        "x": 12,
-        "y": 11
-      },
-      "id": 16,
+      "id": 13,
       "options": {
         "colorMode": "value",
         "graphMode": "area",
@@ -669,81 +541,17 @@
         "textMode": "auto",
         "wideLayout": true
       },
-      "pluginVersion": "11.3.1",
+      "pluginVersion": "11.4.0",
       "targets": [
         {
           "editorMode": "code",
-          "exemplar": false,
-          "expr": "hmac_active_connections_total",
-          "format": "table",
-          "instant": false,
-          "legendFormat": "__auto",
+          "expr": "hmac_file_server_download_errors_total",
+          "legendFormat": "Download Errors",
           "range": true,
           "refId": "A"
         }
       ],
-      "title": "HMAC Active Connections",
-      "type": "stat"
-    },
-    {
-      "datasource": {
-        "default": true,
-        "type": "prometheus"
-      },
-      "fieldConfig": {
-        "defaults": {
-          "color": {
-            "mode": "thresholds"
-          },
-          "mappings": [],
-          "thresholds": {
-            "mode": "absolute",
-            "steps": [
-              {
-                "color": "green",
-                "value": null
-              }
-            ]
-          }
-        },
-        "overrides": []
-      },
-      "gridPos": {
-        "h": 5,
-        "w": 3,
-        "x": 15,
-        "y": 11
-      },
-      "id": 19,
-      "options": {
-        "colorMode": "value",
-        "graphMode": "area",
-        "justifyMode": "auto",
-        "orientation": "auto",
-        "percentChangeColorMode": "standard",
-        "reduceOptions": {
-          "calcs": [
-            "lastNotNull"
-          ],
-          "fields": "",
-          "values": false
-        },
-        "showPercentChange": false,
-        "textMode": "auto",
-        "wideLayout": true
-      },
-      "pluginVersion": "11.3.1",
-      "targets": [
-        {
-          "editorMode": "code",
-          "expr": "hmac_infected_files_total",
-          "format": "table",
-          "legendFormat": "__auto",
-          "range": true,
-          "refId": "A"
-        }
-      ],
-      "title": "HMAC infected file(s)",
+      "title": "HMAC Download Errors",
       "type": "stat"
     },
     {
@@ -771,10 +579,10 @@
         "overrides": []
       },
       "gridPos": {
-        "h": 5,
+        "h": 7,
         "w": 3,
-        "x": 18,
-        "y": 11
+        "x": 16,
+        "y": 7
       },
       "id": 2,
       "options": {
@@ -794,7 +602,7 @@
         "textMode": "auto",
         "wideLayout": true
       },
-      "pluginVersion": "11.3.1",
+      "pluginVersion": "11.4.0",
       "targets": [
         {
           "editorMode": "code",
@@ -832,12 +640,73 @@
         "overrides": []
       },
       "gridPos": {
-        "h": 5,
-        "w": 3,
-        "x": 21,
-        "y": 11
+        "h": 7,
+        "w": 5,
+        "x": 19,
+        "y": 7
       },
-      "id": 13,
+      "id": 21,
+      "options": {
+        "colorMode": "value",
+        "graphMode": "none",
+        "justifyMode": "auto",
+        "orientation": "auto",
+        "percentChangeColorMode": "standard",
+        "reduceOptions": {
+          "calcs": [
+            "lastNotNull"
+          ],
+          "fields": "",
+          "values": false
+        },
+        "showPercentChange": false,
+        "textMode": "value",
+        "wideLayout": true
+      },
+      "pluginVersion": "11.4.0",
+      "targets": [
+        {
+          "editorMode": "code",
+          "expr": "hm",
+          "format": "table",
+          "legendFormat": "__auto",
+          "range": true,
+          "refId": "A"
+        }
+      ],
+      "title": "HMAC FileTTL Deletion(s)",
+      "type": "stat"
+    },
+    {
+      "datasource": {
+        "default": true,
+        "type": "prometheus"
+      },
+      "fieldConfig": {
+        "defaults": {
+          "color": {
+            "mode": "thresholds"
+          },
+          "mappings": [],
+          "thresholds": {
+            "mode": "absolute",
+            "steps": [
+              {
+                "color": "green",
+                "value": null
+              }
+            ]
+          }
+        },
+        "overrides": []
+      },
+      "gridPos": {
+        "h": 7,
+        "w": 3,
+        "x": 0,
+        "y": 14
+      },
+      "id": 19,
       "options": {
         "colorMode": "value",
         "graphMode": "area",
@@ -855,18 +724,254 @@
         "textMode": "auto",
         "wideLayout": true
       },
-      "pluginVersion": "11.3.1",
+      "pluginVersion": "11.4.0",
       "targets": [
         {
           "editorMode": "code",
-          "expr": "hmac_file_server_download_errors_total",
-          "legendFormat": "Download Errors",
+          "expr": "hmac_infected_files_total",
+          "format": "table",
+          "legendFormat": "__auto",
           "range": true,
           "refId": "A"
         }
       ],
-      "title": "HMAC Download Errors",
+      "title": "HMAC infected file(s)",
       "type": "stat"
+    },
+    {
+      "datasource": {
+        "default": true,
+        "type": "prometheus"
+      },
+      "fieldConfig": {
+        "defaults": {
+          "color": {
+            "mode": "palette-classic"
+          },
+          "custom": {
+            "fillOpacity": 80,
+            "gradientMode": "none",
+            "hideFrom": {
+              "legend": false,
+              "tooltip": false,
+              "viz": false
+            },
+            "lineWidth": 1,
+            "stacking": {
+              "group": "A",
+              "mode": "none"
+            }
+          },
+          "fieldMinMax": false,
+          "mappings": [],
+          "thresholds": {
+            "mode": "absolute",
+            "steps": [
+              {
+                "color": "green",
+                "value": null
+              },
+              {
+                "color": "red",
+                "value": 80
+              }
+            ]
+          },
+          "unit": "files"
+        },
+        "overrides": []
+      },
+      "gridPos": {
+        "h": 7,
+        "w": 7,
+        "x": 3,
+        "y": 14
+      },
+      "id": 22,
+      "options": {
+        "legend": {
+          "calcs": [],
+          "displayMode": "list",
+          "placement": "bottom",
+          "showLegend": true
+        },
+        "tooltip": {
+          "mode": "single",
+          "sort": "none"
+        }
+      },
+      "pluginVersion": "11.4.0",
+      "targets": [
+        {
+          "editorMode": "code",
+          "exemplar": false,
+          "expr": "increase(hmac_file_server_clamav_scans_total[24h])",
+          "format": "time_series",
+          "instant": true,
+          "interval": "",
+          "legendFormat": "__auto",
+          "range": true,
+          "refId": "A"
+        }
+      ],
+      "title": "HMAC ClamAV San (24h)",
+      "type": "histogram"
+    },
+    {
+      "datasource": {
+        "default": true,
+        "type": "prometheus"
+      },
+      "fieldConfig": {
+        "defaults": {
+          "color": {
+            "mode": "palette-classic"
+          },
+          "custom": {
+            "fillOpacity": 80,
+            "gradientMode": "none",
+            "hideFrom": {
+              "legend": false,
+              "tooltip": false,
+              "viz": false
+            },
+            "lineWidth": 1,
+            "stacking": {
+              "group": "A",
+              "mode": "none"
+            }
+          },
+          "fieldMinMax": false,
+          "mappings": [],
+          "thresholds": {
+            "mode": "absolute",
+            "steps": [
+              {
+                "color": "green",
+                "value": null
+              },
+              {
+                "color": "red",
+                "value": 80
+              }
+            ]
+          },
+          "unit": "files"
+        },
+        "overrides": []
+      },
+      "gridPos": {
+        "h": 7,
+        "w": 7,
+        "x": 10,
+        "y": 14
+      },
+      "id": 23,
+      "options": {
+        "legend": {
+          "calcs": [],
+          "displayMode": "list",
+          "placement": "bottom",
+          "showLegend": true
+        },
+        "tooltip": {
+          "mode": "single",
+          "sort": "none"
+        }
+      },
+      "pluginVersion": "11.4.0",
+      "targets": [
+        {
+          "editorMode": "code",
+          "exemplar": false,
+          "expr": "increase(hmac_file_server_clamav_errors_total[24h])",
+          "format": "time_series",
+          "instant": true,
+          "interval": "",
+          "legendFormat": "__auto",
+          "range": true,
+          "refId": "A"
+        }
+      ],
+      "title": "HMAC ClamAV SanError(s) (24h)",
+      "type": "histogram"
+    },
+    {
+      "datasource": {
+        "default": true,
+        "type": "prometheus"
+      },
+      "fieldConfig": {
+        "defaults": {
+          "color": {
+            "mode": "palette-classic"
+          },
+          "custom": {
+            "fillOpacity": 80,
+            "gradientMode": "none",
+            "hideFrom": {
+              "legend": false,
+              "tooltip": false,
+              "viz": false
+            },
+            "lineWidth": 1,
+            "stacking": {
+              "group": "A",
+              "mode": "none"
+            }
+          },
+          "mappings": [],
+          "thresholds": {
+            "mode": "absolute",
+            "steps": [
+              {
+                "color": "green",
+                "value": null
+              },
+              {
+                "color": "red",
+                "value": 80
+              }
+            ]
+          }
+        },
+        "overrides": []
+      },
+      "gridPos": {
+        "h": 7,
+        "w": 7,
+        "x": 17,
+        "y": 14
+      },
+      "id": 16,
+      "options": {
+        "legend": {
+          "calcs": [],
+          "displayMode": "list",
+          "placement": "bottom",
+          "showLegend": true
+        },
+        "tooltip": {
+          "mode": "single",
+          "sort": "none"
+        }
+      },
+      "pluginVersion": "11.4.0",
+      "targets": [
+        {
+          "editorMode": "code",
+          "exemplar": false,
+          "expr": "histogram_quantile(0.95, sum(rate(hmac_file_server_request_duration_seconds_bucket[5m])) by (le))",
+          "format": "time_series",
+          "instant": true,
+          "interval": "",
+          "legendFormat": "__auto",
+          "range": true,
+          "refId": "A"
+        }
+      ],
+      "title": "HMAC Request Duration",
+      "type": "histogram"
     }
   ],
   "preload": false,
@@ -876,13 +981,13 @@
     "list": []
   },
   "time": {
-    "from": "now-5m",
+    "from": "now-24h",
     "to": "now"
   },
   "timepicker": {},
   "timezone": "",
   "title": "HMAC File Server Metrics",
   "uid": "de0ye5t0hzq4ge",
-  "version": 129,
+  "version": 153,
   "weekStart": ""
 }

docs/updates.md

@@ -0,0 +1,32 @@
+
+**Important Update Regarding HMAC File Server Version 2.6-Stable**
+
+We've identified and resolved critical issues with the deduplication processes in **Version 2.6-Stable** of the HMAC File Server. This update brings significant improvements to ensure reliable deduplication and overall server performance.
+
+**Impact:**
+- **Resolved Deduplication Issues:** The deduplication feature in **Version 2.6-Stable** has been thoroughly tested and stabilized, ensuring consistent file integrity and optimal server performance.
+- **Enhanced Stability:** Addressed previous unexpected behaviors, including lost links to files and incorrect file handling operations.
+
+**Recommended Actions:**
+- **Upgrade to Version 2.6-Stable:** If you are using any earlier version, we strongly recommend upgrading to **Version 2.6-Stable** to benefit from the latest fixes and improvements.
+- **Rollback Not Required:** Users on older releases can upgrade without the need to rollback, as **Version 2.6-Stable** resolves all critical issues present in prior versions.
+
+**What's New in Version 2.6-Stable:**
+- **Deduplication Enhancements:**
+  - **Reliable Deduplication:** Improved algorithms for accurate and efficient processing.
+  - **Performance Optimizations:** Reduced resource consumption for faster deduplication.
+- **Enhanced Features:**
+  - **Robust Malware Scanning:** Integrated advanced scanning to ensure file safety.
+  - **Enhanced Redis Support:** Improved performance with Redis for better caching.
+  - **Better Configuration Options:** More flexible settings for customization.
+- **Security Improvements:**
+  - **Strengthened Authentication Mechanisms:** Safeguards against unauthorized access.
+  - **Improved Data Encryption:** Advanced encryption standards for data at rest and in transit.
+
+**Stay Updated:**
+For detailed information on addressed issues, fixes, and enhancements introduced in **Version 2.6-Stable**, please visit our repositories:
+
+- [Git uuxo.net](https://git.uuxo.net/uuxo/hmac-file-server)
+- [GitHub](https://github.com/PlusOne/hmac-file-server)
+
+We appreciate your patience and support as we continue improving HMAC File Server. **Version 2.6-Stable** reflects our commitment to delivering a reliable and efficient file server solution. Thank you for choosing HMAC File Server!

go.mod

@@ -1,39 +1,45 @@
 module github.com/PlusOne/hmac-file-server
 
-go 1.21
+go 1.24.0
 
 require (
+	github.com/gdamore/tcell/v2 v2.7.4
 	github.com/go-redis/redis/v8 v8.11.5
+	github.com/pelletier/go-toml v1.9.5
 	github.com/prometheus/client_golang v1.20.5
 	github.com/shirou/gopsutil v3.21.11+incompatible
+	github.com/shirou/gopsutil/v3 v3.24.5
 	github.com/sirupsen/logrus v1.9.3
+	github.com/spf13/viper v1.11.0
 )
 
 require (
 	github.com/dgryski/go-rendezvous v0.0.0-20200823014737-9f7001d12a5f // indirect
 	github.com/fsnotify/fsnotify v1.7.0 // indirect
+	github.com/gdamore/encoding v1.0.0 // indirect
 	github.com/hashicorp/hcl v1.0.0 // indirect
+	github.com/lucasb-eyer/go-colorful v1.2.0 // indirect
+	github.com/lufia/plan9stats v0.0.0-20211012122336-39d0f177ccd0 // indirect
 	github.com/magiconair/properties v1.8.7 // indirect
+	github.com/mattn/go-runewidth v0.0.15 // indirect
 	github.com/mitchellh/mapstructure v1.5.0 // indirect
 	github.com/pelletier/go-toml/v2 v2.2.2 // indirect
-	github.com/sagikazarmark/locafero v0.4.0 // indirect
-	github.com/sagikazarmark/slog-shim v0.1.0 // indirect
-	github.com/sourcegraph/conc v0.3.0 // indirect
+	github.com/power-devops/perfstat v0.0.0-20210106213030-5aafc221ea8c // indirect
+	github.com/rivo/uniseg v0.4.7 // indirect
+	github.com/shoenig/go-m1cpu v0.1.6 // indirect
 	github.com/spf13/afero v1.11.0 // indirect
 	github.com/spf13/cast v1.6.0 // indirect
+	github.com/spf13/jwalterweatherman v1.1.0 // indirect
 	github.com/spf13/pflag v1.0.5 // indirect
-	github.com/spf13/viper v1.19.0 // indirect
 	github.com/subosito/gotenv v1.6.0 // indirect
-	go.uber.org/atomic v1.9.0 // indirect
-	go.uber.org/multierr v1.9.0 // indirect
-	golang.org/x/exp v0.0.0-20230905200255-921286631fa9 // indirect
-	golang.org/x/text v0.18.0 // indirect
+	golang.org/x/term v0.17.0 // indirect
+	golang.org/x/text v0.21.0 // indirect
 	gopkg.in/ini.v1 v1.67.0 // indirect
+	gopkg.in/yaml.v2 v2.4.0 // indirect
 	gopkg.in/yaml.v3 v3.0.1 // indirect
 )
 
 require (
-	github.com/BurntSushi/toml v1.4.0
 	github.com/beorn7/perks v1.0.1 // indirect
 	github.com/cespare/xxhash/v2 v2.3.0 // indirect
 	github.com/dutchcoders/go-clamd v0.0.0-20170520113014-b970184f4d9e
@@ -42,11 +48,13 @@ require (
 	github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 // indirect
 	github.com/patrickmn/go-cache v2.1.0+incompatible
 	github.com/prometheus/client_model v0.6.1 // indirect
-	github.com/prometheus/common v0.60.1 // indirect
+	github.com/prometheus/common v0.61.0
 	github.com/prometheus/procfs v0.15.1 // indirect
+	github.com/rivo/tview v0.0.0-20241103174730-c76f7879f592
 	github.com/tklauser/go-sysconf v0.3.14 // indirect
 	github.com/tklauser/numcpus v0.9.0 // indirect
 	github.com/yusufpapurcu/wmi v1.2.4 // indirect
-	golang.org/x/sys v0.26.0 // indirect
-	google.golang.org/protobuf v1.35.1 // indirect
+	golang.org/x/sys v0.28.0 // indirect
+	google.golang.org/protobuf v1.35.2 // indirect
+	gopkg.in/natefinch/lumberjack.v2 v2.2.1
 )

go.sum

@@ -1,5 +1,3 @@
-github.com/BurntSushi/toml v1.4.0 h1:kuoIxZQy2WRRk1pttg9asf+WVv6tWQuBNVmK8+nqPr0=
-github.com/BurntSushi/toml v1.4.0/go.mod h1:ukJfTF/6rtPPRCnwkur4qwRxa8vTRFBF0uk2lLoLwho=
 github.com/beorn7/perks v1.0.1 h1:VlbKKnNfV8bJzeqoa4cOKqO6bYr3WgKZxO8Z16+hsOM=
 github.com/beorn7/perks v1.0.1/go.mod h1:G2ZrVWU2WbWT9wwq4/hrbKbnv/1ERSJQ0ibhJ6rlkpw=
 github.com/cespare/xxhash/v2 v2.3.0 h1:UL815xU9SqsFlibzuggzjXhog7bL6oX9BbNZnL2UFvs=
@@ -7,30 +5,44 @@ github.com/cespare/xxhash/v2 v2.3.0/go.mod h1:VGX0DQ3Q6kWi7AoAeZDth3/j3BFtOZR5XL
 github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
 github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
 github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
-github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc h1:U9qPSI2PIWSS1VwoXQT9A3Wy9MM3WgvqSxFWenqJduM=
 github.com/dgryski/go-rendezvous v0.0.0-20200823014737-9f7001d12a5f h1:lO4WD4F/rVNCu3HqELle0jiPLLBs70cWOduZpkS1E78=
 github.com/dgryski/go-rendezvous v0.0.0-20200823014737-9f7001d12a5f/go.mod h1:cuUVRXasLTGF7a8hSLbxyZXjz+1KgoB3wDUb6vlszIc=
 github.com/dutchcoders/go-clamd v0.0.0-20170520113014-b970184f4d9e h1:rcHHSQqzCgvlwP0I/fQ8rQMn/MpHE5gWSLdtpxtP6KQ=
 github.com/dutchcoders/go-clamd v0.0.0-20170520113014-b970184f4d9e/go.mod h1:Byz7q8MSzSPkouskHJhX0er2mZY/m0Vj5bMeMCkkyY4=
-github.com/fsnotify/fsnotify v1.4.9 h1:hsms1Qyu0jgnwNXIxa+/V/PDsU6CfLf6CNO8H7IWoS4=
-github.com/fsnotify/fsnotify v1.4.9/go.mod h1:znqG4EE+3YCdAaPaxE2ZRY/06pZUdp0tY4IgpuI1SZQ=
+github.com/frankban/quicktest v1.14.6 h1:7Xjx+VpznH+oBnejlPUj8oUpdxnVs4f8XU8WnHkI4W8=
+github.com/frankban/quicktest v1.14.6/go.mod h1:4ptaffx2x8+WTWXmUCuVU6aPUX1/Mz7zb5vbUoiM6w0=
 github.com/fsnotify/fsnotify v1.7.0 h1:8JEhPFa5W2WU7YfeZzPNqzMP6Lwt7L2715Ggo0nosvA=
 github.com/fsnotify/fsnotify v1.7.0/go.mod h1:40Bi/Hjc2AVfZrqy+aj+yEI+/bRxZnMJyTJwOpGvigM=
+github.com/gdamore/encoding v1.0.0 h1:+7OoQ1Bc6eTm5niUzBa0Ctsh6JbMW6Ra+YNuAtDBdko=
+github.com/gdamore/encoding v1.0.0/go.mod h1:alR0ol34c49FCSBLjhosxzcPHQbf2trDkoo5dl+VrEg=
+github.com/gdamore/tcell/v2 v2.7.4 h1:sg6/UnTM9jGpZU+oFYAsDahfchWAFW8Xx2yFinNSAYU=
+github.com/gdamore/tcell/v2 v2.7.4/go.mod h1:dSXtXTSK0VsW1biw65DZLZ2NKr7j0qP/0J7ONmsraWg=
 github.com/go-ole/go-ole v1.2.6/go.mod h1:pprOEPIfldk/42T2oK7lQ4v4JSDwmV0As9GaiUsvbm0=
 github.com/go-ole/go-ole v1.3.0 h1:Dt6ye7+vXGIKZ7Xtk4s6/xVdGDQynvom7xCFEdWr6uE=
 github.com/go-ole/go-ole v1.3.0/go.mod h1:5LS6F96DhAwUc7C+1HLexzMXY1xGRSryjyPPKW6zv78=
 github.com/go-redis/redis/v8 v8.11.5 h1:AcZZR7igkdvfVmQTPnu9WE37LRrO/YrBH5zWyjDC0oI=
 github.com/go-redis/redis/v8 v8.11.5/go.mod h1:gREzHqY1hg6oD9ngVRbLStwAWKhA0FEgq8Jd4h5lpwo=
+github.com/google/go-cmp v0.5.6/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
 github.com/google/go-cmp v0.6.0 h1:ofyhxvXcZhMsU5ulbFiLKl/XBFqE1GSq7atu8tAmTRI=
 github.com/google/go-cmp v0.6.0/go.mod h1:17dUlkBOakJ0+DkrSSNjCkIjxS6bF9zb3elmeNGIjoY=
 github.com/hashicorp/hcl v1.0.0 h1:0Anlzjpi4vEasTeNFn2mLJgTSwt0+6sfsiTG8qcWGx4=
 github.com/hashicorp/hcl v1.0.0/go.mod h1:E5yfLk+7swimpb2L/Alb/PJmXilQ/rhwaUYs4T20WEQ=
 github.com/klauspost/compress v1.17.11 h1:In6xLpyWOi1+C7tXUUWv2ot1QvBjxevKAaI6IXrJmUc=
 github.com/klauspost/compress v1.17.11/go.mod h1:pMDklpSncoRMuLFrf1W9Ss9KT+0rH90U12bZKk7uwG0=
+github.com/kr/pretty v0.3.1 h1:flRD4NNwYAUpkphVc1HcthR4KEIFJ65n8Mw5qdRn3LE=
+github.com/kr/pretty v0.3.1/go.mod h1:hoEshYVHaxMs3cyo3Yncou5ZscifuDolrwPKZanG3xk=
+github.com/kr/text v0.2.0 h1:5Nx0Ya0ZqY2ygV366QzturHI13Jq95ApcVaJBhpS+AY=
+github.com/kr/text v0.2.0/go.mod h1:eLer722TekiGuMkidMxC/pM04lWEeraHUUmBw8l2grE=
 github.com/kylelemons/godebug v1.1.0 h1:RPNrshWIDI6G2gRW9EHilWtl7Z6Sb1BR0xunSBf0SNc=
 github.com/kylelemons/godebug v1.1.0/go.mod h1:9/0rRGxNHcop5bhtWyNeEfOS8JIWk580+fNqagV/RAw=
+github.com/lucasb-eyer/go-colorful v1.2.0 h1:1nnpGOrhyZZuNyfu1QjKiUICQ74+3FNCN69Aj6K7nkY=
+github.com/lucasb-eyer/go-colorful v1.2.0/go.mod h1:R4dSotOR9KMtayYi1e77YzuveK+i7ruzyGqttikkLy0=
+github.com/lufia/plan9stats v0.0.0-20211012122336-39d0f177ccd0 h1:6E+4a0GO5zZEnZ81pIr0yLvtUWk2if982qA3F3QD6H4=
+github.com/lufia/plan9stats v0.0.0-20211012122336-39d0f177ccd0/go.mod h1:zJYVVT2jmtg6P3p1VtQj7WsuWi/y4VnjVBn7F8KPB3I=
 github.com/magiconair/properties v1.8.7 h1:IeQXZAiQcpL9mgcAe1Nu6cX9LLw6ExEHKjN0VQdvPDY=
 github.com/magiconair/properties v1.8.7/go.mod h1:Dhd985XPs7jluiymwWYZ0G4Z61jb3vdS329zhj2hYo0=
+github.com/mattn/go-runewidth v0.0.15 h1:UNAjwbU9l54TA3KzvqLGxwWjHmMgBUVhBiTjelZgg3U=
+github.com/mattn/go-runewidth v0.0.15/go.mod h1:Jdepj2loyihRzMpdS35Xk/zdY8IAYHsh153qUoGf23w=
 github.com/mitchellh/mapstructure v1.5.0 h1:jeMsZIYE/09sWLaz43PL7Gy6RuMjD2eJVyuac5Z2hdY=
 github.com/mitchellh/mapstructure v1.5.0/go.mod h1:bFUtVrKA4DC2yAKiSyO/QUcy7e+RRV2QTWOzhPopBRo=
 github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 h1:C3w9PqII01/Oq1c1nUAm88MOHcQC9l5mIlSMApZMrHA=
@@ -43,76 +55,124 @@ github.com/onsi/gomega v1.18.1 h1:M1GfJqGRrBrrGGsbxzV5dqM2U2ApXefZCQpkukxYRLE=
 github.com/onsi/gomega v1.18.1/go.mod h1:0q+aL8jAiMXy9hbwj2mr5GziHiwhAIQpFmmtT5hitRs=
 github.com/patrickmn/go-cache v2.1.0+incompatible h1:HRMgzkcYKYpi3C8ajMPV8OFXaaRUnok+kx1WdO15EQc=
 github.com/patrickmn/go-cache v2.1.0+incompatible/go.mod h1:3Qf8kWWT7OJRJbdiICTKqZju1ZixQ/KpMGzzAfe6+WQ=
+github.com/pelletier/go-toml v1.9.5 h1:4yBQzkHv+7BHq2PQUZF3Mx0IYxG7LsP222s7Agd3ve8=
+github.com/pelletier/go-toml v1.9.5/go.mod h1:u1nR/EPcESfeI/szUZKdtJ0xRNbUoANCkoOuaOx1Y+c=
 github.com/pelletier/go-toml/v2 v2.2.2 h1:aYUidT7k73Pcl9nb2gScu7NSrKCSHIDE89b3+6Wq+LM=
 github.com/pelletier/go-toml/v2 v2.2.2/go.mod h1:1t835xjRzz80PqgE6HHgN2JOsmgYu/h4qDAS4n929Rs=
 github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
 github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
-github.com/pmezard/go-difflib v1.0.1-0.20181226105442-5d4384ee4fb2 h1:Jamvg5psRIccs7FGNTlIRMkT8wgtp5eCXdBlqhYGL6U=
+github.com/power-devops/perfstat v0.0.0-20210106213030-5aafc221ea8c h1:ncq/mPwQF4JjgDlrVEn3C11VoGHZN7m8qihwgMEtzYw=
+github.com/power-devops/perfstat v0.0.0-20210106213030-5aafc221ea8c/go.mod h1:OmDBASR4679mdNQnz2pUhc2G8CO2JrUAVFDRBDP/hJE=
 github.com/prometheus/client_golang v1.20.5 h1:cxppBPuYhUnsO6yo/aoRol4L7q7UFfdm+bR9r+8l63Y=
 github.com/prometheus/client_golang v1.20.5/go.mod h1:PIEt8X02hGcP8JWbeHyeZ53Y/jReSnHgO035n//V5WE=
 github.com/prometheus/client_model v0.6.1 h1:ZKSh/rekM+n3CeS952MLRAdFwIKqeY8b62p8ais2e9E=
 github.com/prometheus/client_model v0.6.1/go.mod h1:OrxVMOVHjw3lKMa8+x6HeMGkHMQyHDk9E3jmP2AmGiY=
-github.com/prometheus/common v0.60.1 h1:FUas6GcOw66yB/73KC+BOZoFJmbo/1pojoILArPAaSc=
-github.com/prometheus/common v0.60.1/go.mod h1:h0LYf1R1deLSKtD4Vdg8gy4RuOvENW2J/h19V5NADQw=
+github.com/prometheus/common v0.61.0 h1:3gv/GThfX0cV2lpO7gkTUwZru38mxevy90Bj8YFSRQQ=
+github.com/prometheus/common v0.61.0/go.mod h1:zr29OCN/2BsJRaFwG8QOBr41D6kkchKbpeNH7pAjb/s=
 github.com/prometheus/procfs v0.15.1 h1:YagwOFzUgYfKKHX6Dr+sHT7km/hxC76UB0learggepc=
 github.com/prometheus/procfs v0.15.1/go.mod h1:fB45yRUv8NstnjriLhBQLuOUt+WW4BsoGhij/e3PBqk=
-github.com/sagikazarmark/locafero v0.4.0 h1:HApY1R9zGo4DBgr7dqsTH/JJxLTTsOt7u6keLGt6kNQ=
-github.com/sagikazarmark/locafero v0.4.0/go.mod h1:Pe1W6UlPYUk/+wc/6KFhbORCfqzgYEpgQ3O5fPuL3H4=
-github.com/sagikazarmark/slog-shim v0.1.0 h1:diDBnUNK9N/354PgrxMywXnAwEr1QZcOr6gto+ugjYE=
-github.com/sagikazarmark/slog-shim v0.1.0/go.mod h1:SrcSrq8aKtyuqEI1uvTDTK1arOWRIczQRv+GVI1AkeQ=
+github.com/rivo/tview v0.0.0-20241103174730-c76f7879f592 h1:YIJ+B1hePP6AgynC5TcqpO0H9k3SSoZa2BGyL6vDUzM=
+github.com/rivo/tview v0.0.0-20241103174730-c76f7879f592/go.mod h1:02iFIz7K/A9jGCvrizLPvoqr4cEIx7q54RH5Qudkrss=
+github.com/rivo/uniseg v0.2.0/go.mod h1:J6wj4VEh+S6ZtnVlnTBMWIodfgj8LQOQFoIToxlJtxc=
+github.com/rivo/uniseg v0.4.3/go.mod h1:FN3SvrM+Zdj16jyLfmOkMNblXMcoc8DfTHruCPUcx88=
+github.com/rivo/uniseg v0.4.7 h1:WUdvkW8uEhrYfLC4ZzdpI2ztxP1I582+49Oc5Mq64VQ=
+github.com/rivo/uniseg v0.4.7/go.mod h1:FN3SvrM+Zdj16jyLfmOkMNblXMcoc8DfTHruCPUcx88=
+github.com/rogpeppe/go-internal v1.10.0 h1:TMyTOH3F/DB16zRVcYyreMH6GnZZrwQVAoYjRBZyWFQ=
+github.com/rogpeppe/go-internal v1.10.0/go.mod h1:UQnix2H7Ngw/k4C5ijL5+65zddjncjaFoBhdsK/akog=
 github.com/shirou/gopsutil v3.21.11+incompatible h1:+1+c1VGhc88SSonWP6foOcLhvnKlUeu/erjjvaPEYiI=
 github.com/shirou/gopsutil v3.21.11+incompatible/go.mod h1:5b4v6he4MtMOwMlS0TUMTu2PcXUg8+E1lC7eC3UO/RA=
+github.com/shirou/gopsutil/v3 v3.24.5 h1:i0t8kL+kQTvpAYToeuiVk3TgDeKOFioZO3Ztz/iZ9pI=
+github.com/shirou/gopsutil/v3 v3.24.5/go.mod h1:bsoOS1aStSs9ErQ1WWfxllSeS1K5D+U30r2NfcubMVk=
+github.com/shoenig/go-m1cpu v0.1.6 h1:nxdKQNcEB6vzgA2E2bvzKIYRuNj7XNJ4S/aRSwKzFtM=
+github.com/shoenig/go-m1cpu v0.1.6/go.mod h1:1JJMcUBvfNwpq05QDQVAnx3gUHr9IYF7GNg9SUEw2VQ=
+github.com/shoenig/test v0.6.4 h1:kVTaSd7WLz5WZ2IaoM0RSzRsUD+m8wRR+5qvntpn4LU=
+github.com/shoenig/test v0.6.4/go.mod h1:byHiCGXqrVaflBLAMq/srcZIHynQPQgeyvkvXnjqq0k=
 github.com/sirupsen/logrus v1.9.3 h1:dueUQJ1C2q9oE3F7wvmSGAaVtTmUizReu6fjN8uqzbQ=
 github.com/sirupsen/logrus v1.9.3/go.mod h1:naHLuLoDiP4jHNo9R0sCBMtWGeIprob74mVsIT4qYEQ=
-github.com/sourcegraph/conc v0.3.0 h1:OQTbbt6P72L20UqAkXXuLOj79LfEanQ+YQFNpLA9ySo=
-github.com/sourcegraph/conc v0.3.0/go.mod h1:Sdozi7LEKbFPqYX2/J+iBAM6HpqSLTASQIKqDmF7Mt0=
 github.com/spf13/afero v1.11.0 h1:WJQKhtpdm3v2IzqG8VMqrr6Rf3UYpEF239Jy9wNepM8=
 github.com/spf13/afero v1.11.0/go.mod h1:GH9Y3pIexgf1MTIWtNGyogA5MwRIDXGUr+hbWNoBjkY=
 github.com/spf13/cast v1.6.0 h1:GEiTHELF+vaR5dhz3VqZfFSzZjYbgeKDpBxQVS4GYJ0=
 github.com/spf13/cast v1.6.0/go.mod h1:ancEpBxwJDODSW/UG4rDrAqiKolqNNh2DX3mk86cAdo=
+github.com/spf13/jwalterweatherman v1.1.0 h1:ue6voC5bR5F8YxI5S67j9i582FU4Qvo2bmqnqMYADFk=
+github.com/spf13/jwalterweatherman v1.1.0/go.mod h1:aNWZUN0dPAAO/Ljvb5BEdw96iTZ0EXowPYD95IqWIGo=
 github.com/spf13/pflag v1.0.5 h1:iy+VFUOCP1a+8yFto/drg2CJ5u0yRoB7fZw3DKv/JXA=
 github.com/spf13/pflag v1.0.5/go.mod h1:McXfInJRrz4CZXVZOBLb0bTZqETkiAhM9Iw0y3An2Bg=
-github.com/spf13/viper v1.19.0 h1:RWq5SEjt8o25SROyN3z2OrDB9l7RPd3lwTWU8EcEdcI=
-github.com/spf13/viper v1.19.0/go.mod h1:GQUN9bilAbhU/jgc1bKs99f/suXKeUMct8Adx5+Ntkg=
+github.com/spf13/viper v1.11.0 h1:7OX/1FS6n7jHD1zGrZTM7WtY13ZELRyosK4k93oPr44=
+github.com/spf13/viper v1.11.0/go.mod h1:djo0X/bA5+tYVoCn+C7cAYJGcVn/qYLFTG8gdUsX7Zk=
 github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
 github.com/stretchr/objx v0.4.0/go.mod h1:YvHI0jy2hoMjB+UWwv71VJQ9isScKT/TqJzVSSt89Yw=
 github.com/stretchr/objx v0.5.0/go.mod h1:Yh+to48EsGEfYuaHDzXPcE3xhTkx73EhmCGUpEOglKo=
 github.com/stretchr/objx v0.5.2/go.mod h1:FRsXN1f5AsAjCGJKqEizvkpNtU+EGNCLh3NxZ/8L+MA=
-github.com/stretchr/testify v1.3.0/go.mod h1:M5WIy9Dh21IEIfnGCwXGc5bZfKNJtfHm1UVUgZn+9EI=
+github.com/stretchr/testify v1.2.2/go.mod h1:a8OnRcib4nhh0OaRAV+Yts87kKdq0PP7pXfy6kDkUVs=
 github.com/stretchr/testify v1.7.0/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
 github.com/stretchr/testify v1.7.1/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
 github.com/stretchr/testify v1.8.0/go.mod h1:yNjHg4UonilssWZ8iaSj1OCr/vHnekPRkoO+kdMU+MU=
 github.com/stretchr/testify v1.8.4/go.mod h1:sz/lmYIOXD/1dqDmKjjqLyZ2RngseejIcXlSw2iwfAo=
-github.com/stretchr/testify v1.9.0 h1:HtqpIVDClZ4nwg75+f6Lvsy/wHu+3BoSGCbBAcpTsTg=
 github.com/stretchr/testify v1.9.0/go.mod h1:r2ic/lqez/lEtzL7wO/rwa5dbSLXVDPFyf8C91i36aY=
+github.com/stretchr/testify v1.10.0 h1:Xv5erBjTwe/5IxqUQTdXv5kgmIvbHo3QQyRwhJsOfJA=
+github.com/stretchr/testify v1.10.0/go.mod h1:r2ic/lqez/lEtzL7wO/rwa5dbSLXVDPFyf8C91i36aY=
 github.com/subosito/gotenv v1.6.0 h1:9NlTDc1FTs4qu0DDq7AEtTPNw6SVm7uBMsUCUjABIf8=
 github.com/subosito/gotenv v1.6.0/go.mod h1:Dk4QP5c2W3ibzajGcXpNraDfq2IrhjMIvMSWPKKo0FU=
 github.com/tklauser/go-sysconf v0.3.14 h1:g5vzr9iPFFz24v2KZXs/pvpvh8/V9Fw6vQK5ZZb78yU=
 github.com/tklauser/go-sysconf v0.3.14/go.mod h1:1ym4lWMLUOhuBOPGtRcJm7tEGX4SCYNEEEtghGG/8uY=
 github.com/tklauser/numcpus v0.9.0 h1:lmyCHtANi8aRUgkckBgoDk1nHCux3n2cgkJLXdQGPDo=
 github.com/tklauser/numcpus v0.9.0/go.mod h1:SN6Nq1O3VychhC1npsWostA+oW+VOQTxZrS604NSRyI=
+github.com/yuin/goldmark v1.4.13/go.mod h1:6yULJ656Px+3vBD8DxQVa3kxgyrAnzto9xy5taEt/CY=
 github.com/yusufpapurcu/wmi v1.2.4 h1:zFUKzehAFReQwLys1b/iSMl+JQGSCSjtVqQn9bBrPo0=
 github.com/yusufpapurcu/wmi v1.2.4/go.mod h1:SBZ9tNy3G9/m5Oi98Zks0QjeHVDvuK0qfxQmPyzfmi0=
-go.uber.org/atomic v1.9.0 h1:ECmE8Bn/WFTYwEW/bpKD3M8VtR/zQVbavAoalC1PYyE=
-go.uber.org/atomic v1.9.0/go.mod h1:fEN4uk6kAWBTFdckzkM89CLk9XfWZrxpCo0nPH17wJc=
-go.uber.org/multierr v1.9.0 h1:7fIwc/ZtS0q++VgcfqFDxSBZVv/Xo49/SYnDFupUwlI=
-go.uber.org/multierr v1.9.0/go.mod h1:X2jQV1h+kxSjClGpnseKVIxpmcjrj7MNnI0bnlfKTVQ=
-golang.org/x/exp v0.0.0-20230905200255-921286631fa9 h1:GoHiUyI/Tp2nVkLI2mCxVkOjsbSXD66ic0XW0js0R9g=
-golang.org/x/exp v0.0.0-20230905200255-921286631fa9/go.mod h1:S2oDrQGGwySpoQPVqRShND87VCbxmc6bL1Yd2oYrm6k=
-golang.org/x/net v0.29.0 h1:5ORfpBpCs4HzDYoodCDBbwHzdR5UrLBZ3sOnUJmFoHo=
-golang.org/x/net v0.29.0/go.mod h1:gLkgy8jTGERgjzMic6DS9+SP0ajcu6Xu3Orq/SpETg0=
+golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w=
+golang.org/x/crypto v0.0.0-20210921155107-089bfa567519/go.mod h1:GvvjBRRGRdwPK5ydBHafDWAxML/pGHZbMvKqRZ5+Abc=
+golang.org/x/mod v0.6.0-dev.0.20220419223038-86c51ed26bb4/go.mod h1:jJ57K6gSWd91VN4djpZkiMVwK6gcyfeH4XE8wZrZaV4=
+golang.org/x/mod v0.8.0/go.mod h1:iBbtSCu2XBx23ZKBPSOrRkjjQPZFPuis4dIYUhu/chs=
+golang.org/x/net v0.0.0-20190620200207-3b0461eec859/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
+golang.org/x/net v0.0.0-20210226172049-e18ecbb05110/go.mod h1:m0MpNAwzfU5UDzcl9v0D8zg8gWTRqZa9RBIspLL5mdg=
+golang.org/x/net v0.0.0-20220722155237-a158d28d115b/go.mod h1:XRhObCWvk6IyKnWLug+ECip1KBveYUHfp+8e9klMJ9c=
+golang.org/x/net v0.6.0/go.mod h1:2Tu9+aMcznHK/AK1HMvgo6xiTLG5rD5rZLDS+rp2Bjs=
+golang.org/x/net v0.32.0 h1:ZqPmj8Kzc+Y6e0+skZsuACbx+wzMgo5MQsJh9Qd6aYI=
+golang.org/x/net v0.32.0/go.mod h1:CwU0IoeOlnQQWJ6ioyFrfRuomB8GKF6KbYXZVyeXNfs=
+golang.org/x/sync v0.0.0-20190423024810-112230192c58/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
+golang.org/x/sync v0.0.0-20220722155255-886fb9371eb4/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
+golang.org/x/sync v0.1.0/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
+golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
 golang.org/x/sys v0.0.0-20190916202348-b4ddaad3f8a3/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
+golang.org/x/sys v0.0.0-20201119102817-f84b799fce68/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
+golang.org/x/sys v0.0.0-20201204225414-ed752295db88/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
+golang.org/x/sys v0.0.0-20210615035016-665e8c7367d1/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
+golang.org/x/sys v0.0.0-20220520151302-bc2c85ada10a/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
 golang.org/x/sys v0.0.0-20220715151400-c0bba94af5f8/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
+golang.org/x/sys v0.0.0-20220722155257-8c9f86f7a55f/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
 golang.org/x/sys v0.1.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
-golang.org/x/sys v0.26.0 h1:KHjCJyddX0LoSTb3J+vWpupP9p0oznkqVk/IfjymZbo=
-golang.org/x/sys v0.26.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA=
-golang.org/x/text v0.18.0 h1:XvMDiNzPAl0jr17s6W9lcaIhGUfUORdGCNsuLmPG224=
-golang.org/x/text v0.18.0/go.mod h1:BuEKDfySbSR4drPmRPG/7iBdf8hvFMuRexcpahXilzY=
-google.golang.org/protobuf v1.35.1 h1:m3LfL6/Ca+fqnjnlqQXNpFPABW1UD7mjh8KO2mKFytA=
-google.golang.org/protobuf v1.35.1/go.mod h1:9fA7Ob0pmnwhb644+1+CVWFRbNajQ6iRojtC/QF5bRE=
+golang.org/x/sys v0.5.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
+golang.org/x/sys v0.17.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA=
+golang.org/x/sys v0.28.0 h1:Fksou7UEQUWlKvIdsqzJmUmCX3cZuD2+P3XyyzwMhlA=
+golang.org/x/sys v0.28.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA=
+golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo=
+golang.org/x/term v0.0.0-20210927222741-03fcf44c2211/go.mod h1:jbD1KX2456YbFQfuXm/mYQcufACuNUgVhRMnK/tPxf8=
+golang.org/x/term v0.5.0/go.mod h1:jMB1sMXY+tzblOD4FWmEbocvup2/aLOaQEp7JmGp78k=
+golang.org/x/term v0.17.0 h1:mkTF7LCd6WGJNL3K1Ad7kwxNfYAW6a8a8QqtMblp/4U=
+golang.org/x/term v0.17.0/go.mod h1:lLRBjIVuehSbZlaOtGMbcMncT+aqLLLmKrsjNrUguwk=
+golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
+golang.org/x/text v0.3.3/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
+golang.org/x/text v0.3.7/go.mod h1:u+2+/6zg+i71rQMx5EYifcz6MCKuco9NR6JIITiCfzQ=
+golang.org/x/text v0.7.0/go.mod h1:mrYo+phRRbMaCq/xk9113O4dZlRixOauAjOtrjsXDZ8=
+golang.org/x/text v0.14.0/go.mod h1:18ZOQIKpY8NJVqYksKHtTdi31H5itFRjB5/qKTNYzSU=
+golang.org/x/text v0.21.0 h1:zyQAAkrwaneQ066sspRyJaG9VNi/YJ1NfzcGB3hZ/qo=
+golang.org/x/text v0.21.0/go.mod h1:4IBbMaMmOPCJ8SecivzSH54+73PCFmPWxNTLm+vZkEQ=
+golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
+golang.org/x/tools v0.0.0-20191119224855-298f0cb1881e/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=
+golang.org/x/tools v0.1.12/go.mod h1:hNGJHUnrk76NpqgfD5Aqm5Crs+Hm0VOH/i9J2+nxYbc=
+golang.org/x/tools v0.6.0/go.mod h1:Xwgl3UAJ/d3gWutnCtw505GrjyAbvKui8lOU390QaIU=
+golang.org/x/xerrors v0.0.0-20190717185122-a985d3407aa7/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
+golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
+google.golang.org/protobuf v1.35.2 h1:8Ar7bF+apOIoThw1EdZl0p1oWvMqTHmpA2fRTyZO8io=
+google.golang.org/protobuf v1.35.2/go.mod h1:9fA7Ob0pmnwhb644+1+CVWFRbNajQ6iRojtC/QF5bRE=
 gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
+gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c h1:Hei/4ADfdWqJk1ZMxUNpqntNwaWcugrBjAiHlqqRiVk=
+gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c/go.mod h1:JHkPIbrfpd72SG/EVd6muEfDQjcINNoR0C8j2r3qZ4Q=
 gopkg.in/ini.v1 v1.67.0 h1:Dgnx+6+nfE+IfzjUEISNeydPJh9AXNNsWbGP9KzCsOA=
 gopkg.in/ini.v1 v1.67.0/go.mod h1:pNLf8WUiyNEtQjuu5G5vTm06TEv9tsIgeAvK8hOrP4k=
+gopkg.in/natefinch/lumberjack.v2 v2.2.1 h1:bBRl1b0OH9s/DuPhuXpNl+VtCaJXFZ5/uEFST95x9zc=
+gopkg.in/natefinch/lumberjack.v2 v2.2.1/go.mod h1:YD8tP3GAjkrDg1eZH7EGmyESg/lsYskCTPBJVb9jqSc=
 gopkg.in/tomb.v1 v1.0.0-20141024135613-dd632973f1e7 h1:uRGJdciOHaEIrze2W8Q3AKkepLTh2hOroT7a+7czfdQ=
 gopkg.in/tomb.v1 v1.0.0-20141024135613-dd632973f1e7/go.mod h1:dt/ZhP58zS4L8KSrWDmTeBkI65Dw0HsyUHuEVlX15mw=
 gopkg.in/yaml.v2 v2.4.0 h1:D8xgwECY7CYvx+Y2n4sBz93Jn9JRvxdiyyo8CTfuKaY=

lib/maps/iter.go

@@ -0,0 +1,69 @@
+package maps
+
+// Copyright 2024 The Go Authors. All rights reserved.
+// Use of this source code is governed by a BSD-style
+// license that can be found in the LICENSE file.
+
+type Seq2[K comparable, V any] func(yield func(K, V) bool)
+type Seq[K any] func(yield func(K) bool)
+
+func All[Map ~map[K]V, K comparable, V any](m Map) Seq2[K, V] {
+	return func(yield func(K, V) bool) {
+		for k, v := range m {
+			if !yield(k, v) {
+				return
+			}
+		}
+	}
+}
+
+// All returns an iterator over key-value pairs from m.
+// The iteration order is not specified and is not guaranteed
+// to be the same from one call to the next.
+
+func Insert[Map ~map[K]V, K comparable, V any](m Map, seq Seq2[K, V]) {
+	seq(func(k K, v V) bool {
+		m[k] = v
+		return true
+	})
+}
+
+// Insert adds the key-value pairs from seq to m.
+// If a key in seq already exists in m, its value will be overwritten.
+
+func Collect[K comparable, V any](seq Seq2[K, V]) map[K]V {
+	m := make(map[K]V)
+	Insert(m, seq)
+	return m
+}
+
+// Collect collects key-value pairs from seq into a new map
+// and returns it.
+
+func Keys[Map ~map[K]V, K comparable, V any](m Map) Seq[K] {
+	return func(yield func(K) bool) {
+		for k := range m {
+			if !yield(k) {
+				return
+			}
+		}
+	}
+}
+
+// Keys returns an iterator over keys in m.
+// The iteration order is not specified and is not guaranteed
+// to be the same from one call to the next.
+
+func Values[Map ~map[K]V, K comparable, V any](m Map) Seq[V] {
+	return func(yield func(V) bool) {
+		for _, v := range m {
+			if !yield(v) {
+				return
+			}
+		}
+	}
+}
+
+// Values returns an iterator over values in m.
+// The iteration order is not specified and is not guaranteed
+// to be the same from one call to the next.

test/hmac_test.go

@@ -16,7 +16,7 @@ import (
 
 const (
 	serverURL    = "http://[::1]:8080"				// Replace with your actual server URL
-	secret       = "a-orc-and-a-humans-is-drinking-ale"             // Replace with your HMAC secret key
+	secret       = "hmac-file-server-is-the-win"             // Replace with your HMAC secret key
 	uploadPath   = "hmac_icon.png"                 			// Test file to upload
 	protocolType = "v2"						// Use v2, v, or token as needed
 )