2.5-stable release
This commit is contained in:
parent
74fa394a30
commit
881237a904
294
README.MD
294
README.MD
@ -1,35 +1,287 @@
|
||||
# HMAC File Server
|
||||
A secure file server with HMAC authentication and configurable features.
|
||||
# HMAC File Server 2.5-Stable
|
||||
|
||||
// ...existing code...
|
||||
## Overview
|
||||
The **HMAC File Server** ensures secure file uploads and downloads using HMAC authentication. It incorporates rate limiting, CORS support, retries, file versioning, and Unix socket support for enhanced flexibility. Redis integration provides efficient caching and session management. Prometheus metrics and a graceful shutdown mechanism ensure reliable and efficient file handling.
|
||||
|
||||
Special thanks to **Thomas Leister** for inspiration drawn from [[prosody-filer](https://github.com/ThomasLeister/prosody-filer)](https://github.com/ThomasLeister/prosody-filer).
|
||||
|
||||
## Features
|
||||
- File deduplication
|
||||
- Configurable TTL for automatic file cleanup
|
||||
- Secure HMAC-based authentication
|
||||
- Chunked uploads and downloads
|
||||
- Virus scanning via ClamAV
|
||||
- Prometheus metrics integration
|
||||
- Customizable worker management
|
||||
- Support for thumbnails and ISO-based storage
|
||||
|
||||
- **HMAC Authentication:** Secure file uploads and downloads with HMAC tokens.
|
||||
- **File Versioning:** Enable versioning for uploaded files with configurable retention.
|
||||
- **Chunked and Resumable Uploads:** Handle large files efficiently with support for resumable and chunked uploads.
|
||||
- **Deduplication:** Automatically remove duplicate files based on hashing for storage efficiency.
|
||||
- **ClamAV Scanning:** Optional virus scanning for uploaded files.
|
||||
- **Prometheus Metrics:** Monitor system and application-level metrics.
|
||||
- **Redis Integration:** Use Redis for caching or storing application states.
|
||||
- **File Expiration:** Automatically delete files after a specified TTL.
|
||||
- **Graceful Shutdown:** Handles signals and ensures proper cleanup.
|
||||
- **Auto-Adjust Worker Scaling:** Dynamically optimize worker threads based on system resources when enabled.
|
||||
## Table of Contents
|
||||
1. [Installation](#installation)
|
||||
2. [Configuration](#configuration)
|
||||
3. [Usage](#usage)
|
||||
4. [Setup](#setup)
|
||||
- [Reverse Proxy](#reverse-proxy)
|
||||
- [Systemd Service](#systemd-service)
|
||||
5. [Building](#building)
|
||||
6. [Changelog](#changelog)
|
||||
7. [License](#license)
|
||||
|
||||
// ...existing code...
|
||||
---
|
||||
|
||||
## Installation
|
||||
|
||||
### Prerequisites
|
||||
- Go 1.18 or higher
|
||||
- Redis server (optional, for caching)
|
||||
- ClamAV (optional, for virus scanning)
|
||||
|
||||
### Steps
|
||||
1. Clone the repository:
|
||||
```bash
|
||||
git clone https://github.com/your-repo/hmac-file-server.git
|
||||
cd hmac-file-server
|
||||
```
|
||||
|
||||
2. Build the server:
|
||||
```bash
|
||||
go build -o hmac-file-server
|
||||
```
|
||||
|
||||
3. Create necessary directories:
|
||||
```bash
|
||||
mkdir -p /path/to/hmac-file-server/data/
|
||||
mkdir -p /path/to/hmac-file-server/deduplication/
|
||||
mkdir -p /path/to/hmac-file-server/thumbnails/
|
||||
mkdir -p /path/to/hmac-file-server/iso/
|
||||
```
|
||||
|
||||
4. Copy and edit the configuration file:
|
||||
```bash
|
||||
cp config.example.toml config.toml
|
||||
```
|
||||
|
||||
5. Start the server:
|
||||
```bash
|
||||
./hmac-file-server -config config.toml
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Configuration
|
||||
The server is configured via a `config.toml` file. Key settings include:
|
||||
|
||||
// ...existing code...
|
||||
- **Server Settings**: Port, logging, metrics
|
||||
- **Security**: HMAC secret, TLS options
|
||||
- **File Management**: TTL, deduplication, uploads, and downloads
|
||||
- **Thumbnails and ISO**: Generation and mounting settings
|
||||
- **Workers**: Adjust thread management
|
||||
|
||||
// Removed Thumbnail Creation section
|
||||
For detailed configuration options, refer to the [Wiki](./wiki.md).
|
||||
|
||||
// ...existing code...
|
||||
---
|
||||
|
||||
## Additional Features
|
||||
## Usage
|
||||
Start the server and access it on the configured port. Use curl or a client library to interact with the API.
|
||||
|
||||
// ...existing features...
|
||||
### Example
|
||||
Upload a file:
|
||||
```bash
|
||||
curl -X POST -F 'file=@example.jpg' http://localhost:8080/upload
|
||||
```
|
||||
|
||||
// Removed Thumbnail Creation details
|
||||
---
|
||||
|
||||
// ...existing code...
|
||||
## Setup
|
||||
|
||||
### Reverse Proxy
|
||||
Set up a reverse proxy using Apache2 or Nginx to handle requests.
|
||||
|
||||
#### Apache2 Example
|
||||
```apache
|
||||
<VirtualHost *:80>
|
||||
ServerName your-domain.com
|
||||
|
||||
ProxyPreserveHost On
|
||||
ProxyPass / http://localhost:8080/
|
||||
ProxyPassReverse / http://localhost:8080/
|
||||
</VirtualHost>
|
||||
```
|
||||
|
||||
#### Nginx Example
|
||||
```nginx
|
||||
server {
|
||||
listen 80;
|
||||
server_name your-domain.com;
|
||||
|
||||
location / {
|
||||
proxy_pass http://localhost:8080;
|
||||
proxy_set_header Host $host;
|
||||
proxy_set_header X-Real-IP $remote_addr;
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Systemd Service
|
||||
Create a systemd service file for the HMAC File Server:
|
||||
```ini
|
||||
[Unit]
|
||||
Description=HMAC File Server
|
||||
After=network.target
|
||||
|
||||
[Service]
|
||||
ExecStart=/path/to/hmac-file-server -config /path/to/config.toml
|
||||
WorkingDirectory=/path/to/hmac-file-server
|
||||
Restart=always
|
||||
User=www-data
|
||||
Group=www-data
|
||||
|
||||
[Install]
|
||||
WantedBy=multi-user.target
|
||||
```
|
||||
Enable and start the service:
|
||||
```bash
|
||||
sudo systemctl daemon-reload
|
||||
sudo systemctl enable hmac-file-server
|
||||
sudo systemctl start hmac-file-server
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Building
|
||||
To build for different architectures:
|
||||
|
||||
- **Linux (x86_64)**:
|
||||
```bash
|
||||
GOOS=linux GOARCH=amd64 go build -o hmac-file-server-linux-amd64
|
||||
```
|
||||
|
||||
- **macOS (x86_64)**:
|
||||
```bash
|
||||
GOOS=darwin GOARCH=amd64 go build -o hmac-file-server-darwin-amd64
|
||||
```
|
||||
|
||||
- **Windows (x86_64)**:
|
||||
```bash
|
||||
GOOS=windows GOARCH=amd64 go build -o hmac-file-server-windows-amd64.exe
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Changelog
|
||||
|
||||
### Added
|
||||
- **Deduplication Support:** Automatically remove duplicate files based on SHA256 hashing to save storage space.
|
||||
- **ISO Container Management:** Create and mount ISO containers for specialized storage needs, enhancing flexibility in file management.
|
||||
- **Prometheus Metrics Enhancements:** Added detailed metrics for deduplication and ISO container operations to improve monitoring and observability.
|
||||
- **Redis Integration Improvements:** Enhanced caching mechanisms using Redis for faster access to file metadata and application states.
|
||||
- **Precaching Feature:** Implemented precaching of file structures on startup to reduce access times for frequently used files.
|
||||
- **Configuration Options:** Updated `config.toml` to include new settings for deduplication, thumbnails, ISO management, and worker scaling.
|
||||
|
||||
### Changed
|
||||
- **Worker Pool Scaling:** Implemented dynamic adjustment of worker threads based on system resources to optimize performance.
|
||||
- **Logging Enhancements:** Improved logging for file operations, including detailed information on file extensions and MIME types during uploads.
|
||||
- **Temporary Path Configuration:** Replaced hardcoded temporary upload directories with a configurable `TempPath` parameter in `config.toml` for greater flexibility.
|
||||
|
||||
### Fixed
|
||||
- **Temporary File Handling:** Resolved issues where temporary `.tmp` files caused "Unsupported file type" warnings by enhancing MIME type detection logic.
|
||||
- **MIME Type Detection:** Improved MIME type detection to ensure better compatibility and accuracy during file uploads.
|
||||
|
||||
### Deprecated
|
||||
- **Thumbnail Support (Previous Implementation):** Dropped the previous thumbnail support mechanism. This feature will not return in future releases.
|
||||
|
||||
### Removed
|
||||
- **Old Thumbnail Generation Method:** Removed the outdated thumbnail generation process to streamline the codebase and reduce dependencies.
|
||||
|
||||
---
|
||||
|
||||
**Note:** Ensure to update your `config.toml` with the new configuration options to take full advantage of the added features. Refer to the [Configuration](./config.toml) section in the documentation for detailed settings.
|
||||
|
||||
---
|
||||
|
||||
### Important:
|
||||
- **GitHub Release:** Create a corresponding GitHub release tagged `v2.6.0` with these updated release notes to inform users of the new features and changes.
|
||||
- **Migration Steps:** If upgrading from a previous version, review the configuration changes and deprecated features to adjust your setup accordingly.
|
||||
|
||||
```toml
|
||||
[server]
|
||||
listenport = "8080"
|
||||
unixsocket = false
|
||||
storagepath = "./uploads"
|
||||
metricsenabled = true
|
||||
metricsport = "9090"
|
||||
filettl = "8760h"
|
||||
minfreebytes = "100MB"
|
||||
autoadjustworkers = true
|
||||
networkevents = false
|
||||
temppath = "/tmp"
|
||||
loggingjson = false
|
||||
pidfilepath = "/var/run/hmacfileserver.pid"
|
||||
cleanuponexit = true
|
||||
precaching = true
|
||||
filettlenabled = true
|
||||
deduplicationenabled = true
|
||||
globalextensions = [".txt", ".pdf", ".png", ".jpg"]
|
||||
|
||||
[logging]
|
||||
level = "info"
|
||||
file = "/var/log/hmac-file-server.log"
|
||||
max_size = 100
|
||||
max_backups = 7
|
||||
max_age = 30
|
||||
compress = true
|
||||
|
||||
[deduplication]
|
||||
enabled = true
|
||||
directory = "/deduplication"
|
||||
|
||||
[iso]
|
||||
enabled = true
|
||||
size = "1GB"
|
||||
mountpoint = "/mnt/iso"
|
||||
charset = "utf-8"
|
||||
containerfile = "/mnt/iso/container.iso"
|
||||
|
||||
[timeouts]
|
||||
readtimeout = "4800s"
|
||||
writetimeout = "4800s"
|
||||
idletimeout = "4800s"
|
||||
|
||||
[security]
|
||||
secret = "changeme"
|
||||
|
||||
[versioning]
|
||||
enableversioning = false
|
||||
maxversions = 1
|
||||
|
||||
[uploads]
|
||||
resumableuploadsenabled = true
|
||||
chunkeduploadsenabled = true
|
||||
chunksize = "64MB"
|
||||
allowedextensions = [".txt", ".pdf", ".png", ".jpg"]
|
||||
|
||||
[downloads]
|
||||
resumabledownloadsenabled = true
|
||||
chunkeddownloadsenabled = true
|
||||
chunksize = "64MB"
|
||||
allowedextensions = [".txt", ".pdf", ".png", ".jpg"]
|
||||
|
||||
[clamav]
|
||||
clamavenabled = true
|
||||
clamavsocket = "/var/run/clamav/clamd.ctl"
|
||||
numscanworkers = 2
|
||||
scanfileextensions = [".txt", ".pdf", ".png", ".jpg"]
|
||||
|
||||
[redis]
|
||||
redisenabled = true
|
||||
redisdbindex = 0
|
||||
redisaddr = "localhost:6379"
|
||||
redispassword = ""
|
||||
redishealthcheckinterval = "120s"
|
||||
|
||||
[workers]
|
||||
numworkers = 4
|
||||
uploadqueuesize = 50
|
||||
|
||||
[build]
|
||||
version = "2.5"
|
||||
```
|
||||
|
||||
@ -1,6 +1,38 @@
|
||||
## [2.6.0] - 2025-05-01
|
||||
# Release Notes for HMAC File Server 2.5-Stable
|
||||
|
||||
### Removed
|
||||
- **Thumbnail Support:** The thumbnail generation feature has been removed from the HMAC File Server. This includes the elimination of all related configuration options and dependencies to streamline server operations.
|
||||
## Summary
|
||||
Version 2.5-Stable focuses on improving the overall stability and performance of the HMAC File Server. Significant changes have been made to prioritize reliability and scalability for production environments.
|
||||
|
||||
## Key Changes
|
||||
|
||||
### Breaking Changes
|
||||
- **Thumbnail Generation Dropped**: Support for automatic thumbnail generation has been removed in this release. This decision was made to enhance system stability and reduce resource consumption. Users requiring thumbnails are encouraged to use external tools.
|
||||
|
||||
### New Features
|
||||
- **ISO-Based Storage Support**: Introduced support for ISO-based storage to accommodate specialized use cases.
|
||||
- **Enhanced ClamAV Integration**: Improved ClamAV scanning with concurrent workers, providing better performance for large-scale deployments.
|
||||
- **Timeout Configuration**: Added granular timeout settings for read, write, and idle connections, improving connection management.
|
||||
|
||||
### Improvements
|
||||
- **Worker Management**: Auto-scaling worker threads based on system load for optimal performance.
|
||||
- **Logging Enhancements**: Improved log verbosity control, making debugging and monitoring easier.
|
||||
|
||||
### Bug Fixes
|
||||
- Resolved minor issues affecting deduplication and file upload performance.
|
||||
- Fixed a rare crash scenario during high-concurrency file uploads.
|
||||
|
||||
## Migration Notes
|
||||
1. **Thumbnail Settings**: Remove `[thumbnails]` configuration blocks from your `config.toml` file to avoid errors.
|
||||
2. **Updated Configuration**: Review new timeout settings in `[timeouts]` and adjust as needed.
|
||||
3. **ISO Integration**: Configure the new `[iso]` block for environments utilizing ISO-based storage.
|
||||
|
||||
## Recommendations
|
||||
- **Security**: Ensure that the HMAC secret key in `config.toml` is updated to a strong, unique value.
|
||||
- **Backups**: Regularly back up your `config.toml` and important data directories.
|
||||
- **Monitoring**: Leverage Prometheus metrics for real-time monitoring of server performance.
|
||||
|
||||
For a detailed guide on setting up and configuring the HMAC File Server, refer to the [README.md](./README.md).
|
||||
|
||||
---
|
||||
|
||||
Thank you for using HMAC File Server! If you encounter any issues, feel free to report them on our GitHub repository.
|
||||
|
||||
1
WiKi.md
1
WiKi.md
@ -889,3 +889,4 @@ GOOS=linux GOARCH=arm64 go build -o hmac-file-server-linux-arm64
|
||||
|
||||
- The HMAC File Server is designed to be flexible and configurable. Adjust the settings in the `config.toml` file to match your specific requirements and environment.
|
||||
- For any issues or questions, refer to the project's GitHub repository and documentation.
|
||||
````
|
||||
|
||||
@ -131,6 +131,7 @@ type ServerConfig struct {
|
||||
DeduplicationEnabled bool `mapstructure:"deduplicationenabled"`
|
||||
Logging LoggingConfig `mapstructure:"logging"`
|
||||
GlobalExtensions []string `mapstructure:"globalextensions"`
|
||||
BindIP string `mapstructure:"bind_ip"` // Hinzugefügt: bind_ip
|
||||
}
|
||||
|
||||
type DeduplicationConfig struct {
|
||||
@ -196,7 +197,6 @@ type WorkersConfig struct {
|
||||
}
|
||||
|
||||
type FileConfig struct {
|
||||
FileRevision int `mapstructure:"filerevision"`
|
||||
}
|
||||
|
||||
type BuildConfig struct {
|
||||
@ -449,6 +449,7 @@ func main() {
|
||||
log.Infof("Server PreCaching: %v", conf.Server.PreCaching)
|
||||
log.Infof("Server FileTTLEnabled: %v", conf.Server.FileTTLEnabled)
|
||||
log.Infof("Server DeduplicationEnabled: %v", conf.Server.DeduplicationEnabled)
|
||||
log.Infof("Server BindIP: %s", conf.Server.BindIP) // Hinzugefügt: Logging für BindIP
|
||||
|
||||
err = writePIDFile(conf.Server.PIDFilePath) // Write PID file after config is loaded
|
||||
if err != nil {
|
||||
@ -569,7 +570,7 @@ func main() {
|
||||
}
|
||||
|
||||
server := &http.Server{
|
||||
Addr: ":" + conf.Server.ListenPort,
|
||||
Addr: conf.Server.BindIP + ":" + conf.Server.ListenPort, // Geändert: Nutzung von BindIP
|
||||
Handler: router,
|
||||
ReadTimeout: readTimeout,
|
||||
WriteTimeout: writeTimeout,
|
||||
@ -595,7 +596,7 @@ func main() {
|
||||
go monitorWorkerPerformance(ctx, &conf.Server, &conf.Workers, &conf.ClamAV)
|
||||
}
|
||||
|
||||
versionString = "2.5" // Updated from "2.4"
|
||||
versionString = conf.Build.Version // Updated to use Build.Version from config
|
||||
log.Infof("Running version: %s", versionString)
|
||||
|
||||
log.Infof("Starting HMAC file server %s...", versionString)
|
||||
@ -932,9 +933,6 @@ func validateConfig(conf *Config) error {
|
||||
}
|
||||
|
||||
// Verify File Configuration
|
||||
if conf.File.FileRevision < 0 {
|
||||
return fmt.Errorf("file.filerevision cannot be negative")
|
||||
}
|
||||
|
||||
// Validate Global Extensions
|
||||
if len(conf.Server.GlobalExtensions) == 0 {
|
||||
@ -1338,7 +1336,7 @@ func processUpload(task UploadTask) error {
|
||||
exists, size := fileExists(absFilename)
|
||||
log.Debugf("Exists? %v, Size: %d", exists, size)
|
||||
|
||||
// Gajim and Dino do not require a callback or acknowledgement beyond HTTP success.
|
||||
// Gajim und Dino benötigen keinen Callback oder eine Bestätigung über HTTP-Erfolg hinaus.
|
||||
callbackURL := r.Header.Get("Callback-URL")
|
||||
if callbackURL != "" {
|
||||
log.Warnf("Callback-URL provided (%s) but not needed. Ignoring.", callbackURL)
|
||||
|
||||
23
config.toml
23
config.toml
@ -15,7 +15,7 @@ cleanuponexit = true
|
||||
precaching = true
|
||||
filettlenabled = true
|
||||
globalextensions = ["*"] # Allows all file types globally
|
||||
bind_ip = "0.0.0.0" # New option: Specify the IP address to bind to (IPv4 or IPv6)
|
||||
bind_ip = "0.0.0.0" # Specify the IP address to bind to (IPv4 or IPv6)
|
||||
|
||||
[logging]
|
||||
level = "info"
|
||||
@ -45,29 +45,29 @@ idletimeout = "4800s"
|
||||
secret = "changeme"
|
||||
|
||||
[versioning]
|
||||
enableversioning = true
|
||||
enableversioning = false
|
||||
maxversions = 5
|
||||
|
||||
[uploads]
|
||||
resumableuploadsenabled = true
|
||||
resumableuploadsenabled = false
|
||||
chunkeduploadsenabled = true
|
||||
chunksize = "8192"
|
||||
chunksize = "64mb"
|
||||
allowedextensions = ["*"] # Use ["*"] to allow all or specify extensions
|
||||
|
||||
[downloads]
|
||||
resumabledownloadsenabled = true
|
||||
resumabledownloadsenabled = false
|
||||
chunkeddownloadsenabled = true
|
||||
chunksize = "8192"
|
||||
chunksize = "64mb"
|
||||
allowedextensions = [".jpg", ".png"] # Restricts downloads to specific types
|
||||
|
||||
[clamav]
|
||||
clamavenabled = true
|
||||
clamavenabled = false
|
||||
clamavsocket = "/var/run/clamav/clamd.ctl"
|
||||
numscanworkers = 2
|
||||
scanfileextensions = [".exe", ".dll", ".pdf"]
|
||||
|
||||
[redis]
|
||||
redisenabled = true
|
||||
redisenabled = false
|
||||
redisaddr = "localhost:6379"
|
||||
redispassword = ""
|
||||
redisdbindex = 0
|
||||
@ -75,10 +75,7 @@ redishealthcheckinterval = "120s"
|
||||
|
||||
[workers]
|
||||
numworkers = 4
|
||||
uploadqueuesize = 50
|
||||
|
||||
[file]
|
||||
filerevision = 1
|
||||
uploadqueuesize = 5000
|
||||
|
||||
[build]
|
||||
version = "v2.5"
|
||||
version = "v2.5"
|
||||
89
config.toml.example
Normal file
89
config.toml.example
Normal file
@ -0,0 +1,89 @@
|
||||
### Issue
|
||||
In `config.toml`, comments are using `//`, which is invalid in TOML. TOML uses `#` for comments. This causes a syntax error.
|
||||
|
||||
### Corrected `config.toml`
|
||||
|
||||
```toml
|
||||
# filepath: /home/renz/source/hmac-file-server/config.toml
|
||||
|
||||
[server]
|
||||
listenport = "8080"
|
||||
unixsocket = false
|
||||
storagepath = "./uploads"
|
||||
metricsenabled = true
|
||||
metricsport = "9090"
|
||||
filettl = "8760h"
|
||||
minfreebytes = "100MB"
|
||||
autoadjustworkers = true
|
||||
networkevents = true
|
||||
temppath = "/tmp/hmac-file-server"
|
||||
loggingjson = false
|
||||
pidfilepath = "/var/run/hmacfileserver.pid"
|
||||
cleanuponexit = true
|
||||
precaching = true
|
||||
filettlenabled = true
|
||||
globalextensions = ["*"] # Allows all file types globally
|
||||
bind_ip = "0.0.0.0" # Specify the IP address to bind to (IPv4 or IPv6)
|
||||
|
||||
[logging]
|
||||
level = "info"
|
||||
file = "/var/log/hmac-file-server.log"
|
||||
max_size = 100
|
||||
max_backups = 7
|
||||
max_age = 30
|
||||
compress = true
|
||||
|
||||
[deduplication]
|
||||
enabled = true
|
||||
directory = "./deduplication"
|
||||
|
||||
[iso]
|
||||
enabled = true
|
||||
size = "1GB"
|
||||
mountpoint = "/mnt/iso"
|
||||
charset = "utf-8"
|
||||
containerfile = "/path/to/iso/container.iso"
|
||||
|
||||
[timeouts]
|
||||
readtimeout = "4800s"
|
||||
writetimeout = "4800s"
|
||||
idletimeout = "4800s"
|
||||
|
||||
[security]
|
||||
secret = "changeme"
|
||||
|
||||
[versioning]
|
||||
enableversioning = false
|
||||
maxversions = 5
|
||||
|
||||
[uploads]
|
||||
resumableuploadsenabled = false
|
||||
chunkeduploadsenabled = true
|
||||
chunksize = "64mb"
|
||||
allowedextensions = ["*"] # Use ["*"] to allow all or specify extensions
|
||||
|
||||
[downloads]
|
||||
resumabledownloadsenabled = false
|
||||
chunkeddownloadsenabled = true
|
||||
chunksize = "64mb"
|
||||
allowedextensions = [".jpg", ".png"] # Restricts downloads to specific types
|
||||
|
||||
[clamav]
|
||||
clamavenabled = false
|
||||
clamavsocket = "/var/run/clamav/clamd.ctl"
|
||||
numscanworkers = 2
|
||||
scanfileextensions = [".exe", ".dll", ".pdf"]
|
||||
|
||||
[redis]
|
||||
redisenabled = false
|
||||
redisaddr = "localhost:6379"
|
||||
redispassword = ""
|
||||
redisdbindex = 0
|
||||
redishealthcheckinterval = "120s"
|
||||
|
||||
[workers]
|
||||
numworkers = 4
|
||||
uploadqueuesize = 5000
|
||||
|
||||
[build]
|
||||
version = "v2.5"
|
||||
@ -28,7 +28,7 @@
|
||||
},
|
||||
"gridPos": {
|
||||
"h": 7,
|
||||
"w": 24,
|
||||
"w": 3,
|
||||
"x": 0,
|
||||
"y": 0
|
||||
},
|
||||
@ -77,8 +77,8 @@
|
||||
"gridPos": {
|
||||
"h": 7,
|
||||
"w": 6,
|
||||
"x": 0,
|
||||
"y": 7
|
||||
"x": 3,
|
||||
"y": 0
|
||||
},
|
||||
"id": 14,
|
||||
"options": {
|
||||
@ -143,8 +143,8 @@
|
||||
"gridPos": {
|
||||
"h": 7,
|
||||
"w": 6,
|
||||
"x": 6,
|
||||
"y": 7
|
||||
"x": 9,
|
||||
"y": 0
|
||||
},
|
||||
"id": 18,
|
||||
"options": {
|
||||
@ -191,6 +191,10 @@
|
||||
{
|
||||
"color": "green",
|
||||
"value": null
|
||||
},
|
||||
{
|
||||
"color": "red",
|
||||
"value": 80
|
||||
}
|
||||
]
|
||||
}
|
||||
@ -199,9 +203,67 @@
|
||||
},
|
||||
"gridPos": {
|
||||
"h": 7,
|
||||
"w": 6,
|
||||
"x": 12,
|
||||
"y": 7
|
||||
"w": 5,
|
||||
"x": 15,
|
||||
"y": 0
|
||||
},
|
||||
"id": 10,
|
||||
"options": {
|
||||
"colorMode": "value",
|
||||
"graphMode": "none",
|
||||
"justifyMode": "auto",
|
||||
"orientation": "auto",
|
||||
"percentChangeColorMode": "standard",
|
||||
"reduceOptions": {
|
||||
"calcs": [
|
||||
"lastNotNull"
|
||||
],
|
||||
"fields": "",
|
||||
"values": false
|
||||
},
|
||||
"showPercentChange": false,
|
||||
"textMode": "value",
|
||||
"wideLayout": true
|
||||
},
|
||||
"pluginVersion": "11.4.0",
|
||||
"targets": [
|
||||
{
|
||||
"editorMode": "code",
|
||||
"expr": "go_threads",
|
||||
"format": "table",
|
||||
"legendFormat": "{{hmac-file-server}}",
|
||||
"range": true,
|
||||
"refId": "A"
|
||||
}
|
||||
],
|
||||
"title": "HMAC GO Threads",
|
||||
"type": "stat"
|
||||
},
|
||||
{
|
||||
"datasource": {
|
||||
"default": true,
|
||||
"type": "prometheus"
|
||||
},
|
||||
"fieldConfig": {
|
||||
"defaults": {
|
||||
"mappings": [],
|
||||
"thresholds": {
|
||||
"mode": "absolute",
|
||||
"steps": [
|
||||
{
|
||||
"color": "green",
|
||||
"value": null
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
"overrides": []
|
||||
},
|
||||
"gridPos": {
|
||||
"h": 7,
|
||||
"w": 4,
|
||||
"x": 20,
|
||||
"y": 0
|
||||
},
|
||||
"id": 17,
|
||||
"options": {
|
||||
@ -262,8 +324,8 @@
|
||||
},
|
||||
"gridPos": {
|
||||
"h": 7,
|
||||
"w": 3,
|
||||
"x": 18,
|
||||
"w": 5,
|
||||
"x": 0,
|
||||
"y": 7
|
||||
},
|
||||
"id": 11,
|
||||
@ -288,7 +350,7 @@
|
||||
"targets": [
|
||||
{
|
||||
"editorMode": "code",
|
||||
"expr": "hmac_file_server_uploads_total",
|
||||
"expr": "increase(hmac_file_server_uploads_total[1h])",
|
||||
"format": "table",
|
||||
"legendFormat": "Uploads",
|
||||
"range": true,
|
||||
@ -325,8 +387,8 @@
|
||||
},
|
||||
"gridPos": {
|
||||
"h": 7,
|
||||
"w": 3,
|
||||
"x": 21,
|
||||
"w": 5,
|
||||
"x": 5,
|
||||
"y": 7
|
||||
},
|
||||
"id": 12,
|
||||
@ -360,258 +422,6 @@
|
||||
"title": "HMAC Downloads",
|
||||
"type": "stat"
|
||||
},
|
||||
{
|
||||
"datasource": {
|
||||
"default": true,
|
||||
"type": "prometheus"
|
||||
},
|
||||
"fieldConfig": {
|
||||
"defaults": {
|
||||
"mappings": [],
|
||||
"thresholds": {
|
||||
"mode": "absolute",
|
||||
"steps": [
|
||||
{
|
||||
"color": "green",
|
||||
"value": null
|
||||
},
|
||||
{
|
||||
"color": "red",
|
||||
"value": 80
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
"overrides": []
|
||||
},
|
||||
"gridPos": {
|
||||
"h": 7,
|
||||
"w": 6,
|
||||
"x": 0,
|
||||
"y": 14
|
||||
},
|
||||
"id": 10,
|
||||
"options": {
|
||||
"colorMode": "value",
|
||||
"graphMode": "none",
|
||||
"justifyMode": "auto",
|
||||
"orientation": "auto",
|
||||
"percentChangeColorMode": "standard",
|
||||
"reduceOptions": {
|
||||
"calcs": [
|
||||
"lastNotNull"
|
||||
],
|
||||
"fields": "",
|
||||
"values": false
|
||||
},
|
||||
"showPercentChange": false,
|
||||
"textMode": "value",
|
||||
"wideLayout": true
|
||||
},
|
||||
"pluginVersion": "11.4.0",
|
||||
"targets": [
|
||||
{
|
||||
"editorMode": "code",
|
||||
"expr": "go_threads",
|
||||
"format": "table",
|
||||
"legendFormat": "{{hmac-file-server}}",
|
||||
"range": true,
|
||||
"refId": "A"
|
||||
}
|
||||
],
|
||||
"title": "HMAC GO Threads",
|
||||
"type": "stat"
|
||||
},
|
||||
{
|
||||
"datasource": {
|
||||
"default": true,
|
||||
"type": "prometheus"
|
||||
},
|
||||
"fieldConfig": {
|
||||
"defaults": {
|
||||
"color": {
|
||||
"mode": "thresholds"
|
||||
},
|
||||
"mappings": [],
|
||||
"thresholds": {
|
||||
"mode": "absolute",
|
||||
"steps": [
|
||||
{
|
||||
"color": "green",
|
||||
"value": null
|
||||
},
|
||||
{
|
||||
"color": "red",
|
||||
"value": 80
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
"overrides": []
|
||||
},
|
||||
"gridPos": {
|
||||
"h": 7,
|
||||
"w": 3,
|
||||
"x": 6,
|
||||
"y": 14
|
||||
},
|
||||
"id": 16,
|
||||
"options": {
|
||||
"colorMode": "value",
|
||||
"graphMode": "area",
|
||||
"justifyMode": "auto",
|
||||
"orientation": "auto",
|
||||
"percentChangeColorMode": "standard",
|
||||
"reduceOptions": {
|
||||
"calcs": [
|
||||
"lastNotNull"
|
||||
],
|
||||
"fields": "",
|
||||
"values": false
|
||||
},
|
||||
"showPercentChange": false,
|
||||
"textMode": "auto",
|
||||
"wideLayout": true
|
||||
},
|
||||
"pluginVersion": "11.4.0",
|
||||
"targets": [
|
||||
{
|
||||
"editorMode": "code",
|
||||
"exemplar": false,
|
||||
"expr": "hmac_active_connections_total",
|
||||
"format": "time_series",
|
||||
"instant": true,
|
||||
"interval": "",
|
||||
"legendFormat": "__auto",
|
||||
"range": false,
|
||||
"refId": "A"
|
||||
}
|
||||
],
|
||||
"title": "HMAC Active Connections",
|
||||
"type": "stat"
|
||||
},
|
||||
{
|
||||
"datasource": {
|
||||
"default": true,
|
||||
"type": "prometheus"
|
||||
},
|
||||
"fieldConfig": {
|
||||
"defaults": {
|
||||
"color": {
|
||||
"mode": "thresholds"
|
||||
},
|
||||
"mappings": [],
|
||||
"thresholds": {
|
||||
"mode": "absolute",
|
||||
"steps": [
|
||||
{
|
||||
"color": "green",
|
||||
"value": null
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
"overrides": []
|
||||
},
|
||||
"gridPos": {
|
||||
"h": 7,
|
||||
"w": 3,
|
||||
"x": 9,
|
||||
"y": 14
|
||||
},
|
||||
"id": 19,
|
||||
"options": {
|
||||
"colorMode": "value",
|
||||
"graphMode": "area",
|
||||
"justifyMode": "auto",
|
||||
"orientation": "auto",
|
||||
"percentChangeColorMode": "standard",
|
||||
"reduceOptions": {
|
||||
"calcs": [
|
||||
"lastNotNull"
|
||||
],
|
||||
"fields": "",
|
||||
"values": false
|
||||
},
|
||||
"showPercentChange": false,
|
||||
"textMode": "auto",
|
||||
"wideLayout": true
|
||||
},
|
||||
"pluginVersion": "11.4.0",
|
||||
"targets": [
|
||||
{
|
||||
"editorMode": "code",
|
||||
"expr": "hmac_infected_files_total",
|
||||
"format": "table",
|
||||
"legendFormat": "__auto",
|
||||
"range": true,
|
||||
"refId": "A"
|
||||
}
|
||||
],
|
||||
"title": "HMAC infected file(s)",
|
||||
"type": "stat"
|
||||
},
|
||||
{
|
||||
"datasource": {
|
||||
"default": true,
|
||||
"type": "prometheus"
|
||||
},
|
||||
"fieldConfig": {
|
||||
"defaults": {
|
||||
"mappings": [],
|
||||
"thresholds": {
|
||||
"mode": "absolute",
|
||||
"steps": [
|
||||
{
|
||||
"color": "green",
|
||||
"value": null
|
||||
},
|
||||
{
|
||||
"color": "red",
|
||||
"value": 80
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
"overrides": []
|
||||
},
|
||||
"gridPos": {
|
||||
"h": 7,
|
||||
"w": 3,
|
||||
"x": 12,
|
||||
"y": 14
|
||||
},
|
||||
"id": 2,
|
||||
"options": {
|
||||
"colorMode": "value",
|
||||
"graphMode": "area",
|
||||
"justifyMode": "auto",
|
||||
"orientation": "auto",
|
||||
"percentChangeColorMode": "standard",
|
||||
"reduceOptions": {
|
||||
"calcs": [
|
||||
"lastNotNull"
|
||||
],
|
||||
"fields": "",
|
||||
"values": false
|
||||
},
|
||||
"showPercentChange": false,
|
||||
"textMode": "auto",
|
||||
"wideLayout": true
|
||||
},
|
||||
"pluginVersion": "11.4.0",
|
||||
"targets": [
|
||||
{
|
||||
"editorMode": "code",
|
||||
"expr": "hmac_file_server_upload_errors_total",
|
||||
"legendFormat": "Upload Errors",
|
||||
"range": true,
|
||||
"refId": "A"
|
||||
}
|
||||
],
|
||||
"title": "HMAC Upload Errors",
|
||||
"type": "stat"
|
||||
},
|
||||
{
|
||||
"datasource": {
|
||||
"default": true,
|
||||
@ -643,8 +453,8 @@
|
||||
"gridPos": {
|
||||
"h": 7,
|
||||
"w": 3,
|
||||
"x": 15,
|
||||
"y": 14
|
||||
"x": 10,
|
||||
"y": 7
|
||||
},
|
||||
"id": 15,
|
||||
"options": {
|
||||
@ -710,8 +520,8 @@
|
||||
"gridPos": {
|
||||
"h": 7,
|
||||
"w": 3,
|
||||
"x": 18,
|
||||
"y": 14
|
||||
"x": 13,
|
||||
"y": 7
|
||||
},
|
||||
"id": 13,
|
||||
"options": {
|
||||
@ -771,8 +581,69 @@
|
||||
"gridPos": {
|
||||
"h": 7,
|
||||
"w": 3,
|
||||
"x": 21,
|
||||
"y": 14
|
||||
"x": 16,
|
||||
"y": 7
|
||||
},
|
||||
"id": 2,
|
||||
"options": {
|
||||
"colorMode": "value",
|
||||
"graphMode": "area",
|
||||
"justifyMode": "auto",
|
||||
"orientation": "auto",
|
||||
"percentChangeColorMode": "standard",
|
||||
"reduceOptions": {
|
||||
"calcs": [
|
||||
"lastNotNull"
|
||||
],
|
||||
"fields": "",
|
||||
"values": false
|
||||
},
|
||||
"showPercentChange": false,
|
||||
"textMode": "auto",
|
||||
"wideLayout": true
|
||||
},
|
||||
"pluginVersion": "11.4.0",
|
||||
"targets": [
|
||||
{
|
||||
"editorMode": "code",
|
||||
"expr": "hmac_file_server_upload_errors_total",
|
||||
"legendFormat": "Upload Errors",
|
||||
"range": true,
|
||||
"refId": "A"
|
||||
}
|
||||
],
|
||||
"title": "HMAC Upload Errors",
|
||||
"type": "stat"
|
||||
},
|
||||
{
|
||||
"datasource": {
|
||||
"default": true,
|
||||
"type": "prometheus"
|
||||
},
|
||||
"fieldConfig": {
|
||||
"defaults": {
|
||||
"mappings": [],
|
||||
"thresholds": {
|
||||
"mode": "absolute",
|
||||
"steps": [
|
||||
{
|
||||
"color": "green",
|
||||
"value": null
|
||||
},
|
||||
{
|
||||
"color": "red",
|
||||
"value": 80
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
"overrides": []
|
||||
},
|
||||
"gridPos": {
|
||||
"h": 7,
|
||||
"w": 5,
|
||||
"x": 19,
|
||||
"y": 7
|
||||
},
|
||||
"id": 21,
|
||||
"options": {
|
||||
@ -805,6 +676,302 @@
|
||||
],
|
||||
"title": "HMAC FileTTL Deletion(s)",
|
||||
"type": "stat"
|
||||
},
|
||||
{
|
||||
"datasource": {
|
||||
"default": true,
|
||||
"type": "prometheus"
|
||||
},
|
||||
"fieldConfig": {
|
||||
"defaults": {
|
||||
"color": {
|
||||
"mode": "thresholds"
|
||||
},
|
||||
"mappings": [],
|
||||
"thresholds": {
|
||||
"mode": "absolute",
|
||||
"steps": [
|
||||
{
|
||||
"color": "green",
|
||||
"value": null
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
"overrides": []
|
||||
},
|
||||
"gridPos": {
|
||||
"h": 7,
|
||||
"w": 3,
|
||||
"x": 0,
|
||||
"y": 14
|
||||
},
|
||||
"id": 19,
|
||||
"options": {
|
||||
"colorMode": "value",
|
||||
"graphMode": "area",
|
||||
"justifyMode": "auto",
|
||||
"orientation": "auto",
|
||||
"percentChangeColorMode": "standard",
|
||||
"reduceOptions": {
|
||||
"calcs": [
|
||||
"lastNotNull"
|
||||
],
|
||||
"fields": "",
|
||||
"values": false
|
||||
},
|
||||
"showPercentChange": false,
|
||||
"textMode": "auto",
|
||||
"wideLayout": true
|
||||
},
|
||||
"pluginVersion": "11.4.0",
|
||||
"targets": [
|
||||
{
|
||||
"editorMode": "code",
|
||||
"expr": "hmac_infected_files_total",
|
||||
"format": "table",
|
||||
"legendFormat": "__auto",
|
||||
"range": true,
|
||||
"refId": "A"
|
||||
}
|
||||
],
|
||||
"title": "HMAC infected file(s)",
|
||||
"type": "stat"
|
||||
},
|
||||
{
|
||||
"datasource": {
|
||||
"default": true,
|
||||
"type": "prometheus"
|
||||
},
|
||||
"fieldConfig": {
|
||||
"defaults": {
|
||||
"color": {
|
||||
"mode": "palette-classic"
|
||||
},
|
||||
"custom": {
|
||||
"fillOpacity": 80,
|
||||
"gradientMode": "none",
|
||||
"hideFrom": {
|
||||
"legend": false,
|
||||
"tooltip": false,
|
||||
"viz": false
|
||||
},
|
||||
"lineWidth": 1,
|
||||
"stacking": {
|
||||
"group": "A",
|
||||
"mode": "none"
|
||||
}
|
||||
},
|
||||
"fieldMinMax": false,
|
||||
"mappings": [],
|
||||
"thresholds": {
|
||||
"mode": "absolute",
|
||||
"steps": [
|
||||
{
|
||||
"color": "green",
|
||||
"value": null
|
||||
},
|
||||
{
|
||||
"color": "red",
|
||||
"value": 80
|
||||
}
|
||||
]
|
||||
},
|
||||
"unit": "files"
|
||||
},
|
||||
"overrides": []
|
||||
},
|
||||
"gridPos": {
|
||||
"h": 7,
|
||||
"w": 7,
|
||||
"x": 3,
|
||||
"y": 14
|
||||
},
|
||||
"id": 22,
|
||||
"options": {
|
||||
"legend": {
|
||||
"calcs": [],
|
||||
"displayMode": "list",
|
||||
"placement": "bottom",
|
||||
"showLegend": true
|
||||
},
|
||||
"tooltip": {
|
||||
"mode": "single",
|
||||
"sort": "none"
|
||||
}
|
||||
},
|
||||
"pluginVersion": "11.4.0",
|
||||
"targets": [
|
||||
{
|
||||
"editorMode": "code",
|
||||
"exemplar": false,
|
||||
"expr": "increase(hmac_file_server_clamav_scans_total[24h])",
|
||||
"format": "time_series",
|
||||
"instant": true,
|
||||
"interval": "",
|
||||
"legendFormat": "__auto",
|
||||
"range": true,
|
||||
"refId": "A"
|
||||
}
|
||||
],
|
||||
"title": "HMAC ClamAV San (24h)",
|
||||
"type": "histogram"
|
||||
},
|
||||
{
|
||||
"datasource": {
|
||||
"default": true,
|
||||
"type": "prometheus"
|
||||
},
|
||||
"fieldConfig": {
|
||||
"defaults": {
|
||||
"color": {
|
||||
"mode": "palette-classic"
|
||||
},
|
||||
"custom": {
|
||||
"fillOpacity": 80,
|
||||
"gradientMode": "none",
|
||||
"hideFrom": {
|
||||
"legend": false,
|
||||
"tooltip": false,
|
||||
"viz": false
|
||||
},
|
||||
"lineWidth": 1,
|
||||
"stacking": {
|
||||
"group": "A",
|
||||
"mode": "none"
|
||||
}
|
||||
},
|
||||
"fieldMinMax": false,
|
||||
"mappings": [],
|
||||
"thresholds": {
|
||||
"mode": "absolute",
|
||||
"steps": [
|
||||
{
|
||||
"color": "green",
|
||||
"value": null
|
||||
},
|
||||
{
|
||||
"color": "red",
|
||||
"value": 80
|
||||
}
|
||||
]
|
||||
},
|
||||
"unit": "files"
|
||||
},
|
||||
"overrides": []
|
||||
},
|
||||
"gridPos": {
|
||||
"h": 7,
|
||||
"w": 7,
|
||||
"x": 10,
|
||||
"y": 14
|
||||
},
|
||||
"id": 23,
|
||||
"options": {
|
||||
"legend": {
|
||||
"calcs": [],
|
||||
"displayMode": "list",
|
||||
"placement": "bottom",
|
||||
"showLegend": true
|
||||
},
|
||||
"tooltip": {
|
||||
"mode": "single",
|
||||
"sort": "none"
|
||||
}
|
||||
},
|
||||
"pluginVersion": "11.4.0",
|
||||
"targets": [
|
||||
{
|
||||
"editorMode": "code",
|
||||
"exemplar": false,
|
||||
"expr": "increase(hmac_file_server_clamav_errors_total[24h])",
|
||||
"format": "time_series",
|
||||
"instant": true,
|
||||
"interval": "",
|
||||
"legendFormat": "__auto",
|
||||
"range": true,
|
||||
"refId": "A"
|
||||
}
|
||||
],
|
||||
"title": "HMAC ClamAV SanError(s) (24h)",
|
||||
"type": "histogram"
|
||||
},
|
||||
{
|
||||
"datasource": {
|
||||
"default": true,
|
||||
"type": "prometheus"
|
||||
},
|
||||
"fieldConfig": {
|
||||
"defaults": {
|
||||
"color": {
|
||||
"mode": "palette-classic"
|
||||
},
|
||||
"custom": {
|
||||
"fillOpacity": 80,
|
||||
"gradientMode": "none",
|
||||
"hideFrom": {
|
||||
"legend": false,
|
||||
"tooltip": false,
|
||||
"viz": false
|
||||
},
|
||||
"lineWidth": 1,
|
||||
"stacking": {
|
||||
"group": "A",
|
||||
"mode": "none"
|
||||
}
|
||||
},
|
||||
"mappings": [],
|
||||
"thresholds": {
|
||||
"mode": "absolute",
|
||||
"steps": [
|
||||
{
|
||||
"color": "green",
|
||||
"value": null
|
||||
},
|
||||
{
|
||||
"color": "red",
|
||||
"value": 80
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
"overrides": []
|
||||
},
|
||||
"gridPos": {
|
||||
"h": 7,
|
||||
"w": 7,
|
||||
"x": 17,
|
||||
"y": 14
|
||||
},
|
||||
"id": 16,
|
||||
"options": {
|
||||
"legend": {
|
||||
"calcs": [],
|
||||
"displayMode": "list",
|
||||
"placement": "bottom",
|
||||
"showLegend": true
|
||||
},
|
||||
"tooltip": {
|
||||
"mode": "single",
|
||||
"sort": "none"
|
||||
}
|
||||
},
|
||||
"pluginVersion": "11.4.0",
|
||||
"targets": [
|
||||
{
|
||||
"editorMode": "code",
|
||||
"exemplar": false,
|
||||
"expr": "histogram_quantile(0.95, sum(rate(hmac_file_server_request_duration_seconds_bucket[5m])) by (le))",
|
||||
"format": "time_series",
|
||||
"instant": true,
|
||||
"interval": "",
|
||||
"legendFormat": "__auto",
|
||||
"range": true,
|
||||
"refId": "A"
|
||||
}
|
||||
],
|
||||
"title": "HMAC Request Duration",
|
||||
"type": "histogram"
|
||||
}
|
||||
],
|
||||
"preload": false,
|
||||
@ -821,6 +988,6 @@
|
||||
"timezone": "",
|
||||
"title": "HMAC File Server Metrics",
|
||||
"uid": "de0ye5t0hzq4ge",
|
||||
"version": 145,
|
||||
"version": 153,
|
||||
"weekStart": ""
|
||||
}
|
||||
|
||||
Loading…
x
Reference in New Issue
Block a user