hmac-file-server/wiki.md at 1c6477531cae34db8dd21ee33d346b0e9edd1ec5

Alexander Renz 1c6477531c fix: 2.9 ipv4|ipv6|auto

2025-04-12 09:49:14 +02:00

30 KiB

Raw Blame History

This documentation provides detailed information on configuring, setting up, and maintaining the HMAC File Server. Whether you're a developer, system administrator, or an enthusiast, this guide will help you navigate through the server's features and configurations effectively.

Introduction
Configuration
Example Configuration
Setup Instructions
Building for Different Architectures
Additional Recommendations
Notes
Using HMAC File Server for CI/CD Build Artifacts
Monitoring

Introduction

The HMAC File Server is a secure and efficient file management solution designed to handle file uploads, downloads, deduplication, and more. Built with a focus on security, scalability, and performance, it integrates seamlessly with various tools and services to provide a comprehensive file handling experience.

Configuration

The HMAC File Server is configured using a config.toml file. Below are the detailed explanations of each configuration section and their respective options.

Server Configuration

# Server configuration
listenport = "8080"  # TCP port for incoming requests
unixsocket = false   # Use Unix domain socket instead of TCP
storagepath = "/path/to/hmac-file-server/data/"  # Directory to store uploaded files
loglevel = "debug"   # Logging level: "debug", "info", "warn", "error"
logfile = "/path/to/hmac-file-server.log"  # Path to log file; leave empty to use stdout
metricsenabled = true   # Enable Prometheus metrics
metricsport = "9090"    # Port for Prometheus metrics
deduplicationenabled = true
minfreebytes = "5GB"     # Minimum free disk space required
filettl = "2Y"           # Time-to-live for files
filettlenabled = false   # Enable TTL checks and cleanup
autoadjustworkers = true # Automatically adjust worker threads based on load
networkevents = false    # Enable detailed network event logging
pidfilepath = "./hmac-file-server.pid" # Path to PID file
precaching = true        # Pre-cache file structures on startup

# New option to force network protocol
forceprotocol = "auto"  # Options: "ipv4", "ipv6", "auto"

Configuration Options

listenport:
- Type: String
- Description: Specifies the TCP port on which the server listens for incoming requests.
- Default: "8080"
unixsocket:
- Type: Boolean
- Description: Determines whether to use a Unix domain socket instead of a TCP port for communication.
- Default: false
storagepath:
- Type: String
- Description: Defines the directory path where uploaded files are stored. Ensure this path exists and has appropriate permissions.
- Default: "/path/to/hmac-file-server/data/"
loglevel:
- Type: String
- Description: Sets the verbosity level of logs.
- Options: "debug", "info", "warn", "error"
- Default: "debug"
logfile:
- Type: String
- Description: Specifies the file path for logging. If left empty, logs are output to stdout.
- Default: "/path/to/hmac-file-server.log"
metricsenabled:
- Type: Boolean
- Description: Enables or disables the Prometheus metrics endpoint.
- Default: true
metricsport:
- Type: String
- Description: Defines the port on which Prometheus metrics are exposed.
- Default: "9090"
deduplicationenabled:
- Type: Boolean
- Description: Enables or disables file deduplication to optimize storage usage.
- Default: true
minfreebytes:
- Type: String
- Description: Specifies the minimum free disk space required for the server to operate effectively.
- Default: "5GB"
filettl:
- Type: String
- Description: Sets the default Time-to-Live (TTL) for files, determining how long files are retained before deletion.
- Format: Duration (e.g., "2Y" for two years)
- Default: "2Y"
filettlenabled:
- Type: Boolean
- Description: Enables or disables TTL checks and automatic file cleanup based on the filettl value.
- Default: false
autoadjustworkers:
- Type: Boolean
- Description: Automatically adjusts the number of worker threads based on server load and system resources.
- Default: true
networkevents:
- Type: Boolean
- Description: Enables detailed logging of network events, which can be useful for debugging but may increase log verbosity.
- Default: false
pidfilepath:
- Type: String
- Description: Specifies the file path where the server writes its Process ID (PID) file. This is useful for managing the server process.
- Default: "./hmac-file-server.pid"
precaching:
- Type: Boolean
- Description: Enables pre-caching of file structures on startup to improve access speed and performance.
- Default: true
forceprotocol:
- Type: String
- Description: Specifies the network protocol to use for server communication.
  - "ipv4": Forces the server to use IPv4.
  - "ipv6": Forces the server to use IPv6.
  - "auto": Uses the system's default behavior (dual-stack).
- Default: "auto"

Deduplication Settings

# Deduplication settings
[deduplication]
enabled = true
directory = "/path/to/hmac-file-server/deduplication/"  # Path to deduplication metadata store

Configuration Options

enabled:
- Type: Boolean
- Description: Enables or disables the deduplication feature, which helps in eliminating duplicate files to save storage space.
- Default: true
directory:
- Type: String
- Description: Specifies the directory path where deduplication metadata is stored. Ensure this directory exists and has appropriate permissions.
- Default: "/path/to/hmac-file-server/deduplication/"

ISO Settings

# ISO settings
[iso]
enabled = false
size = "1TB"  # Maximum ISO size
mountpoint = "/path/to/hmac-file-server/iso/"  # ISO mount point
charset = "utf-8"  # Filesystem character set encoding

Configuration Options

enabled:
- Type: Boolean
- Description: Enables or disables the mounting of an ISO-based filesystem for specialized storage needs.
- Default: false
size:
- Type: String
- Description: Defines the maximum allowed size for the ISO container.
- Default: "1TB"
mountpoint:
- Type: String
- Description: Specifies the directory path where the ISO is mounted. Ensure this path exists and has appropriate permissions.
- Default: "/path/to/hmac-file-server/iso/"
charset:
- Type: String
- Description: Sets the filesystem character set encoding for the ISO.
- Default: "utf-8"

Note

: Ensure only one [iso] block is active in your config.toml to avoid configuration conflicts.

Timeout Settings

# Timeout settings
[timeouts]
readtimeout = "3600s"    # Maximum time to read a request
writetimeout = "3600s"   # Maximum time to write a response
idletimeout = "3600s"    # Maximum keep-alive time for idle connections

Configuration Options

readtimeout:
- Type: String
- Description: Sets the maximum duration for reading the entire request, including the body.
- Format: Duration (e.g., "3600s" for one hour)
- Default: "3600s"
writetimeout:
- Type: String
- Description: Defines the maximum duration before timing out writes of the response.
- Format: Duration (e.g., "3600s" for one hour)
- Default: "3600s"
idletimeout:
- Type: String
- Description: Specifies the maximum amount of time to wait for the next request when keep-alives are enabled.
- Format: Duration (e.g., "3600s" for one hour)
- Default: "3600s"

Security Settings

# Security settings
[security]
secret = "your-secure-secret-key"  # HMAC shared secret key (change to a secure value)

Configuration Options

secret:
- Type: String
- Description: The HMAC shared secret key used for signing requests and operations.
- Default: "your-secure-secret-key"
- Warning: Change this immediately to a unique, strong string in production environments to ensure the security of HMAC operations.

Versioning Settings

# Versioning settings
[versioning]
enableversioning = false
maxversions = 1  # Number of file versions to retain

Configuration Options

enableversioning:
- Type: Boolean
- Description: Enables or disables the versioning feature, which maintains multiple versions of the same file.
- Default: false
maxversions:
- Type: Integer
- Description: Specifies the maximum number of versions to retain for each file.
- Default: 1

Uploads Settings

# Upload settings
[uploads]
resumableuploadsenabled = false
chunkeduploadsenabled = true
chunksize = "32MB"  # Chunk size for uploads
allowedextensions = [
    ".txt", ".pdf", ".png", ".jpg", ".jpeg", ".gif",
    ".bmp", ".tiff", ".svg", ".webp", ".wav", ".mp4",
    ".avi", ".mkv", ".mov", ".wmv", ".flv", ".webm",
    ".mpeg", ".mpg", ".m4v", ".3gp", ".3g2", ".mp3", ".ogg"
]

Configuration Options

resumableuploadsenabled:
- Type: Boolean
- Description: Enables or disables support for resumable (chunked) file uploads.
- Default: false
chunkeduploadsenabled:
- Type: Boolean
- Description: Specifically enables or disables chunked uploads.
- Default: true
chunksize:
- Type: String
- Description: Defines the size of each chunk in chunked uploads.
- Format: Size (e.g., "32MB")
- Default: "32MB"

allowedextensions:

Type: Array of Strings
Description: Lists the file extensions permitted for upload.

Default:

allowedextensions = [
    ".txt", ".pdf", ".png", ".jpg", ".jpeg", ".gif",
    ".bmp", ".tiff", ".svg", ".webp", ".wav", ".mp4",
    ".avi", ".mkv", ".mov", ".wmv", ".flv", ".webm",
    ".mpeg", ".mpg", ".m4v", ".3gp", ".3g2", ".mp3", ".ogg"
]

Downloads Settings

# Downloads settings
[downloads]
resumabledownloadsenabled = false
chunkeddownloadsenabled = true
chunksize = "32MB"

Configuration Options

resumabledownloadsenabled:
- Type: Boolean
- Description: Enables or disables support for resumable (chunked) downloads.
- Default: false
chunkeddownloadsenabled:
- Type: Boolean
- Description: Specifically enables or disables chunked downloads.
- Default: true
chunksize:
- Type: String
- Description: Defines the size of each chunk in chunked downloads.
- Format: Size (e.g., "32MB")
- Default: "32MB"

Note

: The allowedextensions key is not part of the [downloads] configuration based on the provided code. Ensure that it is omitted to prevent configuration errors.

ClamAV Settings

# ClamAV settings
[clamav]
clamavenabled = true
clamavsocket = "/path/to/clamav/clamd.ctl"  # Path to ClamAV socket
numscanworkers = 4  # Number of concurrent scan workers
scanfileextensions = [
    ".exe", ".dll", ".bin", ".com", ".bat",
    ".sh", ".php", ".js"
]

Configuration Options

clamavenabled:
- Type: Boolean
- Description: Enables or disables ClamAV integration for virus scanning of uploaded files.
- Default: true
clamavsocket:
- Type: String
- Description: Specifies the file path to the ClamAV socket (.ctl file). Ensure ClamAV is installed and the socket path is correct.
- Default: "/path/to/clamav/clamd.ctl"
numscanworkers:
- Type: Integer
- Description: Sets the number of concurrent workers dedicated to scanning files with ClamAV.
- Default: 4
scanfileextensions:
- Type: Array of Strings
- Description: Lists the file extensions that should be scanned for viruses.
- Default:
```
scanfileextensions = [
    ".exe", ".dll", ".bin", ".com", ".bat",
    ".sh", ".php", ".js"
]
```

Redis Settings

# Redis settings
[redis]
redisenabled = true
redisdbindex = 0
redisaddr = "localhost:6379"  # Redis server address
redispassword = ""            # Redis password if required
redishealthcheckinterval = "120s"  # Interval for Redis health checks

Configuration Options

redisenabled:
- Type: Boolean
- Description: Enables or disables Redis integration for caching or session tracking.
- Default: true
redisaddr:
- Type: String
- Description: Specifies the address of the Redis server (e.g., "localhost:6379").
- Default: "localhost:6379"
redispassword:
- Type: String
- Description: Sets the Redis authentication password, if required.
- Default: ""
redisdbindex:
- Type: Integer
- Description: Specifies the Redis database index to use.
- Default: 0
redishealthcheckinterval:
- Type: String
- Description: Defines the interval for performing health checks on the Redis connection.
- Format: Duration (e.g., "120s" for two minutes)
- Default: "120s"

Worker Settings

# Worker settings
[worker]
numworkers = 10  # Number of worker threads

Configuration Options

numworkers:
- Type: Integer
- Description: Specifies the number of worker threads to handle file operations.
- Default: 10

Configuration Options

maxfilesize:
- Type: String
- Description: Defines the maximum allowed file size for uploads.
- Format: Size (e.g., "10GB")
- Default: "10GB"

Example Configuration

Below is an example config.toml file with default settings:

# Example HMAC File Server configuration

# Server configuration
listenport = "8080"
unixsocket = false
storagepath = "/path/to/hmac-file-server/data/"
loglevel = "debug"
logfile = "/path/to/hmac-file-server.log"
metricsenabled = true
metricsport = "9090"
deduplicationenabled = true
minfreebytes = "5GB"
filettl = "2Y"
filettlenabled = false
autoadjustworkers = true
networkevents = false
pidfilepath = "./hmac-file-server.pid"
precaching = true
forceprotocol = "auto"

# Deduplication settings
[deduplication]
enabled = true
directory = "/path/to/hmac-file-server/deduplication/"

# ISO settings
[iso]
enabled = false
size = "1TB"
mountpoint = "/path/to/hmac-file-server/iso/"
charset = "utf-8"

# Timeout settings
[timeouts]
readtimeout = "3600s"
writetimeout = "3600s"
idletimeout = "3600s"

# Security settings
[security]
secret = "your-secure-secret-key"

# Versioning settings
[versioning]
enableversioning = false
maxversions = 1

# Upload settings
[uploads]
resumableuploadsenabled = false
chunkeduploadsenabled = true
chunksize = "32MB"
allowedextensions = [
    ".txt", ".pdf", ".png", ".jpg", ".jpeg", ".gif",
    ".bmp", ".tiff", ".svg", ".webp", ".wav", ".mp4",
    ".avi", ".mkv", ".mov", ".wmv", ".flv", ".webm",
    ".mpeg", ".mpg", ".m4v", ".3gp", ".3g2", ".mp3", ".ogg"
]

# Download settings
[downloads]
resumabledownloadsenabled = false
chunkeddownloadsenabled = true
chunksize = "32MB"

# ClamAV settings
[clamav]
clamavenabled = true
clamavsocket = "/path/to/clamav/clamd.ctl"
numscanworkers = 4
scanfileextensions = [
    ".exe", ".dll", ".bin", ".com", ".bat",
    ".sh", ".php", ".js"
]

# Redis settings
[redis]
redisenabled = true
redisdbindex = 0
redisaddr = "localhost:6379"
redispassword = ""
redishealthcheckinterval = "120s"

# Worker settings
[worker]
numworkers = 10

Setup Instructions

1. HMAC File Server Installation

To install the HMAC File Server, follow these steps:

Clone the repository:

git clone https://github.com/PlusOne/hmac-file-server.git
cd hmac-file-server

Build the server:
```
go build -o hmac-file-server
```

Create the necessary directories:

mkdir -p /path/to/hmac-file-server/data/
mkdir -p /path/to/hmac-file-server/deduplication/
mkdir -p /path/to/hmac-file-server/iso/

Copy the example configuration file:
```
cp config.example.toml config.toml
```
Edit the config.toml file to match your environment and preferences.
Start the server:
```
./hmac-file-server -config config.toml
```

2. Reverse Proxy Configuration

To set up a reverse proxy for the HMAC File Server, you can use either Apache2 or Nginx. Below are the configuration examples for both.

Apache2 Reverse Proxy

Enable the necessary Apache2 modules:

sudo a2enmod proxy
sudo a2enmod proxy_http
sudo a2enmod headers
sudo a2enmod rewrite

Create a new virtual host configuration file:

sudo nano /etc/apache2/sites-available/hmac-file-server.conf

Add the following configuration to the file:

<VirtualHost *:80>
    ServerName your-domain.com

    ProxyPreserveHost On
    ProxyPass / http://localhost:8080/
    ProxyPassReverse / http://localhost:8080/

    <Location />
        Require all granted
        Header always set X-Content-Type-Options "nosniff"
        Header always set X-Frame-Options "DENY"
        Header always set X-XSS-Protection "1; mode=block"
    </Location>
</VirtualHost>

Enable the new site and restart Apache2:

sudo a2ensite hmac-file-server.conf
sudo systemctl restart apache2

Nginx Reverse Proxy

Install Nginx if not already installed:

sudo apt-get update
sudo apt-get install nginx

Create a new server block configuration file:

sudo nano /etc/nginx/sites-available/hmac-file-server

Add the following configuration to the file:

server {
    listen 80;
    server_name your-domain.com;

    location / {
        proxy_pass http://localhost:8080;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
        proxy_set_header X-Content-Type-Options "nosniff";
        proxy_set_header X-Frame-Options "DENY";
        proxy_set_header X-XSS-Protection "1; mode=block";
    }
}

Enable the new site and restart Nginx:

sudo ln -s /etc/nginx/sites-available/hmac-file-server /etc/nginx/sites-enabled/
sudo systemctl restart nginx

3. ejabberd Configuration

hosts:
  - "your-domain.com"

listen:
  -
    port: 5222
    module: ejabberd_c2s
    certfile: "/etc/ejabberd/ejabberd.pem"
    starttls: true
    starttls_required: true
    protocol_options:
      - "no_sslv3"
      - "no_tlsv1"
      - "no_tlsv1_1"
    ciphers: "HIGH:!aNULL:!eNULL:!3DES:@STRENGTH"
    dhfile: "/etc/ejabberd/dhparams.pem"
    max_stanza_size: 65536
    shaper: c2s_shaper
    access: c2s

  -
    port: 5269
    module: ejabberd_s2s_in
    certfile: "/etc/ejabberd/ejabberd.pem"
    starttls: true
    starttls_required: true
    protocol_options:
      - "no_sslv3"
      - "no_tlsv1"
      - "no_tlsv1_1"
    ciphers: "HIGH:!aNULL:!eNULL:!3DES:@STRENGTH"
    dhfile: "/etc/ejabberd/dhparams.pem"
    max_stanza_size: 131072
    shaper: s2s_shaper
    access: s2s

acl:
  local:
    user_regexp: ""

access_rules:
  local:
    allow: local

mod_http_upload:
    max_size: 1073741824
    thumbnail: true
    put_url: https://share.uuxo.net
    get_url: https://share.uuxo.net
    external_secret: "changeme"
    custom_headers:
      "Access-Control-Allow-Origin": "*"
      "Access-Control-Allow-Methods": "GET,HEAD,PUT,OPTIONS"
      "Access-Control-Allow-Headers": "Content-Type"

Restart ejabberd:
```
sudo systemctl restart ejabberd
```

4. Systemd Service Setup

To set up the HMAC File Server as a systemd service, follow these steps:

Create a new systemd service file:

sudo nano /etc/systemd/system/hmac-file-server.service

Add the following configuration to the file:

[Unit]
Description=HMAC File Server
After=network.target

[Service]
ExecStart=/path/to/hmac-file-server -config /path/to/config.toml
WorkingDirectory=/path/to/hmac-file-server
Restart=always
User=www-data
Group=www-data

[Install]
WantedBy=multi-user.target

Reload systemd and enable the service:

sudo systemctl daemon-reload
sudo systemctl enable hmac-file-server
sudo systemctl start hmac-file-server

Building for Different Architectures

To build the HMAC File Server for different architectures, you can use the following commands:

Building for Linux (x86_64)

GOOS=linux GOARCH=amd64 go build -o hmac-file-server-linux-amd64

Building for ARM (32-bit)

GOOS=linux GOARCH=arm GOARM=7 go build -o hmac-file-server-linux-arm

Building for ARM (64-bit)

GOOS=linux GOARCH=arm64 go build -o hmac-file-server-linux-arm64

Building the Monitoring Tool

The monitoring tool (monitor.go) is located in the server/cmd/monitor/ directory and is compiled separately from the main HMAC File Server. Below are the instructions for building the monitoring tool:

Building for Linux (x86_64)

GOOS=linux GOARCH=amd64 go build -o monitor-linux-amd64 ./server/cmd/monitor/monitor.go

Building for ARM (32-bit)

GOOS=linux GOARCH=arm GOARM=7 go build -o monitor-linux-arm ./server/cmd/monitor/monitor.go

Building for ARM (64-bit)

GOOS=linux GOARCH=arm64 go build -o monitor-linux-arm64 ./server/cmd/monitor/monitor.go

Once built, the monitoring tool can be executed independently to track system performance, Prometheus metrics, and active processes.

Additional Recommendations

Security: Ensure that the secret key in the config.toml file is changed to a unique, strong value to secure HMAC operations.
Backups: Regularly back up the config.toml file and any important data stored by the HMAC File Server.
Monitoring: Use monitoring tools like Prometheus and Grafana to keep track of server performance and metrics.

Notes

The HMAC File Server is designed to be flexible and configurable. Adjust the settings in the config.toml file to match your specific requirements and environment.
For any issues or questions, refer to the project's GitHub repository and documentation.

Using HMAC File Server for CI/CD Build Artifacts

This guide explains how to use HMAC File Server to securely upload and download build artifacts in CI/CD pipelines.

Why Use HMAC File Server?

Secure, HMAC-authenticated access
Self-hosted, no third-party storage needed
Configurable TTL, versioning, and deduplication
Prometheus metrics for monitoring
Easily integrated into GitHub Actions, GitLab CI, Jenkins, etc.

Step 1: Set Up HMAC File Server

Clone and build the server:

git clone https://github.com/PlusOne/hmac-file-server.git
cd hmac-file-server
go build -o hmac-file-server
cp config.example.toml config.toml
mkdir -p /data/artifacts
./hmac-file-server -config config.toml

Update config.toml with:

[hmac]
secret = "your-secret-key"

[upload]
enabled = true
path = "/data/artifacts"

[download]
enabled = true

Step 2: Generate Signed URLs

Use HMAC to generate signed URLs for secure upload/download.

Upload Script

#!/bin/bash

FILE_PATH="./build/output.tar.gz"
FILENAME="output.tar.gz"
SECRET="your-secret-key"
BASE_URL="https://your-hmac-server.com"

TIMESTAMP=$(date +%s)
SIGNATURE=$(echo -n "$FILENAME$TIMESTAMP" | openssl dgst -sha256 -hmac "$SECRET" | sed 's/^.* //')

curl -X PUT "$BASE_URL/upload/$FILENAME?ts=$TIMESTAMP&sig=$SIGNATURE" --data-binary "@$FILE_PATH"

Download Script

#!/bin/bash

FILENAME="output.tar.gz"
SECRET="your-secret-key"
BASE_URL="https://your-hmac-server.com"

TIMESTAMP=$(date +%s)
SIGNATURE=$(echo -n "$FILENAME$TIMESTAMP" | openssl dgst -sha256 -hmac "$SECRET" | sed 's/^.* //')

curl -O "$BASE_URL/download/$FILENAME?ts=$TIMESTAMP&sig=$SIGNATURE"

Step 3: Integrate into CI/CD

GitHub Actions Example

jobs:
  build:
    runs-on: ubuntu-latest
    steps:
      - name: Checkout code
        uses: actions/checkout@v3

      - name: Build
        run: |
          mkdir -p build
          echo "example artifact content" > build/output.tar.gz

      - name: Upload Artifact to HMAC Server
        run: bash scripts/upload-artifact.sh

Optional Features

TTL: Auto-delete artifacts after a set time
Deduplication: Only store unique files
Versioning: Track changes to files over time
Virus Scanning: Integrate with ClamAV

Monitoring

The HMAC File Server provides a built-in monitoring interface to track system performance, Prometheus metrics, and active processes. Below is an overview of the monitoring features:

System Data

The monitoring interface displays key system metrics, including:

CPU Usage: Current CPU usage percentage.
Memory Usage: Current memory usage percentage.
CPU Cores: Number of CPU cores available.

Prometheus Metrics

The server exposes Prometheus metrics for tracking upload and download statistics:

hmac_file_server_upload_errors_total: Total number of upload errors.
hmac_file_server_uploads_total: Total number of successful uploads.
hmac_file_server_downloads_total: Total number of successful downloads.

These metrics can be integrated with Prometheus and visualized using tools like Grafana.

Process List

The monitoring interface also provides a list of active processes, including:

Process ID (PID)
CPU usage percentage
Memory usage percentage
Command or service name

This information helps in identifying resource-intensive processes and debugging performance issues.

Example Monitoring Output

Below is an example of the monitoring interface output:

System Data
Metric                Value
CPU Usage             2.78%
Memory Usage          26.49%
CPU Cores             4

Prometheus Metrics
hmac_file_server_upload_errors_total   1.00
hmac_file_server_uploads_total         4.00
hmac_file_server_downloads_total       15.00

Process List
PID       CPU   MEM   COMMAND
907752    0.12  2.69  /lib/systemd/systemd-journald
4055132   0.12  0.03  /usr/sbin/qemu-ga
2370782   0.11  0.00  kworker/0:2-wg-crypt-wg1
2371119   0.10  0.08  bash
2371096   0.10  0.14  sshd: root@pts/0
2369170   0.09  0.00  kworker/0:0-mm_percpu_wq
2371240   0.07  0.00  kworker/0:1-wg-crypt-wg1
2371099   0.06  0.13  systemd --user
868714    0.05  0.59  php-fpm: pool www

For more details on integrating Prometheus metrics, refer to the Prometheus documentation.

License

HMAC File Server is open-source and MIT licensed.

Resources

Version 3.0 Release Note

Version 2.8 is the last release before we begin integrating additional features and focusing on further stability patches.

CI/CD with HMAC File Server – Summary

Sure! Here is a brief guide on how to use the HMAC File Server in your CI/CD pipeline:

1. Server Setup

git clone https://github.com/PlusOne/hmac-file-server.git
cd hmac-file-server
go build -o hmac-file-server
cp config.example.toml config.toml
mkdir -p /data/artifacts
./hmac-file-server -config config.toml

Update config.toml:

[hmac]
secret = "your-secret-key"

[upload]
enabled = true
path = "/data/artifacts"

[download]
enabled = true

2. Upload & Download with HMAC

Upload Script

FILE="output.tar.gz"
TS=$(date +%s)
SIG=$(echo -n "$FILE$TS" | openssl dgst -sha256 -hmac "$SECRET" | sed 's/^.* //')
curl -X PUT "$URL/upload/$FILE?ts=$TS&sig=$SIG" --data-binary "@build/$FILE"

Download Script

TS=$(date +%s)
SIG=$(echo -n "$FILE$TS" | openssl dgst -sha256 -hmac "$SECRET" | sed 's/^.* //')
curl -O "$URL/download/$FILE?ts=$TS&sig=$SIG"

3. Using in CI/CD (GitHub Actions)

- name: Build
  run: |
    mkdir -p build
    echo "artifact" > build/output.tar.gz

- name: Upload
  env:
    SECRET: ${{ secrets.HMAC_SECRET }}
  run: bash scripts/upload.sh

Advantages

Secure (HMAC)
Self-hosted
Easy to integrate
No dependencies on third-party providers

30 KiB Raw Blame History Unescape Escape

Table of Contents

Introduction

Configuration

Server Configuration

Configuration Options

Deduplication Settings

Configuration Options

ISO Settings

Configuration Options

Timeout Settings

Configuration Options

Security Settings

Configuration Options

Versioning Settings

Configuration Options

Uploads Settings

Configuration Options

Downloads Settings

Configuration Options

ClamAV Settings

Configuration Options

Redis Settings

Configuration Options

Worker Settings

Configuration Options

Configuration Options

Example Configuration

Setup Instructions

1. HMAC File Server Installation

2. Reverse Proxy Configuration

Apache2 Reverse Proxy

Nginx Reverse Proxy

3. ejabberd Configuration

4. Systemd Service Setup

Building for Different Architectures

Building for Linux (x86_64)

Building for ARM (32-bit)

Building for ARM (64-bit)

Building the Monitoring Tool

Building for Linux (x86_64)

Building for ARM (32-bit)

Building for ARM (64-bit)

Additional Recommendations

Notes

Using HMAC File Server for CI/CD Build Artifacts

Why Use HMAC File Server?

Step 1: Set Up HMAC File Server

Step 2: Generate Signed URLs

Upload Script

Download Script

Step 3: Integrate into CI/CD

GitHub Actions Example

Optional Features

Monitoring

System Data

Prometheus Metrics

Process List

Example Monitoring Output

License

Resources

Version 3.0 Release Note

CI/CD with HMAC File Server – Summary

1. Server Setup

2. Upload & Download with HMAC

Upload Script

Download Script

3. Using in CI/CD (GitHub Actions)

Advantages

30 KiB

Raw Blame History