15 Commits

Author SHA1 Message Date
aaa838a520 feat: Optimize metrics fetching and process list management; enhance UI update efficiency 2025-06-12 18:47:43 +02:00
d005543234 clean: cleanup 2025-06-12 18:38:08 +02:00
519091b8d4 Remove obsolete files and scripts; implement multi-architecture build and Debian package creation for HMAC File Server v3.2 2025-06-12 17:52:00 +02:00
c944dd14b1 feat: Add upload test script and enhance Dockerfile for configuration validation 2025-06-12 16:22:46 +02:00
a71d12a681 feat: Add comprehensive installation verification scripts for HMAC File Server 2025-06-12 15:50:17 +02:00
20e9da0413 feat: Enhance configuration validation system and cleanup repository
- Added comprehensive configuration validation framework in HMAC File Server v3.2.
- Implemented core validation checks including error/warning categorization, resource validation, and network connectivity testing.
- Introduced standalone configuration checker script for interactive validation.
- Enhanced installer and uninstaller with data preservation options and smart directory detection.
- Created production fix and quick fix scripts to address systemd startup issues.
- Consolidated documentation and removed temporary files to streamline repository structure.
- Added test scripts for configuration generation, validation, and custom data selection functionality.
- Updated README and INSTALL documentation to reflect new features and usage instructions.
2025-06-12 15:46:07 +02:00
1fda7ef9be Add test script for validating installer configuration generation
- Introduced `test_installer_config.sh` to validate the configuration generated by the installer.
- The script sets up a test environment, simulates installer input, and generates a configuration file.
- It includes a function to create a sample configuration and validates it against the expected structure.
- Added detailed output for build success and configuration validation results.
- Cleans up the test directory after execution.
2025-06-12 15:34:26 +02:00
c566fbe00a feat: add test script for verifying color formatting in installer completion message 2025-06-12 14:33:49 +02:00
38820c401f feat: fix installation message color formatting and add production fix scripts for systemd startup issues 2025-06-12 14:31:43 +02:00
41a0635b43 feat: enhance installer with ClamAV auto-detection and validation file inclusion 2025-06-12 14:13:47 +02:00
ce42a107dc feat: enhance uninstaller with comprehensive data preservation options and interactive interface 2025-06-12 13:58:35 +02:00
130f877d00 Add test configurations with intentional errors for validation checks
- Created `test_config_errors.toml` with various invalid settings including:
  - Invalid port number and IP address
  - Nonexistent storage path
  - Incorrect formats for max upload size and TTL
  - Negative worker count and zero queue size
  - Invalid Redis address format

- Created `test_errors.toml` with additional intentional errors such as:
  - High invalid port number and empty storage path
  - Weak secret and invalid log level
  - Invalid Redis address and zero scan workers for ClamAV
2025-06-12 13:43:07 +02:00
e34c7dde7e feat: add comprehensive configuration validation and testing framework 2025-06-12 13:19:46 +02:00
6d9b062e58 feat: add health check endpoint for monitoring 2025-06-11 19:06:47 +02:00
65e5907b03 fix: cleanup 2025-06-11 13:09:03 +02:00
20 changed files with 3609 additions and 544 deletions

View File

@ -162,21 +162,60 @@ sudo systemctl reload hmac-file-server # Apply changes
### Uninstallation
To completely remove HMAC File Server:
The HMAC File Server installer includes a comprehensive uninstallation system with advanced data preservation options:
```bash
sudo ./installer.sh --uninstall
```
**What gets removed:**
#### Safe Uninstallation Features
🔒 **Interactive Confirmation System**
- Multiple confirmation steps prevent accidental data loss
- Automatic detection of data directories from configuration
- Smart backup system with timestamped backups in `/var/backups/hmac-file-server-*`
- Detailed reporting showing file counts and directory sizes
#### Five Data Handling Options
**1. 🗑️ Complete Removal**
- Deletes all data including uploads, deduplication files, and logs
- Requires typing "DELETE" for final confirmation
- Provides comprehensive warning about permanent data loss
**2. 💾 Preserve Uploads and Deduplication**
- Preserves critical user files and deduplication data
- Removes logs (typically not needed for data recovery)
- Ideal for system migration or reinstallation
**3. 📋 Preserve All Data**
- Keeps uploads, deduplication data, and logs
- Comprehensive data preservation option
- Best for troubleshooting or temporary removal
**4. 🎯 Custom Selection**
- Interactive selection of which directories to preserve
- Shows detailed information for each directory before decision
- Allows granular control over data preservation
**5. ❌ Cancel Operation**
- Safely exits without making any changes
- No system modifications performed
#### What Gets Removed (Service Components)
- ✓ Systemd service (stopped and disabled)
- ✓ Installation directory (`/opt/hmac-file-server/`)
- ✓ Configuration files (`/etc/hmac-file-server/`)
- ✓ All uploaded files (`/var/lib/hmac-file-server/`)
- ✓ Log files (`/var/log/hmac-file-server/`)
- ✓ System user (`hmac-server`)
- ✓ Any remaining binaries
**⚠️ Warning**: This will permanently remove all uploaded files and configuration!
#### Data Backup Location
When data preservation is selected, files are moved to:
- `/var/backups/hmac-file-server-TIMESTAMP/`
- Timestamped directories for multiple backup versions
- Preserves original directory structure
**⚠️ Important**: The uninstaller provides multiple safety checks and data preservation options. Choose wisely based on your needs!
### Security Considerations

View File

@ -50,7 +50,6 @@ The installer will:
- Create system user and directories
- Build and configure the server
- Set up systemd service
- Configure firewall rules
- Optionally install Redis and ClamAV
For detailed installation instructions, see [INSTALL.MD](INSTALL.MD).
@ -98,6 +97,45 @@ For detailed installation instructions, see [INSTALL.MD](INSTALL.MD).
---
## Uninstallation
The installer script provides comprehensive uninstallation options with data preservation:
```bash
sudo ./installer.sh --uninstall
```
### Uninstall Options
The uninstaller offers five data handling options:
1. **🗑️ Delete all data** - Complete removal (requires typing "DELETE" to confirm)
2. **💾 Preserve uploads and deduplication data** - Keeps important files, removes logs
3. **📋 Preserve all data** - Keeps uploads, deduplication data, and logs
4. **🎯 Custom selection** - Choose exactly what to preserve
5. **❌ Cancel** - Exit without making changes
### Data Preservation
When preserving data, the uninstaller:
- Creates timestamped backups in `/var/backups/hmac-file-server-YYYYMMDD-HHMMSS/`
- Shows file counts and sizes before deletion decisions
- Safely moves data to backup locations
- Provides clear feedback on what was preserved or removed
### What Gets Removed
The uninstaller always removes:
- ✓ Systemd service and service file
- ✓ Installation directory (`/opt/hmac-file-server`)
- ✓ Configuration directory (`/etc/hmac-file-server`)
- ✓ System user (`hmac-server`)
- ✓ Binary files in common locations
Data directories are handled according to your selection.
---
## Configuration
The server uses a comprehensive `config.toml` file with the following main sections:
@ -189,6 +227,7 @@ curl -X PUT "http://localhost:8080/upload/file.txt?v3=SIGNATURE&expires=TIMESTAM
- **GET /download/{filename}**: Alternative download endpoint
### Management Endpoints
- **GET /health**: Health check endpoint for monitoring
- **GET /metrics**: Prometheus metrics (if enabled)
- **Various helper endpoints**: Defined in router setup
@ -239,6 +278,11 @@ curl http://localhost:8080/example.jpg -o downloaded_file.jpg
curl -I http://localhost:8080/example.jpg
```
#### Health Check
```bash
curl http://localhost:8080/health
```
---
## Setup

134
WIKI.MD
View File

@ -560,6 +560,140 @@ uploadqueuesize = 5000 # Size of upload queue
---
## Configuration Validation
The HMAC File Server v3.2 includes a comprehensive configuration validation system with specialized command-line flags for different validation scenarios.
### Available Validation Flags
#### Core Validation Commands
**`--validate-config`**
- **Purpose**: Full comprehensive validation of all configuration sections
- **Usage**: `./hmac-file-server --validate-config`
- **Output**: Complete validation report with all errors and warnings
```bash
# Example
./hmac-file-server -config config.toml --validate-config
```
**`--test-config`**
- **Purpose**: Run predefined configuration test scenarios
- **Usage**: `./hmac-file-server --test-config`
- **Output**: Test scenario results for configuration validation
#### Specialized Validation Modes
**`--check-security`**
- **Purpose**: Security-focused validation only
- **Checks**: Secret strength, default values, JWT algorithms, network exposure, file permissions
- **Example**: `./hmac-file-server -config config.toml --check-security`
**`--check-performance`**
- **Purpose**: Performance-focused validation only
- **Checks**: Worker optimization, memory usage, timeout balance, large file handling
- **Example**: `./hmac-file-server -config config.toml --check-performance`
**`--check-connectivity`**
- **Purpose**: Network connectivity validation only
- **Checks**: Redis connections, ClamAV sockets, address validation, DNS resolution
- **Example**: `./hmac-file-server -config config.toml --check-connectivity`
#### Output Control Flags
**`--validate-quiet`**
- **Purpose**: Minimal output, returns only exit codes
- **Usage**: Perfect for automation and scripts
**`--validate-verbose`**
- **Purpose**: Detailed output with comprehensive analysis
- **Usage**: Best for troubleshooting and development
**`--check-fixable`**
- **Purpose**: Show only issues that can be automatically fixed
- **Usage**: Helps prioritize configuration improvements
### Validation Categories
#### Security Checks (6 categories)
- Secret strength analysis
- Default value detection
- Algorithm recommendations
- Network exposure warnings
- File permission analysis
- Debug logging security
#### Performance Checks (5 categories)
- Resource optimization
- Memory usage analysis
- Timeout balancing
- Large file preparation
- Configuration efficiency
#### Connectivity Checks (4 categories)
- Service connectivity
- Socket accessibility
- Address validation
- DNS resolution
#### System Checks (5 categories)
- CPU availability
- Memory monitoring
- Disk space validation
- Permission testing
- Resource constraints
### Integration Examples
#### Shell Script Integration
```bash
#!/bin/bash
CONFIG_FILE="/etc/hmac-file-server/config.toml"
echo "🔍 Validating HMAC File Server configuration..."
# Run validation
if ./hmac-file-server -config "$CONFIG_FILE" --validate-config; then
echo "✅ Configuration validation passed"
# Additional specific checks
echo "🔐 Running security audit..."
./hmac-file-server -config "$CONFIG_FILE" --check-security
echo "⚡ Checking performance settings..."
./hmac-file-server -config "$CONFIG_FILE" --check-performance
else
echo "❌ Configuration validation failed"
echo "💡 Try: ./hmac-file-server -config $CONFIG_FILE --check-fixable"
exit 1
fi
```
#### Docker Integration
```dockerfile
# Add validation step to Dockerfile
RUN ./hmac-file-server -config /etc/config.toml --validate-config && \
./hmac-file-server -config /etc/config.toml --check-security
```
#### Kubernetes Health Check
```yaml
livenessProbe:
exec:
command:
- /usr/local/bin/hmac-file-server
- -config
- /etc/config/config.toml
- --validate-quiet
initialDelaySeconds: 30
periodSeconds: 60
```
The enhanced command-line validation system provides comprehensive coverage with 50+ validation checks across all configuration areas, making HMAC File Server v3.2 production-ready with enterprise-grade configuration management.
---
## Example Configuration
Below is an example `config.toml` file with default settings:

196
build-multi-arch.sh Executable file
View File

@ -0,0 +1,196 @@
#!/bin/bash
# HMAC File Server v3.2 - Multi-Architecture Build Script
# Compiles binaries for AMD64, ARM64, and ARM32 architectures
# Remove set -e to prevent early exit on errors
# Colors for output
GREEN='\033[0;32m'
BLUE='\033[0;34m'
YELLOW='\033[1;33m'
RED='\033[0;31m'
CYAN='\033[0;36m'
NC='\033[0m'
print_status() {
echo -e "${GREEN}[BUILD]${NC} $1"
}
print_info() {
echo -e "${BLUE}[INFO]${NC} $1"
}
print_warning() {
echo -e "${YELLOW}[WARN]${NC} $1"
}
print_error() {
echo -e "${RED}[ERROR]${NC} $1"
}
print_arch() {
echo -e "${CYAN}[ARCH]${NC} $1"
}
# Check if Go is installed
if ! command -v go &> /dev/null; then
print_error "Go is not installed or not in PATH"
exit 1
fi
# Create temp directory if it doesn't exist
TEMP_DIR="./temp"
if [[ ! -d "$TEMP_DIR" ]]; then
mkdir -p "$TEMP_DIR"
print_info "Created temp directory: $TEMP_DIR"
fi
# Source files to compile
SOURCE_FILES="cmd/server/main.go cmd/server/helpers.go cmd/server/config_validator.go cmd/server/config_test_scenarios.go"
print_status "Starting multi-architecture build for HMAC File Server v3.2"
print_info "Source files: $SOURCE_FILES"
print_info "Output directory: $TEMP_DIR"
echo ""
# Build function
build_for_arch() {
local goos=$1
local goarch=$2
local output_name=$3
local arch_description=$4
print_arch "Building for $arch_description ($goos/$goarch)..."
# Set environment variables for cross-compilation
export GOOS=$goos
export GOARCH=$goarch
export CGO_ENABLED=0
# Build the binary
if go build -ldflags="-w -s" -o "$TEMP_DIR/$output_name" $SOURCE_FILES 2>/dev/null; then
# Get file size
if [[ "$OSTYPE" == "darwin"* ]]; then
# macOS
SIZE=$(stat -f%z "$TEMP_DIR/$output_name" | awk '{printf "%.1fMB", $1/1024/1024}')
else
# Linux
SIZE=$(stat -c%s "$TEMP_DIR/$output_name" | awk '{printf "%.1fMB", $1/1024/1024}')
fi
print_status "Build successful: $arch_description"
print_info " Binary: $TEMP_DIR/$output_name"
print_info " Size: $SIZE"
# Test binary (version check)
if timeout 10s "$TEMP_DIR/$output_name" --version >/dev/null 2>&1; then
print_info " Version check: PASSED"
else
print_warning " Version check: SKIPPED (cross-compiled binary)"
fi
return 0
else
print_error "Build failed: $arch_description"
return 1
fi
}
# Track build results
BUILDS_ATTEMPTED=0
BUILDS_SUCCESSFUL=0
FAILED_BUILDS=()
echo "Starting builds..."
echo "===================="
echo ""
# Build for AMD64 (x86_64)
print_arch "AMD64 (Intel/AMD 64-bit)"
((BUILDS_ATTEMPTED++))
if build_for_arch "linux" "amd64" "hmac-file-server-linux-amd64" "AMD64 Linux"; then
((BUILDS_SUCCESSFUL++))
else
FAILED_BUILDS+=("AMD64")
fi
echo ""
# Build for ARM64 (AArch64)
print_arch "ARM64 (AArch64)"
((BUILDS_ATTEMPTED++))
if build_for_arch "linux" "arm64" "hmac-file-server-linux-arm64" "ARM64 Linux"; then
((BUILDS_SUCCESSFUL++))
else
FAILED_BUILDS+=("ARM64")
fi
echo ""
# Build for ARM32 (ARMv7)
print_arch "ARM32 (ARMv7)"
export GOARM=7 # ARMv7 with hardware floating point
((BUILDS_ATTEMPTED++))
if build_for_arch "linux" "arm" "hmac-file-server-linux-arm32" "ARM32 Linux"; then
((BUILDS_SUCCESSFUL++))
else
FAILED_BUILDS+=("ARM32")
fi
echo ""
# Reset environment variables
unset GOOS GOARCH CGO_ENABLED GOARM
# Build summary
echo "Build Summary"
echo "================"
print_info "Builds attempted: $BUILDS_ATTEMPTED"
print_info "Builds successful: $BUILDS_SUCCESSFUL"
if [[ $BUILDS_SUCCESSFUL -eq $BUILDS_ATTEMPTED ]]; then
print_status "ALL BUILDS SUCCESSFUL!"
echo ""
print_info "Generated binaries in $TEMP_DIR:"
ls -lh "$TEMP_DIR"/hmac-file-server-* | while read -r line; do
echo " $line"
done
echo ""
print_info "Usage examples:"
echo " - Copy to target system and run: ./hmac-file-server-linux-amd64 --version"
echo " - Deploy with installer: cp temp/hmac-file-server-linux-amd64 /opt/hmac-file-server/"
echo " - Docker deployment: COPY temp/hmac-file-server-linux-amd64 /usr/local/bin/"
elif [[ $BUILDS_SUCCESSFUL -gt 0 ]]; then
print_warning "PARTIAL SUCCESS: $BUILDS_SUCCESSFUL/$BUILDS_ATTEMPTED builds completed"
if [[ ${#FAILED_BUILDS[@]} -gt 0 ]]; then
print_error "Failed architectures: ${FAILED_BUILDS[*]}"
fi
else
print_error "ALL BUILDS FAILED!"
exit 1
fi
echo ""
print_info "Architecture compatibility:"
echo " - AMD64: Intel/AMD 64-bit servers, desktops, cloud instances"
echo " - ARM64: Apple Silicon, AWS Graviton, modern ARM servers"
echo " - ARM32: Raspberry Pi, embedded systems, older ARM devices"
echo ""
print_status "Multi-architecture build completed!"
# Final verification
echo ""
print_info "Final verification:"
for binary in "$TEMP_DIR"/hmac-file-server-*; do
if [[ -f "$binary" ]]; then
filename=$(basename "$binary")
if file "$binary" >/dev/null 2>&1; then
file_info=$(file "$binary" | cut -d: -f2- | sed 's/^ *//')
print_info " OK $filename: $file_info"
else
print_info " OK $filename: Binary file"
fi
fi
done
exit 0

407
builddebian.sh Executable file
View File

@ -0,0 +1,407 @@
#!/bin/bash
# HMAC File Server v3.2 - Debian Package Builder
# Creates .deb packages for AMD64 and ARM64 architectures
set -e
# Colors for output
GREEN='\033[0;32m'
BLUE='\033[0;34m'
YELLOW='\033[1;33m'
RED='\033[0;31m'
NC='\033[0m'
print_status() {
echo -e "${GREEN}[BUILD]${NC} $1"
}
print_info() {
echo -e "${BLUE}[INFO]${NC} $1"
}
print_warning() {
echo -e "${YELLOW}[WARN]${NC} $1"
}
print_error() {
echo -e "${RED}[ERROR]${NC} $1"
}
# Project configuration
PROJECT_DIR=$(pwd)
BUILD_DIR=$PROJECT_DIR/build
DEB_DIR=$PROJECT_DIR/debian
PACKAGE_NAME="hmac-file-server"
VERSION="3.2.0"
MAINTAINER="Alex Renz <renz@uuxo.net>"
# Source files for compilation
SOURCE_FILES="cmd/server/main.go cmd/server/helpers.go cmd/server/config_validator.go cmd/server/config_test_scenarios.go"
print_status "Starting Debian package build for HMAC File Server v$VERSION"
print_info "Building packages for: AMD64, ARM64"
# Check if Go is installed
if ! command -v go &> /dev/null; then
print_error "Go is not installed or not in PATH"
exit 1
fi
# Check if dpkg-deb is available
if ! command -v dpkg-deb &> /dev/null; then
print_error "dpkg-deb is not installed. Please install dpkg-dev package"
exit 1
fi
# Clean and create required directories
print_info "Setting up build directories..."
rm -rf $BUILD_DIR $DEB_DIR
mkdir -p $BUILD_DIR/{amd64,arm64}
mkdir -p $DEB_DIR/DEBIAN
mkdir -p $DEB_DIR/usr/local/bin
mkdir -p $DEB_DIR/etc/hmac-file-server
mkdir -p $DEB_DIR/var/lib/hmac-file-server/{uploads,deduplication,runtime}
mkdir -p $DEB_DIR/var/log/hmac-file-server
mkdir -p $DEB_DIR/usr/share/doc/hmac-file-server
mkdir -p $DEB_DIR/lib/systemd/system
# Compile Go binaries for both architectures
print_status "Compiling binaries..."
for ARCH in amd64 arm64; do
print_info "Building for $ARCH..."
# Set cross-compilation environment
export GOOS=linux
export GOARCH=$ARCH
export CGO_ENABLED=0
# Build hmac-file-server
if go build -ldflags="-w -s" -o $BUILD_DIR/$ARCH/hmac-file-server $SOURCE_FILES; then
SIZE=$(stat -c%s "$BUILD_DIR/$ARCH/hmac-file-server" | awk '{printf "%.1fMB", $1/1024/1024}')
print_info " $ARCH binary built successfully ($SIZE)"
else
print_error "Failed to build $ARCH binary"
exit 1
fi
done
# Reset environment variables
unset GOOS GOARCH CGO_ENABLED
# Prepare Debian control file template
print_info "Creating package metadata..."
CONTROL_TEMPLATE=$DEB_DIR/DEBIAN/control.template
cat <<EOF > $CONTROL_TEMPLATE
Package: $PACKAGE_NAME
Version: $VERSION
Architecture: ARCH_PLACEHOLDER
Maintainer: $MAINTAINER
Depends: redis-server, clamav, clamav-daemon
Recommends: nginx
Section: net
Priority: optional
Homepage: https://github.com/PlusOne/hmac-file-server
Description: HMAC File Server v3.2 - Enterprise XMPP File Sharing
A lightweight, secure file server designed for XMPP environments with
enterprise-grade features including:
.
* HMAC-based authentication and JWT support
* Redis integration for session management
* ClamAV virus scanning for uploaded files
* Prometheus metrics for monitoring
* Chunked upload/download support
* File deduplication capabilities
* Comprehensive configuration validation
.
Perfect for Prosody, Ejabberd, and other XMPP servers requiring
secure file sharing capabilities with professional deployment features.
EOF
# Prepare systemd service file
print_info "Creating systemd service configuration..."
cat <<EOF > $DEB_DIR/lib/systemd/system/hmac-file-server.service
[Unit]
Description=HMAC File Server 3.2
Documentation=https://github.com/PlusOne/hmac-file-server
After=network.target
Wants=network-online.target
After=redis.service
After=clamav-daemon.service
[Service]
Type=simple
User=hmac-file-server
Group=hmac-file-server
ExecStart=/usr/local/bin/hmac-file-server -config /etc/hmac-file-server/config.toml
ExecReload=/bin/kill -SIGHUP \$MAINPID
WorkingDirectory=/var/lib/hmac-file-server
Restart=always
RestartSec=10
StandardOutput=journal
StandardError=journal
SyslogIdentifier=hmac-file-server
# Security settings
NoNewPrivileges=true
PrivateTmp=true
ProtectSystem=strict
ProtectHome=true
ReadWritePaths=/var/lib/hmac-file-server /var/log/hmac-file-server
CapabilityBoundingSet=CAP_NET_BIND_SERVICE
AmbientCapabilities=CAP_NET_BIND_SERVICE
# Resource limits
LimitNOFILE=65536
LimitNPROC=4096
[Install]
WantedBy=multi-user.target
EOF
# Prepare example configuration file
print_info "Creating example configuration..."
cat <<EOF > $DEB_DIR/etc/hmac-file-server/config.toml
# HMAC File Server v3.2 Configuration
# Complete configuration reference: https://github.com/PlusOne/hmac-file-server/blob/main/WIKI.MD
[server]
bind_ip = "127.0.0.1"
listenport = "8080"
unixsocket = false
storagepath = "/var/lib/hmac-file-server/uploads"
metricsenabled = true
metricsport = "9090"
deduplicationenabled = true
deduplicationpath = "/var/lib/hmac-file-server/deduplication"
filenaming = "HMAC"
force_protocol = "auto"
sslenabled = false
pidfilepath = "/var/lib/hmac-file-server/runtime/hmac-file-server.pid"
[security]
secret = "CHANGE_THIS_SECRET_IN_PRODUCTION_USE_48_CHARS_MIN"
enablejwt = false
jwtsecret = ""
jwtalgorithm = "HS256"
jwtexpiration = "24h"
[uploads]
allowedextensions = [".txt", ".pdf", ".jpg", ".jpeg", ".png", ".gif", ".webp", ".zip", ".tar", ".gz", ".7z", ".mp4", ".webm", ".ogg", ".mp3", ".wav", ".flac", ".doc", ".docx", ".xls", ".xlsx", ".ppt", ".pptx", ".odt", ".ods", ".odp"]
maxfilesize = "100MB"
chunkeduploadsenabled = true
chunksize = "10MB"
resumableuploadsenabled = true
ttlenabled = false
ttl = "168h"
[downloads]
chunkeddownloadsenabled = true
chunksize = "10MB"
[logging]
level = "INFO"
file = "/var/log/hmac-file-server/hmac-file-server.log"
max_size = 100
max_backups = 3
max_age = 30
compress = true
[workers]
numworkers = 10
uploadqueuesize = 1000
autoscaling = true
[timeouts]
readtimeout = "30s"
writetimeout = "30s"
idletimeout = "120s"
shutdown = "30s"
[clamav]
enabled = false
socket = "/var/run/clamav/clamd.ctl"
timeout = "30s"
[redis]
enabled = false
address = "localhost:6379"
database = 0
password = ""
EOF
# Prepare post-installation script
print_info "Creating installation scripts..."
cat <<EOF > $DEB_DIR/DEBIAN/postinst
#!/bin/bash
set -e
# Create hmac-file-server user and group if they do not exist
if ! id -u hmac-file-server >/dev/null 2>&1; then
useradd --system --no-create-home --shell /usr/sbin/nologin --home-dir /var/lib/hmac-file-server hmac-file-server
echo "Created system user: hmac-file-server"
fi
# Set proper ownership and permissions
chown -R hmac-file-server:hmac-file-server /var/lib/hmac-file-server
chown -R hmac-file-server:hmac-file-server /var/log/hmac-file-server
chown hmac-file-server:hmac-file-server /etc/hmac-file-server/config.toml
# Set directory permissions
chmod 755 /var/lib/hmac-file-server
chmod 755 /var/lib/hmac-file-server/uploads
chmod 755 /var/lib/hmac-file-server/deduplication
chmod 755 /var/lib/hmac-file-server/runtime
chmod 755 /var/log/hmac-file-server
chmod 640 /etc/hmac-file-server/config.toml
# Reload systemd and enable service
systemctl daemon-reload
systemctl enable hmac-file-server.service
echo ""
echo "HMAC File Server v3.2 installed successfully!"
echo ""
echo "Next steps:"
echo "1. Edit /etc/hmac-file-server/config.toml (CHANGE THE SECRET!)"
echo "2. Enable Redis/ClamAV if needed: systemctl enable redis-server clamav-daemon"
echo "3. Start the service: systemctl start hmac-file-server"
echo "4. Check status: systemctl status hmac-file-server"
echo ""
echo "Documentation: https://github.com/PlusOne/hmac-file-server"
echo ""
EOF
chmod 0755 $DEB_DIR/DEBIAN/postinst
# Prepare pre-removal script
cat <<EOF > $DEB_DIR/DEBIAN/prerm
#!/bin/bash
set -e
# Stop the service before removal
if systemctl is-active --quiet hmac-file-server.service; then
echo "Stopping HMAC File Server service..."
systemctl stop hmac-file-server.service || true
fi
EOF
chmod 0755 $DEB_DIR/DEBIAN/prerm
# Prepare post-removal script
cat <<EOF > $DEB_DIR/DEBIAN/postrm
#!/bin/bash
set -e
case "\$1" in
purge)
# Remove systemd service
systemctl disable hmac-file-server.service >/dev/null 2>&1 || true
rm -f /lib/systemd/system/hmac-file-server.service
systemctl daemon-reload >/dev/null 2>&1 || true
# Remove user and group
if id -u hmac-file-server >/dev/null 2>&1; then
userdel hmac-file-server || true
fi
if getent group hmac-file-server >/dev/null 2>&1; then
groupdel hmac-file-server || true
fi
# Remove data directories (ask user)
echo ""
echo "HMAC File Server has been removed."
echo "Data directories remain at:"
echo " - /var/lib/hmac-file-server/"
echo " - /var/log/hmac-file-server/"
echo " - /etc/hmac-file-server/"
echo ""
echo "Remove them manually if no longer needed:"
echo " sudo rm -rf /var/lib/hmac-file-server"
echo " sudo rm -rf /var/log/hmac-file-server"
echo " sudo rm -rf /etc/hmac-file-server"
echo ""
;;
remove)
# Just disable service
systemctl disable hmac-file-server.service >/dev/null 2>&1 || true
systemctl daemon-reload >/dev/null 2>&1 || true
;;
esac
EOF
chmod 0755 $DEB_DIR/DEBIAN/postrm
# Prepare documentation
print_info "Including documentation..."
cp README.MD $DEB_DIR/usr/share/doc/hmac-file-server/
cp INSTALL.MD $DEB_DIR/usr/share/doc/hmac-file-server/
cp WIKI.MD $DEB_DIR/usr/share/doc/hmac-file-server/
cp CHANGELOG.MD $DEB_DIR/usr/share/doc/hmac-file-server/
cp config-example-xmpp.toml $DEB_DIR/usr/share/doc/hmac-file-server/
# Create .deb packages
print_status "Building Debian packages..."
for ARCH in amd64 arm64; do
print_info "Creating package for $ARCH..."
# Update control file for the current architecture
sed "s/ARCH_PLACEHOLDER/$ARCH/" $CONTROL_TEMPLATE > $DEB_DIR/DEBIAN/control
# Copy binary for current architecture
cp $BUILD_DIR/$ARCH/hmac-file-server $DEB_DIR/usr/local/bin/
# Calculate installed size
INSTALLED_SIZE=$(du -sk $DEB_DIR | cut -f1)
echo "Installed-Size: $INSTALLED_SIZE" >> $DEB_DIR/DEBIAN/control
# Ensure proper permissions
find $DEB_DIR -type d -exec chmod 755 {} \;
find $DEB_DIR -type f -exec chmod 644 {} \;
chmod 0755 $DEB_DIR/usr/local/bin/hmac-file-server
chmod 0755 $DEB_DIR/DEBIAN/postinst
chmod 0755 $DEB_DIR/DEBIAN/prerm
chmod 0755 $DEB_DIR/DEBIAN/postrm
# Build the .deb package
PACKAGE_FILE="${PACKAGE_NAME}_${VERSION}_${ARCH}.deb"
if dpkg-deb --build $DEB_DIR $PACKAGE_FILE; then
SIZE=$(stat -c%s "$PACKAGE_FILE" | awk '{printf "%.1fMB", $1/1024/1024}')
print_info " Package created: $PACKAGE_FILE ($SIZE)"
else
print_error "Failed to create package for $ARCH"
exit 1
fi
# Clean up binary for next build
rm -f $DEB_DIR/usr/local/bin/hmac-file-server
rm -f $DEB_DIR/DEBIAN/control
done
# Cleanup temporary directories
print_info "Cleaning up build directories..."
rm -rf $BUILD_DIR $DEB_DIR
# Show results
print_status "Debian package build completed!"
echo ""
print_info "Generated packages:"
for PACKAGE in ${PACKAGE_NAME}_${VERSION}_*.deb; do
if [[ -f "$PACKAGE" ]]; then
SIZE=$(stat -c%s "$PACKAGE" | awk '{printf "%.1fMB", $1/1024/1024}')
print_info " $PACKAGE ($SIZE)"
fi
done
echo ""
print_info "Installation commands:"
echo " sudo dpkg -i ${PACKAGE_NAME}_${VERSION}_amd64.deb"
echo " sudo dpkg -i ${PACKAGE_NAME}_${VERSION}_arm64.deb"
echo ""
print_info "Package information:"
echo " dpkg -I ${PACKAGE_NAME}_${VERSION}_amd64.deb"
echo " dpkg -c ${PACKAGE_NAME}_${VERSION}_amd64.deb"
echo ""
print_warning "Remember to:"
echo " 1. Edit /etc/hmac-file-server/config.toml"
echo " 2. Change the default secret"
echo " 3. Configure Redis/ClamAV if needed"
echo " 4. Start the service: systemctl start hmac-file-server"
exit 0

View File

@ -29,8 +29,8 @@ if ! command -v go &> /dev/null; then
fi
# Build the application
print_status "Building HMAC File Server..."
go build -o hmac-file-server cmd/server/main.go cmd/server/helpers.go
print_status "Building HMAC File Server v3.2..."
go build -o hmac-file-server cmd/server/main.go cmd/server/helpers.go cmd/server/config_validator.go cmd/server/config_test_scenarios.go
if [ $? -eq 0 ]; then
print_status "Build successful! Binary created: ./hmac-file-server"

View File

@ -1,18 +1,18 @@
package main
import (
"bufio"
"context"
"fmt"
"io"
"log"
"net/http"
"os"
"sort"
"strconv"
"strings"
"time"
"context"
"io"
"sync"
"bufio"
"time"
"github.com/gdamore/tcell/v2"
"github.com/pelletier/go-toml"
@ -24,11 +24,11 @@ import (
)
var (
prometheusURL string
configFilePath string // Pfad der gefundenen Konfiguration
logFilePath string // Pfad der Logdatei aus der Konfiguration
metricsEnabled bool // Neue Variable für die Aktivierung von Metriken
bindIP string // Neue Variable für die gebundene IP-Adresse
prometheusURL string
configFilePath string // Pfad der gefundenen Konfiguration
logFilePath string // Pfad der Logdatei aus der Konfiguration
metricsEnabled bool // Neue Variable für die Aktivierung von Metriken
bindIP string // Neue Variable für die gebundene IP-Adresse
)
func init() {
@ -128,68 +128,94 @@ const (
// ProcessInfo holds information about a process
type ProcessInfo struct {
PID int32
Name string
CPUPercent float64
MemPercent float32
CommandLine string
Uptime string // Neues Feld für die Uptime
Status string // Neues Feld für den Status
ErrorCount int // Neues Feld für die Anzahl der Fehler
TotalRequests int64 // Neues Feld für die Gesamtanzahl der Anfragen
ActiveConnections int // Neues Feld für aktive Verbindungen
AverageResponseTime float64 // Neues Feld für die durchschnittliche Antwortzeit in Millisekunden
PID int32
Name string
CPUPercent float64
MemPercent float32
CommandLine string
Uptime string // Neues Feld für die Uptime
Status string // Neues Feld für den Status
ErrorCount int // Neues Feld für die Anzahl der Fehler
TotalRequests int64 // Neues Feld für die Gesamtanzahl der Anfragen
ActiveConnections int // Neues Feld für aktive Verbindungen
AverageResponseTime float64 // Neues Feld für die durchschnittliche Antwortzeit in Millisekunden
}
// Function to fetch and parse Prometheus metrics
// Optimized metrics fetching with timeout and connection reuse
func fetchMetrics() (map[string]float64, error) {
resp, err := http.Get(prometheusURL)
// Create HTTP client with timeout and connection reuse
client := &http.Client{
Timeout: 5 * time.Second,
Transport: &http.Transport{
MaxIdleConns: 10,
IdleConnTimeout: 30 * time.Second,
DisableCompression: true,
},
}
resp, err := client.Get(prometheusURL)
if err != nil {
return nil, fmt.Errorf("failed to fetch metrics: %w", err)
}
defer resp.Body.Close()
// Limit response body size to prevent memory issues
limitedReader := io.LimitReader(resp.Body, 1024*1024) // 1MB limit
parser := &expfmt.TextParser{}
metricFamilies, err := parser.TextToMetricFamilies(resp.Body)
metricFamilies, err := parser.TextToMetricFamilies(limitedReader)
if err != nil {
return nil, fmt.Errorf("failed to parse metrics: %w", err)
}
metrics := make(map[string]float64)
// More selective metric filtering to reduce processing
relevantPrefixes := []string{
"hmac_file_server_",
"memory_usage_bytes",
"cpu_usage_percent",
"active_connections_total",
"goroutines_count",
"total_requests",
"average_response_time_ms",
}
for name, mf := range metricFamilies {
// Filter the metrics you're interested in
if strings.HasPrefix(name, "hmac_file_server_") ||
name == "memory_usage_bytes" ||
name == "cpu_usage_percent" ||
name == "active_connections_total" ||
name == "goroutines_count" ||
name == "total_requests" ||
name == "average_response_time_ms" {
// Quick prefix check to skip irrelevant metrics
relevant := false
for _, prefix := range relevantPrefixes {
if strings.HasPrefix(name, prefix) || name == prefix {
relevant = true
break
}
}
if !relevant {
continue
}
for _, m := range mf.GetMetric() {
var value float64
if m.GetGauge() != nil {
value = m.GetGauge().GetValue()
} else if m.GetCounter() != nil {
value = m.GetCounter().GetValue()
} else if m.GetUntyped() != nil {
value = m.GetUntyped().GetValue()
} else {
// If the metric type is not handled, skip it
continue
}
for _, m := range mf.GetMetric() {
var value float64
if m.GetGauge() != nil {
value = m.GetGauge().GetValue()
} else if m.GetCounter() != nil {
value = m.GetCounter().GetValue()
} else if m.GetUntyped() != nil {
value = m.GetUntyped().GetValue()
} else {
continue
}
// Handle metrics with labels
if len(m.GetLabel()) > 0 {
labels := make([]string, 0)
for _, label := range m.GetLabel() {
labels = append(labels, fmt.Sprintf("%s=\"%s\"", label.GetName(), label.GetValue()))
}
metricKey := fmt.Sprintf("%s{%s}", name, strings.Join(labels, ","))
metrics[metricKey] = value
} else {
metrics[name] = value
// Simplified label handling
if len(m.GetLabel()) > 0 {
labels := make([]string, 0, len(m.GetLabel()))
for _, label := range m.GetLabel() {
labels = append(labels, fmt.Sprintf("%s=\"%s\"", label.GetName(), label.GetValue()))
}
metricKey := fmt.Sprintf("%s{%s}", name, strings.Join(labels, ","))
metrics[metricKey] = value
} else {
metrics[name] = value
}
}
}
@ -222,47 +248,82 @@ func fetchSystemData() (float64, float64, int, error) {
return v.UsedPercent, cpuUsage, cores, nil
}
// Funktion zum Abrufen der Prozessliste mit paralleler Verarbeitung
// Optimized process list fetching with better resource management
func fetchProcessList() ([]ProcessInfo, error) {
processes, err := process.Processes()
if err != nil {
return nil, fmt.Errorf("failed to fetch processes: %w", err)
}
var processList []ProcessInfo
// Pre-allocate slice with reasonable capacity
processList := make([]ProcessInfo, 0, len(processes))
var mu sync.Mutex
var wg sync.WaitGroup
// Begrenzung der gleichzeitigen Goroutinen auf 10
sem := make(chan struct{}, 10)
// Limit concurrent goroutines to prevent resource exhaustion
sem := make(chan struct{}, 5) // Reduced from 10 to 5
timeout := time.After(10 * time.Second) // Add timeout
// Process only a subset of processes to reduce load
maxProcesses := 200
if len(processes) > maxProcesses {
processes = processes[:maxProcesses]
}
for _, p := range processes {
select {
case <-timeout:
log.Printf("Process list fetch timeout, returning partial results")
return processList, nil
default:
}
wg.Add(1)
sem <- struct{}{} // Eintritt in semaphor
sem <- struct{}{} // Enter semaphore
go func(p *process.Process) {
defer wg.Done()
defer func() { <-sem }() // Austritt aus semaphor
defer func() {
<-sem // Exit semaphore
// Recover from any panics in process info fetching
if r := recover(); r != nil {
log.Printf("Process info fetch panic: %v", r)
}
}()
cpuPercent, err := p.CPUPercent()
// Set shorter timeout for individual process operations
ctx, cancel := context.WithTimeout(context.Background(), 1*time.Second)
defer cancel()
// Use context for process operations where possible
cpuPercent, err := p.CPUPercentWithContext(ctx)
if err != nil {
return
}
memPercent, err := p.MemoryPercent()
memPercent, err := p.MemoryPercentWithContext(ctx)
if err != nil {
return
}
name, err := p.Name()
name, err := p.NameWithContext(ctx)
if err != nil {
return
}
cmdline, err := p.Cmdline()
// Skip if CPU and memory usage are both very low to reduce noise
if cpuPercent < 0.1 && memPercent < 0.1 {
return
}
// Limit command line length to prevent memory bloat
cmdline, err := p.CmdlineWithContext(ctx)
if err != nil {
cmdline = ""
}
if len(cmdline) > 100 {
cmdline = cmdline[:100] + "..."
}
info := ProcessInfo{
PID: p.Pid,
@ -278,7 +339,20 @@ func fetchProcessList() ([]ProcessInfo, error) {
}(p)
}
wg.Wait()
// Wait with timeout
done := make(chan struct{})
go func() {
wg.Wait()
close(done)
}()
select {
case <-done:
// All goroutines completed
case <-time.After(15 * time.Second):
log.Printf("Process list fetch timeout after 15 seconds, returning partial results")
}
return processList, nil
}
@ -369,201 +443,245 @@ func fetchHmacFileServerInfo() (*ProcessInfo, error) {
return nil, fmt.Errorf("hmac-file-server process not found")
}
// Neue Funktion zur Zählung der Fehler in den Logs
// Optimized error counting with caching and limits
var (
errorCountCache int
errorCountCacheTime time.Time
errorCountMutex sync.RWMutex
)
func countHmacErrors() (int, error) {
logFilePath := "/var/log/hmac-file-server.log" // Pfad zur Logdatei
// Use cached value if recent (within 30 seconds)
errorCountMutex.RLock()
if time.Since(errorCountCacheTime) < 30*time.Second {
count := errorCountCache
errorCountMutex.RUnlock()
return count, nil
}
errorCountMutex.RUnlock()
// Use the configured log file path
file, err := os.Open(logFilePath)
if err != nil {
return 0, err
}
defer file.Close()
// Get file size to limit reading for very large files
stat, err := file.Stat()
if err != nil {
return 0, err
}
// Limit to last 1MB for large log files
var startPos int64 = 0
if stat.Size() > 1024*1024 {
startPos = stat.Size() - 1024*1024
file.Seek(startPos, io.SeekStart)
}
scanner := bufio.NewScanner(file)
errorCount := 0
for scanner.Scan() {
lineCount := 0
maxLines := 1000 // Limit lines scanned
for scanner.Scan() && lineCount < maxLines {
line := scanner.Text()
if strings.Contains(line, "level=error") {
errorCount++
}
lineCount++
}
if err := scanner.Err(); err != nil {
return 0, err
}
// Update cache
errorCountMutex.Lock()
errorCountCache = errorCount
errorCountCacheTime = time.Now()
errorCountMutex.Unlock()
return errorCount, nil
}
// Funktion zur Aktualisierung der UI mit paralleler Datenbeschaffung
// Optimized data structure for caching
type cachedData struct {
systemData systemData
metrics map[string]float64
processes []ProcessInfo
hmacInfo *ProcessInfo
lastUpdate time.Time
mu sync.RWMutex
}
type systemData struct {
memUsage float64
cpuUsage float64
cores int
}
var cache = &cachedData{}
// Optimized updateUI with reduced frequency and better resource management
func updateUI(ctx context.Context, app *tview.Application, pages *tview.Pages, sysPage, hmacPage tview.Primitive) {
ticker := time.NewTicker(2 * time.Second)
defer ticker.Stop()
// Reduce update frequency significantly
fastTicker := time.NewTicker(5 * time.Second) // UI updates
slowTicker := time.NewTicker(15 * time.Second) // Process list updates
defer fastTicker.Stop()
defer slowTicker.Stop()
// Einführung von Channels für verschiedene Daten
systemDataCh := make(chan struct {
memUsage float64
cpuUsage float64
cores int
err error
})
var metricsCh chan struct {
metrics map[string]float64
err error
}
if metricsEnabled {
metricsCh = make(chan struct {
metrics map[string]float64
err error
})
}
processListCh := make(chan struct {
processes []ProcessInfo
err error
})
hmacInfoCh := make(chan struct {
info *ProcessInfo
metrics map[string]float64
err error
})
// Worker pool to limit concurrent operations
workerPool := make(chan struct{}, 3) // Max 3 concurrent operations
// Goroutine zur Datenbeschaffung
// Single goroutine for data collection
go func() {
defer func() {
if r := recover(); r != nil {
log.Printf("Data collection goroutine recovered from panic: %v", r)
}
}()
for {
select {
case <-ctx.Done():
close(systemDataCh)
if metricsEnabled {
close(metricsCh)
}
close(processListCh)
close(hmacInfoCh)
return
case <-ticker.C:
// Systemdaten abrufen asynchron
go func() {
memUsage, cpuUsage, cores, err := fetchSystemData()
systemDataCh <- struct {
memUsage float64
cpuUsage float64
cores int
err error
}{memUsage, cpuUsage, cores, err}
}()
if metricsEnabled {
// Metriken abrufen asynchron
case <-fastTicker.C:
// Only update system data and metrics (lightweight operations)
select {
case workerPool <- struct{}{}:
go func() {
metrics, err := fetchMetrics()
metricsCh <- struct {
metrics map[string]float64
err error
}{metrics, err}
defer func() { <-workerPool }()
updateSystemAndMetrics()
}()
default:
// Skip if worker pool is full
}
case <-slowTicker.C:
// Update process list less frequently (expensive operation)
select {
case workerPool <- struct{}{}:
go func() {
defer func() { <-workerPool }()
updateProcessData()
}()
default:
// Skip if worker pool is full
}
// Prozessliste abrufen asynchron
go func() {
processes, err := fetchProcessList()
processListCh <- struct {
processes []ProcessInfo
err error
}{processes, err}
}()
// hmac-file-server Informationen abrufen asynchron
go func() {
hmacInfo, err := fetchHmacFileServerInfo()
var metrics map[string]float64
if metricsEnabled {
metrics, err = fetchMetrics()
}
hmacInfoCh <- struct {
info *ProcessInfo
metrics map[string]float64
err error
}{hmacInfo, metrics, err}
}()
}
}
}()
// UI update loop
uiTicker := time.NewTicker(2 * time.Second)
defer uiTicker.Stop()
for {
select {
case <-ctx.Done():
return
case data, ok := <-systemDataCh:
if !ok {
systemDataCh = nil
continue
}
if data.err != nil {
log.Printf("Fehler beim Abrufen der Systemdaten: %v\n", data.err)
continue
}
// UI aktualisieren mit Systemdaten
case <-uiTicker.C:
app.QueueUpdateDraw(func() {
if currentPage, _ := pages.GetFrontPage(); currentPage == "system" {
sysFlex := sysPage.(*tview.Flex)
sysTable := sysFlex.GetItem(0).(*tview.Table)
updateSystemTable(sysTable, data.memUsage, data.cpuUsage, data.cores)
}
})
case data, ok := <-metricsCh:
if !ok {
metricsCh = nil
continue
}
if data.err != nil {
log.Printf("Fehler beim Abrufen der Metriken: %v\n", data.err)
continue
}
// UI aktualisieren mit Metriken
app.QueueUpdateDraw(func() {
if currentPage, _ := pages.GetFrontPage(); currentPage == "system" {
sysFlex := sysPage.(*tview.Flex)
metricsTable := sysFlex.GetItem(1).(*tview.Table)
updateMetricsTable(metricsTable, data.metrics)
}
})
case data, ok := <-processListCh:
if !ok {
processListCh = nil
continue
}
if data.err != nil {
log.Printf("Fehler beim Abrufen der Prozessliste: %v\n", data.err)
continue
}
// UI aktualisieren mit Prozessliste
app.QueueUpdateDraw(func() {
if currentPage, _ := pages.GetFrontPage(); currentPage == "system" {
sysFlex := sysPage.(*tview.Flex)
processTable := sysFlex.GetItem(2).(*tview.Table)
updateProcessTable(processTable, data.processes)
}
})
case data, ok := <-hmacInfoCh:
if !ok {
hmacInfoCh = nil
continue
}
if data.err != nil {
log.Printf("Fehler beim Abrufen der hmac-file-server Informationen: %v\n", data.err)
continue
}
// UI aktualisieren mit hmac-file-server Informationen
app.QueueUpdateDraw(func() {
if currentPage, _ := pages.GetFrontPage(); currentPage == "hmac" && data.info != nil {
hmacFlex := hmacPage.(*tview.Flex)
hmacTable := hmacFlex.GetItem(0).(*tview.Table)
updateHmacTable(hmacTable, data.info, data.metrics)
}
updateUIComponents(pages, sysPage, hmacPage)
})
}
}
}
// Abbruchbedingung, wenn alle Channels geschlossen sind
if systemDataCh == nil && (!metricsEnabled || metricsCh == nil) && processListCh == nil && hmacInfoCh == nil {
break
// Separate function to update system data and metrics
func updateSystemAndMetrics() {
defer func() {
if r := recover(); r != nil {
log.Printf("updateSystemAndMetrics recovered from panic: %v", r)
}
}()
// Get system data
memUsage, cpuUsage, cores, err := fetchSystemData()
if err != nil {
log.Printf("Error fetching system data: %v", err)
return
}
// Get metrics if enabled
var metrics map[string]float64
if metricsEnabled {
metrics, err = fetchMetrics()
if err != nil {
log.Printf("Error fetching metrics: %v", err)
metrics = make(map[string]float64) // Use empty map on error
}
}
// Update cache
cache.mu.Lock()
cache.systemData = systemData{memUsage, cpuUsage, cores}
cache.metrics = metrics
cache.lastUpdate = time.Now()
cache.mu.Unlock()
}
// Separate function to update process data (expensive operation)
func updateProcessData() {
defer func() {
if r := recover(); r != nil {
log.Printf("updateProcessData recovered from panic: %v", r)
}
}()
// Get process list
processes, err := fetchProcessList()
if err != nil {
log.Printf("Error fetching process list: %v", err)
return
}
// Get HMAC info
hmacInfo, err := fetchHmacFileServerInfo()
if err != nil {
log.Printf("Error fetching HMAC info: %v", err)
}
// Update cache
cache.mu.Lock()
cache.processes = processes
cache.hmacInfo = hmacInfo
cache.mu.Unlock()
}
// Update UI components with cached data
func updateUIComponents(pages *tview.Pages, sysPage, hmacPage tview.Primitive) {
currentPage, _ := pages.GetFrontPage()
cache.mu.RLock()
defer cache.mu.RUnlock()
switch currentPage {
case "system":
sysFlex := sysPage.(*tview.Flex)
// Update system table
sysTable := sysFlex.GetItem(0).(*tview.Table)
updateSystemTable(sysTable, cache.systemData.memUsage, cache.systemData.cpuUsage, cache.systemData.cores)
// Update metrics table
if metricsEnabled && len(cache.metrics) > 0 {
metricsTable := sysFlex.GetItem(1).(*tview.Table)
updateMetricsTable(metricsTable, cache.metrics)
}
// Update process table
if len(cache.processes) > 0 {
processTable := sysFlex.GetItem(2).(*tview.Table)
updateProcessTable(processTable, cache.processes)
}
case "hmac":
if cache.hmacInfo != nil {
hmacFlex := hmacPage.(*tview.Flex)
hmacTable := hmacFlex.GetItem(0).(*tview.Table)
updateHmacTable(hmacTable, cache.hmacInfo, cache.metrics)
}
}
}
@ -665,13 +783,13 @@ func updateHmacTable(hmacTable *tview.Table, hmacInfo *ProcessInfo, metrics map[
hmacTable.SetCell(4, 0, tview.NewTableCell("Command"))
hmacTable.SetCell(4, 1, tview.NewTableCell(hmacInfo.CommandLine))
hmacTable.SetCell(5, 0, tview.NewTableCell("Uptime"))
hmacTable.SetCell(5, 1, tview.NewTableCell(hmacInfo.Uptime)) // Neue Zeile für Uptime
hmacTable.SetCell(6, 0, tview.NewTableCell("Status"))
hmacTable.SetCell(6, 1, tview.NewTableCell(hmacInfo.Status)) // Neue Zeile für Status
hmacTable.SetCell(7, 0, tview.NewTableCell("Error Count"))
hmacTable.SetCell(7, 1, tview.NewTableCell(fmt.Sprintf("%d", hmacInfo.ErrorCount))) // Neue Zeile für Error Count
@ -740,41 +858,73 @@ func createLogsPage(ctx context.Context, app *tview.Application, logFilePath str
SetWordWrap(true)
logsTextView.SetTitle(" [::b]Logs ").SetBorder(true)
const numLines = 100 // Number of lines to read from the end of the log file
const numLines = 50 // Reduced from 100 to 50 lines
// Read logs periodically
// Cache for log content to avoid reading file too frequently
var lastLogUpdate time.Time
var logMutex sync.RWMutex
// Read logs less frequently and only when on logs page
go func() {
ticker := time.NewTicker(5 * time.Second) // Increased from 2 to 5 seconds
defer ticker.Stop()
for {
select {
case <-ctx.Done():
return
default:
case <-ticker.C:
// Only update if we haven't updated recently
logMutex.RLock()
timeSinceUpdate := time.Since(lastLogUpdate)
logMutex.RUnlock()
if timeSinceUpdate < 4*time.Second {
continue
}
content, err := readLastNLines(logFilePath, numLines)
if err != nil {
app.QueueUpdateDraw(func() {
logsTextView.SetText(fmt.Sprintf("[red]Error reading log file: %v[white]", err))
})
} else {
// Process the log content to add colors
lines := strings.Split(content, "\n")
var coloredLines []string
for _, line := range lines {
if strings.Contains(line, "level=info") {
coloredLines = append(coloredLines, "[green]"+line+"[white]")
} else if strings.Contains(line, "level=warn") {
coloredLines = append(coloredLines, "[yellow]"+line+"[white]")
} else if strings.Contains(line, "level=error") {
coloredLines = append(coloredLines, "[red]"+line+"[white]")
} else {
// Default color
coloredLines = append(coloredLines, line)
}
}
app.QueueUpdateDraw(func() {
logsTextView.SetText(strings.Join(coloredLines, "\n"))
})
continue
}
time.Sleep(2 * time.Second) // Refresh interval for logs
// Process the log content with color coding
lines := strings.Split(content, "\n")
var coloredLines []string
// Limit the number of lines processed
maxLines := min(len(lines), numLines)
coloredLines = make([]string, 0, maxLines)
for i := len(lines) - maxLines; i < len(lines); i++ {
if i < 0 {
continue
}
line := lines[i]
if strings.Contains(line, "level=info") {
coloredLines = append(coloredLines, "[green]"+line+"[white]")
} else if strings.Contains(line, "level=warn") {
coloredLines = append(coloredLines, "[yellow]"+line+"[white]")
} else if strings.Contains(line, "level=error") {
coloredLines = append(coloredLines, "[red]"+line+"[white]")
} else {
coloredLines = append(coloredLines, line)
}
}
logContent := strings.Join(coloredLines, "\n")
// Update cache
logMutex.Lock()
lastLogUpdate = time.Now()
logMutex.Unlock()
app.QueueUpdateDraw(func() {
logsTextView.SetText(logContent)
})
}
}
}()
@ -782,6 +932,14 @@ func createLogsPage(ctx context.Context, app *tview.Application, logFilePath str
return logsTextView
}
// Helper function for min
func min(a, b int) int {
if a < b {
return a
}
return b
}
// Optimized readLastNLines to handle large files efficiently
func readLastNLines(filePath string, n int) (string, error) {
file, err := os.Open(filePath)
@ -889,4 +1047,4 @@ func main() {
log.Fatalf("Error running application: %v", err)
log.Fatalf("Error running application: %v", err)
}
}
}

View File

@ -1,102 +0,0 @@
[server]
#bind_ip = "127.0.0.1"
listenport = "8081"
unixsocket = false
storagepath = "./uploads/"
metricsenabled = true
metricsport = "9092"
deduplicationenabled = true
minfreebytes = "5GB"
filettl = "2y"
filettlenabled = false
autoadjustworkers = true
networkevents = false
pidfilepath = "/tmp/hmac-file-server.pid"
precaching = false
force_protocol = "auto"
#globalextensions = ["*"]
[deduplication]
enabled = true
directory = "./deduplication/"
[logging]
level = "debug"
file = "./hmac-file-server.log"
max_size = 100
max_backups = 7
max_age = 30
compress = true
[thumbnails]
enabled = false
directory = "./thumbnails/"
size = "200x200"
thumbnailintervalscan = "1h"
concurrency = 5
[iso]
enabled = false
size = "1TB"
mountpoint = "/mnt/nfs_vol01/hmac-file-server/iso/"
charset = "utf-8"
[timeouts]
readtimeout = "3600s"
writetimeout = "3600s"
idletimeout = "3600s"
[security]
secret = "matrix-does-not-solve-thinking"
enablejwt = false
jwtsecret = "jwt-secret-key-for-testing"
jwtalgorithm = "HS256"
jwtexpiration = "24h"
[versioning]
enableversioning = false
maxversions = 1
[uploads]
resumableuploadsenabled = false
chunkeduploadsenabled = true
chunksize = "32MB"
allowedextensions = [
".txt", ".pdf", ".png", ".jpg", ".jpeg", ".gif",
".bmp", ".tiff", ".svg", ".webp", ".wav", ".mp4",
".avi", ".mkv", ".mov", ".wmv", ".flv", ".webm",
".mpeg", ".mpg", ".m4v", ".3gp", ".3g2", ".mp3", ".ogg"
]
[downloads]
chunkeddownloadsenabled = false
chunksize = "32MB"
allowedextensions = [
".txt", ".pdf", ".png", ".jpg", ".jpeg", ".gif",
".bmp", ".tiff", ".svg", ".webp", ".wav", ".mp4",
".avi", ".mkv", ".mov", ".wmv", ".flv", ".webm",
".mpeg", ".mpg", ".m4v", ".3gp", ".3g2", ".mp3", ".ogg"
]
[clamav]
clamavenabled = false
clamavsocket = "/var/run/clamav/clamd.ctl"
numscanworkers = 4
scanfileextensions = [
".exe", ".dll", ".bin", ".com", ".bat",
".sh", ".php", ".js"
]
[redis]
redisenabled = false
redisdbindex = 0
redisaddr = "localhost:6379"
redispassword = ""
redishealthcheckinterval = "120s"
[workers]
numworkers = 4
uploadqueuesize = 5000
[file]
filerevision = 1

View File

@ -0,0 +1,294 @@
// config_test_scenarios.go
package main
import (
"fmt"
"os"
"path/filepath"
)
// ConfigTestScenario represents a test scenario for configuration validation
type ConfigTestScenario struct {
Name string
Config Config
ShouldPass bool
ExpectedErrors []string
ExpectedWarnings []string
}
// GetConfigTestScenarios returns a set of test scenarios for configuration validation
func GetConfigTestScenarios() []ConfigTestScenario {
baseValidConfig := Config{
Server: ServerConfig{
ListenAddress: "8080",
BindIP: "0.0.0.0",
StoragePath: "/tmp/test-storage",
MetricsEnabled: true,
MetricsPort: "9090",
FileTTLEnabled: true,
FileTTL: "24h",
MinFreeBytes: "1GB",
FileNaming: "HMAC",
ForceProtocol: "auto",
PIDFilePath: "/tmp/test.pid",
},
Security: SecurityConfig{
Secret: "test-secret-key-32-characters",
EnableJWT: false,
},
Logging: LoggingConfig{
Level: "info",
File: "/tmp/test.log",
MaxSize: 100,
MaxBackups: 3,
MaxAge: 30,
},
Timeouts: TimeoutConfig{
Read: "30s",
Write: "30s",
Idle: "60s",
},
Workers: WorkersConfig{
NumWorkers: 4,
UploadQueueSize: 50,
},
Uploads: UploadsConfig{
AllowedExtensions: []string{".txt", ".pdf", ".jpg"},
ChunkSize: "10MB",
},
Downloads: DownloadsConfig{
AllowedExtensions: []string{".txt", ".pdf", ".jpg"},
ChunkSize: "10MB",
},
}
return []ConfigTestScenario{
{
Name: "Valid Basic Configuration",
Config: baseValidConfig,
ShouldPass: true,
},
{
Name: "Missing Listen Address",
Config: func() Config {
c := baseValidConfig
c.Server.ListenAddress = ""
return c
}(),
ShouldPass: false,
ExpectedErrors: []string{"server.listen_address is required"},
},
{
Name: "Invalid Port Number",
Config: func() Config {
c := baseValidConfig
c.Server.ListenAddress = "99999"
return c
}(),
ShouldPass: false,
ExpectedErrors: []string{"invalid port number"},
},
{
Name: "Invalid IP Address",
Config: func() Config {
c := baseValidConfig
c.Server.BindIP = "999.999.999.999"
return c
}(),
ShouldPass: false,
ExpectedErrors: []string{"invalid IP address format"},
},
{
Name: "Same Port for Server and Metrics",
Config: func() Config {
c := baseValidConfig
c.Server.ListenAddress = "8080"
c.Server.MetricsPort = "8080"
return c
}(),
ShouldPass: false,
ExpectedErrors: []string{"metrics port cannot be the same as main listen port"},
},
{
Name: "JWT Enabled Without Secret",
Config: func() Config {
c := baseValidConfig
c.Security.EnableJWT = true
c.Security.JWTSecret = ""
return c
}(),
ShouldPass: false,
ExpectedErrors: []string{"JWT secret is required when JWT is enabled"},
},
{
Name: "Short JWT Secret",
Config: func() Config {
c := baseValidConfig
c.Security.EnableJWT = true
c.Security.JWTSecret = "short"
c.Security.JWTAlgorithm = "HS256"
return c
}(),
ShouldPass: true,
ExpectedWarnings: []string{"JWT secret should be at least 32 characters"},
},
{
Name: "Invalid Log Level",
Config: func() Config {
c := baseValidConfig
c.Logging.Level = "invalid"
return c
}(),
ShouldPass: false,
ExpectedErrors: []string{"invalid log level"},
},
{
Name: "Invalid Timeout Format",
Config: func() Config {
c := baseValidConfig
c.Timeouts.Read = "invalid"
return c
}(),
ShouldPass: false,
ExpectedErrors: []string{"invalid read timeout format"},
},
{
Name: "Negative Worker Count",
Config: func() Config {
c := baseValidConfig
c.Workers.NumWorkers = -1
return c
}(),
ShouldPass: false,
ExpectedErrors: []string{"number of workers must be positive"},
},
{
Name: "Extensions Without Dots",
Config: func() Config {
c := baseValidConfig
c.Uploads.AllowedExtensions = []string{"txt", "pdf"}
return c
}(),
ShouldPass: false,
ExpectedErrors: []string{"file extensions must start with a dot"},
},
{
Name: "High Worker Count Warning",
Config: func() Config {
c := baseValidConfig
c.Workers.NumWorkers = 100
return c
}(),
ShouldPass: true,
ExpectedWarnings: []string{"very high worker count may impact performance"},
},
{
Name: "Deduplication Without Directory",
Config: func() Config {
c := baseValidConfig
c.Deduplication.Enabled = true
c.Deduplication.Directory = ""
return c
}(),
ShouldPass: false,
ExpectedErrors: []string{"deduplication directory is required"},
},
}
}
// RunConfigTests runs all configuration test scenarios
func RunConfigTests() {
scenarios := GetConfigTestScenarios()
passed := 0
failed := 0
fmt.Println("🧪 Running Configuration Test Scenarios")
fmt.Println("=======================================")
fmt.Println()
for i, scenario := range scenarios {
fmt.Printf("Test %d: %s\n", i+1, scenario.Name)
// Create temporary directories for testing
tempDir := filepath.Join(os.TempDir(), fmt.Sprintf("hmac-test-%d", i))
os.MkdirAll(tempDir, 0755)
defer os.RemoveAll(tempDir)
// Update paths in config to use temp directory
scenario.Config.Server.StoragePath = filepath.Join(tempDir, "storage")
scenario.Config.Logging.File = filepath.Join(tempDir, "test.log")
scenario.Config.Server.PIDFilePath = filepath.Join(tempDir, "test.pid")
if scenario.Config.Deduplication.Enabled {
scenario.Config.Deduplication.Directory = filepath.Join(tempDir, "dedup")
}
result := ValidateConfigComprehensive(&scenario.Config)
// Check if test passed as expected
testPassed := true
if scenario.ShouldPass && result.HasErrors() {
fmt.Printf(" ❌ Expected to pass but failed with errors:\n")
for _, err := range result.Errors {
fmt.Printf(" • %s\n", err.Message)
}
testPassed = false
} else if !scenario.ShouldPass && !result.HasErrors() {
fmt.Printf(" ❌ Expected to fail but passed\n")
testPassed = false
} else if !scenario.ShouldPass && result.HasErrors() {
// Check if expected errors are present
expectedFound := true
for _, expectedError := range scenario.ExpectedErrors {
found := false
for _, actualError := range result.Errors {
if contains([]string{actualError.Message}, expectedError) ||
contains([]string{actualError.Error()}, expectedError) {
found = true
break
}
}
if !found {
fmt.Printf(" ❌ Expected error not found: %s\n", expectedError)
expectedFound = false
}
}
if !expectedFound {
testPassed = false
}
}
// Check expected warnings
if len(scenario.ExpectedWarnings) > 0 {
for _, expectedWarning := range scenario.ExpectedWarnings {
found := false
for _, actualWarning := range result.Warnings {
if contains([]string{actualWarning.Message}, expectedWarning) ||
contains([]string{actualWarning.Error()}, expectedWarning) {
found = true
break
}
}
if !found {
fmt.Printf(" ⚠️ Expected warning not found: %s\n", expectedWarning)
}
}
}
if testPassed {
fmt.Printf(" ✅ Passed\n")
passed++
} else {
failed++
}
fmt.Println()
}
// Summary
fmt.Printf("📊 Test Results: %d passed, %d failed\n", passed, failed)
if failed > 0 {
fmt.Printf("❌ Some tests failed. Please review the implementation.\n")
os.Exit(1)
} else {
fmt.Printf("✅ All tests passed!\n")
}
}

File diff suppressed because it is too large Load Diff

View File

@ -1 +0,0 @@
Hello, HMAC File Server! Mo 9. Jun 20:09:03 CEST 2025

Binary file not shown.

View File

@ -136,26 +136,26 @@ type ServerConfig struct {
}
type UploadsConfig struct {
AllowedExtensions []string `toml:"allowed_extensions"`
ChunkedUploadsEnabled bool `toml:"chunked_uploads_enabled"`
ChunkSize string `toml:"chunk_size"`
ResumableUploadsEnabled bool `toml:"resumable_uploads_enabled"`
MaxResumableAge string `toml:"max_resumable_age"`
AllowedExtensions []string `toml:"allowedextensions" mapstructure:"allowedextensions"`
ChunkedUploadsEnabled bool `toml:"chunkeduploadsenabled" mapstructure:"chunkeduploadsenabled"`
ChunkSize string `toml:"chunksize" mapstructure:"chunksize"`
ResumableUploadsEnabled bool `toml:"resumableuploadsenabled" mapstructure:"resumableuploadsenabled"`
MaxResumableAge string `toml:"max_resumable_age" mapstructure:"max_resumable_age"`
}
type DownloadsConfig struct {
AllowedExtensions []string `toml:"allowed_extensions"`
ChunkedDownloadsEnabled bool `toml:"chunked_downloads_enabled"`
ChunkSize string `toml:"chunk_size"`
ResumableDownloadsEnabled bool `toml:"resumable_downloads_enabled"`
AllowedExtensions []string `toml:"allowedextensions" mapstructure:"allowedextensions"`
ChunkedDownloadsEnabled bool `toml:"chunkeddownloadsenabled" mapstructure:"chunkeddownloadsenabled"`
ChunkSize string `toml:"chunksize" mapstructure:"chunksize"`
ResumableDownloadsEnabled bool `toml:"resumable_downloads_enabled" mapstructure:"resumable_downloads_enabled"`
}
type SecurityConfig struct {
Secret string `toml:"secret"`
EnableJWT bool `toml:"enablejwt"` // Added EnableJWT field
JWTSecret string `toml:"jwtsecret"`
JWTAlgorithm string `toml:"jwtalgorithm"`
JWTExpiration string `toml:"jwtexpiration"`
Secret string `toml:"secret" mapstructure:"secret"`
EnableJWT bool `toml:"enablejwt" mapstructure:"enablejwt"` // Added EnableJWT field
JWTSecret string `toml:"jwtsecret" mapstructure:"jwtsecret"`
JWTAlgorithm string `toml:"jwtalgorithm" mapstructure:"jwtalgorithm"`
JWTExpiration string `toml:"jwtexpiration" mapstructure:"jwtexpiration"`
}
type LoggingConfig struct {
@ -174,16 +174,17 @@ type DeduplicationConfig struct {
type ISOConfig struct {
Enabled bool `mapstructure:"enabled"`
MountPoint string `mapstructure:"mount_point"`
MountPoint string `mapstructure:"mountpoint"`
Size string `mapstructure:"size"`
Charset string `mapstructure:"charset"`
ContainerFile string `mapstructure:"containerfile"` // Added missing field
}
type TimeoutConfig struct {
Read string `mapstructure:"readtimeout" toml:"readtimeout"` // Corrected to match example config
Write string `mapstructure:"writetimeout" toml:"writetimeout"` // Corrected to match example config
Idle string `mapstructure:"idletimeout" toml:"idletimeout"` // Corrected to match example config
Read string `mapstructure:"readtimeout" toml:"readtimeout"`
Write string `mapstructure:"writetimeout" toml:"writetimeout"`
Idle string `mapstructure:"idletimeout" toml:"idletimeout"`
Shutdown string `mapstructure:"shutdown" toml:"shutdown"`
}
type VersioningConfig struct {
@ -437,10 +438,41 @@ func main() {
flag.StringVar(&configFile, "config", "./config.toml", "Path to configuration file \"config.toml\".")
var genConfig bool
var genConfigPath string
var validateOnly bool
var runConfigTests bool
var validateQuiet bool
var validateVerbose bool
var validateFixable bool
var validateSecurity bool
var validatePerformance bool
var validateConnectivity bool
var listValidationChecks bool
var showVersion bool
flag.BoolVar(&genConfig, "genconfig", false, "Print example configuration and exit.")
flag.StringVar(&genConfigPath, "genconfig-path", "", "Write example configuration to the given file and exit.")
flag.BoolVar(&validateOnly, "validate-config", false, "Validate configuration and exit without starting server.")
flag.BoolVar(&runConfigTests, "test-config", false, "Run configuration validation test scenarios and exit.")
flag.BoolVar(&validateQuiet, "validate-quiet", false, "Only show errors during validation (suppress warnings and info).")
flag.BoolVar(&validateVerbose, "validate-verbose", false, "Show detailed validation information including system checks.")
flag.BoolVar(&validateFixable, "check-fixable", false, "Only show validation issues that can be automatically fixed.")
flag.BoolVar(&validateSecurity, "check-security", false, "Run only security-related validation checks.")
flag.BoolVar(&validatePerformance, "check-performance", false, "Run only performance-related validation checks.")
flag.BoolVar(&validateConnectivity, "check-connectivity", false, "Run only network connectivity validation checks.")
flag.BoolVar(&listValidationChecks, "list-checks", false, "List all available validation checks and exit.")
flag.BoolVar(&showVersion, "version", false, "Show version information and exit.")
flag.Parse()
if showVersion {
fmt.Printf("HMAC File Server v3.2\n")
os.Exit(0)
}
if listValidationChecks {
printValidationChecks()
os.Exit(0)
}
if genConfig {
printExampleConfig()
os.Exit(0)
@ -458,6 +490,10 @@ func main() {
fmt.Printf("Example config written to %s\n", genConfigPath)
os.Exit(0)
}
if runConfigTests {
RunConfigTests()
os.Exit(0)
}
// Initialize Viper
viper.SetConfigType("toml")
@ -503,6 +539,34 @@ func main() {
}
log.Info("Configuration validated successfully.")
// Perform comprehensive configuration validation
validationResult := ValidateConfigComprehensive(&conf)
PrintValidationResults(validationResult)
if validationResult.HasErrors() {
log.Fatal("Cannot start server due to configuration errors. Please fix the above issues and try again.")
}
// Handle specialized validation flags
if validateSecurity || validatePerformance || validateConnectivity || validateQuiet || validateVerbose || validateFixable {
runSpecializedValidation(&conf, validateSecurity, validatePerformance, validateConnectivity, validateQuiet, validateVerbose, validateFixable)
os.Exit(0)
}
// If only validation was requested, exit now
if validateOnly {
if validationResult.HasErrors() {
log.Error("Configuration validation failed with errors. Review the errors above.")
os.Exit(1)
} else if validationResult.HasWarnings() {
log.Info("Configuration is valid but has warnings. Review the warnings above.")
os.Exit(0)
} else {
log.Info("Configuration validation completed successfully!")
os.Exit(0)
}
}
// Set log level based on configuration
level, err := logrus.ParseLevel(conf.Logging.Level)
if err != nil {
@ -1074,6 +1138,7 @@ func setDefaults() {
viper.SetDefault("timeouts.read", "60s")
viper.SetDefault("timeouts.write", "60s")
viper.SetDefault("timeouts.idle", "120s")
viper.SetDefault("timeouts.shutdown", "30s")
// Versioning defaults
viper.SetDefault("versioning.enabled", false)
@ -1855,3 +1920,73 @@ func handleLegacyDownload(w http.ResponseWriter, r *http.Request) {
downloadSizeBytes.Observe(float64(n))
log.Infof("Successfully downloaded %s (%s) in %s", absFilename, formatBytes(n), duration)
}
// printValidationChecks prints all available validation checks
func printValidationChecks() {
fmt.Println("HMAC File Server Configuration Validation Checks")
fmt.Println("=================================================")
fmt.Println()
fmt.Println("🔍 CORE VALIDATION CHECKS:")
fmt.Println(" ✓ server.* - Server configuration (ports, paths, protocols)")
fmt.Println(" ✓ security.* - Security settings (secrets, JWT, authentication)")
fmt.Println(" ✓ logging.* - Logging configuration (levels, files, rotation)")
fmt.Println(" ✓ timeouts.* - Timeout settings (read, write, idle)")
fmt.Println(" ✓ uploads.* - Upload configuration (extensions, chunk size)")
fmt.Println(" ✓ downloads.* - Download configuration (extensions, chunk size)")
fmt.Println(" ✓ workers.* - Worker pool configuration (count, queue size)")
fmt.Println(" ✓ redis.* - Redis configuration (address, credentials)")
fmt.Println(" ✓ clamav.* - ClamAV antivirus configuration")
fmt.Println(" ✓ versioning.* - File versioning configuration")
fmt.Println(" ✓ deduplication.* - File deduplication configuration")
fmt.Println(" ✓ iso.* - ISO filesystem configuration")
fmt.Println()
fmt.Println("🔐 SECURITY CHECKS:")
fmt.Println(" ✓ Secret strength analysis (length, entropy, patterns)")
fmt.Println(" ✓ Default/example value detection")
fmt.Println(" ✓ JWT algorithm security recommendations")
fmt.Println(" ✓ Network binding security (0.0.0.0 warnings)")
fmt.Println(" ✓ File permission analysis")
fmt.Println(" ✓ Debug logging security implications")
fmt.Println()
fmt.Println("⚡ PERFORMANCE CHECKS:")
fmt.Println(" ✓ Worker count vs CPU cores optimization")
fmt.Println(" ✓ Queue size vs memory usage analysis")
fmt.Println(" ✓ Timeout configuration balance")
fmt.Println(" ✓ Large file handling preparation")
fmt.Println(" ✓ Memory-intensive configuration detection")
fmt.Println()
fmt.Println("🌐 CONNECTIVITY CHECKS:")
fmt.Println(" ✓ Redis server connectivity testing")
fmt.Println(" ✓ ClamAV socket accessibility")
fmt.Println(" ✓ Network address format validation")
fmt.Println(" ✓ DNS resolution testing")
fmt.Println()
fmt.Println("💾 SYSTEM RESOURCE CHECKS:")
fmt.Println(" ✓ CPU core availability analysis")
fmt.Println(" ✓ Memory usage monitoring")
fmt.Println(" ✓ Disk space validation")
fmt.Println(" ✓ Directory write permissions")
fmt.Println(" ✓ Goroutine count analysis")
fmt.Println()
fmt.Println("🔄 CROSS-SECTION VALIDATION:")
fmt.Println(" ✓ Path conflict detection")
fmt.Println(" ✓ Extension compatibility checks")
fmt.Println(" ✓ Configuration consistency validation")
fmt.Println()
fmt.Println("📋 USAGE EXAMPLES:")
fmt.Println(" hmac-file-server --validate-config # Full validation")
fmt.Println(" hmac-file-server --check-security # Security checks only")
fmt.Println(" hmac-file-server --check-performance # Performance checks only")
fmt.Println(" hmac-file-server --check-connectivity # Network checks only")
fmt.Println(" hmac-file-server --validate-quiet # Errors only")
fmt.Println(" hmac-file-server --validate-verbose # Detailed output")
fmt.Println(" hmac-file-server --check-fixable # Auto-fixable issues")
fmt.Println()
}

View File

@ -1 +0,0 @@
Hello, HMAC File Server! Mo 9. Jun 20:09:03 CEST 2025

View File

@ -1,136 +0,0 @@
# HMAC File Server 3.2 Example Configuration for XMPP Operators
# This configuration is optimized for production XMPP deployments
[server]
# Server binding configuration
bind_ip = "127.0.0.1" # Bind to localhost only (use reverse proxy for public access)
listenport = "8080" # HTTP port for file operations
unixsocket = false # Use TCP instead of Unix socket
# Storage configuration
storagepath = "/var/lib/hmac-file-server/uploads"
deduplicationenabled = true # Save space by detecting duplicate files
deduplicationpath = "/var/lib/hmac-file-server/deduplication"
# File naming strategy
filenaming = "HMAC" # Use HMAC-based filenames for security
# Protocol enforcement (auto-detect by default)
forceprotocol = "auto"
# Metrics for monitoring
metricsenabled = true
metricsport = "9090"
# SSL/TLS configuration (optional - use reverse proxy recommended)
sslenabled = false
# sslcert = "/path/to/certificate.pem"
# sslkey = "/path/to/private-key.pem"
[security]
# HMAC secret - MUST match your XMPP server configuration
# Generate with: openssl rand -base64 32
secret = "CHANGE_THIS_TO_A_STRONG_SECRET_AT_LEAST_32_CHARS_LONG"
# JWT authentication (recommended for enhanced security)
enablejwt = true
jwtsecret = "CHANGE_THIS_TO_A_DIFFERENT_STRONG_JWT_SECRET"
jwtalgorithm = "HS256" # Supported: HS256, HS384, HS512
jwtexpiration = "24h" # Token validity period
[uploads]
# Allowed file extensions (customize based on your needs)
allowedextensions = [
# Documents
".txt", ".pdf", ".doc", ".docx", ".odt",
".xls", ".xlsx", ".ods", ".ppt", ".pptx", ".odp",
# Images
".jpg", ".jpeg", ".png", ".gif", ".webp", ".bmp", ".svg",
# Audio
".mp3", ".wav", ".ogg", ".flac", ".m4a", ".aac",
# Video
".mp4", ".webm", ".avi", ".mkv", ".mov", ".wmv",
# Archives
".zip", ".tar", ".gz", ".7z", ".rar", ".bz2"
]
# File size limits
maxfilesize = "50MB" # Maximum file size per upload
# Chunked uploads for large files
chunkeduploadsenabled = true
chunksize = "10MB" # Size of each chunk
# Automatic file cleanup (TTL)
ttlenabled = true # Enable automatic deletion of old files
ttl = "720h" # Delete files after 30 days (720 hours)
[downloads]
# Chunked downloads for large files
chunkeddownloadsenabled = true
chunksize = "10MB"
[logging]
# Log level: DEBUG, INFO, WARN, ERROR
loglevel = "INFO"
# File logging
logtofile = true
logfilepath = "/var/log/hmac-file-server/hmac-file-server.log"
# Log rotation
logrotation = true
maxlogsize = "100MB" # Rotate when log file exceeds this size
maxlogfiles = 5 # Keep this many rotated log files
maxage = 30 # Delete log files older than this (days)
compress = true # Compress rotated log files
[workers]
# Worker pool configuration for handling concurrent requests
maxworkers = 100 # Maximum number of worker goroutines
minworkers = 10 # Minimum number of worker goroutines
autoscaling = true # Automatically scale workers based on load
[timeouts]
# Network timeouts
readtimeout = "60s" # Maximum time to read request
writetimeout = "60s" # Maximum time to write response
idletimeout = "120s" # Maximum time for idle connections
[redis]
# Redis integration for session management and caching
enabled = false # Set to true if you have Redis available
host = "localhost"
port = 6379
database = 0
password = "" # Set if Redis requires authentication
timeout = "5s"
# Connection pooling
maxidle = 10
maxactive = 100
idletimeout = "240s"
[clamav]
# Virus scanning integration
enabled = false # Set to true to enable virus scanning
socket = "/var/run/clamav/clamd.ctl" # ClamAV socket path
timeout = "30s" # Maximum time to wait for scan
# XMPP Server Integration Examples:
#
# Prosody Configuration:
# Component "upload.yourdomain.com" "http_file_share"
# http_file_share_url = "https://upload.yourdomain.com"
# http_file_share_secret = "SAME_SECRET_AS_ABOVE"
#
# Ejabberd Configuration:
# mod_http_file_share:
# external_secret: "SAME_SECRET_AS_ABOVE"
# service_url: "https://upload.yourdomain.com"
#
# Note: Always use HTTPS in production with proper SSL certificates

View File

@ -6,7 +6,7 @@ RUN apk add --no-cache git
COPY go.mod go.sum ./
RUN go mod download
COPY . .
RUN CGO_ENABLED=0 go build -o hmac-file-server cmd/server/main.go cmd/server/helpers.go
RUN CGO_ENABLED=0 go build -o hmac-file-server cmd/server/main.go cmd/server/helpers.go cmd/server/config_validator.go cmd/server/config_test_scenarios.go
# Stage 2: Runtime
FROM alpine:latest

View File

@ -354,8 +354,10 @@ get_user_input() {
read -p "Enable ClamAV virus scanning? (y/N): " ENABLE_CLAMAV
if [[ $ENABLE_CLAMAV =~ ^[Yy]$ ]]; then
ENABLE_CLAMAV="true"
CLAMAV_CONFIG="socket = \"/var/run/clamav/clamd.ctl\"" # Default, will be updated during installation
else
ENABLE_CLAMAV="false"
CLAMAV_CONFIG=""
fi
# SSL/TLS settings
@ -410,6 +412,7 @@ create_directories() {
mkdir -p "$DEFAULT_CONFIG_DIR"
mkdir -p "$DATA_DIR/uploads"
mkdir -p "$DATA_DIR/deduplication"
mkdir -p "$DATA_DIR/runtime"
mkdir -p "$DEFAULT_LOG_DIR"
# Set ownership
@ -429,7 +432,7 @@ build_server() {
# Build the server
cd "$(dirname "$0")"
go build -o "$INSTALL_DIR/hmac-file-server" cmd/server/main.go cmd/server/helpers.go
go build -o "$INSTALL_DIR/hmac-file-server" cmd/server/main.go cmd/server/helpers.go cmd/server/config_validator.go cmd/server/config_test_scenarios.go
# Set ownership and permissions
chown "$HMAC_USER:$HMAC_USER" "$INSTALL_DIR/hmac-file-server"
@ -456,7 +459,8 @@ metricsport = "$METRICS_PORT"
deduplicationenabled = true
deduplicationpath = "$DATA_DIR/deduplication"
filenaming = "HMAC"
forceprotocol = "auto"
force_protocol = "auto"
pidfilepath = "$DATA_DIR/runtime/hmac-file-server.pid"
EOF
if [[ $ENABLE_TLS == "true" ]]; then
@ -501,24 +505,22 @@ chunkeddownloadsenabled = true
chunksize = "10MB"
[logging]
loglevel = "INFO"
logfile = "$DEFAULT_LOG_DIR/hmac-file-server.log"
enablerotation = true
maxsize = 100
maxbackups = 3
maxage = 30
level = "INFO"
file = "$DEFAULT_LOG_DIR/hmac-file-server.log"
max_size = 100
max_backups = 3
max_age = 30
compress = true
[workers]
initial = 10
max = 100
idletimeout = "60s"
numworkers = 10
uploadqueuesize = 1000
autoscaling = true
[timeouts]
read = "30s"
write = "30s"
idle = "120s"
readtimeout = "30s"
writetimeout = "30s"
idletimeout = "120s"
shutdown = "30s"
EOF
@ -527,7 +529,7 @@ EOF
[clamav]
enabled = true
socket = "/var/run/clamav/clamd.ctl"
${CLAMAV_CONFIG}
timeout = "30s"
EOF
else
@ -639,6 +641,40 @@ install_dependencies() {
systemctl enable clamav-daemon
# Update virus definitions
freshclam || true
# Detect ClamAV configuration and configure accordingly
echo -e "${YELLOW}Configuring ClamAV connection...${NC}"
# Check if ClamAV daemon is running and detect socket/port
if systemctl is-active --quiet clamav-daemon; then
echo " ✓ ClamAV daemon is running"
# Check for Unix socket (preferred)
if [[ -S "/var/run/clamav/clamd.ctl" ]]; then
echo " ✓ Unix socket detected: /var/run/clamav/clamd.ctl"
CLAMAV_CONFIG="socket = \"/var/run/clamav/clamd.ctl\""
elif [[ -S "/run/clamav/clamd.ctl" ]]; then
echo " ✓ Unix socket detected: /run/clamav/clamd.ctl"
CLAMAV_CONFIG="socket = \"/run/clamav/clamd.ctl\""
elif [[ -S "/tmp/clamd" ]]; then
echo " ✓ Unix socket detected: /tmp/clamd"
CLAMAV_CONFIG="socket = \"/tmp/clamd\""
# Check for TCP port
elif netstat -ln | grep -q ":3310"; then
echo " ✓ TCP port detected: 127.0.0.1:3310"
CLAMAV_CONFIG="address = \"127.0.0.1:3310\""
else
echo " ⚠ ClamAV socket/port not detected, using default Unix socket"
CLAMAV_CONFIG="socket = \"/var/run/clamav/clamd.ctl\""
fi
else
echo " ⚠ ClamAV daemon not running, using default configuration"
CLAMAV_CONFIG="socket = \"/var/run/clamav/clamd.ctl\""
# Try to start the daemon
echo " 🔄 Attempting to start ClamAV daemon..."
systemctl start clamav-daemon || echo " ⚠ Failed to start ClamAV daemon"
fi
fi
elif command -v yum &> /dev/null; then
if [[ $ENABLE_REDIS == "true" ]]; then
@ -804,38 +840,300 @@ print_completion_info() {
echo -e " Status: ${YELLOW}sudo systemctl status hmac-file-server${NC}"
echo ""
echo -e "${BLUE}🔧 Next Steps for XMPP Integration:${NC}"
echo "1. ${YELLOW}Configure firewall${NC} to allow ports $SERVER_PORT (server) and $METRICS_PORT (metrics)"
echo "2. Configure your reverse proxy (nginx/apache) with SSL"
echo "3. Update your Prosody/Ejabberd configuration:"
echo -e "1. ${YELLOW}Configure firewall${NC} to allow ports $SERVER_PORT (server) and $METRICS_PORT (metrics)"
echo -e "2. Configure your reverse proxy (nginx/apache) with SSL"
echo -e "3. Update your Prosody/Ejabberd configuration:"
echo -e " ${YELLOW}http_file_share = \"http://localhost:$SERVER_PORT\"${NC}"
echo "4. Set up monitoring and log rotation"
echo "5. Test file uploads with your XMPP client"
echo -e "4. Set up monitoring and log rotation"
echo -e "5. Test file uploads with your XMPP client"
echo ""
echo -e "${BLUE}📚 Documentation & Support:${NC}"
echo " README: https://github.com/PlusOne/hmac-file-server/blob/main/README.MD"
echo " Wiki: https://github.com/PlusOne/hmac-file-server/blob/main/WIKI.MD"
echo " Issues: https://github.com/PlusOne/hmac-file-server/issues"
echo -e " README: https://github.com/PlusOne/hmac-file-server/blob/main/README.MD"
echo -e " Wiki: https://github.com/PlusOne/hmac-file-server/blob/main/WIKI.MD"
echo -e " Issues: https://github.com/PlusOne/hmac-file-server/issues"
echo ""
echo -e "${GREEN}────────────────────────────────────────────────────────────────${NC}"
echo -e "${GREEN} Thank you for choosing HMAC File Server for your XMPP setup! ${NC}"
echo -e "${GREEN}────────────────────────────────────────────────────────────────${NC}"
}
# Helper function to safely preserve a directory
preserve_directory() {
local source_dir="$1"
local backup_path="$2"
if [[ -d "$source_dir" ]]; then
local parent_dir=$(dirname "$backup_path")
mkdir -p "$parent_dir"
if mv "$source_dir" "$backup_path" 2>/dev/null; then
echo " ✓ Preserved: $source_dir$backup_path"
else
# Fallback to copy if move fails
if cp -r "$source_dir" "$backup_path" 2>/dev/null; then
echo " ✓ Copied: $source_dir$backup_path"
rm -rf "$source_dir"
echo " ✓ Removed original: $source_dir"
else
echo " ⚠ Failed to preserve: $source_dir"
fi
fi
else
echo " ⚠ Directory not found: $source_dir"
fi
}
# Custom data selection for option 4
custom_data_selection() {
echo ""
echo -e "${BLUE}Custom Data Selection:${NC}"
echo "Choose which data directories to preserve:"
echo ""
CUSTOM_PRESERVE_UPLOADS=""
CUSTOM_PRESERVE_DEDUP=""
CUSTOM_PRESERVE_LOGS=""
# Ask about uploads
if [[ -d "$UPLOAD_DIR" ]]; then
FILE_COUNT=$(find "$UPLOAD_DIR" -type f 2>/dev/null | wc -l)
DIR_SIZE=$(du -sh "$UPLOAD_DIR" 2>/dev/null | cut -f1)
echo -e "${GREEN}📤 Upload Directory: ${UPLOAD_DIR}${NC} (Files: $FILE_COUNT, Size: $DIR_SIZE)"
read -p "Preserve upload directory? (y/N): " PRESERVE_UPLOADS
if [[ $PRESERVE_UPLOADS =~ ^[Yy]$ ]]; then
CUSTOM_PRESERVE_UPLOADS="yes"
echo " ✓ Will preserve uploads"
else
echo " ✗ Will delete uploads"
fi
else
echo -e "${YELLOW}📤 Upload Directory: Not found${NC}"
fi
echo ""
# Ask about deduplication
if [[ -d "$DEDUP_DIR" ]]; then
FILE_COUNT=$(find "$DEDUP_DIR" -type f 2>/dev/null | wc -l)
DIR_SIZE=$(du -sh "$DEDUP_DIR" 2>/dev/null | cut -f1)
echo -e "${GREEN}🔗 Deduplication Directory: ${DEDUP_DIR}${NC} (Files: $FILE_COUNT, Size: $DIR_SIZE)"
read -p "Preserve deduplication directory? (y/N): " PRESERVE_DEDUP
if [[ $PRESERVE_DEDUP =~ ^[Yy]$ ]]; then
CUSTOM_PRESERVE_DEDUP="yes"
echo " ✓ Will preserve deduplication data"
else
echo " ✗ Will delete deduplication data"
fi
else
echo -e "${YELLOW}🔗 Deduplication Directory: Not found${NC}"
fi
echo ""
# Ask about logs
if [[ -d "$LOG_DIR" ]]; then
FILE_COUNT=$(find "$LOG_DIR" -type f 2>/dev/null | wc -l)
DIR_SIZE=$(du -sh "$LOG_DIR" 2>/dev/null | cut -f1)
echo -e "${GREEN}📄 Log Directory: ${LOG_DIR}${NC} (Files: $FILE_COUNT, Size: $DIR_SIZE)"
read -p "Preserve log directory? (y/N): " PRESERVE_LOGS
if [[ $PRESERVE_LOGS =~ ^[Yy]$ ]]; then
CUSTOM_PRESERVE_LOGS="yes"
echo " ✓ Will preserve logs"
else
echo " ✗ Will delete logs"
fi
else
echo -e "${YELLOW}📄 Log Directory: Not found${NC}"
fi
# Store custom selection for later processing
PRESERVE_DATA="custom"
echo ""
echo -e "${BLUE}Custom selection complete:${NC}"
[[ "$CUSTOM_PRESERVE_UPLOADS" == "yes" ]] && echo " 📤 Uploads: Preserve" || echo " 📤 Uploads: Delete"
[[ "$CUSTOM_PRESERVE_DEDUP" == "yes" ]] && echo " 🔗 Deduplication: Preserve" || echo " 🔗 Deduplication: Delete"
[[ "$CUSTOM_PRESERVE_LOGS" == "yes" ]] && echo " 📄 Logs: Preserve" || echo " 📄 Logs: Delete"
echo ""
}
# Handle custom preservation choices
handle_custom_preservation() {
# Check if any data needs to be preserved
if [[ "$CUSTOM_PRESERVE_UPLOADS" == "yes" || "$CUSTOM_PRESERVE_DEDUP" == "yes" || "$CUSTOM_PRESERVE_LOGS" == "yes" ]]; then
BACKUP_DIR="/var/backups/hmac-file-server-$(date +%Y%m%d-%H%M%S)"
mkdir -p "$BACKUP_DIR"
echo " ✓ Created backup directory: $BACKUP_DIR"
fi
# Handle uploads
if [[ "$CUSTOM_PRESERVE_UPLOADS" == "yes" ]]; then
preserve_directory "$UPLOAD_DIR" "$BACKUP_DIR/uploads"
elif [[ -d "$UPLOAD_DIR" ]]; then
rm -rf "$UPLOAD_DIR"
echo " ✓ Removed uploads: $UPLOAD_DIR"
fi
# Handle deduplication
if [[ "$CUSTOM_PRESERVE_DEDUP" == "yes" ]]; then
preserve_directory "$DEDUP_DIR" "$BACKUP_DIR/deduplication"
elif [[ -d "$DEDUP_DIR" ]]; then
rm -rf "$DEDUP_DIR"
echo " ✓ Removed deduplication: $DEDUP_DIR"
fi
# Handle logs
if [[ "$CUSTOM_PRESERVE_LOGS" == "yes" ]]; then
preserve_directory "$LOG_DIR" "$BACKUP_DIR/logs"
elif [[ -d "$LOG_DIR" ]]; then
rm -rf "$LOG_DIR"
echo " ✓ Removed logs: $LOG_DIR"
fi
# Remove the main data directory if it's separate and empty
if [[ -d "$DEFAULT_DATA_DIR" ]]; then
# Only remove if it's different from preserved directories and if it's empty or only contains subdirs we've handled
if [[ "$DEFAULT_DATA_DIR" != "$UPLOAD_DIR" && "$DEFAULT_DATA_DIR" != "$DEDUP_DIR" && "$DEFAULT_DATA_DIR" != "$LOG_DIR" ]]; then
# Check if directory is effectively empty (only contains directories we've already handled)
remaining_files=$(find "$DEFAULT_DATA_DIR" -type f 2>/dev/null | wc -l)
if [[ $remaining_files -eq 0 ]]; then
rm -rf "$DEFAULT_DATA_DIR"
echo " ✓ Removed empty data directory: $DEFAULT_DATA_DIR"
else
echo " ⚠ Data directory contains additional files: $DEFAULT_DATA_DIR"
fi
fi
fi
}
# Uninstaller function (can be called with ./installer.sh --uninstall)
uninstall() {
echo ""
echo -e "${RED} █ HMAC File Server Uninstaller █${NC}"
echo -e "${RED}────────────────────────────────────────────────────────────────${NC}"
echo -e "${RED} Warning: This will remove all server data! ${NC}"
echo -e "${RED} Warning: This will remove the server installation! ${NC}"
echo -e "${RED}────────────────────────────────────────────────────────────────${NC}"
echo ""
read -p "Are you sure you want to uninstall HMAC File Server? This will remove all data! (y/N): " CONFIRM_UNINSTALL
read -p "Are you sure you want to uninstall HMAC File Server? (y/N): " CONFIRM_UNINSTALL
if [[ ! $CONFIRM_UNINSTALL =~ ^[Yy]$ ]]; then
echo -e "${YELLOW}Uninstall cancelled${NC}"
exit 0
fi
echo ""
echo -e "${BLUE}📁 Data Preservation Options:${NC}"
echo -e "${BLUE}────────────────────────────────────────────────────────────────${NC}"
echo ""
echo "The following data directories may contain important files:"
# Check what data directories exist and show their contents
PRESERVE_DATA=""
UPLOAD_DIR=""
DEDUP_DIR=""
LOG_DIR=""
# Find upload directory from config if it exists
if [[ -f "$DEFAULT_CONFIG_DIR/config.toml" ]]; then
UPLOAD_DIR=$(grep -E "^storagepath\s*=" "$DEFAULT_CONFIG_DIR/config.toml" 2>/dev/null | sed 's/.*=\s*"*\([^"]*\)"*.*/\1/' | xargs)
DEDUP_DIR=$(grep -E "^directory\s*=" "$DEFAULT_CONFIG_DIR/config.toml" 2>/dev/null | sed 's/.*=\s*"*\([^"]*\)"*.*/\1/' | xargs)
fi
# Fallback to default locations
[[ -z "$UPLOAD_DIR" ]] && UPLOAD_DIR="$DEFAULT_DATA_DIR/uploads"
[[ -z "$DEDUP_DIR" ]] && DEDUP_DIR="$DEFAULT_DATA_DIR/deduplication"
LOG_DIR="$DEFAULT_LOG_DIR"
# Show upload directory status
if [[ -d "$UPLOAD_DIR" ]]; then
FILE_COUNT=$(find "$UPLOAD_DIR" -type f 2>/dev/null | wc -l)
DIR_SIZE=$(du -sh "$UPLOAD_DIR" 2>/dev/null | cut -f1)
echo -e "${GREEN} 📤 Upload Directory: ${UPLOAD_DIR}${NC}"
echo -e " Files: $FILE_COUNT, Size: $DIR_SIZE"
else
echo -e "${YELLOW} 📤 Upload Directory: Not found or empty${NC}"
fi
# Show deduplication directory status
if [[ -d "$DEDUP_DIR" ]]; then
FILE_COUNT=$(find "$DEDUP_DIR" -type f 2>/dev/null | wc -l)
DIR_SIZE=$(du -sh "$DEDUP_DIR" 2>/dev/null | cut -f1)
echo -e "${GREEN} 🔗 Deduplication Directory: ${DEDUP_DIR}${NC}"
echo -e " Files: $FILE_COUNT, Size: $DIR_SIZE"
else
echo -e "${YELLOW} 🔗 Deduplication Directory: Not found or empty${NC}"
fi
# Show log directory status
if [[ -d "$LOG_DIR" ]]; then
FILE_COUNT=$(find "$LOG_DIR" -type f 2>/dev/null | wc -l)
DIR_SIZE=$(du -sh "$LOG_DIR" 2>/dev/null | cut -f1)
echo -e "${GREEN} 📄 Log Directory: ${LOG_DIR}${NC}"
echo -e " Files: $FILE_COUNT, Size: $DIR_SIZE"
else
echo -e "${YELLOW} 📄 Log Directory: Not found or empty${NC}"
fi
echo ""
echo -e "${BLUE}Choose data handling option:${NC}"
echo " 1) 🗑️ Delete all data (complete removal)"
echo " 2) 💾 Preserve upload and deduplication data only"
echo " 3) 📋 Preserve all data (uploads, deduplication, and logs)"
echo " 4) 🎯 Custom selection (choose what to preserve)"
echo " 5) ❌ Cancel uninstallation"
echo ""
while true; do
read -p "Select option (1-5): " DATA_OPTION
case $DATA_OPTION in
1)
echo -e "${RED}Selected: Delete all data${NC}"
PRESERVE_DATA="none"
break
;;
2)
echo -e "${GREEN}Selected: Preserve uploads and deduplication data${NC}"
PRESERVE_DATA="uploads_dedup"
break
;;
3)
echo -e "${GREEN}Selected: Preserve all data${NC}"
PRESERVE_DATA="all"
break
;;
4)
echo -e "${BLUE}Custom selection:${NC}"
custom_data_selection
break
;;
5)
echo -e "${YELLOW}Uninstall cancelled${NC}"
exit 0
;;
*)
echo -e "${RED}Invalid option. Please choose 1-5.${NC}"
;;
esac
done
# Final confirmation for complete deletion
if [[ "$PRESERVE_DATA" == "none" ]]; then
echo ""
echo -e "${RED}⚠️ FINAL WARNING: This will permanently delete ALL data!${NC}"
echo -e "${RED} This includes all uploaded files, deduplication data, and logs.${NC}"
echo -e "${RED} This action cannot be undone!${NC}"
echo ""
read -p "Type 'DELETE' to confirm complete data removal: " FINAL_CONFIRM
if [[ "$FINAL_CONFIRM" != "DELETE" ]]; then
echo -e "${YELLOW}Uninstall cancelled - confirmation failed${NC}"
exit 0
fi
fi
echo ""
echo -e "${YELLOW}🔄 Starting uninstallation process...${NC}"
echo ""
echo -e "${YELLOW}Stopping and disabling service...${NC}"
if systemctl is-active --quiet hmac-file-server.service; then
systemctl stop hmac-file-server.service || true
@ -861,15 +1159,72 @@ uninstall() {
systemctl daemon-reload
echo " ✓ Systemd reloaded"
echo -e "${YELLOW}Removing installation directories...${NC}"
for dir in "$DEFAULT_INSTALL_DIR" "$DEFAULT_CONFIG_DIR" "$DEFAULT_DATA_DIR" "$DEFAULT_LOG_DIR"; do
if [[ -d "$dir" ]]; then
rm -rf "$dir"
echo " ✓ Removed $dir"
else
echo " ⚠ Directory $dir not found"
fi
done
echo -e "${YELLOW}Removing installation and configuration...${NC}"
# Always remove installation directory
if [[ -d "$DEFAULT_INSTALL_DIR" ]]; then
rm -rf "$DEFAULT_INSTALL_DIR"
echo " ✓ Removed installation directory: $DEFAULT_INSTALL_DIR"
else
echo " ⚠ Installation directory not found: $DEFAULT_INSTALL_DIR"
fi
# Always remove configuration directory
if [[ -d "$DEFAULT_CONFIG_DIR" ]]; then
rm -rf "$DEFAULT_CONFIG_DIR"
echo " ✓ Removed configuration directory: $DEFAULT_CONFIG_DIR"
else
echo " ⚠ Configuration directory not found: $DEFAULT_CONFIG_DIR"
fi
# Handle data directories based on user choice
echo -e "${YELLOW}Processing data directories...${NC}"
case $PRESERVE_DATA in
"none")
# Delete everything
for dir in "$UPLOAD_DIR" "$DEDUP_DIR" "$LOG_DIR" "$DEFAULT_DATA_DIR"; do
if [[ -d "$dir" ]]; then
rm -rf "$dir"
echo " ✓ Removed: $dir"
fi
done
;;
"uploads_dedup")
# Preserve uploads and deduplication, remove logs
if [[ -d "$LOG_DIR" ]]; then
rm -rf "$LOG_DIR"
echo " ✓ Removed logs: $LOG_DIR"
fi
# Move preserved data to a safe location
BACKUP_DIR="/var/backups/hmac-file-server-$(date +%Y%m%d-%H%M%S)"
mkdir -p "$BACKUP_DIR"
preserve_directory "$UPLOAD_DIR" "$BACKUP_DIR/uploads"
preserve_directory "$DEDUP_DIR" "$BACKUP_DIR/deduplication"
# Remove original data directory structure but keep preserved data
if [[ -d "$DEFAULT_DATA_DIR" && "$DEFAULT_DATA_DIR" != "$UPLOAD_DIR" && "$DEFAULT_DATA_DIR" != "$DEDUP_DIR" ]]; then
rm -rf "$DEFAULT_DATA_DIR"
echo " ✓ Removed data directory (preserved content moved to $BACKUP_DIR)"
fi
;;
"all")
# Preserve everything
BACKUP_DIR="/var/backups/hmac-file-server-$(date +%Y%m%d-%H%M%S)"
mkdir -p "$BACKUP_DIR"
preserve_directory "$UPLOAD_DIR" "$BACKUP_DIR/uploads"
preserve_directory "$DEDUP_DIR" "$BACKUP_DIR/deduplication"
preserve_directory "$LOG_DIR" "$BACKUP_DIR/logs"
# Remove original data directory structure but keep preserved data
if [[ -d "$DEFAULT_DATA_DIR" ]]; then
rm -rf "$DEFAULT_DATA_DIR"
echo " ✓ Removed data directory (all content preserved in $BACKUP_DIR)"
fi
;;
"custom")
# Handle custom selection
handle_custom_preservation
;;
esac
echo -e "${YELLOW}Removing system user...${NC}"
if id "$DEFAULT_USER" &>/dev/null; then
@ -889,8 +1244,17 @@ uninstall() {
done
echo ""
echo -e "${GREEN}✅ HMAC File Server uninstalled successfully${NC}"
echo -e "${BLUE}All files, services, and user accounts have been removed.${NC}"
if [[ "$PRESERVE_DATA" != "none" ]]; then
echo -e "${GREEN}✅ HMAC File Server uninstalled successfully with data preservation${NC}"
if [[ -d "$BACKUP_DIR" ]]; then
echo -e "${BLUE}📁 Preserved data location: $BACKUP_DIR${NC}"
echo -e "${BLUE} You can safely delete this directory if you no longer need the data.${NC}"
fi
else
echo -e "${GREEN}✅ HMAC File Server uninstalled completely${NC}"
echo -e "${BLUE}All files, services, and user accounts have been removed.${NC}"
fi
echo ""
}
# Check for help flag

173
test/test_installer_config.sh Executable file
View File

@ -0,0 +1,173 @@
#!/bin/bash
# Test script to validate installer configuration generation
# Tests that the installer generates config compatible with fixed struct definitions
set -e
# Colors for output
RED='\033[0;31m'
GREEN='\033[0;32m'
YELLOW='\033[1;33m'
BLUE='\033[0;34m'
NC='\033[0m' # No Color
echo -e "${BLUE}🔍 Testing Installer Configuration Generation${NC}"
echo "============================================="
echo ""
# Test configuration values that simulate installer input
export HMAC_SECRET="test-hmac-secret-32-characters-long-minimum"
export JWT_SECRET="test-jwt-secret-also-32-characters-long-minimum"
# Create a test directory
TEST_DIR="/tmp/hmac-installer-test-$$"
mkdir -p "$TEST_DIR"
cd "$TEST_DIR"
echo -e "${YELLOW}📁 Test directory: $TEST_DIR${NC}"
echo ""
# Copy necessary files for testing
cp /home/renz/source/hmac-file-server-uuxo/go.mod .
cp /home/renz/source/hmac-file-server-uuxo/go.sum .
cp -r /home/renz/source/hmac-file-server-uuxo/cmd .
# Extract the generate_config function and create a test version
cat > test_config_generation.sh << 'EOF'
#!/bin/bash
# Simulated installer variables
DEFAULT_CONFIG_DIR="./test-config"
DATA_DIR="./test-data"
DEFAULT_LOG_DIR="./test-logs"
SERVER_PORT="8080"
METRICS_PORT="9090"
ENABLE_TLS="false"
HMAC_SECRET="test-hmac-secret-32-characters-long-minimum"
ENABLE_JWT="false"
ENABLE_CLAMAV="false"
ENABLE_REDIS="false"
# Create directories
mkdir -p "$DEFAULT_CONFIG_DIR"
mkdir -p "$DATA_DIR/runtime"
mkdir -p "$DEFAULT_LOG_DIR"
# Generate configuration (extracted from installer)
generate_config() {
echo "Generating test configuration..."
cat > "$DEFAULT_CONFIG_DIR/config.toml" << EOFCONFIG
# HMAC File Server Configuration
# Generated by installer test on $(date)
[server]
bind_ip = "0.0.0.0"
listenport = "$SERVER_PORT"
unixsocket = false
storagepath = "$DATA_DIR/uploads"
metricsenabled = true
metricsport = "$METRICS_PORT"
deduplicationenabled = true
deduplicationpath = "$DATA_DIR/deduplication"
filenaming = "HMAC"
force_protocol = "auto"
pidfilepath = "$DATA_DIR/runtime/hmac-file-server.pid"
sslenabled = false
[security]
secret = "$HMAC_SECRET"
enablejwt = false
[uploads]
allowedextensions = [".txt", ".pdf", ".jpg", ".jpeg", ".png", ".gif", ".webp", ".zip", ".tar", ".gz", ".7z", ".mp4", ".webm", ".ogg", ".mp3", ".wav", ".flac", ".doc", ".docx", ".xls", ".xlsx", ".ppt", ".pptx", ".odt", ".ods", ".odp"]
maxfilesize = "100MB"
chunkeduploadsenabled = true
chunksize = "10MB"
ttlenabled = false
ttl = "168h"
[downloads]
chunkeddownloadsenabled = true
chunksize = "10MB"
[logging]
level = "INFO"
file = "$DEFAULT_LOG_DIR/hmac-file-server.log"
max_size = 100
max_backups = 3
max_age = 30
compress = true
[workers]
numworkers = 10
uploadqueuesize = 1000
autoscaling = true
[timeouts]
readtimeout = "30s"
writetimeout = "30s"
idletimeout = "120s"
shutdown = "30s"
[clamav]
enabled = false
[redis]
enabled = false
EOFCONFIG
echo "Configuration file created: $DEFAULT_CONFIG_DIR/config.toml"
}
# Call the function
generate_config
EOF
chmod +x test_config_generation.sh
./test_config_generation.sh
echo -e "${YELLOW}📋 Generated test configuration:${NC}"
echo ""
cat ./test-config/config.toml
echo ""
# Build a test binary to validate the configuration
echo -e "${YELLOW}🔨 Building test binary...${NC}"
if go build -o hmac-test-server ./cmd/server/*.go; then
echo -e "${GREEN}✅ Build successful${NC}"
else
echo -e "${RED}❌ Build failed${NC}"
exit 1
fi
echo ""
echo -e "${YELLOW}🔍 Testing configuration validation...${NC}"
# Test configuration validation
if ./hmac-test-server -config ./test-config/config.toml -validate-config -validate-quiet; then
echo -e "${GREEN}✅ Configuration validation PASSED!${NC}"
echo ""
echo -e "${GREEN}🎉 All critical fixes verified:${NC}"
echo -e "${GREEN} ✓ Workers: numworkers/uploadqueuesize (not initial/max)${NC}"
echo -e "${GREEN} ✓ Protocol: force_protocol (not forceprotocol)${NC}"
echo -e "${GREEN} ✓ PID file: pidfilepath configured${NC}"
echo -e "${GREEN} ✓ Timeouts: read/write/idle/shutdown${NC}"
echo -e "${GREEN} ✓ Logging: level/file/max_size/max_backups/max_age${NC}"
VALIDATION_RESULT=0
else
echo -e "${RED}❌ Configuration validation FAILED!${NC}"
echo ""
echo -e "${YELLOW}Running detailed validation for diagnosis...${NC}"
./hmac-test-server -config ./test-config/config.toml -validate-config -validate-verbose || true
VALIDATION_RESULT=1
fi
echo ""
echo -e "${YELLOW}🧹 Cleaning up test directory...${NC}"
cd /
rm -rf "$TEST_DIR"
echo -e "${BLUE}Test completed.${NC}"
exit $VALIDATION_RESULT

230
verify_installation.sh Executable file
View File

@ -0,0 +1,230 @@
#!/bin/bash
# HMAC File Server v3.2 - Installation Verification Script
# Run this script on your production server to verify the installation
set -e
# Colors for output
RED='\033[0;31m'
GREEN='\033[0;32m'
YELLOW='\033[1;33m'
BLUE='\033[0;34m'
CYAN='\033[0;36m'
NC='\033[0m' # No Color
echo -e "${BLUE}🔍 HMAC File Server v3.2 - Installation Verification${NC}"
echo "======================================================"
echo ""
# Check if running as root
if [[ $EUID -ne 0 ]]; then
echo -e "${RED}❌ This script must be run as root (use sudo)${NC}"
exit 1
fi
ERRORS=0
WARNINGS=0
# Function to report status
report_status() {
local status=$1
local message=$2
local details=$3
case $status in
"OK")
echo -e "${GREEN}$message${NC}"
[[ -n "$details" ]] && echo -e " ${CYAN}$details${NC}"
;;
"WARNING")
echo -e "${YELLOW}⚠️ $message${NC}"
[[ -n "$details" ]] && echo -e " ${YELLOW}$details${NC}"
((WARNINGS++))
;;
"ERROR")
echo -e "${RED}$message${NC}"
[[ -n "$details" ]] && echo -e " ${RED}$details${NC}"
((ERRORS++))
;;
"INFO")
echo -e "${CYAN} $message${NC}"
[[ -n "$details" ]] && echo -e " $details"
;;
esac
}
# 1. Check SystemD Service Status
echo -e "${YELLOW}🔧 Checking SystemD Service...${NC}"
if systemctl is-active --quiet hmac-file-server; then
service_status=$(systemctl status hmac-file-server --no-pager -l | head -10)
uptime=$(systemctl show hmac-file-server --property=ActiveEnterTimestamp --value)
report_status "OK" "HMAC File Server service is running" "Active since: $uptime"
else
service_status=$(systemctl status hmac-file-server --no-pager -l | head -10)
report_status "ERROR" "HMAC File Server service is not running" "$service_status"
fi
if systemctl is-enabled --quiet hmac-file-server; then
report_status "OK" "Service is enabled (will start on boot)"
else
report_status "WARNING" "Service is not enabled for auto-start"
fi
echo ""
# 2. Check Installation Files
echo -e "${YELLOW}📁 Checking Installation Files...${NC}"
# Binary
if [[ -f "/opt/hmac-file-server/hmac-file-server" ]]; then
binary_info=$(ls -lh /opt/hmac-file-server/hmac-file-server)
report_status "OK" "Binary installed" "$binary_info"
# Check if binary has version flag (indicates correct build)
if /opt/hmac-file-server/hmac-file-server --version >/dev/null 2>&1; then
version=$(/opt/hmac-file-server/hmac-file-server --version 2>/dev/null || echo "Unknown")
report_status "OK" "Binary supports --version flag" "Version: $version"
else
report_status "WARNING" "Binary doesn't support --version flag (may be old build)"
fi
else
report_status "ERROR" "Binary not found at /opt/hmac-file-server/hmac-file-server"
fi
# Configuration
if [[ -f "/etc/hmac-file-server/config.toml" ]]; then
config_info=$(ls -lh /etc/hmac-file-server/config.toml)
report_status "OK" "Configuration file exists" "$config_info"
else
report_status "ERROR" "Configuration file not found at /etc/hmac-file-server/config.toml"
fi
# Data directories
data_dirs=("/var/lib/hmac-file-server" "/var/log/hmac-file-server")
for dir in "${data_dirs[@]}"; do
if [[ -d "$dir" ]]; then
dir_info=$(ls -lhd "$dir")
report_status "OK" "Directory exists: $dir" "$dir_info"
else
report_status "WARNING" "Directory missing: $dir"
fi
done
echo ""
# 3. Check Configuration Validation
echo -e "${YELLOW}⚙️ Checking Configuration Validation...${NC}"
if [[ -f "/opt/hmac-file-server/hmac-file-server" ]]; then
echo -e "${CYAN}Running configuration validation...${NC}"
# Run validation with timeout
if timeout 30s /opt/hmac-file-server/hmac-file-server -config /etc/hmac-file-server/config.toml --validate-config >/tmp/hmac_validation.log 2>&1; then
report_status "OK" "Configuration validation passed"
# Check for warnings in validation output
if grep -q "WARNING\|WARN" /tmp/hmac_validation.log; then
warning_count=$(grep -c "WARNING\|WARN" /tmp/hmac_validation.log)
report_status "WARNING" "Configuration validation has $warning_count warnings" "Check logs for details"
fi
else
validation_error=$(tail -5 /tmp/hmac_validation.log)
report_status "ERROR" "Configuration validation failed" "$validation_error"
fi
rm -f /tmp/hmac_validation.log
fi
echo ""
# 4. Check Network Connectivity
echo -e "${YELLOW}🌐 Checking Network Connectivity...${NC}"
# Extract ports from config
if [[ -f "/etc/hmac-file-server/config.toml" ]]; then
server_port=$(grep -E "^listenport\s*=" /etc/hmac-file-server/config.toml | cut -d'"' -f2 | tr -d '"' || echo "8080")
metrics_port=$(grep -E "^metricsport\s*=" /etc/hmac-file-server/config.toml | cut -d'"' -f2 | tr -d '"' || echo "9090")
# Check if ports are listening
if netstat -tln 2>/dev/null | grep -q ":$server_port "; then
report_status "OK" "Server port $server_port is listening"
else
report_status "ERROR" "Server port $server_port is not listening"
fi
if netstat -tln 2>/dev/null | grep -q ":$metrics_port "; then
report_status "OK" "Metrics port $metrics_port is listening"
else
report_status "WARNING" "Metrics port $metrics_port is not listening"
fi
# Test HTTP connectivity
if curl -s --connect-timeout 5 "http://localhost:$server_port" >/dev/null 2>&1; then
report_status "OK" "HTTP server responding on port $server_port"
elif curl -s --connect-timeout 5 "http://localhost:$server_port" 2>&1 | grep -q "404\|401\|403"; then
report_status "OK" "HTTP server responding (expected auth required)"
else
report_status "WARNING" "HTTP server not responding on port $server_port"
fi
fi
echo ""
# 5. Check System Resources
echo -e "${YELLOW}💾 Checking System Resources...${NC}"
# Memory usage
memory_usage=$(ps -o pid,ppid,cmd,%mem,%cpu --sort=-%mem -C hmac-file-server | tail -n +2)
if [[ -n "$memory_usage" ]]; then
report_status "OK" "Process running and using resources" "$memory_usage"
else
report_status "WARNING" "No process information available"
fi
# Disk space
storage_path=$(grep -E "^storagepath\s*=" /etc/hmac-file-server/config.toml 2>/dev/null | cut -d'"' -f2 | tr -d '"' || echo "/var/lib/hmac-file-server")
if [[ -d "$storage_path" ]]; then
disk_usage=$(df -h "$storage_path" | tail -1)
report_status "INFO" "Storage directory disk usage" "$disk_usage"
fi
echo ""
# 6. Check Logs
echo -e "${YELLOW}📋 Checking Recent Logs...${NC}"
# SystemD logs
recent_logs=$(journalctl -u hmac-file-server --since "5 minutes ago" --no-pager -q)
if [[ -n "$recent_logs" ]]; then
report_status "INFO" "Recent SystemD logs available"
echo -e "${CYAN}Last 5 log entries:${NC}"
echo "$recent_logs" | tail -5
else
report_status "INFO" "No recent SystemD logs (service may be stable)"
fi
echo ""
# 7. Final Summary
echo -e "${BLUE}📊 Verification Summary${NC}"
echo "========================"
if [[ $ERRORS -eq 0 && $WARNINGS -eq 0 ]]; then
echo -e "${GREEN}🎉 PERFECT! HMAC File Server installation is working correctly!${NC}"
echo -e "${GREEN} No errors or warnings found.${NC}"
elif [[ $ERRORS -eq 0 ]]; then
echo -e "${YELLOW}✅ GOOD! HMAC File Server is working with $WARNINGS warning(s).${NC}"
echo -e "${YELLOW} Review warnings above for optimization opportunities.${NC}"
else
echo -e "${RED}❌ ISSUES FOUND! $ERRORS error(s) and $WARNINGS warning(s) detected.${NC}"
echo -e "${RED} Please address the errors above before using in production.${NC}"
fi
echo ""
echo -e "${CYAN}💡 Additional Checks You Can Perform:${NC}"
echo " • Test file upload: curl -X POST -F \"file=@testfile.txt\" http://localhost:$server_port/"
echo " • Check metrics: curl http://localhost:$metrics_port/metrics"
echo " • Review full logs: journalctl -u hmac-file-server -f"
echo " • Test configuration: /opt/hmac-file-server/hmac-file-server --validate-config"
exit $ERRORS