Operations & Diagnostics Handbook

This guide details how to monitor, scale, maintain, and debug the ChronoSeal daemon (chronoseal) in production environments.

Systemd Service Configuration

On single-node Linux systems, run the daemon under systemd. The standard service unit file is configured at /etc/systemd/system/chronoseal.service:

[Unit]
Description=ChronoSeal Attestation Daemon
After=network.target

[Service]
Type=simple
User=chronoseal
Group=chronoseal
WorkingDirectory=/var/lib/chronoseal
ExecStart=/usr/local/bin/chronoseal run --config /etc/chronoseal.toml
Restart=always
RestartSec=5
LimitNOFILE=65536

# Hardening
ProtectSystem=full
ProtectHome=true
NoNewPrivileges=true

[Install]
WantedBy=multi-user.target

Reload and enable the service:

sudo systemctl daemon-reload
sudo systemctl enable --now chronoseal

Reverse Proxy & TLS Termination

Do not expose the raw ChronoSeal port directly to the internet. Always terminate TLS using Nginx, HAProxy, or a cloud load balancer. A sample Nginx setup is shown below:

server {
    listen 443 ssl http2;
    server_name attestation.example.com;

    ssl_certificate /etc/letsencrypt/live/example.com/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/example.com/privkey.pem;
    ssl_protocols TLSv1.2 TLSv1.3;
    ssl_ciphers HIGH:!aNULL:!MD5;

    location / {
        proxy_pass http://127.0.0.1:3000;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
    }
}

Storage & Horizontal Scaling

Single Node (SQLite)

Set db_type = "sqlite-in-disk" and configure a writeable path in db_path. Optimistic Compare-And-Swap (CAS) locking prevents concurrency errors, but high transaction loads may face write lock queuing.

Multi-Node Scaling (Valkey / Redis)

For high availability, run multiple stateless chronoseal instances behind a load balancer and set db_type = "valkey". Connect all nodes to the same shared Valkey/Redis instance using CHRONOSEAL_VALKEY_ADDR. This keeps sessions consistent across all server nodes.

Monitoring & Observability

Configure Prometheus to scrape operational metrics from `/metrics`:

scrape_configs:
  - job_name: 'chronoseal'
    static_configs:
      - targets: ['localhost:3000']

Set alerts for these key metrics:

• chronoseal_sessions: Spike suggests rate limit thresholds should be reviewed or a scraper campaign is initiating.
• chronoseal_max_chain_length: Tracks session duration. Low values suggest clients are failing attestation.

Failure Diagnostic Guide

Because ChronoSeal returns a silent success response {"status":"ok"} on heartbeat rejections, check the following variables when debugging client integration issues:

A. Clock Drift (TimestampDrift)

Cause: Client machine clock differs from the server clock by more than 30 seconds (max_timestamp_drift_ms). Sync both clocks using NTP.

B. Out-of-Sequence Replays (ChainBroken)

Cause: Client submitted a prev_hash that does not match the server's last_hash. This occurs when network packet drops cause retries from outdated states, or if an attacker attempts replay. The client must restart liveness via /init.

C. Signature Mismatch (Signature)

Cause: The Ed25519 signature over the canonical JSON payload is invalid. Verify that the client orders JSON keys alphabetically (entropyData, fingerprint, geneCommitment, mutationStep, prevHash, sessionId, stackState, timestamp) and serializes types correctly.

D. VM State Mismatch (VmStackMismatch)

Cause: Client's VM stack_state differs from server re-execution results. Ensure the browser VM implementation wraps 32-bit math correctly matching shared/src/vm.rs.