The Hackathon Organizer Node

Configuration Reference

Detailed configuration options for THON and Lemonade Server

Configuration Reference

thon.yaml (Unified Config)

The recommended way to configure THON is via a single thon.yaml file, created by python -m thon init. This file is the single source of truth for all settings — the CLI, API, and dashboard all read from it (or its .env export).

Full Example

# THON Configuration
# Generated by `python -m thon init` — edit freely or re-run the wizard.

# ── Network ───────────────────────────────────────────────────
# External IP for SSL cert SAN and URLs (auto-detected if empty)
external_ip: ""

# Demo mode (default workspace when no groups)
demo: false

# Logging level: DEBUG, INFO, WARNING, ERROR
log_level: "INFO"

# ── Groups & Users ────────────────────────────────────────────
# Each user gets their own VS Code sandbox with workspace at
# /workspace/{group}/{username}
groups:
  alpha:
    - alice
    - bob
    - carol
  beta:
    - dave
    - eve
  gamma:
    - frank

# ── Sandbox ───────────────────────────────────────────────────
sandbox:
  domain: "localhost:8080"
  api_key: ""
  image: "waterpistol/thon:latest"
  starting_port: 8443
  timeout_minutes: 0

# ── VS Code Instances ────────────────────────────────────────
vscode:
  secure: false
  settings_file: ""
  extensions_file: ""

# ── Nginx & SSL ──────────────────────────────────────────────
nginx:
  enabled: true
  ssl_dir: "/etc/nginx/ssl"
  domain: ""                   # Domain for server_name (enables Let's Encrypt)
  ssl_provider: "auto"         # auto, certbot, mkcert, openssl
  certbot_email: ""            # Email for Let's Encrypt registration

# ── Workspace ─────────────────────────────────────────────────
workspace:
  dir: ""                      # Host directory for bind mounts (empty = ephemeral)

# ── Lemonade Server (Local LLM Inference) ────────────────────
lemonade:
  enabled: false
  host: "0.0.0.0"
  port: 13305
  model: "unsloth/gemma-4-31B-it-GGUF:Q8_K_XL"
  model_name: "gemma-4-31b-it"
  mmproj: "mmproj-BF16.gguf"
  ctx_size_per_user: 262144
  embedding_model: "SuperPauly/harrier-oss-v1-0.6b-gguf:harrier-oss-v1-0.6B-BF16"
  embedding_model_name: "harrier-oss-v1-0.6b"
  embedding_ctx_size_per_user: 32768
  embedding_dimensions: 0
  llamacpp_backend: "auto"     # auto, vulkan, cpu
  prefer_system: true
  llamacpp_bin: "builtin"
  rocm_channel: "preview"
  generate_keys: true
  api_key: ""
  admin_api_key: ""
  llamacpp:                    # Inference tuning parameters
    ctk: "q8_0"
    ctv: "q8_0"
    batch_size: 8192
    ubatch_size: 8192
    split_mode: ""
    main_gpu: -1
    cpu_moe: false
    n_cpu_moe: 0
    min_p: 0.0
    presence_penalty: 0.0
  chat_models:                 # Additional models for selection
    - name: "gemma-4-31b-it"
      checkpoint: "unsloth/gemma-4-31B-it-GGUF:Q8_K_XL"
      context: 262144
      output: 4096
    - name: "qwen3.6-27b"
      checkpoint: "unsloth/Qwen3.6-27B-GGUF:Q8_K_XL"
      context: 262144
      output: 4096

# ── Kilo Code ─────────────────────────────────────────────────
kilo:
  config_file: ""              # Output path (default: ~/.thon/kilo.jsonc)
  skeleton_file: "config/kilo.jsonc.skeleton"
  chat_model: "lemonade/user.gemma-4-31b-it"
  small_model: ""

# ── AI Gateway (APISIX Rate Limiting) ────────────────────────
gateway:
  enabled: false
  mode: "per-user"             # per-user or per-group
  rate_limit_scope: "per-user" # per-user or per-model
  admin_key: ""
  redis_host: ""
  redis_port: 6379
  concurrency_limit: 1
  token_limit: 0               # 0 = unlimited
  token_window: 60
  model_concurrency:           # Per-model limits (when rate_limit_scope=per-model)
    - model: "gemma-4-31b-it"
      concurrency_limit: 1
      token_limit: 0
      token_window: 60
      priority: 0

# ── Langfuse Observability ────────────────────────────────────
langfuse:
  enabled: false
  public_key: ""
  secret_key: ""
  base_url: "https://cloud.langfuse.com"

# ── Dashboard ─────────────────────────────────────────────────
dashboard:
  host: "0.0.0.0"
  port: 8100
  debug: false

# ── Authentication (OIDC) ─────────────────────────────────────
auth:
  enabled: false
  session_secret: ""
  github:
    client_id: ""
    client_secret: ""
  gitlab:
    client_id: ""
    client_secret: ""
  linkedin:
    client_id: ""
    client_secret: ""

Config Sections

SectionDescription
external_ipExternal IP for SSL cert SAN and URLs
demoDemo mode (default workspace when no groups)
log_levelLogging level (DEBUG, INFO, WARNING, ERROR)
groupsGroup name → list of usernames mapping
sandboxSandbox server connection and instance settings
vscodeVS Code security and customization
nginxNginx reverse proxy, SSL, domain, and provider selection
workspacePersistent workspace bind-mounts
lemonadeLocal LLM inference server with llamacpp tuning
kiloKilo Code extension config injection with skeleton merging
gatewayAPISIX AI Gateway with rate limiting and per-model concurrency
langfuseLangfuse LLM observability integration
dashboardWeb dashboard settings
authOIDC authentication providers

Generating .env from thon.yaml

# Export config as .env file for the API and dashboard
python -m thon config env --output .env

The .env file maps thon.yaml fields to environment variables (e.g., lemonade.portLEMONADE_PORT). Values already set in the environment are not overwritten.


groups.yaml

Defines users and groups for VS Code instance creation. Used by main.py --groups and can also be imported into the database via the dashboard.

Structure

groups:
  <group-name>:
    users:
      - <username>
      - <username>

Example

groups:
  alpha:
    users:
      - alice
      - bob
      - carol
  beta:
    users:
      - dave
      - eve
  gamma:
    users:
      - frank

Behavior

  • Each user gets a unique sandbox instance
  • Workspace path: /workspace/{group}/{username}
  • URL path: https://{ip}/{endpoint_port}/proxy/{code_server_port}/
  • Port assignment: Sequential from --port (default 8443)

nginx Configuration

SSL Providers

ProviderDescriptionBest For
auto (default)Tries mkcert → certbot → openssl in orderMost setups
certbotLet's Encrypt certificates (requires domain + port 80)Production with domain
mkcertCA-trusted certs, filename includes IP hashDevelopment
opensslSelf-signed certs with IP/domain in SANFallback

Select via nginx.ssl_provider in thon.yaml or THON_SSL_PROVIDER env var.

Domain Configuration

When nginx.domain is set:

  • nginx uses the domain as server_name
  • Let's Encrypt is preferred for CA-trusted certificates
  • InstanceInfo.domain_url is populated for domain-based URLs

Directory Structure

/etc/nginx/
├── sites-available/
│   ├── sandbox-thon-8443
│   ├── sandbox-thon-8444
│   └── ...
├── sites-enabled/
│   ├── sandbox-thon-8443 -> ../sites-available/sandbox-thon-8443
│   └── ...
└── ssl/
    ├── server-<ip-hash>.crt
    ├── server-<ip-hash>.key
    └── ca.crt (if mkcert)

Per-Port Server Block

Each VS Code instance gets its own nginx config:

server {
    listen 80;
    listen 443 ssl;
    server_name _;

    ssl_certificate /etc/nginx/ssl/server.crt;
    ssl_certificate_key /etc/nginx/ssl/server.key;
    ssl_protocols TLSv1.2 TLSv1.3;
    ssl_ciphers HIGH:!aNULL:!MD5;

    location / {
        proxy_pass http://127.0.0.1:52322/;

        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "upgrade";
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto https;
        proxy_redirect off;

        add_header Service-Worker-Allowed /;

        proxy_read_timeout 86400;
        proxy_send_timeout 86400;
        proxy_buffering off;
        proxy_request_buffering off;
    }

    location = /ca.crt {
        alias /path/to/mkcert/rootCA.pem;
    }
}

Key Configuration Points

SettingValueReason
proxy_passhttp://127.0.0.1:{port}/No upstream path (prevents doubling)
Service-Worker-Allowed/Fixes SW scope errors
proxy_read_timeout8640024h for long-lived WebSocket
proxy_bufferingoffReal-time terminal output

Lemonade Server Configuration

config.json

Location: /var/lib/lemonade/.cache/lemonade/config.json

{
  "config_version": 1,
  "port": 13305,
  "host": "0.0.0.0",
  "log_level": "info",
  "global_timeout": 300,
  "max_loaded_models": 2,
  "no_broadcast": false,
  "extra_models_dir": "",
  "models_dir": "auto",
  "ctx_size": 1572864,
  "offline": false,
  "disable_model_filtering": false,
  "enable_dgpu_gtt": false,
  "llamacpp": {
    "backend": "auto",
    "args": "",
    "prefer_system": true,
    "rocm_bin": "/usr/local/bin/llama-server",
    "vulkan_bin": "/usr/local/bin/llama-server",
    "cpu_bin": "/usr/local/bin/llama-server"
  },
  "whispercpp": {
    "backend": "auto",
    "args": "",
    "cpu_bin": "builtin",
    "npu_bin": "builtin"
  },
  "sdcpp": {
    "backend": "auto",
    "args": "",
    "steps": 20,
    "cfg_scale": 7.0,
    "width": 512,
    "height": 512,
    "cpu_bin": "builtin",
    "rocm_bin": "builtin",
    "vulkan_bin": "builtin"
  },
  "flm": { "args": "" },
  "ryzenai": { "server_bin": "builtin" },
  "kokoro": { "cpu_bin": "builtin" }
}

When embedding is enabled, max_loaded_models is automatically set to 2 (1 chat model + 1 embedding model).

Config Fields

FieldTypeDefaultDescription
portint13305HTTP server port
hoststringlocalhostBind address
log_levelstringinfotrace, debug, info, warning, error
global_timeoutint300Timeout in seconds
max_loaded_modelsint2Max models per type (-1 for unlimited); auto-set to 2 when embedding enabled
ctx_sizeint4096Default context size
offlineboolfalseSkip model downloads
llamacpp.backendstringautoauto, vulkan, cpu
llamacpp.prefer_systemboolfalsePrefer system llama.cpp
llamacpp.*_binstringbuiltinPath to binary or "builtin"

user_models.json

Location: /var/lib/lemonade/.cache/lemonade/user_models.json

{
  "gemma-4-31b-it": {
    "model_name": "gemma-4-31b-it",
    "checkpoint": "unsloth/gemma-4-31B-it-GGUF:Q8_K_XL",
    "recipe": "llamacpp",
    "suggested": true,
    "labels": ["custom", "vision"],
    "mmproj": "mmproj-BF16.gguf"
  },
  "harrier-oss-v1-0.6b": {
    "model_name": "harrier-oss-v1-0.6b",
    "checkpoint": "SuperPauly/harrier-oss-v1-0.6b-gguf:harrier-oss-v1-0.6B-BF16",
    "recipe": "llamacpp",
    "suggested": true,
    "labels": ["custom", "embedding"]
  }
}

Model Entry Fields

FieldRequiredDescription
model_nameNoDisplay name (matches key)
checkpointYesHuggingFace checkpoint (org/repo:variant)
recipeYesBackend engine (llamacpp, whispercpp, etc.)
suggestedNoShow as suggested model
labelsNoTags (custom, vision, embedding)
mmprojNoVision model mmproj filename
sizeNoModel size in GB (informational)

recipe_options.json

Location: /var/lib/lemonade/.cache/lemonade/recipe_options.json

{
  "user.gemma-4-31b-it": {
    "ctx_size": 1572864,
    "llamacpp_backend": "auto",
    "llamacpp_args": "-b 8192 -ub 8192 -to 3600 -ctk q8_0 -ctv q8_0 --temp 1.0 --top-k 64 --top-p 0.95 --min-p 0.0 --repeat-penalty 1.0 --no-webui --threads-http -1 --threads -1 -np 6"
  },
  "user.harrier-oss-v1-0.6b": {
    "ctx_size": 196608,
    "llamacpp_backend": "auto",
    "llamacpp_args": "-b 8192 -ub 8192 -to 3600 -ctk q8_0 -ctv q8_0 --no-webui --threads-http -1 --threads -1 -np 6"
  }
}

Embedding Model Scaling

When embedding is enabled, the embedding model gets its own entry in recipe_options.json with scaled parameters:

ParameterChat ModelEmbedding Model
ctx_size262144 per user32768 per user
-npnum_usersnum_users

Dynamic Rescaling

Adjust context size and parallel slots without restarting the server:

# Via CLI
python ./scripts/lemonade_server.py rescale --num-users 8

# Via API
curl -X POST http://localhost:8100/api/lemonade/rescale?num_users=8

This rewrites recipe_options.json and reloads models.

llamacpp Tuning Parameters

The lemonade.llamacpp section in thon.yaml controls inference tuning:

ParameterDefaultDescription
ctkq8_0Cache type K
ctvq8_0Cache type V
batch_size8192Batch size
ubatch_size8192Micro-batch size
split_mode""Multi-GPU split mode
main_gpu-1Primary GPU index
cpu_moefalseOffload MoE to CPU
n_cpu_moe0Number of CPU MoE experts
min_p0.0Minimum P sampling
presence_penalty0.0Presence penalty

Recipe Options Fields

FieldDescription
ctx_sizeTotal context size (for all parallel slots)
llamacpp_backendOverride backend for this model
llamacpp_argsCustom llama.cpp arguments

server_models.json

Location: /var/lib/lemonade/.cache/lemonade/server_models.json

Same structure as user_models.json. Used for server-suggested models that appear in the Lemonade desktop app.


kilo.jsonc (Kilo Code Config)

Skeleton Merging

The config/kilo.jsonc.skeleton file provides a base config. Dynamic fields (provider, apiKey, model, indexing) are deep-merged on top. Template variables are substituted per-user:

VariableReplacement
$THON_USERNAMEUser's username
$THON_USER_EMAILUser's email or {username}@thon.local
$WORKSPACEUser's workspace path

Deployment Modes

ModeConfig SourceKilo Points To
lemonade-direct--lemonade kilo.jsoncLemonade server directly
gateway-per-user--gatewayAPISIX gateway (per-user API key)
gateway-per-group--gateway --gateway-per-groupAPISIX gateway (shared group API key)

Location in Sandbox

/home/vscode/.config/kilo/config.json

Structure

{
  "provider": {
    "lemonade": {
      "models": {
        "user.gemma-4-31b-it": {
          "name": "unsloth/gemma-4-31B-it-GGUF:Q8_K_XL",
          "limit": {
            "context": 262144,
            "output": 4096
          }
        }
      },
      "options": {
        "apiKey": "your-api-key",
        "baseURL": "http://YOUR_IP:13305/v1"
      }
    }
  },
  "model": "lemonade/user.gemma-4-31b-it",
  "experimental": {
    "batch_tool": false,
    "codebase_search": true,
    "openTelemetry": false,
    "continue_loop_on_deny": true,
    "semantic_indexing": true,
    "agent_manager_tool": true
  },
  "indexing": {
    "enabled": true,
    "provider": "openai-compatible",
    "vectorStore": "lancedb",
    "openai-compatible": {
      "baseUrl": "http://YOUR_IP:13305/v1",
      "apiKey": "your-api-key",
      "model": "user.harrier-oss-v1-0.6b"
    }
  }
}

Fields

FieldDescription
provider.<name>.models.<id>.nameDisplay name (checkpoint)
provider.<name>.models.<id>.limit.contextMax context tokens
provider.<name>.models.<id>.limit.outputMax output tokens
provider.<name>.options.apiKeyAPI key for authentication
provider.<name>.options.baseURLOpenAI-compatible API endpoint
modelActive model (provider/model-id format)
experimental.batch_toolEnable batch tool calling
experimental.codebase_searchEnable codebase search
experimental.openTelemetryEnable OpenTelemetry tracing
experimental.continue_loop_on_denyContinue agent loop on tool deny
experimental.semantic_indexingEnable semantic code indexing
experimental.agent_manager_toolEnable agent manager tool
indexing.enabledEnable semantic indexing
indexing.providerIndexing provider (openai-compatible)
indexing.vectorStoreVector store type (lancedb)
indexing.openai-compatible.baseUrlEmbedding API base URL
indexing.openai-compatible.apiKeyEmbedding API key
indexing.openai-compatible.modelEmbedding model ID (e.g., user.harrier-oss-v1-0.6b)

Gateway-aware kilo.jsonc

When the AI Gateway is enabled, the kilo.jsonc points to the gateway instead of directly to Lemonade, and the indexing config uses the gateway URL for embedding requests:

{
  "providers": {
    "lemonade-gateway": {
      "baseUrl": "http://1.2.3.4:9080",
      "apiKey": "<consumer-api-key>"
    }
  },
  "models": {
    "gemma-4-31b-it": {
      "provider": "lemonade-gateway",
      "modelId": "user.gemma-4-31b-it"
    }
  },
  "experimental": {
    "batch_tool": false,
    "codebase_search": true,
    "openTelemetry": false,
    "continue_loop_on_deny": true,
    "semantic_indexing": true,
    "agent_manager_tool": true
  },
  "indexing": {
    "enabled": true,
    "provider": "openai-compatible",
    "vectorStore": "lancedb",
    "openai-compatible": {
      "baseUrl": "http://1.2.3.4:9080/v1",
      "apiKey": "<consumer-api-key>",
      "model": "user.harrier-oss-v1-0.6b"
    }
  }
}

VS Code Settings

vscode-settings.jsonc

Injected into each sandbox at /home/vscode/.local/share/code-server/User/settings.json.

Key settings for Lemonade integration:

{
  "kilo-code.new.showTaskTimeline": true,
  "kilo-code.new.browserAutomation.enabled": true,
  "telemetry.telemetryLevel": "off"
}

Config File Storage

VS Code settings can also be stored in the database (via the Settings page in the dashboard or the /api/config-files REST API). When main.py runs without --vscode-settings, it reads the stored config from the database.

DB KeyLabelDescription
config_vscode_settingsVS Code SettingsCode-server settings JSON
config_kilo_jsonKilo Code ConfigKilo Code provider config
config_groups_yamlGroups YAMLGroups and users definition

Priority: CLI flag (e.g., --vscode-settings) > database > none.


Authentication

THON supports two authentication mechanisms that operate independently:

Streamlit Dashboard: Local Password

A single shared password that gates access to the Streamlit dashboard.

VariableDefaultDescription
AUTH_LOCAL_PASSWORD(none)Set a password to require login; unset = no auth

When set, visitors see a login form before any dashboard content renders. Suitable for internal or hackathon use. Not a replacement for OIDC in production.

FastAPI REST API: OIDC/OAuth2

Full OIDC/OAuth2 authentication via GitHub, GitLab, or LinkedIn.

VariableDefaultDescription
AUTH_ENABLEDfalseEnable OIDC authentication on the REST API
AUTH_SESSION_SECRET(none)HMAC secret for signing session tokens (required when enabled)
AUTH_GITHUB_CLIENT_ID(none)GitHub OAuth App client ID
AUTH_GITHUB_CLIENT_SECRET(none)GitHub OAuth App client secret
AUTH_GITLAB_CLIENT_ID(none)GitLab OAuth App client ID
AUTH_GITLAB_CLIENT_SECRET(none)GitLab OAuth App client secret
AUTH_LINKEDIN_CLIENT_ID(none)LinkedIn OIDC client ID
AUTH_LINKEDIN_CLIENT_SECRET(none)LinkedIn OIDC client secret

When AUTH_ENABLED=true, unauthenticated requests to /api/* return 401. The OAuth flow uses PKCE (S256) for security. Sessions are HMAC-signed tokens stored as HttpOnly cookies with 24-hour expiry.

See Dashboard → Authentication for the full OAuth flow diagram, session details, and provider setup instructions.


Systemd Override

Location

/etc/systemd/system/lemonade-server.service.d/override.conf

Structure

[Service]
Environment="LEMONADE_API_KEY=your-api-key"
Environment="LEMONADE_ADMIN_API_KEY=your-admin-key"

Apply Changes

sudo systemctl daemon-reload
sudo systemctl restart lemonade-server

PVC Workspace Volumes

When users are created via the dashboard or imported from YAML, each user is assigned a Docker named volume (PVC) for persistent workspace storage.

Volume Naming

  • Workspace: thon-workspace-{group}-{username}
  • Storage: thon-storage-{group}-{username}

Behavior

  • Volumes are created automatically via docker volume create before sandbox creation
  • When a sandbox is recreated (via the dashboard or --from-db), the same PVC volume is reattached so files persist across instance lifecycles
  • PVC volumes take precedence over --workspace-dir bind mounts

From main.py

# When using --from-db, PVC volumes from the database are used automatically
python ./scripts/main.py --from-db --external-ip 1.2.3.4

Dashboard Configuration

The Streamlit dashboard reads the same environment variables as the backend services. No separate configuration file is needed — or use python -m thon config env to generate the .env file from thon.yaml.

Environment Variables

VariableDefaultDescription
SANDBOX_DOMAINlocalhost:8080Sandbox server address
SANDBOX_API_KEY(none)Sandbox API key
SANDBOX_IMAGEwaterpistol/thon:latestDocker image for sandboxes
LEMONADE_HOST0.0.0.0Lemonade server bind address
LEMONADE_PORT13305Lemonade server port
LEMONADE_API_KEY(none)Lemonade API key
LEMONADE_ADMIN_API_KEY(none)Lemonade admin API key
THON_DB_PATH~/.thon/thon.dbSQLite database path
THON_WORKSPACE_DIR~/.thon/workspaceWorkspace directory for groups
THON_DOMAIN(none)Domain name for nginx and Let's Encrypt
THON_SSL_PROVIDERautoSSL provider: auto, certbot, mkcert, openssl
THON_CERTBOT_EMAIL(none)Email for Let's Encrypt registration
THON_KILO_CONFIG(none)Path to kilo.jsonc
THON_VSCODE_SETTINGS(none)Path to VS Code settings file
THON_LOG_LEVELINFOLogging level
DASHBOARD_HOST0.0.0.0FastAPI bind address
DASHBOARD_PORT8100FastAPI port
DASHBOARD_DEBUGfalseEnable FastAPI debug/reload mode

AI Gateway Configuration

VariableDefaultDescription
GATEWAY_ENABLEDfalseEnable APISIX AI Gateway
GATEWAY_ADMIN_URLhttp://127.0.0.1:9180APISIX Admin API URL
GATEWAY_ADMIN_KEY(auto-detected)APISIX Admin API key
GATEWAY_PROXY_PORT9080APISIX proxy port
GATEWAY_REDIS_HOST(none)Redis host for shared rate limiting
GATEWAY_REDIS_PORT6379Redis port
GATEWAY_REDIS_PASSWORD(none)Redis password
GATEWAY_RATE_LIMIT_TOKENS500Token limit per consumer per time window
GATEWAY_RATE_LIMIT_WINDOW60Rate limit time window in seconds
GATEWAY_MODEper-userConsumer mode: per-user or per-group
GATEWAY_RATE_LIMIT_SCOPEper-userRate limit scope: per-user or per-model
GATEWAY_CONCURRENCY_LIMIT1Concurrency limit per consumer

Langfuse Configuration

VariableDefaultDescription
LANGFUSE_ENABLEDfalseEnable Langfuse observability
LANGFUSE_PUBLIC_KEY(none)Langfuse public key
LANGFUSE_SECRET_KEY(none)Langfuse secret key
LANGFUSE_BASEURLhttps://cloud.langfuse.comLangfuse API base URL

Running

# Streamlit dashboard (port 8501)
streamlit run dashboard/streamlit_app.py --server.port 8501

# FastAPI REST API (port 8100, optional)
python -m thon run

See Dashboard for full documentation including pages and implementation details.

On this page