CLI reference
Kapkan is a single binary. What it does is selected by command-line flags at startup, plus a
small number of commands that inspect the running system and exit. All logs go to
stderr; the REST API and Prometheus metrics are served over HTTP on api.listen (default
127.0.0.1:8080), while the utility flags (-version, -dump-schema, -dump-zones-schema)
and the commands print their output to stdout.
kapkan [flags] # run the daemon
kapkan [flags] <command> [args] # run a command and exit
Global flags come before the command; the command's own flags come after it. So
kapkan -config /etc/kapkan/config.yaml dataplane status -json reads the config path from the
global -config and passes -json to dataplane status. A flag that already exits on its own
(-version, -check-config, -dump-schema, -dump-zones-schema, -check-update, -s) cannot
be combined with a command — Kapkan refuses the combination rather than silently honouring one
of them.
Flags
| Flag | Default | Description |
|---|---|---|
-config <path> | configs/dev.yaml | Path to the YAML configuration file. |
-config-overlay <path> | — | Optional YAML overlay merged onto -config. Mappings merge recursively; lists and scalar values replace the base values. An empty overlay changes nothing. |
-log-format <fmt> | json | Log output format: json (structured, for log collectors) or text (human-readable, for local runs). Any value other than text is treated as json. |
-log-level <lvl> | info | Minimum log level: debug, info, warn, error. An unrecognized value falls back to info. |
-check-config <path> | — | Validate the config file at <path> and exit — does not start the daemon. After the OK line it prints warnings for legal but unwise config: static rules that can never fire, agent tokens not bound to a node (api.tokens[].node), and zones no edge node's scope covers; with an edge block it also prints each node's scope, zone count and token binding. See below. |
-dump-schema | false | Print the configuration JSON schema to stdout and exit. See below. |
-dump-zones-schema | false | Print the JSON schema of the zones file an edge brain serves (edge.zones_file) to stdout and exit. |
-version | false | Print the build version and exit. The same version is exposed at runtime via /api/v1/status and the kapkan_build_info metric. |
-check-update | false | Run the opt-in update check once and exit — 0 up to date, 10 an update is available, 1 on error. Polls the GitHub Releases API; sends only the request itself, never node identity or config. |
-pid-file <path> | /run/kapkan/kapkan.pid | Where the daemon writes its process id on start (removed on clean shutdown); read by -s to locate the running daemon. A write failure is non-fatal — the daemon still runs, only -s is unavailable. |
-s <signal> | — | Signal a running daemon and exit, by reading -pid-file: reload, stop or quit. Does not start a daemon. See below. |
ijson or text?
Use -log-format text for local development — it is the format the
Quickstart uses. Keep the default json in production so a log
collector can parse the structured fields; the systemd unit ships with
-log-format json -log-level info.
Running the daemon
With no utility flag, Kapkan opens its flow listeners, REST API and (when not in dry-run) BGP
speaker, then runs until it receives SIGINT or SIGTERM, at which point it shuts down
cleanly. It exits non-zero only on a fatal startup or runtime error.
# Local, human-readable logs against the bundled dev config:
kapkan -config configs/dev.yaml -log-format text
# Production-style: structured logs at info level:
kapkan -config /etc/kapkan/config.yaml -log-format json -log-level info
# Development differences layered onto the production config:
kapkan -config /etc/kapkan/config.yaml -config-overlay /path/to/dev.yaml -log-format text
The config file and optional overlay are reloadable at runtime without a restart — run kapkan -s reload,
send SIGHUP directly (systemctl reload kapkan), or POST /api/v1/config/reload. See the
configuration reference for what reloads and what is fixed at startup.
Controlling a running daemon
kapkan -s reload # re-read the config file (SIGHUP)
kapkan -s stop # graceful shutdown (SIGTERM)
kapkan -s quit # alias of stop
-s reads the process id from -pid-file (default /run/kapkan/kapkan.pid, written by the
daemon on start) and delivers the matching signal to that process — the same nginx-style local
control as nginx -s reload. It needs no network connection and no API token, so it works even
if the REST API is down; it does need permission to signal the daemon, so run it as root or the
kapkan user.
-s value | Signal | Effect |
|---|---|---|
reload | SIGHUP | Hot-reload the config file. Same as systemctl reload kapkan or POST /api/v1/config/reload. |
stop / quit | SIGTERM | Graceful shutdown. Same as systemctl stop kapkan. The two are aliases — Kapkan's shutdown is always graceful (it asks BGP peers to retain mitigation routes via Graceful Restart). |
If the pid file is missing or stale (no such process), or you lack permission to signal it, -s
prints the reason to stderr and exits 1.
iWhere the pid file lives
The bundled systemd unit sets RuntimeDirectory=kapkan, so /run/kapkan
exists and is writable by the kapkan user; the daemon writes /run/kapkan/kapkan.pid there and
removes it on a clean stop. In a local dev run that directory may not exist — the daemon logs a
warning and continues, and -s will report "no pid file". Pass -pid-file /tmp/kapkan.pid to both
the daemon and -s if you want the shortcut locally.
Utility flags
These flags exit before the daemon starts — they never open a listener or send a route
announcement, so they are safe to run on a production host. (-check-update makes a single
outbound request to the releases API; the others are fully offline.)
Validating a config
kapkan -check-config /etc/kapkan/config.yaml
Runs the engine's exact parse and validation — including cross-field rules a static schema cannot express — then prints the resolved configuration (mode, protected networks, the configured flow listeners, the group count, and the effective mitigation per hostgroup) and exits:
| Exit code | Meaning |
|---|---|
0 | The config is valid. |
1 | The config is invalid; the exact error is printed to stderr. |
The clean 0/1 split drops straight into CI or a pre-deploy gate. See Validating the configuration.
A valid config can still print a WARNING block after the OK line — legal config that the
daemon will run, but almost certainly not what you meant. Today there are three: a
static rule that can never fire, naming the
rule and what takes its packets; an agent token not
bound to a node (api.tokens[].node),
naming each such token; and a zone of the zones file that no edge node's
placement scope covers, naming the zone and its hostgroup.
With an edge block the resolved-configuration section also lists each node's scope, how many
zones it covers and the token bound to it (or SHARED while an unbound agent token exists). The
exit code stays 0, so an existing gate is unaffected; if
you want warnings to fail the build too, check the output for WARNING as well as the code.
Dumping the schema
kapkan -dump-schema
Prints the configuration's JSON schema to stdout and exits 0 (or 1 on an internal error).
The config builder consumes this schema to validate fields in your browser.
kapkan -dump-zones-schema
The same for the zones file an edge brain serves (edge.zones_file): its JSON
schema, with the zone vocabulary's enums and bounds, for your editor or a tenant's tooling.
Printing the version
kapkan -version
Prints the build version (release tag plus short VCS revision when known) to stdout and exits.
The same string is served at runtime on /api/v1/status and carried by the kapkan_build_info
metric — so you can confirm a binary's version locally with zero egress.
Checking for updates
kapkan -check-update
Runs the update_check once on demand and exits: 0 if up to date, 10
if a newer release is available, 1 on error. It queries the GitHub Releases API and sends only
the request itself (your IP and a generic User-Agent) — never node identity, config or attack
data. The periodic in-process check is off by default; this flag works regardless of that
setting.
Commands
dataplane status
kapkan dataplane status # human-readable report
kapkan dataplane status -json # the same inspection as JSON
Reports whether the XDP data plane is actually filtering, and why not when it isn't. Two properties make it usable during an incident:
- It is strictly read-only. It opens the pinned program and maps read-only — the map file
descriptors carry
BPF_F_RDONLY, so the kernel refuses a write through them — and it never loads a program, creates a map, writes a map value, attaches, detaches or removes a pin. Starting the daemon adopts-or-rebuilds a pin set; this command cannot, so it can never cost you the mitigation rules you are trying to diagnose. - It works with the daemon stopped. That is the main case, not a bonus: with the default
dataplane.on_exit: keep, the kernel goes on enforcing static policy with no Kapkan process at all, and this command reads the pins directly to tell you so.
The report leads with the verdict and the remedy, then gives the detail: attached interfaces with the mode actually in force, the kernel and map-schema versions, the live generation, static and dynamic rule counts, per-map size/occupancy/memory, the dry-run flag, and the verdict counters.
Two things in that output are worth knowing about in advance:
GENERICattach mode is called out on the first line.xdp_mode: autofalls back from the driver path to the generic (skb) path without failing, and that costs roughly an order of magnitude of capacity, so it is a headline and not a field among twenty.- Verdict counters are printed in two blocks. The terminal counters partition the traffic —
exactly one is bumped per packet — so they are totalled. The observation counters are bumped
alongside a terminal one for the same packet (
dryrun_would_drop,pass_rule_expired,pass_frag_noports,err_policy_missing); adding them to the total would produce a packet count larger than the number of packets, so they are listed separately and never summed in.
| Flag | Default | Description |
|---|---|---|
-json | false | Print the whole inspection as JSON, including the state field the exit codes are derived from. |
-pin-path <dir> | from -config | The bpffs directory to inspect. Defaults to dataplane.pin_path from the config named by the global -config, falling back to /sys/fs/bpf/kapkan when that file cannot be read. The report always states which path it looked at and where that path came from. |
Exit codes
| Exit code | Reported state | Meaning and what to do |
|---|---|---|
0 | enforcing | At least one interface has a live XDP attachment. Nothing to do. |
10 | no_pin_path, no_program, detached | The data plane is not filtering, and nothing about it is broken — it has never run here, or it is stopped, or its attachments are gone. Start Kapkan, or check that the configured interfaces exist. |
11 | not_bpffs, torn, schema_skew | Something must be fixed before it can work: bpffs is not mounted, the pin set is torn, or the pinned maps are a different schema version than this binary. The printed reason names the fix (mount bpffs; restart Kapkan). |
1 | attach_unknown, or any read failure | The command could not answer — most often permission (see below). Deliberately not reported as "not filtering", because it may well be filtering. |
2 | — | Usage error: an unknown flag, command or stray argument. |
Only enforcing exits 0, so kapkan dataplane status >/dev/null drops straight into a
monitoring check. The 10-for-a-meaningful-state convention is the same one
-check-update uses.
Permissions
Reading pins is much cheaper than creating them: on a kernel with
kernel.unprivileged_bpf_disabled=0, no capability at all is required — neither
CAP_NET_ADMIN nor CAP_PERFMON, which the daemon needs in order to load and attach, is
involved. What does gate it:
- The pin directory is mode
0700, owned by the user the daemon runs as, so in practice you wantsudo kapkan dataplane status. - On a kernel with
kernel.unprivileged_bpf_disabledset — the default on Debian and Ubuntu —bpf(2)additionally needsCAP_BPF, even to open an existing pin. - Link pins need write permission, not just read: the kernel refuses
BPF_OBJ_GETon abpf_linkunless the descriptor is openedO_RDWR. A non-root reader therefore sees every map and no attachment, which is reported asattach_unknown(exit1) rather than as "detached".
iIt never mutates the thing it is diagnosing
This is why dataplane status is a separate read-only path instead of reusing the daemon's own
startup code. Starting the data plane adopts an existing pin set — or, when the schema does not
match, tears it down and rebuilds it, discarding every dynamic mitigation rule in the kernel. A
diagnostic that could do that mid-attack would be worse than no diagnostic.
scrub
kapkan scrub -config /etc/kapkan/scrub.yaml
Runs the scrub-node role: the same binary, a different job. There is no detection, no BGP and no telemetry listener — the box receives traffic the brain diverted to it, long-polls the brain's rule table, and keeps its local XDP data plane enforcing exactly what the brain says to drop. The poll doubles as the node's liveness signal.
Everything role-specific lives in its own scrub.yaml; the daemon's config.yaml is not
read (-config here defaults to /etc/kapkan/scrub.yaml). It has three parts: the controller to
reach, the node's identity, and the same dataplane: block a daemon carries.
dry_run: true # the remote-role default — counts, drops nothing
controller:
url: "https://kapkan.example.net:8443" # the brain's API base (no path)
token_env: KAPKAN_AGENT_TOKEN # env var holding the agent token (required)
name: scrub-fra1 # must equal a scrubbing.nodes[] name on the brain
dataplane:
interfaces: [eth0] # the dirty side (diverted traffic arrives here)
xdp_mode: auto
pin_path: /sys/fs/bpf/kapkan
!A scrub node defaults to dry-run
dry_run defaults to true for a remote role — the node installs the rules and counts what
they would drop, but drops nothing, until you set dry_run: false explicitly. It never
enforces a rule the brain marked dry-run while it is live, and it requires a real agent token:
without one its polls carry no identity and the brain would count it dead. Use https outside a
lab — a plaintext controller.url sends the token in the clear.
| Flag | Default | Description |
|---|---|---|
-config <path> | /etc/kapkan/scrub.yaml | The scrub-node config. |
-log-format <fmt> | json | json or text. |
-log-level <lvl> | info | debug, info, warn or error. |
Confirm a running node with kapkan dataplane status on the node itself — it reports ENFORCING
and the installed rule count — and the brain's Nodes view shows it alive.
Getting the diverted traffic to the node is the network integration guide.
edge
kapkan edge -config /etc/kapkan/edge.yaml
Runs the edge-node role: the box's nginx or Angie becomes a managed HTTPS front
for the zones the brain serves. The role long-polls the brain's zone document, renders the
terminator's configuration and installs it behind nginx -t, answers nginx's auth_request
from a local decision service, reads the access log into per-source rollups, issues and renews
the zones' certificates on the box, and reports itself. The poll doubles as the node's liveness
signal. The step-by-step is the installation guide.
Everything role-specific lives in its own edge.yaml; the daemon's config.yaml is not
read (-config here defaults to /etc/kapkan/edge.yaml).
dry_run: true # the remote-role default — marks, refuses nothing
controller:
url: https://kapkan.example.net:8443 # the brain's API base (no path); https outside a lab
token_env: KAPKAN_EDGE_TOKEN # env var holding the agent token (required)
name: edge-1 # must equal an edge.nodes[] name on the brain
report_interval_seconds: 10 # self-report cadence
state_dir: /var/lib/kapkan-edge # document cache, generations, ACME keys and certificates
sockets_dir: /run/kapkan-edge # edge-decide.sock, edge-clearance.sock, edge-log.sock, edge-challenge.sock
socket_group: nginx # the terminator's worker group (nginx | www-data | angie)
terminator:
binary: nginx # nginx (default) | angie
main_conf: /etc/nginx/nginx.conf # -c for `nginx -t` and `nginx -s reload`; empty = compiled default
reload: exec # exec (default) | signal | command
pid_file: /run/nginx.pid # liveness check; required for reload: signal
# command: [systemctl, reload, nginx]
acme:
directory: "" # default CA; empty = Let's Encrypt production
fallback: "" # optional second CA after three consecutive failures
contact: ["mailto:ops@example.net"]
# eab: # External Account Binding for CAs that require one
# - directory: https://acme.zerossl.com/v2/DV90
# kid: your-kid
# hmac_key_env: KAPKAN_EDGE_ZEROSSL_HMAC
# disabled: true # issue nothing (zones stay on :80 answering 503)
status_listen: 127.0.0.1:9102 # /healthz + /metrics; unauthenticated, keep it private
omit_catch_all: false # true when nginx.conf has its own default_server
disable_ipv6: false # true on hosts without an IPv6 stack
quic:
h3: auto # auto (default) | off — may this box render HTTP/3 at all
# retry: true # nginx's quic_retry, node-wide (default on)
# omit_anchor: false # with omit_catch_all: your own server carries `listen 443 quic reuseport`
| Key | Meaning |
|---|---|
dry_run | Absent or true: decisions are counted and marked (X-Kapkan-Mark: would-deny:<reason> reaches the origin), none enforced. Set false to refuse. |
controller.url / token_env / name | The brain's API base (no path, no credentials in the URL), the environment variable holding the agent token, and this node's name — it must equal an edge.nodes[] entry on the brain or every poll is a loud 404. |
state_dir / sockets_dir | The role's own directories (defaults /var/lib/kapkan-edge, /run/kapkan-edge) — never the brain's /var/lib/kapkan and /run/kapkan, which systemd would re-chown and remove under it on a shared host. Absolute, no characters nginx would misread. state_dir/tls/quic_host.key is the terminator's QUIC host key (32 random bytes, 0600), minted once at the node's first start and kept across restarts so Retry and stateless-reset tokens survive a reload; delete it to mint a new one. |
socket_group | The terminator's worker group. The decision, log and clearance-page sockets are 0660 for it; the ACME challenge socket is 0666 (it answers public tokens). Empty leaves the three group-restricted sockets owner-only. |
terminator.binary / main_conf | What to run for -t and reloads, and the main configuration to pass as -c. That file must include /var/lib/kapkan-edge/conf/live/*.conf; once, inside http {}. |
terminator.reload | exec runs <binary> -s reload (needs to signal nginx's master — the unit runs as root); signal sends HUP to the pid in pid_file; command runs terminator.command, e.g. [systemctl, reload, nginx]. |
terminator.pid_file | When set, read on every report tick and signalled with 0: /healthz answers 503 and the report says alive: false while no process has that pid. |
acme.directory / fallback / contact | The node's default CA (a zone's own acme block overrides directory and fallback), the fallback tried after three consecutive failures (must differ from the resolved primary), and the account contact — each entry a mailto: URL (mailto:ops@example.net), or the file is refused. |
acme.eab[] | External Account Binding per directory URL (directory, kid, hmac_key_env) for CAs that require one — ZeroSSL, Google Trust Services. The HMAC key is a secret and is read from the named environment variable. |
acme.disabled | Issue nothing: zones stay on :80 answering 503 (a lab that exercises everything but issuance). Operator-supplied certificates are not a feature yet. |
status_listen | Serves /healthz and /metrics when set. Unauthenticated — loopback or a private address. |
omit_catch_all / disable_ipv6 | Drop Kapkan's default_server catch-all (your nginx.conf declares one) / drop the [::] listeners (no IPv6 stack). |
quic.h3 | auto (default) or off. The node probes the binary at start (nginx -V): with auto, HTTP/3 is rendered for zones that ask when the build carries --with-http_v3_module; off never renders it on this node — the switch for a build whose HTTP/3 you do not trust, for instance one the probe flags with a CVE advisory. Either way the node's report and /healthz carry h3.state: ready, no_module, node_off or unknown (the probe failed; the node renders no QUIC then). Zones asking for HTTP/3 are served over TCP on a node that is not ready. |
quic.retry | nginx's quic_retry, node-wide — nginx decides it from the address's default server before SNI names a zone, so it cannot be per zone. true (default): every new client address proves it can receive before the handshake costs the node anything (a Retry is smaller than the Initial it answers, so a spoofed source amplifies nothing). false saves that round trip. Changing it is a new tested generation on a node that renders QUIC for at least one zone; on a node with no h3 zone the value simply waits for the first h3 render. |
quic.omit_anchor | Only with omit_catch_all: true. Kapkan's catch-all normally carries the address's one listen 443 quic reuseport; without the catch-all a bare QUIC anchor server carries it. Set true when your own default server already listens 443 quic reuseport — two reuseports on one address fail nginx -t and the generation is refused (the previous one keeps serving). Your server then carries the anchor's lines itself: reuseport, ssl_protocols TLSv1.3 and the zones' session cache (ssl_session_cache shared:kapkan_ssl:10m; ssl_session_timeout 1d; ssl_session_tickets off;), the cache being where HTTP/3 sessions started on it live. |
!An edge node defaults to dry-run
dry_run defaults to true for a remote role — the node counts and marks what it would
refuse but refuses nothing, until you set dry_run: false explicitly. It requires a real agent
token: without one its polls carry no identity and the brain would count it dead. Use https
outside a lab — a plaintext controller.url sends the token in the clear.
| Flag | Default | Description |
|---|---|---|
-config <path> | /etc/kapkan/edge.yaml | The edge-node config. |
-check | off | Validate the file and what it names on this box — the socket group must exist and any EAB key be well-formed (problems); an absent token, terminator binary or main_conf is a warning — and, when the binary is on PATH, print what its -V says: kind, version, nginx core, TLS library, whether it has the HTTP/3 module and could do 0-RTT, the node's HTTP/3 readiness and the effective quic.retry (a CVE advisory for the build's core is a warning, unless quic.h3: off has already turned HTTP/3 off here) — then exit. |
-log-format <fmt> | json | json or text. |
-log-level <lvl> | info | debug, info, warn or error. |
Confirm a running node with curl 127.0.0.1:9102/healthz — 200 with "converged":true once
the document is rendered and live — and the brain's inventory (GET /api/v1/edge/nodes) shows
it alive. The brain-side schema of the zones file is kapkan -dump-zones-schema.
nginx-exporter
KAPKAN_API_TOKEN=... kapkan nginx-exporter -log /var/log/nginx/kapkan.json.log -api http://127.0.0.1:8080
Tails an nginx JSON access log, measures per-source request rates per window, and posts verdicts
to the brain's source-block channel. The required
log_format, the thresholds and the operational bounds live with the feature:
the reference exporter. The
daemon's global flags are not read; everything is its own:
| Flag | Default | Description |
|---|---|---|
-log <path> | — (required) | The nginx access log, in the documented JSON log_format. |
-api <url> | http://127.0.0.1:8080 | The brain's API base URL. |
-token-env <name> | KAPKAN_API_TOKEN | Env var holding an operator token. |
-victim <ip> | — | Fixed victim; overrides the log's dst field. |
-window <dur> | 10s | Measurement window. |
-rps <n> | 50 | Per-source requests/second that arms a block. |
-min-requests <n> | 100 | Floor per window before rate arithmetic applies. |
-error-ratio <r> | 0 | Additionally require this 4xx/5xx share (0 disables the axis). |
-ttl <dur> | 5m | Block TTL; a still-hot source is refreshed before it lapses. Must be at least 2×-window. |
-observe | off | Log verdicts without posting them. |
-log-format / -log-level | json / info | The exporter's own logging. |
Next steps
- Quickstart — download Kapkan and run it in dry-run.
- Configuration reference — every key in the YAML file.
- Production deployment — the systemd unit and its flags.
- Troubleshooting — symptom-keyed fixes, including the data plane.