Run the agent as a standalone CLI
Serve a model, boot the demo fleet, and talk to the operator agent from the terminal
Overview
@tetherto/mdk-agent runs as a small library and CLI for local development,
talking to a served model and an MCP tool server directly, with no Gateway in front of it.
This is one of two ways to run the agent.
Prerequisites
-
Node.js ≥ 24
-
npm 11 (< 12)
-
GPU:
- macOS: Metal
- Linux and Windows: Requires Vulkan 1.4+. On macOS and Linux, a missing GPU backend falls back to CPU (slower but functional). On Windows, the native runtime requires Vulkan 1.4+ even for CPU inference and fails outright if missing or on an older version. Verify with
vulkaninfo --summarybefore running.
-
The MDK repository cloned, with dependencies installed by a single
npm installat its root, not one run inside the package (the agent's prerequisites explain why):git clone https://github.com/tetherto/mdk.git cd mdk npm install -
Language model: this guide uses QVAC serving an OpenAI-compatible API on port
11500 -
For the optional demo fleet: PM2 installed globally (
npm install -g pm2, once per machine)
Optional: Start the demo fleet
If you already have an MCP tool server, skip this and point the agent at its URL when you start the agent.
Start the fleet before the model server. On its first run, QVAC downloads the model over a DHT that takes UDP port 49737
by default, the port the fleet's private DHT bootstrap needs. Started after the fleet, QVAC uses another port; started
first, it breaks the fleet's tools (see troubleshooting).
Navigate to examples/mvp-site in your mdk clone and boot the MVP site example. It brings up a
simulated fleet (miners, a powermeter, pools) under PM2 and exposes the MDK tools over MCP on port 3101:
cd <path-to>/mdk/examples/mvp-site
npm run setup # first time only. NOT npm install: examples/mvp-site is a root workspace member,
# and this also builds the UI toolkit and installs its own nested ui/ app.
# It starts by writing config/devices.json and config/site.deploy.json
npm start # kernel + workers + gateway + MCP server on :3101, under PM2npm start returns once PM2 has the processes; PM2 keeps them running in the background. Check them with pm2 list,
and stop the fleet with npm run stop:pm2 when you're done.
Serve the model with QVAC
The agent needs a language model listening locally. Already have a QVAC server running elsewhere?
Skip this step, then add --base-url <its-url> when you start the agent.
2.1 Install the QVAC CLI
QVAC installs into a separate directory to keep the repository's dependencies and lockfile clean:
mkdir -p ~/qvac-runtime && cd ~/qvac-runtime && npm init -y # on macOS/Linux
# On Windows (PowerShell): md ~/qvac-runtime; cd ~/qvac-runtime; npm init -y
npm i @qvac/cli@^0.12.0 # ~6 GB of prebuilt enginesWhy install @qvac/cli outside this package?
@qvac/cli is an optional peer dependency because the agent only needs it to serve a model, not
to talk to one. Installing its ~6 GB into this tree perturbs a package-lock.json that CI
gates on. External mode never loads @qvac/cli at all, so a separate directory costs nothing and
keeps the lockfile clean.
Optional: Free ~2.5 to 4.4 GB of disk space
@qvac/cli bundles native binaries for multiple operating systems and architectures. If disk space is tight, remove the platforms you do not need:
- macOS:
rm -rf node_modules/@qvac/*/prebuilds/{win32,linux,android,ios}* - Linux (x64):
rm -rf node_modules/@qvac/*/prebuilds/{win32,darwin,android,ios,linux-arm64}* - Windows (PowerShell):
Remove-Item "node_modules/@qvac/*/prebuilds/*" -Exclude "win32*" -Recurse -Force
(Note: This is disk relief rather than a permanent fix: running npm install again will restore them.)
2.2 Copy the model configuration
Return to the root of your mdk clone, then copy the pre-configured model definition (qvac.config.json)
into your runtime directory:
cd <path-to>/mdk
cp backend/core/agent/qvac-runtime/qvac.config.json ~/qvac-runtime/2.3 Start the model server
Confirm your host capabilities and start the server:
cd ~/qvac-runtime
npx qvac doctor # confirms GPU backend and host capabilities
# --config pins the model: Qwen3-4B, 4-bit quantized, 16k context
npx qvac serve openai \
--config qvac.config.json \
--port 11500 --verbose --no-cancel-load-on-disconnect # --verbose: download and load steps; wait until listening on 11500The port and flag above are not arbitrary: 11500 avoids Ollama's default 11434 (whichever
starts second on a machine with both dies on EADDRINUSE), and --no-cancel-load-on-disconnect
stops a lazily loaded model's first request from failing with a 503 that isn't really about a
disconnect.
If this doesn't work first try, see troubleshooting.
Start the agent and ask
In another terminal, navigate to backend/core/agent in your mdk clone and start the CLI, pointing it at the model
(port 11500) and the MCP tool server (the demo fleet's is on port 3101; if you skipped it, use your own server's URL
in --mcp-url):
cd <path-to>/mdk/backend/core/agent
node bin/mdk-agent.js --model qwen3-4b --base-url http://127.0.0.1:11500/v1 --mcp-url http://127.0.0.1:3101/mcpThe demo fleet's MCP tool server checks no token. A stack's mdk run mcp checks every agent against spec.rbac: give
the agent its own service account, then add --mcp-credential <project>/.mdk/credentials/agents/<name>.json
and --gateway-url with the stack's Gateway, which the agent's flags cover.
You may now query the site in plain language, for example:
you › how many miners are on the site?
→ tool count_devices({"family":"miner","state":"all"})
← data { "summary": "10 miners.", "count": 10, … }
▌ 10 miners.
you › list the devices that are not ready
you › reboot AM-003 ← a write: the agent stops and asks you to approveREPL commands: /about (what this is) · /tools · /info · /new · /exit.
Troubleshooting
The CLI exits with the MCP server at <url> answered 401
The MCP server checks agents against spec.rbac, and the agent sent no access token, or one it refused. Start the CLI with
--mcp-credential and --gateway-url, pointing at the credential mdk sa add wrote and the Gateway of the same stack.
npx qvac fails with a 404
When inside ~/qvac-runtime, npx qvac resolves from the local ./node_modules/.bin/qvac binary. If run outside that directory, npx searches the npm registry for a package named qvac (which does not exist; the package is @qvac/cli), failing with a 404. Run cd ~/qvac-runtime first.
qvac serve openai dies with EADDRINUSE
qvac serve openai defaults to port 11434, the same as Ollama's. Pass --port 11500 (as the
command above does) if both run on this machine.
The first request 503s with model_load_failed, blaming a disconnect that didn't happen
serve.load.cancelOnDisconnect defaults to true, which cancels a lazily loaded model's first
request. --no-cancel-load-on-disconnect fixes it. Models declared preload: true (like
qwen3-4b in qvac.config.json) never hit this, but the flag costs nothing.
The first run looks stalled, or the server "never starts"
The first run fetches ~2.5 GB peer-to-peer. The log says registry://s3/…, but the transfer is
actually Hyperswarm (a DHT plus UDP hole-punching), and takes anywhere from a few minutes to ~25 minutes depending on the network. There's
no HTTP fallback, so a network that blocks UDP stalls it, and since preload: true means the port
doesn't open until the model is resident, a stalled download presents as a dead server, not a
failed fetch. Three things help:
- Copy
qvac.config.jsonand setpreload: false. The port opens in ~5 s and a stalled fetch surfaces as a503on the first request instead of a server that looks dead. Switch back once~/.qvac/modelsholds the weights. - Always pass
--verbose: it logs when the download starts and completes. There's no progress bar, so watch the size of~/.qvacto see that the download is moving. - Bypass the registry entirely:
serve.modelsaccepts an explicit{ "src", "type" }entry, and*ModelSrcfields accept URLs and filesystem paths, so a GGUF fetched by hand over HTTPS, or copied from a machine that already has one, works. This is the path for a locked-down site.
Disk usage is larger than expected
Budget ~6 GB for the CLI, ~5 GB for the model (~2.5 GB of weights in ~/.qvac/models, plus a second copy the registry
keeps in ~/.qvac/registry-corestore), and a KV cache in ~/.qvac/kv-cache that grows over time.
Of the CLI's ~6 GB, ~4.6 GB consists of native prebuilt binaries across multiple platforms and architectures.
Deleting the foreign prebuilds reclaims ~2.5 to 4.4 GB safely — see Free foreign prebuilds in Step 2.1 for the commands.
Every tool call returns ERR_MDK_CLIENT_UNAVAILABLE: CHANNEL_CLOSED
On its first run, qvac serve joins a DHT to download the model, on UDP port 49737 by default, and keeps the port after
the download completes. The demo fleet's private DHT bootstrap uses the same port, so when QVAC starts first, the fleet's
dht-bootstrap process can't bind it and keeps restarting. This guide starts the fleet first
to avoid this. npm start still reports success and pm2 list shows the rest of the fleet online, but dht-bootstrap sits
in waiting restart (or errored) with its restart count (↺) climbing, and every tool call fails.
To confirm, run lsof -nP -iUDP:49737: a bare process holding the port is QVAC's worker.
Once the model has finished downloading (the server logs listening on http://127.0.0.1:11500):
-
Stop
qvac servewithCtrl+Cin its terminal. -
Restart the demo fleet from
examples/mvp-site:npm run stop:pm2 && npm startDo this even if
dht-bootstrapcomes back online by itself once QVAC stops: the tools still fail until the fleet restarts. -
Start
qvac serveagain with the same command. With the weights cached, it loads in seconds and doesn't join the DHT. -
In the agent CLI, start a fresh conversation with
/newand repeat your question.
The fleet has fewer devices than expected
Each Worker seeds its device list once, into a persistent on-disk store, and only when that store
is empty. A later npm start reuses whatever's already there rather than reseeding from the current
config/devices.json, so a fleet from an earlier run (a different device list, or an interrupted
seed) sticks around. Run npm run stop:pm2, then clear examples/mvp-site/.site-data for a fresh fleet on the next start.
Next steps
- Look up a flag, the capability table, or a hosted-model setup: the CLI's full reference
- Deploy the agent behind the Gateway instead, for a chat API an operator UI can call