Troubleshooting
Every entry in this catalog follows one shape: symptom → diagnosis → safe command → expected output → recovery → escalate when. Commands are exact names from the generated CLI reference or tools from the generated tool registry; they never touch infrastructure you do not control and never require credentials beyond your own account.
Working as designed, not broken
Before treating an empty or unusual result as a fault, check whether it is one of these:
- Abstention — recall refuses to answer when evidence is missing, stale, or
contradictory (
no-evidence,unresolved-contradiction,insufficient-confidence,semantic_unavailable,stale_beyond_tolerance, and others). That is correct behaviour, not a recall bug. - Not evaluated — a health metric whose measurement plane produced no value is rendered
not_evaluated, never as a number. - Degraded — a third-party surface that needs the managed cloud returns a structured
degradedresponse when the plane is unreachable. - Typed absence — a graph query that matches nothing returns
absent_unindexed(corpus not built) orabsent_verified(built and covered) — never a bare empty list. - Fails closed — surfaces that need a credential or executor refuse explicitly
(
capability_required,not_wired,credential_required) instead of half-working.
The general diagnosis loop is at the end: Incident recovery loop.
Install
Symptom: install or build fails early. Diagnosis. The workspace pins Node 24
(engines.node: >=24 <25) and pnpm 11.x; a mismatched runtime fails at build, not at first
tool call. Linux x64 is the only CI-proven platform. Safe command. node --version then
pnpm --version. Expected output. Node 24.x and pnpm 11.x. Recovery. Install the
pinned versions and retry pnpm install --frozen-lockfile — no other package manager is
supported. Escalate when. You are on the pinned versions and install still fails.
Symptom: no working npx zephr. Diagnosis. Working as designed — npm publication is
not shipped today; the working path is build from source (see
Install). Recovery. Build from a clone; the server entry
point is produced under apps/mcp/dist (source build main.js; packaged bundle main.cjs).
Escalate when. Distribution status changes — watch Releases.
Symptom: search or code-intelligence tools report unavailable. Diagnosis. zephr_grep
(lexical search) uses the optional ug (ugrep) binary and zephr_lsp (structural search)
uses typescript-language-server; features degrade gracefully when the host-level binary is
absent. Safe command. ugrep --version and typescript-language-server --version.
Recovery. Install the missing optional binary. Escalate when. Installed and on PATH
but still reported unavailable.
Connect
Symptom: the client reports the server failed to start. Diagnosis. The MCP server
command/args do not launch the built server, or a path is relative. Safe command.
ls -l /absolute/path/to/zephr/apps/mcp/dist/. Expected output. The built entry point
exists at the absolute path. Recovery. Use the exact configuration shape from
Configure your client with absolute paths and
ZEPHR_DB_PATH in the env block. Escalate when. Path correct, file exists, launch
still fails.
Symptom: the server starts but every tool call is refused. Diagnosis. The process is
running without an admitted worktree — a bare zephr-mcp process with only ZEPHR_DB_PATH
is useful for a smoke test only. Recovery. Run the connect flow
(Connect your client) to admit a worktree — see
Scope and admission.
Symptom: features work in one client but not another. Diagnosis. Adapters are wired
per client; the Cursor adapter (@zephr-ai/adapter-cursor) is a mapper library the operator
wires into the brokering process, not an editor bundle. Recovery. Wire the adapter per
its documentation. Escalate when. Wired per docs and a tool call still fails only there.
Auth
Symptom: a command reports you are not authenticated. Diagnosis. No valid credential;
the CLI refuses rather than guessing (for example "Not authenticated. Run zephr login
first."). Safe command. zephr login. Expected output. A device code to authorize
(OAuth 2.0 device flow, RFC 8628). Recovery. Complete the flow; re-enroll a stale
enrollment with zephr connect --force. Escalate when. The flow completes and the
command still reports not_authenticated.
Symptom: the device code expires mid-flow. Diagnosis. Device codes are short-lived;
the flow reports expired_token on expiry or timeout. Recovery. zephr login again, or
zephr login --force to re-authenticate from scratch. slow_down needs no action (the
client backs off); access_denied means the authorization was declined — start a fresh
login. Escalate when. Fresh codes keep expiring before authorization.
Symptom: roles are missing or a change does not apply. Safe command. zephr role list, zephr role show <role>, zephr role use <role>. Recovery. Select the role you
intend to operate under. A revoked role will not work again until re-granted (error
vocabulary includes role_revoked, role_deleted).
Symptom: an administrative operation is refused. Diagnosis. zephr_admin is an
MCP tool, not a CLI command. Its actions require both the operator-set ZEPHR_ADMIN_TOKEN
and trusted user presence. Local stdio does not bind a presence surface, so it refuses admin
actions with presence_attestation_unavailable even when the token is set. Recovery.
For local data, use zephr memory retract <id> for beliefs or zephr files rm --id <id>
for files; export and run zephr uninstall --force when removing the whole installation. Use
zephr_admin only through a surface that supplies trusted presence. Escalate when. A presence-enabled
surface refuses a correctly configured action.
Scope and admission
Symptom: tool calls refused with scope_violation. Diagnosis. A worktreeId was
passed outside the admitted scope, or the binding points at a checkout you are no longer in.
Safe command. Re-run zephr connect — on success it prints the resolved project and the
admitted binding (Project: …, Admitted: binding=…; see
Connect your client). Expected output. The project and
binding you intend, for the checkout you ran it from. Recovery. Run the connect flow
against the correct checkout and restart the client; re-point a stale binding with
zephr connect --force (re-enroll) — never by editing a file. Escalate when. A fresh,
correct binding still returns scope_violation.
Symptom: launches report no active binding. Diagnosis. No active path binding exists for the project — the server has no admitted worktree to scope against. Recovery. Run the connect flow to approve the project and bind its checkout. Detaching a binding does not delete data, and re-running the connect flow re-attaches it.
Daemon
Symptom: CLI commands exit with code 69. Diagnosis. A required dependency — the
on-box zephrd daemon, a backing binary, or a network endpoint — is not reachable. Exit 69
is the dedicated "unavailable" code; the caller must not fabricate data as a substitute.
Safe command. zephr health and zephr status. Expected output. Whether zephrd
is running and the handshake state. Recovery. zephr daemon start; if the binary is
missing, place zephrd on PATH or point at it with zephr daemon start --bin <path>
(ZEPHR_DAEMON_BIN and ZEPHR_DAEMON_SOCKET are honoured). Escalate when. The daemon
reports running and commands still exit 69.
Symptom: zephr health says the handshake is down. Safe command. zephr doctor.
Expected output. Per-check diagnostics with stable ids (binary.identity,
binary.attestation, daemon.socket — best-effort, client.handshake.<name>,
install.cold_runner_risk, schema — best-effort, skipped locally), each
pass / warn / fail with a recovery hint; the report is ok, degraded, or
unhealthy; exit 0 when all pass, 1 when any fail, 2 for unsatisfiable usage. Recovery.
Follow the failing check's hint (for daemon.socket: start the daemon or correct
ZEPHR_DAEMON_SOCKET). Escalate when. unhealthy after following the hint.
Symptom: the daemon crash-loops. Diagnosis. A crash breaker counts crash-marked
starts within a short window (default 5 in 60s) and refuses to continue as a daemon beyond
that, writing marker files next to its runtime state: zephrd.crash when a process
panicked, zephrd.crash-loop when the breaker tripped. Recovery. Read the daemon log
(zephrd.log) first — forced restarts inside the window just feed the breaker. Fix the
cause, then clear the stale zephrd.crash marker so the next start is not counted. The
marker files are nothing but counters' evidence — no store data lives in them. The daemon
also auto-exits after an idle period.
Recall
Symptom: recall returns nothing you expected. Diagnosis. One of: the belief is not
eligible (only the core-eligible set — confirmed and valid — is recallable by default); a
policy narrowed eligibility further (zephr_rules, recallEligibleStates can only narrow);
the query matched nothing; or the belief belongs to a different worktree. Safe command.
zephr recall, then zephr why for the belief's review state and provenance, and the MCP
zephr_status report for retrieval modes. Recovery. Confirm state and scope before
treating it as a fault. Escalate when. A confirmed, valid, in-scope belief with a
matching query is never returned.
Symptom: recall returns a structured abstention. Diagnosis. Recall abstains rather
than guessing; the code names the reason (no-evidence, unresolved-contradiction,
insufficient-confidence, semantic_unavailable, provenance_fail,
stale_beyond_tolerance, calibration_stale, budget_exhausted). Safe command. Ask
the client for the MCP zephr_status report. Expected output. zephr_status reports
the retrieval flags (lexical, semantic); an embedding block is present only when the
semantic tier is available (capture responses additionally carry a per-belief embedding
lifecycle state — pending | available | unavailable). Recovery. Treat the code as
the diagnosis — e.g. semantic_unavailable means the semantic tier is required by the call
and not usable, while default requests disclose a lexical_fallback instead. Escalate
when. A code contradicts the report (for example semantic_unavailable while
zephr_status reports the embedding block with retrieval.semantic: true).
Symptom: recall quality is poor. Diagnosis. Often eligibility or provenance, not
ranking: unreviewed captures rank below confirmed ones and unprovenanced beliefs are reported
honestly as unproven. Safe command. zephr timeline (chronological handles) and zephr why. Escalate when. A clearly matching, confirmed, provenanced belief ranks far below
weaker matches — report with a reproducible example.
Indexes
Symptom: a graph query returns absent_unindexed. Diagnosis. The entity-relation
edge graph was never built for this installation — the typed reason distinguishes that
from absent_verified (built and covered, but no edge matched). Recovery. Confirm the
corpus build that owns the graph ran; absent_verified is a genuine "no answer".
Escalate when. The corpus build completed and the query still reports
absent_unindexed.
Symptom: semantic recall degrades or abstains. Diagnosis. The semantic tier needs the
embedding capability. When it is unavailable, default requests still return lexical results
but disclose the fallback (lexical_fallback mode); calls that require semantic abstain
with semantic_unavailable. Nothing is fabricated and nothing degrades silently. Safe
command. Ask the client for the MCP zephr_status report. Recovery. While embeddings
are unavailable, default recall still answers lexically (disclosed); semantic-only requests
abstain honestly. Escalate when. zephr_status reports the embedding block with
retrieval.semantic: true and recall still abstains or falls back.
Symptom: hosted docs answers look stale. Diagnosis. Third-party docs come from the
hosted catalog; project docs come from the admitted worktree in local mode. A hosted action
whose plane is unreachable returns degraded — never a fabricated answer. Recovery. For
project docs, re-import after the change; for hosted sources, retry when reachable and use
the staleness probe. Escalate when. Project-docs actions return stale content from an
unchanged worktree.
Trust
Symptom: a policy change made recall return less. Diagnosis. The Trust Firewall
(zephr_rules) only narrows the core-eligible set — it never broadens retrieval.
Safe command. Read the current policy with zephr_rules. Recovery. Widen the policy
back toward the core-eligible set. Escalate when. A valid policy is rejected (the error
names the offending state).
Symptom: zephr verify fails. Diagnosis. Structural (T1) and consistency (T2)
checks — current-version pointer, contiguous revisions, non-overlapping transaction
intervals, one open version, provenance anchors, review/validity state — do not hold, or the
belief is not yet in a well-formed, provenanced, confirmed state. Safe command. zephr verify --claim-id <id>
(alias verify-claim --claim <id>), then zephr why. Expected output. A verification
profile; only well-formed, provenanced, confirmed beliefs are reported verified.
Escalate when. A previously verified, unchanged belief starts failing.
Symptom: governance or review actions are refused. Diagnosis. The constraint surface
fails closed when the governance ports are not bound — the refusal names the missing wiring
explicitly ("zephr_constraint is not wired (no governance ports bound).", code
capability_required); human-gated confirmations refuse non-human principals
(confirm_denied); zephr_review returns a structured abstention when the substrate is
unavailable — a verdict is never fabricated. Recovery. Wire the missing governance port;
run human-gated confirmations as a human; for review abstentions read the reason
(no_worktree_scope, change_resolver_required). Escalate when. Ports bound, human
principal, still refused.
Sync
Symptom: a sync run exits with code 5. Diagnosis. Drift — the run completed with
partial success; some clients/surfaces applied, others were skipped or conflicted. Safe
command. zephr sync status --json and zephr sync push --dry-run. Expected output.
Per-client outcomes (synced, drifted, conflict, not-installed) and what a push would
apply. Recovery. Reconcile the surfaces that did not reach synced. Escalate when.
Everything reports synced and the run still exits 5.
Symptom: a client is stuck at drifted or conflict. Diagnosis. Drift classification
is explicit: a region inside the snapshot's fence that contradicts the projection is a
managed-edit and halts apply (conflict); a region outside any fence is a hand-edit
(your private data) and is preserved. Recovery. Re-adopt the projection with zephr sync adopt. Escalate when. Re-adoption loses your hand-edits — report with the before/after
projection.
Symptom: a client shows not-installed. Diagnosis. The client config file is absent,
or no projection is recorded yet (first sync). A not-installed client is not an error
and does not affect the exit code. Recovery. If installed, sync once so a projection is
recorded. Escalate when. Config present, sync ran, state unchanged.
Providers
Symptom: a hosted tool returns not_wired. Diagnosis. No executor is configured for
the tool (Linear and Supabase "fail closed with not_wired when no executor is configured").
The response carries remedies. Safe command. zephr integrations. Expected output.
Installed integration connections and status. Recovery. Wire the executor per the
remedies and restart the server. Escalate when. Wired, restarted, still not_wired.
Symptom: a call returns 401 with a connect_ref. Diagnosis. Credentials are missing;
the handler mints an opaque single-use connect_ref (credential_required) — never a
device code or verification URI. Recovery. Run the connect flow that redeems it; a BYOK
key (tools.<provider>.key) makes the call a cost-only pass-through against your key.
Escalate when. Redemption succeeds and the call still returns credential_required.
Symptom: a call returns 404, 403, or 500 from the executor. Diagnosis. Explicit
vocabulary: 404 not_found = unknown tool (no connect_ref); 403 forbidden_operation =
known tool, disallowed operation; 500 outbound_host_denied = target host not on the
allowlist, refused before any network request. Recovery. Fix the tool name, use an
allowed operation, or an allowlisted host — do not bypass the host allowlist.
Escalate when. A previously working call starts returning one of these.
Symptom: a docs action returns degraded. Diagnosis. zephr_docs third-party
actions depend on the hosted plane; project-docs actions are unaffected (they operate against
the admitted worktree locally). Recovery. Retry when the plane is reachable; never
substitute fabricated content. Escalate when. The plane is healthy and the action still
degrades.
Files
Symptom: a file's markdown is not available. Diagnosis. The file is unparsed (or
parsing pending). The vault never fabricates markdown: an unparsed file returns
file_unparsed naming its parse status. Safe command. zephr files ls. Recovery.
Wait for parsing or re-upload; the file itself is intact. Escalate when. Unparsed long
after upload.
Symptom: a file id returns "file not found". Diagnosis. The id does not exist in your
vault or belongs to another scope. Safe command. zephr files ls. Recovery. Use an
id from the list; erasing a missing id reports erased: false, reason: 'not_found' honestly.
Symptom: an upload is refused. Diagnosis. One of the admission rules fired: per-file
cap (100 MB), storage quota (storage_limit_exceeded, 413, with a remedy), empty uploads
(refused), or no object plane (503 object_plane_unavailable). Recovery. Match the rule:
shrink the file, free storage or raise the quota, send content, or confirm the deployment has
an object plane. Escalate when. Under the cap, storage has headroom, and upload still
fails.
Quota and billing
Symptom: an action is refused for budget or capacity. Diagnosis. The refusal names
the condition — budget_exhausted (recall abstains on budget), capacity_exceeded,
storage_limit_exceeded, or a metered hosted tool that is band-gated and fails closed on
meter outage (no un-metered path). Safe command. zephr usage review (with --json for
machine output). Expected output. The usage review view: allowance, consumed, remaining,
the current band (ok, soft, degraded, suspended, not_configured), breakdowns, and
remedies. Recovery. Compare the refusal against the view; if the view shows headroom the
refusal is a metering problem, not spend. Escalate when. Headroom shown and refusal
persists.
Symptom: a review refuses on affordability. Diagnosis. Entitlement-before-work:
zephr_review_usage is the read-only pre-flight returning the same review-usage view, and a
review that would exceed the entitlement is refused before work starts. Recovery. Treat
the pre-flight as authoritative — do not bypass it by calling the provider directly.
Symptom: billing looks unconnected on-box. Diagnosis. When the ledger is not
reachable the view reports band not_configured with available: false (reason
ledger_unreachable_on_box) — honest, not a fault. Note on pricing. Only the Free tier
is a committed offer; the commercial price list is still planning, and the only shipped,
public pricing artifacts are the per-invocation credit rows in the generated
MCP tools reference.
Workers
Symptom: a pod command is refused. Diagnosis. Pod lifecycle verbs are
capability-gated: launch, pause, resume, terminate, exec, mcp, and acp require
consent or return 403 capability_denied; 404 tenant_not_found means no tenant resolved.
Inspection verbs (list, show, logs, egress-audit) are ungated. Safe command.
zephr pod list, zephr pod show <pod-id>. Recovery. Obtain consent for the gated verb;
check account context on tenant_not_found. Escalate when. Consent granted and the pod
is yours, still refused.
Symptom: a pod launch fails or misbehaves. Diagnosis. pod launch requires a
template — the error pod launch requires --template <id> appears when it is missing.
Safe command. zephr pod logs <pod-id>, zephr pod egress-audit <pod-id>, then pause,
resume, or terminate. Expected output. egress-audit lists the outbound hosts a pod
attempted to reach, with reason and time. Escalate when. A pod cannot be terminated.
Note the compute plane is evolving — end-to-end hosted execution depends on the plane
being wired for your deployment.
Symptom. An agent does not run after you trigger it.
Diagnosis. zephr agent run <id> only enqueues a run — the agent worker is the sole
entry point to execution, and runs never execute in-process.
Safe command. zephr agent list, zephr agent logs.
Expected output. Agent definitions and run history.
Recovery. Confirm the run was enqueued and inspect run history. The agent surface is evolving — a missing execution means the execution plane did not claim the run, not that your CLI executed it.
Escalate when. A run never appears in run history after enqueueing.
Cloud API (OpenAPI)
Symptom: a cloud API call fails. Diagnosis. Shared error responses: 401 Unauthorized ("Authentication required"), 400 BadRequest ("Invalid input"), 404 NotFound
("Resource not found"). Recovery. 401 → zephr login (see
Auth); 400 → validate the payload against the generated
Cloud API reference; 404 → confirm the
operation is in the public manifest. Escalate when. A payload that validates against the
published schema returns 400.
Symptom: the served API disagrees with the generated reference. Diagnosis. A parity gate keeps the runtime document aligned with the canonical source; the reference covers only operations classified public, and coverage reflects checked parity only. Recovery. Check the operation's owner, status, and evidence in the generated reference. Escalate when. A shipped operation behaves differently from its published reference — report the drift with the operation id and observed response.
Privacy and deletion
Symptom: data "disappeared" after exit. Diagnosis. The server ran on the in-memory
store — with ZEPHR_DB_PATH unset, a plain process falls back to :memory: and discards
everything at exit. Recovery. Set ZEPHR_DB_PATH to an absolute path in the client's
env block before capturing anything you want to keep (see
Configuration).
Symptom: a deleted memory still appears somewhere. Diagnosis. Memory deletion
tombstones the active version; the immutable version chain keeps history verifiable.
Physical purge is separate and requires both an admin token and trusted user presence; local
stdio cannot perform it. Recovery. Locally, retract/tombstone the belief or remove the
whole installation after exporting. Use MCP zephr_admin purge only through a trusted
presence-enabled surface (see Data lifecycle). Escalate
when. A presence-enabled purge reports success and recall still returns the record.
Symptom: you need to delete everything. Diagnosis. zephr uninstall removes local
Zephr data while preserving SQLite backups; it is a dry run by default (--force
executes). Recovery. Export first (zephr pack, zephr handoff --file, zephr receipts export), then uninstall; the preserved .db/.sqlite files are your recovery
source. Escalate when. You believe erasure did not erase — report the surface and the
operation that reported success. Do not delete store files by hand while the server is
running.
Incident recovery loop
When a symptom does not fit one area, run the loop:
- Triage.
zephr health→zephr doctor→zephr status, in order.doctorgives per-check ids withrecoveryhints — follow the failing check's hint first. - Read the exit code. 0 success, 1 catch-all, 2 usage, 3 data integrity, 4 I/O, 5 drift (partial sync — see Sync), 69 unavailable (daemon/binary unreachable — see Daemon), 130 interrupted (SIGINT); on-box evidence commands add 64 (usage) and 77 (honest abstention).
- Check "working as designed" (top of this page) before escalating — abstention,
not_evaluated,degraded, typed absence, and fail-closed refusals are correct behaviour. - Recover the store per Data lifecycle: restore the
verified recovery snapshot, never hand-edit SQLite or seal files, and verify a recovered
belief with
zephr verify --claim-id <id>. - Escalate honestly. Collect the evidence (doctor report, exit code, exact error code, smallest reproduction), confirm it is not a designed state, and report through the public repository's issue tracker (see Install for the repository) — without credentials, tokens, or private infrastructure details.
Do not route around a fail-closed refusal — bypassing an allowlist, fabricating a result, or running hosted-plane recovery commands from a local installation is how small incidents become data-loss incidents.