Zephr Docs

Troubleshooting

Every entry in this catalog follows one shape: symptom → diagnosis → safe command → expected output → recovery → escalate when. Commands are exact names from the generated CLI reference or tools from the generated tool registry; they never touch infrastructure you do not control and never require credentials beyond your own account.

Working as designed, not broken

Before treating an empty or unusual result as a fault, check whether it is one of these:

  • Abstention — recall refuses to answer when evidence is missing, stale, or contradictory (no-evidence, unresolved-contradiction, insufficient-confidence, semantic_unavailable, stale_beyond_tolerance, and others). That is correct behaviour, not a recall bug.
  • Not evaluated — a health metric whose measurement plane produced no value is rendered not_evaluated, never as a number.
  • Degraded — a third-party surface that needs the managed cloud returns a structured degraded response when the plane is unreachable.
  • Typed absence — a graph query that matches nothing returns absent_unindexed (corpus not built) or absent_verified (built and covered) — never a bare empty list.
  • Fails closed — surfaces that need a credential or executor refuse explicitly (capability_required, not_wired, credential_required) instead of half-working.

The general diagnosis loop is at the end: Incident recovery loop.

Install

Symptom: install or build fails early. Diagnosis. The workspace pins Node 24 (engines.node: >=24 <25) and pnpm 11.x; a mismatched runtime fails at build, not at first tool call. Linux x64 is the only CI-proven platform. Safe command. node --version then pnpm --version. Expected output. Node 24.x and pnpm 11.x. Recovery. Install the pinned versions and retry pnpm install --frozen-lockfile — no other package manager is supported. Escalate when. You are on the pinned versions and install still fails.

Symptom: no working npx zephr. Diagnosis. Working as designed — npm publication is not shipped today; the working path is build from source (see Install). Recovery. Build from a clone; the server entry point is produced under apps/mcp/dist (source build main.js; packaged bundle main.cjs). Escalate when. Distribution status changes — watch Releases.

Symptom: search or code-intelligence tools report unavailable. Diagnosis. zephr_grep (lexical search) uses the optional ug (ugrep) binary and zephr_lsp (structural search) uses typescript-language-server; features degrade gracefully when the host-level binary is absent. Safe command. ugrep --version and typescript-language-server --version. Recovery. Install the missing optional binary. Escalate when. Installed and on PATH but still reported unavailable.

Connect

Symptom: the client reports the server failed to start. Diagnosis. The MCP server command/args do not launch the built server, or a path is relative. Safe command. ls -l /absolute/path/to/zephr/apps/mcp/dist/. Expected output. The built entry point exists at the absolute path. Recovery. Use the exact configuration shape from Configure your client with absolute paths and ZEPHR_DB_PATH in the env block. Escalate when. Path correct, file exists, launch still fails.

Symptom: the server starts but every tool call is refused. Diagnosis. The process is running without an admitted worktree — a bare zephr-mcp process with only ZEPHR_DB_PATH is useful for a smoke test only. Recovery. Run the connect flow (Connect your client) to admit a worktree — see Scope and admission.

Symptom: features work in one client but not another. Diagnosis. Adapters are wired per client; the Cursor adapter (@zephr-ai/adapter-cursor) is a mapper library the operator wires into the brokering process, not an editor bundle. Recovery. Wire the adapter per its documentation. Escalate when. Wired per docs and a tool call still fails only there.

Auth

Symptom: a command reports you are not authenticated. Diagnosis. No valid credential; the CLI refuses rather than guessing (for example "Not authenticated. Run zephr login first."). Safe command. zephr login. Expected output. A device code to authorize (OAuth 2.0 device flow, RFC 8628). Recovery. Complete the flow; re-enroll a stale enrollment with zephr connect --force. Escalate when. The flow completes and the command still reports not_authenticated.

Symptom: the device code expires mid-flow. Diagnosis. Device codes are short-lived; the flow reports expired_token on expiry or timeout. Recovery. zephr login again, or zephr login --force to re-authenticate from scratch. slow_down needs no action (the client backs off); access_denied means the authorization was declined — start a fresh login. Escalate when. Fresh codes keep expiring before authorization.

Symptom: roles are missing or a change does not apply. Safe command. zephr role list, zephr role show <role>, zephr role use <role>. Recovery. Select the role you intend to operate under. A revoked role will not work again until re-granted (error vocabulary includes role_revoked, role_deleted).

Symptom: an administrative operation is refused. Diagnosis. zephr_admin is an MCP tool, not a CLI command. Its actions require both the operator-set ZEPHR_ADMIN_TOKEN and trusted user presence. Local stdio does not bind a presence surface, so it refuses admin actions with presence_attestation_unavailable even when the token is set. Recovery. For local data, use zephr memory retract <id> for beliefs or zephr files rm --id <id> for files; export and run zephr uninstall --force when removing the whole installation. Use zephr_admin only through a surface that supplies trusted presence. Escalate when. A presence-enabled surface refuses a correctly configured action.

Scope and admission

Symptom: tool calls refused with scope_violation. Diagnosis. A worktreeId was passed outside the admitted scope, or the binding points at a checkout you are no longer in. Safe command. Re-run zephr connect — on success it prints the resolved project and the admitted binding (Project: …, Admitted: binding=…; see Connect your client). Expected output. The project and binding you intend, for the checkout you ran it from. Recovery. Run the connect flow against the correct checkout and restart the client; re-point a stale binding with zephr connect --force (re-enroll) — never by editing a file. Escalate when. A fresh, correct binding still returns scope_violation.

Symptom: launches report no active binding. Diagnosis. No active path binding exists for the project — the server has no admitted worktree to scope against. Recovery. Run the connect flow to approve the project and bind its checkout. Detaching a binding does not delete data, and re-running the connect flow re-attaches it.

Daemon

Symptom: CLI commands exit with code 69. Diagnosis. A required dependency — the on-box zephrd daemon, a backing binary, or a network endpoint — is not reachable. Exit 69 is the dedicated "unavailable" code; the caller must not fabricate data as a substitute. Safe command. zephr health and zephr status. Expected output. Whether zephrd is running and the handshake state. Recovery. zephr daemon start; if the binary is missing, place zephrd on PATH or point at it with zephr daemon start --bin <path> (ZEPHR_DAEMON_BIN and ZEPHR_DAEMON_SOCKET are honoured). Escalate when. The daemon reports running and commands still exit 69.

Symptom: zephr health says the handshake is down. Safe command. zephr doctor. Expected output. Per-check diagnostics with stable ids (binary.identity, binary.attestation, daemon.socket — best-effort, client.handshake.<name>, install.cold_runner_risk, schema — best-effort, skipped locally), each pass / warn / fail with a recovery hint; the report is ok, degraded, or unhealthy; exit 0 when all pass, 1 when any fail, 2 for unsatisfiable usage. Recovery. Follow the failing check's hint (for daemon.socket: start the daemon or correct ZEPHR_DAEMON_SOCKET). Escalate when. unhealthy after following the hint.

Symptom: the daemon crash-loops. Diagnosis. A crash breaker counts crash-marked starts within a short window (default 5 in 60s) and refuses to continue as a daemon beyond that, writing marker files next to its runtime state: zephrd.crash when a process panicked, zephrd.crash-loop when the breaker tripped. Recovery. Read the daemon log (zephrd.log) first — forced restarts inside the window just feed the breaker. Fix the cause, then clear the stale zephrd.crash marker so the next start is not counted. The marker files are nothing but counters' evidence — no store data lives in them. The daemon also auto-exits after an idle period.

Recall

Symptom: recall returns nothing you expected. Diagnosis. One of: the belief is not eligible (only the core-eligible set — confirmed and valid — is recallable by default); a policy narrowed eligibility further (zephr_rules, recallEligibleStates can only narrow); the query matched nothing; or the belief belongs to a different worktree. Safe command. zephr recall, then zephr why for the belief's review state and provenance, and the MCP zephr_status report for retrieval modes. Recovery. Confirm state and scope before treating it as a fault. Escalate when. A confirmed, valid, in-scope belief with a matching query is never returned.

Symptom: recall returns a structured abstention. Diagnosis. Recall abstains rather than guessing; the code names the reason (no-evidence, unresolved-contradiction, insufficient-confidence, semantic_unavailable, provenance_fail, stale_beyond_tolerance, calibration_stale, budget_exhausted). Safe command. Ask the client for the MCP zephr_status report. Expected output. zephr_status reports the retrieval flags (lexical, semantic); an embedding block is present only when the semantic tier is available (capture responses additionally carry a per-belief embedding lifecycle state — pending | available | unavailable). Recovery. Treat the code as the diagnosis — e.g. semantic_unavailable means the semantic tier is required by the call and not usable, while default requests disclose a lexical_fallback instead. Escalate when. A code contradicts the report (for example semantic_unavailable while zephr_status reports the embedding block with retrieval.semantic: true).

Symptom: recall quality is poor. Diagnosis. Often eligibility or provenance, not ranking: unreviewed captures rank below confirmed ones and unprovenanced beliefs are reported honestly as unproven. Safe command. zephr timeline (chronological handles) and zephr why. Escalate when. A clearly matching, confirmed, provenanced belief ranks far below weaker matches — report with a reproducible example.

Indexes

Symptom: a graph query returns absent_unindexed. Diagnosis. The entity-relation edge graph was never built for this installation — the typed reason distinguishes that from absent_verified (built and covered, but no edge matched). Recovery. Confirm the corpus build that owns the graph ran; absent_verified is a genuine "no answer". Escalate when. The corpus build completed and the query still reports absent_unindexed.

Symptom: semantic recall degrades or abstains. Diagnosis. The semantic tier needs the embedding capability. When it is unavailable, default requests still return lexical results but disclose the fallback (lexical_fallback mode); calls that require semantic abstain with semantic_unavailable. Nothing is fabricated and nothing degrades silently. Safe command. Ask the client for the MCP zephr_status report. Recovery. While embeddings are unavailable, default recall still answers lexically (disclosed); semantic-only requests abstain honestly. Escalate when. zephr_status reports the embedding block with retrieval.semantic: true and recall still abstains or falls back.

Symptom: hosted docs answers look stale. Diagnosis. Third-party docs come from the hosted catalog; project docs come from the admitted worktree in local mode. A hosted action whose plane is unreachable returns degraded — never a fabricated answer. Recovery. For project docs, re-import after the change; for hosted sources, retry when reachable and use the staleness probe. Escalate when. Project-docs actions return stale content from an unchanged worktree.

Trust

Symptom: a policy change made recall return less. Diagnosis. The Trust Firewall (zephr_rules) only narrows the core-eligible set — it never broadens retrieval. Safe command. Read the current policy with zephr_rules. Recovery. Widen the policy back toward the core-eligible set. Escalate when. A valid policy is rejected (the error names the offending state).

Symptom: zephr verify fails. Diagnosis. Structural (T1) and consistency (T2) checks — current-version pointer, contiguous revisions, non-overlapping transaction intervals, one open version, provenance anchors, review/validity state — do not hold, or the belief is not yet in a well-formed, provenanced, confirmed state. Safe command. zephr verify --claim-id <id> (alias verify-claim --claim <id>), then zephr why. Expected output. A verification profile; only well-formed, provenanced, confirmed beliefs are reported verified. Escalate when. A previously verified, unchanged belief starts failing.

Symptom: governance or review actions are refused. Diagnosis. The constraint surface fails closed when the governance ports are not bound — the refusal names the missing wiring explicitly ("zephr_constraint is not wired (no governance ports bound).", code capability_required); human-gated confirmations refuse non-human principals (confirm_denied); zephr_review returns a structured abstention when the substrate is unavailable — a verdict is never fabricated. Recovery. Wire the missing governance port; run human-gated confirmations as a human; for review abstentions read the reason (no_worktree_scope, change_resolver_required). Escalate when. Ports bound, human principal, still refused.

Sync

Symptom: a sync run exits with code 5. Diagnosis. Drift — the run completed with partial success; some clients/surfaces applied, others were skipped or conflicted. Safe command. zephr sync status --json and zephr sync push --dry-run. Expected output. Per-client outcomes (synced, drifted, conflict, not-installed) and what a push would apply. Recovery. Reconcile the surfaces that did not reach synced. Escalate when. Everything reports synced and the run still exits 5.

Symptom: a client is stuck at drifted or conflict. Diagnosis. Drift classification is explicit: a region inside the snapshot's fence that contradicts the projection is a managed-edit and halts apply (conflict); a region outside any fence is a hand-edit (your private data) and is preserved. Recovery. Re-adopt the projection with zephr sync adopt. Escalate when. Re-adoption loses your hand-edits — report with the before/after projection.

Symptom: a client shows not-installed. Diagnosis. The client config file is absent, or no projection is recorded yet (first sync). A not-installed client is not an error and does not affect the exit code. Recovery. If installed, sync once so a projection is recorded. Escalate when. Config present, sync ran, state unchanged.

Providers

Symptom: a hosted tool returns not_wired. Diagnosis. No executor is configured for the tool (Linear and Supabase "fail closed with not_wired when no executor is configured"). The response carries remedies. Safe command. zephr integrations. Expected output. Installed integration connections and status. Recovery. Wire the executor per the remedies and restart the server. Escalate when. Wired, restarted, still not_wired.

Symptom: a call returns 401 with a connect_ref. Diagnosis. Credentials are missing; the handler mints an opaque single-use connect_ref (credential_required) — never a device code or verification URI. Recovery. Run the connect flow that redeems it; a BYOK key (tools.<provider>.key) makes the call a cost-only pass-through against your key. Escalate when. Redemption succeeds and the call still returns credential_required.

Symptom: a call returns 404, 403, or 500 from the executor. Diagnosis. Explicit vocabulary: 404 not_found = unknown tool (no connect_ref); 403 forbidden_operation = known tool, disallowed operation; 500 outbound_host_denied = target host not on the allowlist, refused before any network request. Recovery. Fix the tool name, use an allowed operation, or an allowlisted host — do not bypass the host allowlist. Escalate when. A previously working call starts returning one of these.

Symptom: a docs action returns degraded. Diagnosis. zephr_docs third-party actions depend on the hosted plane; project-docs actions are unaffected (they operate against the admitted worktree locally). Recovery. Retry when the plane is reachable; never substitute fabricated content. Escalate when. The plane is healthy and the action still degrades.

Files

Symptom: a file's markdown is not available. Diagnosis. The file is unparsed (or parsing pending). The vault never fabricates markdown: an unparsed file returns file_unparsed naming its parse status. Safe command. zephr files ls. Recovery. Wait for parsing or re-upload; the file itself is intact. Escalate when. Unparsed long after upload.

Symptom: a file id returns "file not found". Diagnosis. The id does not exist in your vault or belongs to another scope. Safe command. zephr files ls. Recovery. Use an id from the list; erasing a missing id reports erased: false, reason: 'not_found' honestly. Symptom: an upload is refused. Diagnosis. One of the admission rules fired: per-file cap (100 MB), storage quota (storage_limit_exceeded, 413, with a remedy), empty uploads (refused), or no object plane (503 object_plane_unavailable). Recovery. Match the rule: shrink the file, free storage or raise the quota, send content, or confirm the deployment has an object plane. Escalate when. Under the cap, storage has headroom, and upload still fails.

Quota and billing

Symptom: an action is refused for budget or capacity. Diagnosis. The refusal names the condition — budget_exhausted (recall abstains on budget), capacity_exceeded, storage_limit_exceeded, or a metered hosted tool that is band-gated and fails closed on meter outage (no un-metered path). Safe command. zephr usage review (with --json for machine output). Expected output. The usage review view: allowance, consumed, remaining, the current band (ok, soft, degraded, suspended, not_configured), breakdowns, and remedies. Recovery. Compare the refusal against the view; if the view shows headroom the refusal is a metering problem, not spend. Escalate when. Headroom shown and refusal persists.

Symptom: a review refuses on affordability. Diagnosis. Entitlement-before-work: zephr_review_usage is the read-only pre-flight returning the same review-usage view, and a review that would exceed the entitlement is refused before work starts. Recovery. Treat the pre-flight as authoritative — do not bypass it by calling the provider directly.

Symptom: billing looks unconnected on-box. Diagnosis. When the ledger is not reachable the view reports band not_configured with available: false (reason ledger_unreachable_on_box) — honest, not a fault. Note on pricing. Only the Free tier is a committed offer; the commercial price list is still planning, and the only shipped, public pricing artifacts are the per-invocation credit rows in the generated MCP tools reference.

Workers

Symptom: a pod command is refused. Diagnosis. Pod lifecycle verbs are capability-gated: launch, pause, resume, terminate, exec, mcp, and acp require consent or return 403 capability_denied; 404 tenant_not_found means no tenant resolved. Inspection verbs (list, show, logs, egress-audit) are ungated. Safe command. zephr pod list, zephr pod show <pod-id>. Recovery. Obtain consent for the gated verb; check account context on tenant_not_found. Escalate when. Consent granted and the pod is yours, still refused.

Symptom: a pod launch fails or misbehaves. Diagnosis. pod launch requires a template — the error pod launch requires --template <id> appears when it is missing. Safe command. zephr pod logs <pod-id>, zephr pod egress-audit <pod-id>, then pause, resume, or terminate. Expected output. egress-audit lists the outbound hosts a pod attempted to reach, with reason and time. Escalate when. A pod cannot be terminated. Note the compute plane is evolving — end-to-end hosted execution depends on the plane being wired for your deployment.

Symptom. An agent does not run after you trigger it.

Diagnosis. zephr agent run <id> only enqueues a run — the agent worker is the sole entry point to execution, and runs never execute in-process.

Safe command. zephr agent list, zephr agent logs.

Expected output. Agent definitions and run history.

Recovery. Confirm the run was enqueued and inspect run history. The agent surface is evolving — a missing execution means the execution plane did not claim the run, not that your CLI executed it.

Escalate when. A run never appears in run history after enqueueing.

Cloud API (OpenAPI)

Symptom: a cloud API call fails. Diagnosis. Shared error responses: 401 Unauthorized ("Authentication required"), 400 BadRequest ("Invalid input"), 404 NotFound ("Resource not found"). Recovery. 401 → zephr login (see Auth); 400 → validate the payload against the generated Cloud API reference; 404 → confirm the operation is in the public manifest. Escalate when. A payload that validates against the published schema returns 400.

Symptom: the served API disagrees with the generated reference. Diagnosis. A parity gate keeps the runtime document aligned with the canonical source; the reference covers only operations classified public, and coverage reflects checked parity only. Recovery. Check the operation's owner, status, and evidence in the generated reference. Escalate when. A shipped operation behaves differently from its published reference — report the drift with the operation id and observed response.

Privacy and deletion

Symptom: data "disappeared" after exit. Diagnosis. The server ran on the in-memory store — with ZEPHR_DB_PATH unset, a plain process falls back to :memory: and discards everything at exit. Recovery. Set ZEPHR_DB_PATH to an absolute path in the client's env block before capturing anything you want to keep (see Configuration).

Symptom: a deleted memory still appears somewhere. Diagnosis. Memory deletion tombstones the active version; the immutable version chain keeps history verifiable. Physical purge is separate and requires both an admin token and trusted user presence; local stdio cannot perform it. Recovery. Locally, retract/tombstone the belief or remove the whole installation after exporting. Use MCP zephr_admin purge only through a trusted presence-enabled surface (see Data lifecycle). Escalate when. A presence-enabled purge reports success and recall still returns the record.

Symptom: you need to delete everything. Diagnosis. zephr uninstall removes local Zephr data while preserving SQLite backups; it is a dry run by default (--force executes). Recovery. Export first (zephr pack, zephr handoff --file, zephr receipts export), then uninstall; the preserved .db/.sqlite files are your recovery source. Escalate when. You believe erasure did not erase — report the surface and the operation that reported success. Do not delete store files by hand while the server is running.

Incident recovery loop

When a symptom does not fit one area, run the loop:

  1. Triage. zephr health → zephr doctor → zephr status, in order. doctor gives per-check ids with recovery hints — follow the failing check's hint first.
  2. Read the exit code. 0 success, 1 catch-all, 2 usage, 3 data integrity, 4 I/O, 5 drift (partial sync — see Sync), 69 unavailable (daemon/binary unreachable — see Daemon), 130 interrupted (SIGINT); on-box evidence commands add 64 (usage) and 77 (honest abstention).
  3. Check "working as designed" (top of this page) before escalating — abstention, not_evaluated, degraded, typed absence, and fail-closed refusals are correct behaviour.
  4. Recover the store per Data lifecycle: restore the verified recovery snapshot, never hand-edit SQLite or seal files, and verify a recovered belief with zephr verify --claim-id <id>.
  5. Escalate honestly. Collect the evidence (doctor report, exit code, exact error code, smallest reproduction), confirm it is not a designed state, and report through the public repository's issue tracker (see Install for the repository) — without credentials, tokens, or private infrastructure details.

Do not route around a fail-closed refusal — bypassing an allowlist, fabricating a result, or running hosted-plane recovery commands from a local installation is how small incidents become data-loss incidents.