feat: jrag watch — warm, self-refreshing index daemon

agents/explorer-rag-cli.md

@@ -95,6 +95,8 @@ The Decision Framework above tells you *which* command; reach for `--help` only
 
 **Prerequisite.** `jrag` needs an index — unindexed, every command exits 2 (`jrag status` checks; the file-system tools work without one).
 
+**Tip.** Run `jrag watch` once per session for fast, fresh queries — it keeps the index fresh on file change and serves every read command warm (no per-call model load). Optional; with no daemon running, all reads take the cold path byte-identically.
+
 **Resolve-first contract.** Every `<query>` command resolves the identifier first, then maps `one` / `many` / `none` onto one envelope: `one` → run; `many` → return candidates and stop, **no silent guess across distinct types** (a class sharing its simple name with its own constructor still resolves to the type — narrow with `--kind` / `--role` / `--fqn-contains` / `--service`); `none` → `status: not_found` (exit 0), fall back to `search` or `Grep`. Pass names (FQN / simple name / route path / topic) or prior `sym:`/`route:`/`client:`/`producer:` ids — never raw node ids. `--kind` is a true resolve input; `--role` / `--java-kind` / `--fqn-contains` post-filter client-side.
 
 **Output.** Default is compact text; `--format json` emits `{status, nodes, edges, candidates, truncated, agent_next_actions, file_location}` (empty fields dropped; `file_location` is a `filename:line` string; `agent_next_actions` suggests ≤5 next commands). `truncated` pages via `--limit` / `--offset` (`find` / `search` only). Output-shaping flags (every query / listing / traversal command — not `status` / `microservices` / `vocab-index`, which reject them): `--count` prints just the result count (bare int in text; `{"status","count"}` in json), `--exists` prints `true`/`false` (`{"status","exists"}` in json) and exits 0 on a hit / 2 on a miss (scriptable existence gate — `find X --exists`, `inspect X --exists`), `--fields fqn,role,…` projects each node to a comma-separated field allowlist (overrides `--detail`; ignored with `--count`/`--exists`; primarily a `--format json` lever).

docs/ARCHITECTURE.md

@@ -31,9 +31,10 @@ Core library = **top-level `.py` modules** (`py-modules`); the installable **`ja
 | --- | --- |
 | Write path | `java_codebase_rag/cli.py`, `java_codebase_rag/pipeline.py`, `java_codebase_rag/lance_optimize.py`, `java_index_flow_lancedb.py`, `build_ast_graph.py` |
 | Parse + ontology | `ast_java.py` (`ONTOLOGY_VERSION=18`), `java_ontology.py` (`EDGE_SCHEMA` + label sets), `graph_enrich.py`, `chunk_heuristics.py` |
-| Read path | `server.py`, `mcp_v2.py`, `ladybug_queries.py`, `search_lancedb.py`, `search_lexical.py`, `search_scoring.py`, `resolve_service.py` |
+| Read path | `server.py`, `mcp_v2.py`, `ladybug_queries.py`, `search_lancedb.py`, `search_lexical.py`, `search_scoring.py`, `resolve_service.py`, `java_codebase_rag/read_payloads.py` |
 | Hints + absence | `mcp_hints.py`, `graph_types.py`, `absence_types.py`, `absence_vocab.py`, `absence_diagnosis.py` |
 | Config + paths | `java_codebase_rag/config.py`, `path_filtering.py`, `index_common.py`, `brownfield_events.py` |
+| Watch daemon | `java_codebase_rag/watch/` (`lock`, `paths`, `protocol`, `warm`, `server`, `client`, `watcher`, `daemon`) |
 | Surfaces | `java_codebase_rag/{cli,jrag,installer}.py` |
 | Shipped artifacts | `skills/`, `agents/` (deployed verbatim to agent host via `install`/`update`) |
 
@@ -85,6 +86,23 @@ MCP tool call (server.py)  ──asyncio.to_thread──▶  mcp_v2.*
 
 **Lexical fallback** is selected by import availability (`mcp_v2` guards `from search_lancedb import …`): same row contract, flagged via `lexical_mode` + advisory. **`jrag` CLI** calls the same `mcp_v2.*` functions — identical backends, only rendering differs.
 
+### Watch path (`jrag watch`) — warm reads + freshness
+
+When a `jrag watch` daemon is running, the **read path gains a warm hop**: the `jrag` read handlers ask the daemon over a Unix socket for the already-built payload instead of cold-loading the model and graph. The daemon reuses the MCP server's warm-cache posture — a process-singleton `_st_model` (SBERT) and a `LadybugGraph` — served to the CLI, so each query skips the per-call torch/model load. Output is byte-identical to the cold path (the same payload cores in `read_payloads.py` run either way). With no daemon running, the client transparently takes the cold path — the daemon is a pure accelerator, never a dependency.
+
+| Concern | Module | Notes |
+| --- | --- | --- |
+| Project lock | `watch/lock.py` | `ProjectLock` (pidfile + stdlib `flock`). New — the codebase had no locking. One daemon per index dir; also blocks a concurrent manual `increment`. Unix-only (`WatchUnsupportedPlatform` on Windows). |
+| Runtime paths | `watch/paths.py` | socket, pidfile, state file under the index dir. |
+| IPC protocol | `watch/protocol.py` | `Request`/`Response`/`ErrorShape`; `ERR_*` kinds. |
+| Warm resources | `watch/warm.py` | `WarmResources` holds the model + graph; `LadybugGraph.reset_for_path` swaps the live graph handle after a reindex. |
+| Socket server | `watch/server.py` | `WatchServer` dispatches read payloads (serialized, not rendered). |
+| IPC client | `watch/client.py` | `is_daemon_alive` / `get_payload`; any error → cold fallback. |
+| Watcher | `watch/watcher.py` | `SourceWatcher` (watchdog native + polling fallback), lossless debounce, per-type routing. |
+| Daemon | `watch/daemon.py` | `WatchDaemon` lifecycle: lock → warm → server → watcher → serve loop → teardown (`os._exit(0)`). |
+
+**Reindexing is subprocessed**, never in-process: cocoindex for vectors, `build_ast_graph.py --incremental` for the graph. **Concurrency:** searches never wait and never see partial state — Lance commits are atomic per version (fresh per-query reads are consistent), and the graph (LadybugDB — no transactions, single writer) is kept readable via a **copy-on-write file snapshot** of `code_graph.lbug` taken around each graph reindex: reads continue from the snapshot while the subprocess writes the original, then `reset_for_path` repoints the live handle.
+
 ## Stores
 
 **LanceDB** (index dir, e.g. `.java-codebase-rag/`) — 3 tables (`LANCE_TABLE_NAMES`): `javacodeindex_java_code` (Java chunks w/ role · module · microservice · generated), `sqlschemaindex_sql_schema`, `yamlconfigindex_yaml_config`. cocoindex state in `cocoindex.db/`.
@@ -101,6 +119,7 @@ Precedence **CLI flag > env > YAML (`.java-codebase-rag.yml`) > default**; each
 - **New role/capability** → inference tables in `ast_java.py` + valid sets in `java_ontology.py`.
 - **New node kind** → Ladybug schema (`_create_schema`) + extraction pass + `NodeFilter` / resolve generators in `mcp_v2.py` / `resolve_service.py`.
 - **Semantic extraction change** → bump `ONTOLOGY_VERSION` (`ast_java.py:87`); read guard + incremental fallback follow automatically; note reindex in [`docs/CONFIGURATION.md`](./CONFIGURATION.md).
+- **Watch surface** → `java_codebase_rag/watch/` (warm reads + debounced reindex). New read command? add a payload core in `read_payloads.py`, then wire `server.py` dispatch + the `jrag` handler (cold path stays the default). The cold path must stay byte-identical — the daemon is an accelerator, never a dependency.
 
 Dev workflow (editable install, test-reset ritual, full-suite discipline) — see [`CLAUDE.md`](../CLAUDE.md).
 

docs/CONFIGURATION.md

@@ -295,6 +295,23 @@ generated_detection:
   # fix the generator or filter with exclude_generated at query time.
   exclude_fqns:
     - 'com.example.manual.ExampleImpl'
+
+# -------- Watch mode (`jrag watch`) --------
+# The `jrag watch` daemon (index freshness + warm-query). No env vars are
+# introduced for watch — precedence is CLI flag (--debounce-ms / --backend)
+# > YAML > built-in default. See JAVA-CODEBASE-RAG-CLI.md § `jrag watch`.
+watch:
+  # watch.debounce_ms — reindex debounce window in ms. Default 1500; floor 100
+  # (a value at/below 100 ms falls back to 1500 with a stderr warning).
+  debounce_ms: 1500
+
+  # watch.backend — file-watch backend. `auto` picks watchdog when available,
+  # else polling. An invalid value falls back to `auto` with a warning.
+  backend: auto          # auto | watchdog | polling
+
+  # watch.poll_interval_ms — poll interval for the polling backend, in ms.
+  # Default 2000; floor 200 (below → 2000 + warning). Polling-only; ignored by the native watchdog observer.
+  poll_interval_ms: 2000
 ```
 
 **Path expansion (what gets `~` / `$VAR` treatment):**

docs/DESIGN.md

@@ -43,6 +43,7 @@ One repo, two stores, two audiences:
 | --- | --- | --- |
 | MCP server (`server.py`) | agents | `search` / `find` / `describe` / `neighbors` / `resolve` |
 | `jrag` CLI | agents / humans | same five tools, terminal rendering |
+| `jrag watch` daemon | agents / humans | index freshness + warm-query accelerator over a Unix socket (one per project); pure accelerator, cold path stays byte-identical when no daemon runs |
 | `java-codebase-rag` CLI | operators | index lifecycle, `meta` / `tables` / `diagnose-ignore`, `analyze-pr` |
 
 ## Non-goals (by design)
@@ -51,5 +52,9 @@ One repo, two stores, two audiences:
 - Not a reflection / dynamic-dispatch oracle — `CALLS` is static only.
 - Not git history — use `git log` / `blame`.
 - Not re-indexable from MCP — only the operator CLI rebuilds.
+- **`jrag watch` is Unix-only (macOS/Linux)** — the project lock uses stdlib `fcntl`; on Windows `jrag watch` exits 2 and the cold read path is unaffected.
+- **`jrag watch` is not a boot/persistent service** — it is a foreground-or-detached process the operator starts per coding session; nothing auto-starts on boot or survives logout.
+- **`jrag watch` is one daemon per index dir, not multi-project** — a pidfile + `flock` enforces a single watcher (and blocks a concurrent manual `increment`) per project; run one process per project.
+- **`jrag watch` is not network/remote and not for the MCP surface** — it serves warm reads over a local Unix socket to the `jrag` CLI only; the MCP server has its own warm-cache posture and is untouched.
 
 Non-goal detail: [`docs/AGENT-GUIDE.md`](./AGENT-GUIDE.md) (§ "What this MCP is not"). Roadmap and future direction live in [`docs/PRODUCT-VISION.md`](./PRODUCT-VISION.md), not here.

docs/JAVA-CODEBASE-RAG-CLI.md

@@ -486,3 +486,33 @@ jrag search "service" --limit 20 --offset 20
 - `--generated-only` — Show only generated sources.
 
 **Breaking change (PR-SEARCH-2):** By default, `jrag search` now returns one row per `primary_type_fqn` (symbol/type) to prevent a single type from flooding the page. The `--chunks` flag restores the previous chunk-level output. When deduped, each hit shows a `chunks=N` field indicating how many chunks were collapsed into that hit.
+
+### `jrag watch`
+
+A single long-running daemon that does two things at once **while it runs**: (a) **keeps the index fresh** — it watches the source tree and re-runs a debounced per-type reindex (vectors via cocoindex, graph via `build_ast_graph.py --incremental`) on file change; and (b) **serves every read command warm** — `search` / `find` / `inspect` / `callers` / `callees` / `flow` are served over a Unix socket from a pre-loaded model + graph, so each query skips the per-call torch/model load and is effectively instant. The "run it while you code" workflow: start it once per coding session (foreground or detached), then keep issuing the normal `jrag` read commands — they are accelerated automatically.
+
+```bash
+# Foreground — Ctrl+C (or SIGTERM) stops it
+jrag watch
+
+# Background — returns once the daemon is ready to serve
+jrag watch --detach
+
+# Is it running? (exit 0 if up, 1 if down)
+jrag watch --status
+
+# Graceful stop (SIGTERM; SIGKILL after 5 s); cleans the socket
+jrag watch --stop
+```
+
+**Modes / flags:**
+- *(default, foreground)* — runs in the foreground; **Ctrl+C** or **SIGTERM** stops it. On a TTY it renders a status panel showing the socket path, reindex count, and last reindex.
+- `--detach` — start as a background daemon and return once it is ready to serve (logs to a file under the index dir).
+- `--stop` — gracefully stop a running watcher (pidfile + signal; SIGKILL after 5 s) and clean up its socket.
+- `--status` — print `up`/`down` with pid, socket path, and last reindex. Exits **0** if up, **1** if down. Does **not** acquire the project lock.
+- `--debounce-ms N` — reindex debounce window in ms (overrides YAML `watch:debounce_ms`).
+- `--backend {auto,watchdog,polling}` — file-watch backend (overrides YAML `watch:backend`).
+
+**Cold-fallback guarantee.** With **no daemon running**, every read command behaves byte-identically to today — the daemon is a pure accelerator + freshness layer, never a dependency. If the daemon is down or unreachable, each `jrag` read silently takes the cold path (identical output, identical exit codes; you only pay the one-off cold-start model/graph load). See [`CONFIGURATION.md`](./CONFIGURATION.md) § 2 for the `watch:` block.
+
+**Unix-only.** `jrag watch` relies on `fcntl`, so it runs on **macOS / Linux** only. On Windows it prints `jrag watch: watch mode requires macOS/Linux` to stderr and exits **2**; the cold read path is unaffected on every platform. One daemon per index dir — a pidfile + `flock` prevents two watchers (or a concurrent manual `increment`) on the same project.