C/C++ cross-LSP short-name fallback scans the full type registry on every miss

[astandrik]

Version

codebase-memory-mcp dev, built from cb0dfa64971a793e55170a3de9d2bd4c84f3d996 (v0.10.3-38-gcb0dfa64). The affected source files are unchanged on origin/main at 49d928be67322184fe25c8d15acd6f0e80b7e648.

Platform

macOS (Apple Silicon), 36 GiB unified memory, 18 logical CPUs.

Install channel

Built from source.

Binary variant

standard

What happened, and what did you expect?

Fast indexing of a large C/C++ repository spends most of its time in cross-file resolution. In c_lookup_member_depth, the namespaced-type fallback scans every registered type whenever direct, module-prefixed, alias, and base-class resolution miss:

for (int i = 0; i < ctx->registry->type_count; i++) {
    // compare the final qualified-name segment
}

On the public YDB corpus below, parallel_resolve took 857-908 seconds and accounted for about 71-72% of the pipeline. Worker sampling reached roughly 16 fully utilized CPU cores. I expected repeated short-name fallback lookups to use the finalized registry indexes instead of paying O(type_count) per lookup.

This is related to #1669's cross-LSP investigation, but it is not the same root cause. #1669 isolates per-file registry reconstruction for Java/Kotlin/PHP. This report isolates a C/C++ lookup over the shared prebuilt registry; the proposed index is deliberately built only for C/C++ so it does not add work to those per-file paths.

Reproduction

1. Check out the public corpus:

   git clone https://github.com/ydb-platform/ydb.git
   git -C ydb checkout be88c989a45f90ea7b2372cb0f9098e0fa23a3d9

2. Build codebase-memory-mcp from cb0dfa64971a793e55170a3de9d2bd4c84f3d996, use a fresh cache, and run:

   CBM_PROFILE=1 CBM_WORKERS=18 CBM_CACHE_DIR=/tmp/cbm-ydb-baseline-cache \
     ./codebase-memory-mcp cli index_repository \
     '{"repo_path":"/path/to/ydb","name":"ydb-cpp-short-name-profile","mode":"fast","persistence":false}'

3. Inspect the pass.timing rows. Two byte-identical baseline runs and one local proof-of-concept run produced:

   build	pipeline	CLI wall	parallel_resolve
   baseline 1	1,255.481 s	1,261.89 s	907.684 s
   baseline 2	1,212.979 s	1,219.50 s	857.105 s
   short-name index	456.974 s	463.22 s	131.926 s

   Depending on the baseline, the proof of concept is 2.63-2.72x faster end to end and 6.50-6.88x faster in resolution.

The proof of concept builds an optional hash index from the final segment of each type's qualified name to an ascending chain of registry indices. C/C++ builds it once after registry finalization. The lookup walks that chain plus any post-finalize tail; allocation failure or an unindexed registry falls back to the original complete scan. The caller retains its exact dotted-suffix predicate, duplicate exclusion, and current-module preference.

Logs

No raw logs attached because they include local filesystem paths. Relevant aggregate profile rows are reproduced in the table above.

The baseline pipeline also reported:

parallel_extract=114.186 s
lsp_cross_prepare=78.333 s
parallel_resolve=907.684 s
dump_and_persist=106.995 s

Writer index sorting was 2.011 seconds (0.16% of the pipeline), so persistence or GPU work is not the first-order fix for this workload.

Diagnostics trajectory (memory / performance / leak issues)

Five-second process-family sampling observed baseline peak CPU at 1,594.7% and peak RSS at 13.20 GiB. Optimized runs reached 1,600.5% CPU; sampled peak RSS varied from 14.28 to 19.70 GiB, so this report makes no memory-improvement claim.

Both baselines used the same binary SHA-256 and all compared runs retained the same 106,939 file hashes and coverage counts. Full graph identity is naturally order-sensitive in this pipeline: the two byte-identical baseline runs themselves changed 222,076 common node payloads and tens of thousands of symmetric edge identities. The direct semantic guard is therefore a differential iterator test against the former linear scan, not a claim of byte-for-byte graph equality.

Local regression evidence for the proof of concept:

• 512 generated registry types across 23 repeated short names compare indexed output to the former ascending linear scan;
• post-finalize tail entries and unavailable-index fallback preserve complete candidate coverage;
• the caller still applies the exact predicate and current-module tie-break;
• the focused native C-LSP suite passes 760/760 with -Werror.

Project scale (if relevant)

106,939 file hashes / 2,408,291 nodes / 11,397,892 edges in the first baseline; 225,399 Class nodes.

Proposed acceptance criteria

• C/C++ short-name fallback no longer scans the full finalized type registry for each lookup.
• Candidate order, exact filtering, module preference, post-finalize additions, and allocation-failure behavior remain unchanged.
• Languages that do not consume the index pay no new registry-finalization allocation or scan.
• Focused and repository-wide tests, lint, security, build, and DCO checks pass.

Confirmations

• I searched existing issues and this is not a duplicate.
• My reproduction uses shareable code (the public YDB repository), not proprietary code.