Disable TopK/filtered pruning, preserve legacy threshold behavior

[tlwillke]

This is a hotfix for JV 4.0.0-rc.8. Resolves #678.

Summary

This hotfix disables the unsafe threshold == 0 graph-search pruning path while preserving legacy threshold-search behavior for compatibility.

GraphSearcher.usePruning(boolean) is now deprecated and retained only for source and binary compatibility. Calls to this method no longer enable TopK or filtered-search pruning.

Threshold searches, where threshold > 0, continue to use the existing legacy threshold early-termination path. This is intentional because threshold search may already be used by customers, and changing that behavior in this hotfix could be a compatibility break.

In short:

threshold == 0:
  RelaxedMonotonicityTracker is not used
  usePruning(true) is ignored
  plain and filtered TopK use standard graph-search termination

threshold > 0:
  legacy TwoPhaseTracker behavior is preserved
  threshold search behavior remains unchanged for compatibility

Why This Change Is Needed

The existing TopK/filtered pruning implementation did not demonstrate reliable value and carried recall risk.

The original pruning design had several problems:

1. Pruning was used to skip graph expansion.
   The search could pop a candidate and skip expanding its neighbors, then continue searching. In graph search, expansion is how better candidates are discovered, so this changes the traversal and can reduce recall.

2. TopK pruning was redundant with standard graph-search termination.
   Plain TopK already has the standard frontier-vs-result-queue termination condition. Adding another relaxed-monotonicity signal can only terminate earlier than the normal search and therefore risks recall loss.

3. Filtered TopK did not show useful benefit.
   This implementation already applies acceptOrds during traversal. In low-cardinality filtering tests, additional pruning saved almost no visits while sometimes reducing recall.

4. Threshold pruning is imperfect but may be in use.
   Diagnostic threshold testing showed that the legacy threshold path can save substantial work for very selective thresholds, but it can lose recall for less-selective thresholds. Because this behavior existed previously, this hotfix preserves it rather than changing threshold semantics.

Evidence From Filtering Tests

Filtered TopK was the primary case where pruning was expected to help. The observed benefit was negligible.

Filter selectivity	Hierarchy	Visit reduction from pruning	Recall delta from pruning
50%	off	0.81%	-0.0082
10%	off	0.05%	-0.0006
2%	off	0.01%	-0.0003
50%	on	0.06%	-0.0006
10%	on	0.02%	-0.0001
2%	on	0.00%	0.0000

This indicates that the existing filtered-search implementation already gets the useful behavior from applying acceptOrds during traversal, and the additional relaxed pruning tracker does not provide meaningful value.

Evidence From Threshold Diagnostics

Threshold pruning remains preserved for compatibility, but diagnostic testing showed why it should not be expanded or treated as a validated general pruning mechanism.

Expected threshold matches	Visit reduction from legacy threshold pruning	Recall with pruning
25	79.05%	1.000
100	79.05%	0.932
500	68.50%	0.618

Implementation Notes

GraphSearcher.usePruning(boolean)

The API is deprecated and retained for compatibility.

/**
 * @deprecated TopK and filtered graph-search pruning is disabled because the
 * existing heuristic can reduce recall and has not shown reliable production
 * value. This method is retained for API compatibility and has no effect.
 *
 * Threshold searches, where {@code threshold > 0}, continue to use their
 * legacy threshold early-termination behavior.
 */
@Deprecated
public void usePruning(boolean usage) {
    pruneSearch = false;
}

Compatibility Notes

• The public usePruning(boolean) method still exists.
• Existing callers will not get source or binary breakage from this change.
• Calls to usePruning(true) no longer enable TopK or filtered pruning.
• Threshold search behavior is preserved for compatibility.
• Callers that previously relied on TopK/filtered pruning may see slightly higher visited counts, but recall behavior should be safer and more predictable.

Tests Added / Updated

TestPruningCompatibility

Adds explicit coverage for the new compatibility contract.

testScoreTrackerFactoryPolicy

Verifies tracker selection:

threshold == 0:
  pruneSearch=false -> NoOpTracker
  pruneSearch=true  -> NoOpTracker

threshold > 0:
  pruneSearch=false -> TwoPhaseTracker
  pruneSearch=true  -> TwoPhaseTracker

This confirms that RelaxedMonotonicityTracker is disabled for threshold == 0, while legacy threshold behavior is preserved.

testUsePruningIgnoredForTopKAndFilteredTopK

Verifies that usePruning(true) has no effect for:

plain TopK
filtered TopK
hierarchy off
hierarchy on

The test compares pruning-on and pruning-off searches over the same graph, queries, and filters, and asserts:

same returned nodes
same visited count

testUsePruningIgnoredForThresholdSearch

Verifies that usePruning(true) and usePruning(false) produce identical threshold-search behavior.

This confirms that the public pruning API is ignored, while the legacy threshold path remains unchanged.

TestLowCardinalityFiltering

Keeps coverage for filtered TopK search behavior and filter correctness.

The test evaluates filtered search across multiple selectivities:

50%
10%
2%

For each selectivity, the test compares pruning-off and pruning-on behavior over the same graph, queries, and filter set. It validates that returned nodes satisfy acceptOrds and records the visit/recall effect of pruning.

This test documents why filtered TopK pruning is being disabled: the measured visit reduction was negligible while recall could decrease slightly.

Test2DThreshold

Continues to validate legacy threshold-search behavior.

This test is retained because threshold searches are intentionally preserved for compatibility. It checks that threshold queries continue to return sufficient matching nodes and meet the existing visited/recall expectations.

Release Note Text

Deprecated and disabled `GraphSearcher.usePruning(boolean)` for TopK and filtered searches.

The existing TopK/filter pruning heuristic could reduce recall and did not show reliable production value. The method remains available for source and binary compatibility, but enabling it no longer activates pruning for `threshold == 0` searches.

Legacy threshold-search behavior is preserved: searches with `threshold > 0` continue to use the existing threshold early-termination path.

Future Work

Any future pruning implementation should be reintroduced only after a separate redesign and validation effort. A replacement should:

• never skip graph expansion for popped candidates;
• avoid upper-layer pruning;
• avoid redundant pruning for plain TopK;
• clearly separate TopK, filtered TopK, and threshold/range semantics;
• define the traversal signal it tracks;
• include representative recall and latency benchmarks across production-like datasets, dimensions, filter selectivities, and threshold selectivities.

[MarkWolters]

Code looks good, as do the tests. There is a minor conflict in GraphSearcher that needs to be resolved before this can be merged.

[tlwillke]

This branch is a hotfix for 4.0.0-rc.8. We need to cherry pick the fix for the latest main.

[ashkrisk]

I ran a test with cohere-1M, everything looks as expected. There's a big recall improvement with hierarchy off.

topK	overquery	usePruning	addHierarchy	recall_main	recall_pr	recall_change
10	1.0	True	True	0.57925	0.59661	0.01736
10	2.0	True	True	0.75996	0.77704	0.01708
10	1.0	False	True	0.59821	0.59661	-0.00160
10	2.0	False	True	0.77833	0.77704	-0.00129
10	1.0	True	False	0.45247	0.58810	0.13563
10	2.0	True	False	0.69523	0.77637	0.08114
10	1.0	False	False	0.57372	0.58810	0.01438
10	2.0	False	False	0.77301	0.77637	0.00336

[ashkrisk]

A clarification might be helpful here since this tripped me up a bit.

Suggested change

	* Threshold searches, where {@code threshold > 0}, continue to use their
	* legacy threshold early-termination behavior.
	* Threshold searches, where {@code threshold > 0}, continue to use their
	* legacy threshold early-termination behavior which disregards this setting.

[ashkrisk]

nit: should we be sorting the nodes here? If we're expecting the exact same results that would include the order of the results as well.

[ashkrisk]

nit: A comment here would improve readability

// `usePruning` parameter is deprecated and should have no effect.

Suggested change

	assertTrue(factory.getScoreTracker(false, RERANK_K, 0.0f) instanceof ScoreTracker.NoOpTracker);
	assertTrue(factory.getScoreTracker(true, RERANK_K, 0.0f) instanceof ScoreTracker.NoOpTracker);
	// `usePruning` parameter is deprecated and should have no effect.
	assertTrue(factory.getScoreTracker(false, RERANK_K, 0.0f) instanceof ScoreTracker.NoOpTracker);
	assertTrue(factory.getScoreTracker(true, RERANK_K, 0.0f) instanceof ScoreTracker.NoOpTracker);