Fix TokenOverlap crash when references list is empty

[romainsc]

Fix TokenOverlap crash when references list is empty

Summary

TokenOverlap.compute() raises ValueError: max() iterable argument is empty when references is an
empty list. This adds a guard that returns zero scores
instead of crashing.

Problem

When TokenOverlap is used via a metric pipeline that
maps a potentially empty field to references, the list
comprehension produces an empty results list, and
max(r[i] for r in results) raises ValueError.

This occurs in practice with the faithfulness metric
(metrics.rag.external_rag.faithfulness), which copies
contexts to references. If a vector store query
returns zero chunks for a given question (e.g. due to
eventual consistency in the vector database, or a query
with no semantic match), contexts is an empty list,
and TokenOverlap crashes instead of returning a zero
score.

Reproduction

import pandas as pd
from unitxt.eval_utils import evaluate

data = [{
    "question": "Q1",
    "answer": "Some answer",
    "ground_truths": ["Ground truth"],
    "contexts": [],  # no chunks retrieved
    "context_ids": [],
    "question_id": "q1",
    "additional_data": None,
    "ground_truths_context_ids": ["doc1"],
}]
df = pd.DataFrame(data)
evaluate(
    df,
    metric_names=[
        "metrics.rag.external_rag.faithfulness"
    ],
    compute_conf_intervals=True,
)
# ValueError: max() iterable argument is empty

Root cause in practice

This was encountered using the AutoRAG pipeline in
Red Hat OpenShift AI 3.4 with a Milvus vector store
configured with consistency_level=Bounded. With fast
GPU-based embedding, chunks are inserted and queried
within ~100ms. Bounded consistency does not guarantee
immediate visibility of inserted data, so some queries
return zero results. The faithfulness metric then
receives contexts=[], which maps to references=[],
triggering the crash.

While the operational fix is to use Strong consistency
in Milvus, TokenOverlap should handle empty references
gracefully regardless of the upstream cause.

Fix

Add if not results: return {"precision": 0, "recall": 0, "f1": 0} before the max() call, consistent with the
zero-score behavior already present in
_compute_single_ref when num_same == 0.

Test

Added test_token_overlap_empty_references that verifies
the metric returns zero scores when references is empty,
instead of crashing.

Developed with AI assistance (Claude). Code reviewed and empirically validated by the author.