bench(parquet): cover nullable and repeated large-value writes

[adriangb]

Why

Every large-value benchmark in arrow_writer builds its array with StringArray::from_iter_values. With no nulls present, RecordBatch::try_from_iter marks the field non-nullable, the column has max_def_level == 0, and its definition levels are absent.

The writer resolves an absent-level chunk's value count in O(1) and never inspects levels, so nothing here covers a nullable column whose values exceed data_page_size_limit — the path that decides how many values share a data page, and on DELTA_BYTE_ARRAY whether prefix deduplication survives at all. The repeated case is a third level shape, also uncovered: records cannot span data pages, so mini-batches must step whole records.

What

One nullable case, then five that each vary a single property of it, so a movement can be attributed rather than guessed at:

group	varies
large_string_shared_prefix_nullable	—	2 MiB values, one null in 16
..._nullable_dense	null density → one in two	ratio is exactly two levels per value, so window arithmetic is exact — the case that should not move
..._nullable_trailing	null placement, count fixed	a run of nulls at the end leaves every earlier window holding values only
large_string_distinct_nullable	prefix	removes what deduplication has to work with
medium_string_shared_prefix_nullable	value size → 256 KiB	several values share a page budget instead of one overrunning it
..._shared_prefix_list	level shape	repeated column, records cannot span pages

PLAIN is measured where page count alone drives the result and omitted where it would restate a neighbour: it never reads the previous value, so prefix, null placement and value size do not change its per-page work.

Validation

Run against #10554, which changes exactly this path, as base → branch → base on an idle machine so the two base passes give a per-benchmark noise floor. Base is #10505's head, so the effect is #10554 alone.

benchmark	effect	noise floor
..._nullable_trailing/delta_byte_array	−27.8%	1.2%
..._nullable/plain	+27.6%	2.1%
..._nullable/delta_byte_array	+7.6%	1.0%
..._nullable_dense/delta_byte_array	−0.2%	0.1%
large_string_distinct_nullable/delta_byte_array	+3.1%	3.1%
medium_string_shared_prefix_nullable/delta_byte_array	+1.3%	2.9%
..._shared_prefix_list/delta_byte_array	+1.3%	0.9%
list_struct_with_list/* (untouched control)	−0.1% to +0.8%	0.1–2.4%

The design holds up: the two cases built to move do, the case built as a control lands at −0.2% against a 0.1% floor, and the cases whose axis should not matter stay inside their own noise.

Time alone understates what is happening, so the same batches written to a file:

case	#10505	#10505 + #10554
uniform_1in16 / DELTA_BYTE_ARRAY	16 MiB	2 MiB
trailing_8 / DELTA_BYTE_ARRAY	120 MiB	2 MiB
either / PLAIN	240 MiB	240 MiB

Which is why the throughput numbers split the way they do. _trailing gets faster because 118 MiB of writing disappears; the uniform case pays a small amount because its output was already close to deduplicated and value-exact windows roughly double the mini-batch count; PLAIN pays the most because it doubles pages for no reduction in output at all.

Notes

Benchmark-only; no library code touched.

CI clippy is currently red on main for an unrelated reason — arrow-arith/src/numeric.rs:756 trips collapsible_if, from #10409 — so this PR's Clippy job will fail until that lands a fix. cargo clippy -p parquet --all-features --benches -- -D warnings is clean.

🤖 Generated with Claude Code