Skip to content

Fix #758: Clear ARROW_FLAG_NULLABLE on the entries struct of MAP columns - #767

Open
adsharma wants to merge 1 commit into
mainfrom
fix/issue-758-map-arrow-entries-nullable
Open

Fix #758: Clear ARROW_FLAG_NULLABLE on the entries struct of MAP columns#767
adsharma wants to merge 1 commit into
mainfrom
fix/issue-758-map-arrow-entries-nullable

Conversation

@adsharma

@adsharma adsharma commented Aug 1, 2026

Copy link
Copy Markdown
Contributor

Fixes #758.

Problem

Any query returning a `MAP` column is unreadable through the Arrow result
API. Consumers (e.g. arrow-java's `MapVector`) reject the schema Ladybug
produces with:

```
IllegalArgumentException: Map data should be a non-nullable struct type
```

This also breaks `RETURN n.*` and `RETURN n`, since including a single
`MAP` column makes the whole Arrow read path unusable for that table.

Root cause

`ArrowConverter::setArrowFormat`, `LogicalTypeID::MAP` case
(`src/common/arrow/arrow_converter.cpp`), only cleared
`ARROW_FLAG_NULLABLE` on the key child of the entries struct. The Arrow
columnar map layout requires both the entries struct and the key child
to be non-nullable:

The entries struct itself must be non-nullable, as is its key child.
https://arrow.apache.org/docs/format/Columnar.html#map-layout

The fix is a single added line, applied immediately before the existing
key-child clear:

```cpp
child.children[0]->flags &=
~ARROW_FLAG_NULLABLE; // Map's entries struct must be non-nullable
```

Test

Added `ArrowTest.mapColumnArrowSchemaHasNonNullableEntriesAndKey` in
`test/api/arrow_test.cpp`. The test creates a `MAP(STRING, STRING)` column,
exports the Arrow schema through `getArrowSchema()`, and asserts:

  • the map column has format `+m` and is itself nullable,
  • the entries struct is named `entries` and is non-nullable,
  • the key child is non-nullable, and
  • the value child is nullable (values follow the value type's nullability).

Notes

  • The C data interface does not treat child names as normative, so the
    upper-case `KEY`/`VALUE` naming mentioned in the issue is left for a
    separate change.
  • A Java round trip with a locally patched `liblbug` cannot be verified
    end to end because the released `com.ladybugdb:lbug` artifact statically
    links its own engine build, but the flag is what `arrow-java`'s exception
    names, and it is now correct.

The Arrow columnar map layout requires both the entries struct of a map
and the key child of that struct to be non-nullable. The existing
converter only cleared the flag on the key child, so consumers such as
arrow-java's MapVector rejected the schema with

  IllegalArgumentException: Map data should be a non-nullable struct type

making every query that returns a MAP column unreadable through the
Arrow API. Adding the flag clear on the entries struct brings the
exported schema back in line with the spec.
@adsharma
adsharma force-pushed the fix/issue-758-map-arrow-entries-nullable branch from a897a28 to d061eb6 Compare August 1, 2026 01:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Bug: MAP export leaves the Arrow entries struct nullable, so no MAP column can be read through the Arrow API

1 participant