Skip to content

Fix #757: accept FixedSizeBinary(16) with arrow.uuid for UUID PK - #768

Open
adsharma wants to merge 1 commit into
mainfrom
fix/issue-757-arrow-rel-uuid-keyed
Open

Fix #757: accept FixedSizeBinary(16) with arrow.uuid for UUID PK#768
adsharma wants to merge 1 commit into
mainfrom
fix/issue-757-arrow-rel-uuid-keyed

Conversation

@adsharma

@adsharma adsharma commented Aug 1, 2026

Copy link
Copy Markdown
Contributor

Summary

Closes #757.

createArrowRelTable required an exact match between the Arrow endpoint
column's logical type and the node table's primary key type. No Arrow wire
type mapped to a UUID logical type, so createArrowRelTable was unable to
address any node table keyed on UUID. CSV COPY could resolve STRING
endpoints against a UUID PK, so the restriction was an asymmetry, not a
limitation of the storage layer.

This implements option 2 from the issue: accept
FixedSizeBinary(16) carrying the arrow.uuid Arrow extension as a
UUID primary key. The 16-byte physical layout is identical to lbug's
internal UUID storage, so no conversion is needed in the read path beyond
the byte order and the MSB flip that UUIDs already require internally.

Changes

  • Recognize the arrow.uuid Arrow extension on FixedSizeBinary(16) as
    a UUID in tryGetArrowLogicalTypeInfo, so the schema round-trips to
    LogicalType::UUID instead of BLOB.
  • Route a FixedSizeBinary(16) UUID column through a new
    scanArrowArrayUuid that byte-reverses the 16-byte big-endian buffer
    to land in the int128_t layout and re-flips the UUID MSB. This mirrors
    the byte order the existing UUID write path produces.
  • Replace the exact-type check in the ArrowRelTable constructor with
    isArrowEndpointCompatibleWithPK, which still accepts an exact
    logical-type match and additionally accepts a FixedSizeBinary(16)
    UUID-extension column for a UUID PK. The error message is updated to
    describe the new acceptance rule.
  • Test utilities gain createUuidSchema and createUuidArray helpers.

Tests

  • ArrowRelTableTest.ScanArrowRelTableOverNativeUuidNodeTable is the
    end-to-end regression test: a CREATE NODE TABLE ... PRIMARY KEY(id UUID), then createRelTableFromArrowTable with w:16+arrow.uuid
    endpoint columns. Asserts the edges load, the weights sum, and the
    endpoints round-trip back to the right nodes.
  • ArrowRelTableTest.RejectFixedSizeBinaryWithoutUuidExtensionForUuidPK
    asserts that a bare w:16 (no extension) is still rejected for a UUID
    PK, since the current "exact match" message contract is preserved for
    types that cannot be reconciled.
  • ArrowConverterTest.bindsArrowUuidExtensionMetadataAsUuid,
    ...fixedSizeBinaryWithoutUuidExtensionIsNotUuid, and
    ...rejectsNon16ByteArrowUuidExtensionAsUuid cover the metadata
    parser directly: a correctly-annotated 16-byte column binds to UUID,
    a bare 16-byte column stays BLOB, and a non-16-byte column with the
    UUID extension is rejected (a UUID is exactly 16 bytes).

Notes

  • Byte order on the wire is big-endian (network), matching the Arrow
    UUID extension spec. The 16 bytes written by lbug's own UUID
    serialization in arrow_row_batch.cpp are already in this order; the
    read path mirrors the byte reversal performed on write.
  • A caller now needs to know the byte order, as the issue notes, and
    must include the ARROW:extension:name=arrow.uuid extension metadata
    on the endpoint columns. The new error message spells that out.
  • fromArrowSchema now returns UUID for w:16 columns carrying the
    extension, so this is also a prerequisite fix for reading UUID values
    from Arrow anywhere else (e.g. node tables).

@adsharma
adsharma force-pushed the fix/issue-757-arrow-rel-uuid-keyed branch 3 times, most recently from ab84c72 to dc49997 Compare August 1, 2026 01:54
The ArrowRelTable constructor required an exact match between the Arrow
endpoint column's logical type and the node table's primary key type.
No Arrow wire type mapped to a UUID logical type, so createArrowRelTable
was unable to address any node table keyed on UUID. CSV and the COPY
subquery path can both resolve STRING endpoints against a UUID PK, so
this restriction was a needless asymmetry.

Recognize the 'arrow.uuid' Arrow extension on FixedSizeBinary(16) as a
UUID, route that column through a UUID-specific read path that handles
the 16-byte big-endian layout and re-flips the UUID MSB, and accept the
extension as a valid endpoint type for a UUID primary key in the rel
table constructor. The error message is updated to reflect the new
acceptance rule.
@adsharma
adsharma force-pushed the fix/issue-757-arrow-rel-uuid-keyed branch from dc49997 to 62d10f2 Compare August 1, 2026 03:23
@alvorithm

Copy link
Copy Markdown
Contributor

Reporter of #757 here. Tested this branch at 62d10f224, relwithdebinfo, C API only. The accepted path works end to end: the rel table loads, both endpoints resolve to the right nodes, and the payload column reads back intact.

Two things worth folding in.

The reproducer attached to #757 cannot show this fix. arrow_rel_pk.c builds its FixedSizeBinary(16) case with no field metadata, so against this branch all four of its cases still fail. That is correct behaviour, but anyone who re-runs the attached file to check the fix will read it as a failure. The new message makes the reason clear:

Runtime exception: Arrow 'from' column type w:16 is not compatible with source node PK
type UUID; supported: matching logical type, or FixedSizeBinary(16) with the
'arrow.uuid' extension for UUID

I wrote a second reproducer that builds the ARROW:extension:name metadata blob by hand and drives the accepted path. Happy to attach it.

The UUID fixtures cannot catch a byte-order regression. ScanArrowRelTableOverNativeUuidNodeTable uses 11111111-1111-1111-1111-111111111111, 22222222-... and 33333333-.... Every byte inside each of those values is identical, so the 16-byte reversal in scanArrowArrayUuid is a no-op on them: the test passes whether or not the byte order is correct. The MSB flip is still covered, since flipping it changes the value. Only the ordering is invisible.

One asymmetric fixture closes the gap. I ran that against the branch:

check result
w:16 with arrow.uuid accepted
w:16 without the extension (control) rejected, as designed
edge lands on the right two nodes a -> b
source id round trip 01234567-89ab-cdef-0123-456789abcdef
target id round trip fedcba98-7654-3210-fedc-ba9876543210

The byte order is right today. The suggestion is only about keeping it that way if the read path is ever touched again.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Bug: createArrowRelTable cannot address a UUID-keyed node table, though COPY can

2 participants