Fix #757: accept FixedSizeBinary(16) with arrow.uuid for UUID PK - #768
Fix #757: accept FixedSizeBinary(16) with arrow.uuid for UUID PK#768adsharma wants to merge 1 commit into
Conversation
ab84c72 to
dc49997
Compare
The ArrowRelTable constructor required an exact match between the Arrow endpoint column's logical type and the node table's primary key type. No Arrow wire type mapped to a UUID logical type, so createArrowRelTable was unable to address any node table keyed on UUID. CSV and the COPY subquery path can both resolve STRING endpoints against a UUID PK, so this restriction was a needless asymmetry. Recognize the 'arrow.uuid' Arrow extension on FixedSizeBinary(16) as a UUID, route that column through a UUID-specific read path that handles the 16-byte big-endian layout and re-flips the UUID MSB, and accept the extension as a valid endpoint type for a UUID primary key in the rel table constructor. The error message is updated to reflect the new acceptance rule.
dc49997 to
62d10f2
Compare
|
Reporter of #757 here. Tested this branch at Two things worth folding in. The reproducer attached to #757 cannot show this fix. I wrote a second reproducer that builds the The UUID fixtures cannot catch a byte-order regression. One asymmetric fixture closes the gap. I ran that against the branch:
The byte order is right today. The suggestion is only about keeping it that way if the read path is ever touched again. |
Summary
Closes #757.
createArrowRelTablerequired an exact match between the Arrow endpointcolumn's logical type and the node table's primary key type. No Arrow wire
type mapped to a UUID logical type, so
createArrowRelTablewas unable toaddress any node table keyed on UUID. CSV
COPYcould resolve STRINGendpoints against a UUID PK, so the restriction was an asymmetry, not a
limitation of the storage layer.
This implements option 2 from the issue: accept
FixedSizeBinary(16)carrying thearrow.uuidArrow extension as aUUID primary key. The 16-byte physical layout is identical to lbug's
internal UUID storage, so no conversion is needed in the read path beyond
the byte order and the MSB flip that UUIDs already require internally.
Changes
arrow.uuidArrow extension onFixedSizeBinary(16)asa UUID in
tryGetArrowLogicalTypeInfo, so the schema round-trips toLogicalType::UUIDinstead ofBLOB.FixedSizeBinary(16)UUID column through a newscanArrowArrayUuidthat byte-reverses the 16-byte big-endian bufferto land in the
int128_tlayout and re-flips the UUID MSB. This mirrorsthe byte order the existing UUID write path produces.
ArrowRelTableconstructor withisArrowEndpointCompatibleWithPK, which still accepts an exactlogical-type match and additionally accepts a
FixedSizeBinary(16)UUID-extension column for a UUID PK. The error message is updated to
describe the new acceptance rule.
createUuidSchemaandcreateUuidArrayhelpers.Tests
ArrowRelTableTest.ScanArrowRelTableOverNativeUuidNodeTableis theend-to-end regression test: a
CREATE NODE TABLE ... PRIMARY KEY(id UUID), thencreateRelTableFromArrowTablewithw:16+arrow.uuidendpoint columns. Asserts the edges load, the weights sum, and the
endpoints round-trip back to the right nodes.
ArrowRelTableTest.RejectFixedSizeBinaryWithoutUuidExtensionForUuidPKasserts that a bare
w:16(no extension) is still rejected for a UUIDPK, since the current "exact match" message contract is preserved for
types that cannot be reconciled.
ArrowConverterTest.bindsArrowUuidExtensionMetadataAsUuid,...fixedSizeBinaryWithoutUuidExtensionIsNotUuid, and...rejectsNon16ByteArrowUuidExtensionAsUuidcover the metadataparser directly: a correctly-annotated 16-byte column binds to UUID,
a bare 16-byte column stays BLOB, and a non-16-byte column with the
UUID extension is rejected (a UUID is exactly 16 bytes).
Notes
UUID extension spec. The 16 bytes written by lbug's own UUID
serialization in
arrow_row_batch.cppare already in this order; theread path mirrors the byte reversal performed on write.
must include the
ARROW:extension:name=arrow.uuidextension metadataon the endpoint columns. The new error message spells that out.
fromArrowSchemanow returnsUUIDforw:16columns carrying theextension, so this is also a prerequisite fix for reading UUID values
from Arrow anywhere else (e.g. node tables).