Skip to content
Draft
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
156 changes: 156 additions & 0 deletions _posts/2026-08-01-nanoarrow-0.9.0-release.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,156 @@
---
layout: post
title: "Apache Arrow nanoarrow 0.9.0 Release"
date: "2026-08-01 00:00:00"
author: pmc
categories: [release]
---
<!--
{% comment %}
Licensed to the Apache Software Foundation (ASF) under one or more
contributor license agreements. See the NOTICE file distributed with
this work for additional information regarding copyright ownership.
The ASF licenses this file to you under the Apache License, Version 2.0
(the "License"); you may not use this file except in compliance with
the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
{% endcomment %}
-->

The Apache Arrow team is pleased to announce the 0.9.0 release of
Apache Arrow nanoarrow. This release consists of 38 resolved GitHub issues from
5 contributors.

## Release Highlights

In addition to a number of bugfixes and minor build system improvements, we
added several new features in nanoarrow 0.9.0.

- Dictionary decoding support in IPC reader
- Reference-counted array/buffer support
- LZ4 decompression support in R and Python bindings

See the
[Changelog](https://github.com/apache/arrow-nanoarrow/blob/apache-arrow-nanoarrow-0.9.0/CHANGELOG.md)
for a detailed list of contributions to this release.

## Features

### Dictionary decode support

Whereas the nanoarrow IPC reader suppports most Arrow IPC features, dictionary support
was a long requested gap in the reader functionality (mostly requested by users of
the [DuckDB nanoarrow extension](https://github.com/paleolimbot/duckdb-nanoarrow), which uses nanoarrow's reader). Dictionary encoding is used to reduce the size of frequently
repeated values and is the serialized equivalent of the "dictionary" data type that
is exposed in most Arrow implementations.

In nanoarrow 0.9.0 built with the IPC feature enabled, streams that include the most
common forms of dictionary encoding (i.e., dictionary replacement) should now work
out of the box. This includes nested/complex dictionary types and dictionary
replacement but does not include "delta" dictionaries (i.e., dictionaires that
grow larger as more values are encountered in the encoded values).

In R this is accessible via `read_nanoarrow()`; in Python this is accessible via
`nanoarrow.ArrayStream.from_readable()`; in C this is available via the
higher level `ArrowIpcArrayStreamReader` API. Lower level users of the
`ArrowIpcDecoder` will have to update existing usage to use the
`ArrowIpcDecoder...WithDictionaries()` variants of some functions
to support input with dictionary schemas or batches.

### Reference-counted array/buffer support

In previous versions (since the introduction of the IPC reader), the
`ArrowIpcSharedBuffer` has supported reading IPC streams and sharing an
underlying set of data buffers for a group of arrays; however, this wasn't
quite sufficient for the more complex case of decoding a dictionary and
attaching cheaply-cloned shared "values" arrays for potentially many batches.
Version 0.9.0 moves this functionality to the `ArrowSharedBuffer` and expands it
to support moving all of an array's buffers into a shared state that can be
more cheaply cloned.

### LZ4 decompression support in R and Python

While LZ4 decompression support was has long been available via the pluggable decoder
framework (and available since 0.8.0 as a built-in compile time option), reading
IPC streams with LZ4 buffer compression was not possible in the R or Python bindings.
In 0.9.0, the requisite configuration options were added such that the packages are
built with LZ4 when it is available on the system.

```python
import io
import nanoarrow as na
import pyarrow as pa

buf = io.BytesIO()
batch = pa.record_batch({"x": range(1000)})
with pa.ipc.new_stream(buf, batch.schema, options=pa.ipc.IpcWriteOptions(compression="lz4")) as w:
w.write_batch(batch)

buf.seek(0)
na.ArrayStream.from_readable(buf).read_all()
# nanoarrow.Array<non-nullable struct<x: int64>>[1000]
# {'x': 0}
# {'x': 1}
# {'x': 2}
# {'x': 3}
# {'x': 4}
# {'x': 5}
# {'x': 6}
# {'x': 7}
# {'x': 8}
# {'x': 9}
# ...and 990 more items
```

```r
library(nanoarrow)
library(reticulate)

# IPC Write with compression not available in arrow/R
pa <- reticulate::import("pyarrow")
io <- reticulate::import("io")

buf <- io$BytesIO()
batch <- arrow::record_batch(x = 1:1000)
writer <- pa$ipc$new_stream(buf, batch$schema, options = pa$ipc$IpcWriteOptions(compression = "lz4"))
writer$write_batch(batch)
writer$close()

nanoarrow::read_nanoarrow(as.raw(buf$getvalue())) |>
tibble::as_tibble()
#> # A tibble: 1,000 × 1
#> x
#> <int>
#> 1 1
#> 2 2
#> 3 3
#> 4 4
#> 5 5
#> 6 6
#> 7 7
#> 8 8
#> 9 9
#> 10 10
#> # ℹ 990 more rows
```

## Contributors

This release consists of contributions from 5 contributors in addition
to the invaluable advice and support of the Apache Arrow community.

```console
$ git shortlog -sn apache-arrow-nanoarrow-0.9.0.dev..apache-arrow-nanoarrow-0.9.0
36 Dewey Dunnington
2 Andrew Kane
2 Bryce Mecum
1 Michael Osipov
1 Oliver Borchert
```