Skip to content

GH-50913: [C++][Dataset] Replace RapidJSON with JsonWriter - #50914

Merged
HuaHuaY merged 3 commits into
apache:mainfrom
Reranko05:gh-35460-dataset
Aug 25, 2026
Merged

GH-50913: [C++][Dataset] Replace RapidJSON with JsonWriter#50914
HuaHuaY merged 3 commits into
apache:mainfrom
Reranko05:gh-35460-dataset

Conversation

@Reranko05

@Reranko05 Reranko05 commented Aug 19, 2026

Copy link
Copy Markdown
Collaborator

Rationale for this change

This PR continues the RapidJSON → simdjson migration by replacing the remaining RapidJSON usage in the Dataset JSON test with Arrow's existing JsonWriter.

Changes

  • Replace rapidjson::Writer and rapidjson::OStreamWrapper with arrow::json::JsonWriter.
  • Replace RapidJSON-specific SizeType usage with the JsonWriter interface.
  • Preserve the existing newline-delimited JSON test fixture generation.
  • Remove the unused RapidJSON dependency from the Dataset test in CMake and Meson.
  • Preserve the existing Dataset JSON test coverage and behavior.

Fixes: #50913

@Reranko05 Reranko05 added the CI: Extra: C++ Run extra C++ CI label Aug 19, 2026
@Reranko05
Reranko05 marked this pull request as ready for review August 19, 2026 19:51
@Reranko05
Reranko05 requested a review from pitrou as a code owner August 19, 2026 19:51
Copilot AI lite review requested due to automatic review settings August 19, 2026 19:51

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@Reranko05
Reranko05 requested review from kou and rok August 19, 2026 19:51
@kou

kou commented Aug 19, 2026

Copy link
Copy Markdown
Member

Can we fix the Meson job failure as a separated task to test Meson related changes in this PR before we proceed this PR?

https://github.com/apache/arrow/actions/runs/32293675583/job/96201011745?pr=50914#step:6:1103

FAILED: [code=1] subprojects/bzip2-1.0.8/libbz2.a.p/bzlib.c.o 
/usr/local/bin/sccache cc -Isubprojects/bzip2-1.0.8/libbz2.a.p -Isubprojects/bzip2-1.0.8 -I../../arrow/cpp/subprojects/bzip2-1.0.8 -fdiagnostics-color=always -D_FILE_OFFSET_BITS=64 -Wall -Winvalid-pch -Wextra -std=c11 -O0 -g -fPIC -MD -MQ subprojects/bzip2-1.0.8/libbz2.a.p/bzlib.c.o -MF subprojects/bzip2-1.0.8/libbz2.a.p/bzlib.c.o.d -o subprojects/bzip2-1.0.8/libbz2.a.p/bzlib.c.o -c ../../arrow/cpp/subprojects/bzip2-1.0.8/bzlib.c
../../arrow/cpp/subprojects/bzip2-1.0.8/bzlib.c: In function 'default_bzalloc':
../../arrow/cpp/subprojects/bzip2-1.0.8/bzlib.c:102:31: warning: unused parameter 'opaque' [-Wunused-parameter]
  102 | void* default_bzalloc ( void* opaque, Int32 items, Int32 size )
      |                         ~~~~~~^~~~~~
../../arrow/cpp/subprojects/bzip2-1.0.8/bzlib.c: In function 'default_bzfree':
../../arrow/cpp/subprojects/bzip2-1.0.8/bzlib.c:109:29: warning: unused parameter 'opaque' [-Wunused-parameter]
  109 | void default_bzfree ( void* opaque, void* addr )
      |                       ~~~~~~^~~~~~
../../arrow/cpp/subprojects/bzip2-1.0.8/bzlib.c: In function 'bzopen_or_bzdopen':
../../arrow/cpp/subprojects/bzip2-1.0.8/bzlib.c:1431:12: error: implicit declaration of function 'fdopen'; did you mean 'fopen'? [-Wimplicit-function-declaration]
 1431 |       fp = fdopen(fd,mode2);
      |            ^~~~~~
      |            fopen
...

Copilot AI review requested due to automatic review settings August 24, 2026 07:41

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

Copilot AI review requested due to automatic review settings August 24, 2026 08:21

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@pitrou
pitrou requested a review from HuaHuaY August 24, 2026 08:36

@HuaHuaY HuaHuaY left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry, I wasn't following the rapidjson replacement work before today. But as far as this PR goes, it looks good to me.

test_name,
sources: value['sources'],
dependencies: [arrow_dataset_test_dep, val.get('dependencies', [])],
dependencies: [arrow_dataset_test_dep, value.get('dependencies', [])],

@HuaHuaY HuaHuaY Aug 24, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just provide information to other reviewers: It is a historical issue, but it wasn't exposed because rapidjson only requires header files and doesn't need to link libraries.

@github-actions github-actions Bot added awaiting committer review Awaiting committer review and removed awaiting review Awaiting review labels Aug 24, 2026
@Reranko05

Copy link
Copy Markdown
Collaborator Author

@HuaHuaY Thanks for the review!

@pitrou

pitrou commented Aug 25, 2026

Copy link
Copy Markdown
Member

@HuaHuaY Do you want to merge this?

@HuaHuaY

HuaHuaY commented Aug 25, 2026

Copy link
Copy Markdown
Contributor

@HuaHuaY Do you want to merge this?

Yes. Let me merge this.

@pitrou

pitrou commented Aug 25, 2026

Copy link
Copy Markdown
Member

Yes. Let me merge this.

You need to use the merge script by the way.

@HuaHuaY
HuaHuaY merged commit 9979ec5 into apache:main Aug 25, 2026
74 of 79 checks passed
@HuaHuaY HuaHuaY removed the awaiting committer review Awaiting committer review label Aug 25, 2026
@HuaHuaY

HuaHuaY commented Aug 25, 2026

Copy link
Copy Markdown
Contributor

Thanks to @Reranko05 for the fix and to @pitrou for the help with merging the code.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[C++][Dataset] Replace RapidJSON with JsonWriter in JSON Dataset

5 participants