Skip to content

[experiment] Use DenseBitSetStorage to store MixedBitSet::Small inline - #161454

Draft
panstromek wants to merge 9 commits into
rust-lang:mainfrom
panstromek:mixed-bitset-inline
Draft

[experiment] Use DenseBitSetStorage to store MixedBitSet::Small inline#161454
panstromek wants to merge 9 commits into
rust-lang:mainfrom
panstromek:mixed-bitset-inline

Conversation

@panstromek

Copy link
Copy Markdown
Contributor

Depends on the base from #161345, uses slightly different design with a new DenseBitSetStorage trait, that makes it somewhat easier to thread generic params through the codebase.

We use this new design to store MixedBitSet::Small variant inline in 2-word array, instead of Heap.

I'm not super happy about this test case for 2 reasons

  1. MixedBitSet is not really a good example of the target use case for this abstraction

But it's also hottest usage of small DenseBitSet so I hope I'll see some effect compared to #161345 (which was tested on roughly second hottest small bitset on serde).

  1. This requires me to change threshold for Small bitsets, which might have side effects that mask the potential improvements

But according to my measurements, ::Small variant is almost always <=2 words in our benchmark suite. I believe there's at most 1 case where it is 4 words (maybe none, but line numbers changed since I measured this so I'm not sure).

@rustbot rustbot added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. labels Aug 21, 2026
@panstromek

Copy link
Copy Markdown
Contributor Author

@bors try @rust-timer queue

@rust-timer

This comment has been minimized.

@rustbot rustbot added the S-waiting-on-perf Status: Waiting on a perf run to be completed. label Aug 21, 2026
@rust-bors

This comment has been minimized.

rust-bors Bot pushed a commit that referenced this pull request Aug 21, 2026
[experiment] Use DenseBitSetStorage to store MixedBitSet::Small inline
@rust-bors

rust-bors Bot commented Aug 21, 2026

Copy link
Copy Markdown
Contributor

☀️ Try build successful (CI)
Build commit: 477da19 (477da195437b12e5c9752a247a097be731237842)
Base parent: 16a623a (16a623ad672a92409b5c04beb303583c6cf72a7e)

@rust-timer

This comment has been minimized.

@rust-log-analyzer

Copy link
Copy Markdown
Collaborator

The job aarch64-gnu-llvm-21-1 failed! Check out the build log: (web) (plain enhanced) (plain)

Click to see the possible cause of the failure (guessed by this bot)

---- [mir-opt] tests/mir-opt/dataflow.rs stdout ----
------rustc stdout------------------------------

------rustc stderr------------------------------

thread 'rustc' (186454) panicked at compiler/rustc_mir_dataflow/src/framework/fmt.rs:251:52:
index out of bounds: the len is 1 but the index is 1
stack backtrace:
   0: __rustc::rust_begin_unwind
   1: core::panicking::panic_fmt
   2: core::panicking::panic_bounds_check
   3: <rustc_mir_dataflow::move_paths::MovePathIndex as rustc_mir_dataflow::framework::fmt::DebugWithContext<rustc_mir_dataflow::impls::initialized::MaybeUninitializedPlaces>>::fmt_with
   4: <core::fmt::builders::DebugList>::entry
   5: <core::fmt::builders::DebugSet>::entries::<rustc_mir_dataflow::framework::fmt::DebugWithAdapter<rustc_mir_dataflow::move_paths::MovePathIndex, rustc_mir_dataflow::impls::initialized::MaybeUninitializedPlaces>, core::iter::adapters::map::Map<rustc_index::bit_set::BitIter<rustc_mir_dataflow::move_paths::MovePathIndex>, <rustc_index::bit_set::DenseBitSet<rustc_mir_dataflow::move_paths::MovePathIndex, rustc_index::bit_set::ArrayStorage<2>> as rustc_mir_dataflow::framework::fmt::DebugWithContext<rustc_mir_dataflow::impls::initialized::MaybeUninitializedPlaces>>::fmt_with::{closure#0}>>
   6: <&rustc_index::bit_set::MixedBitSet<rustc_mir_dataflow::move_paths::MovePathIndex> as rustc_mir_dataflow::framework::fmt::DebugWithContext<rustc_mir_dataflow::impls::initialized::MaybeUninitializedPlaces>>::fmt_with
   7: core::fmt::write
   8: alloc::fmt::format::format_inner
   9: <rustc_mir_dataflow::framework::graphviz::BlockFormatter<rustc_mir_dataflow::impls::initialized::MaybeUninitializedPlaces>>::write_row::<alloc::vec::Vec<u8>, <rustc_mir_dataflow::framework::graphviz::BlockFormatter<rustc_mir_dataflow::impls::initialized::MaybeUninitializedPlaces>>::write_row_with_full_state<alloc::vec::Vec<u8>>::{closure#0}>
  10: <rustc_mir_dataflow::framework::graphviz::Formatter<rustc_mir_dataflow::impls::initialized::MaybeUninitializedPlaces> as rustc_graphviz::Labeller>::node_label
  11: rustc_graphviz::render_opts::<rustc_middle::mir::BasicBlock, rustc_mir_dataflow::framework::graphviz::CfgEdge, rustc_mir_dataflow::framework::graphviz::Formatter<rustc_mir_dataflow::impls::initialized::MaybeUninitializedPlaces>, alloc::vec::Vec<u8>>
  12: rustc_mir_dataflow::framework::graphviz::write_graphviz_results::<rustc_mir_dataflow::impls::initialized::MaybeUninitializedPlaces>
  13: <rustc_mir_dataflow::impls::initialized::MaybeUninitializedPlaces as rustc_mir_dataflow::framework::Analysis>::iterate_to_fixpoint
  14: rustc_borrowck::borrowck_check_region_constraints
  15: <rustc_borrowck::root_cx::BorrowCheckRootCtxt>::do_mir_borrowck
  16: rustc_borrowck::mir_borrowck
      [... omitted 2 frames ...]
  17: <rustc_middle::ty::context::TyCtxt>::par_hir_body_owners::<rustc_interface::passes::run_required_analyses::{closure#2}::{closure#0}>::{closure#0}
  18: std::panicking::catch_unwind::<(), core::panic::unwind_safe::AssertUnwindSafe<rustc_data_structures::sync::parallel::par_for_each_in<&rustc_span::def_id::LocalDefId, &[rustc_span::def_id::LocalDefId], <rustc_middle::ty::context::TyCtxt>::par_hir_body_owners<rustc_interface::passes::run_required_analyses::{closure#2}::{closure#0}>::{closure#0}>::{closure#0}::{closure#1}::{closure#0}>>
  19: rustc_data_structures::sync::parallel::par_for_each_in::<&rustc_span::def_id::LocalDefId, &[rustc_span::def_id::LocalDefId], <rustc_middle::ty::context::TyCtxt>::par_hir_body_owners<rustc_interface::passes::run_required_analyses::{closure#2}::{closure#0}>::{closure#0}>
  20: <rustc_session::session::Session>::time::<(), rustc_interface::passes::run_required_analyses::{closure#2}>
  21: rustc_interface::passes::analysis
      [... omitted 2 frames ...]
  22: std::panicking::catch_unwind::<core::option::Option<rustc_interface::queries::Linker>, core::panic::unwind_safe::AssertUnwindSafe<rustc_interface::passes::create_and_enter_global_ctxt<core::option::Option<rustc_interface::queries::Linker>, rustc_driver_impl::run_compiler::{closure#0}::{closure#1}>::{closure#2}::{closure#0}>>
  23: <std::thread::local::LocalKey<core::cell::Cell<*const ()>>>::with::<rustc_middle::ty::context::tls::enter_context<<rustc_middle::ty::context::GlobalCtxt>::enter<rustc_interface::passes::create_and_enter_global_ctxt<core::option::Option<rustc_interface::queries::Linker>, rustc_driver_impl::run_compiler::{closure#0}::{closure#1}>::{closure#2}, core::option::Option<rustc_interface::queries::Linker>>::{closure#1}, core::option::Option<rustc_interface::queries::Linker>>::{closure#0}, core::option::Option<rustc_interface::queries::Linker>>
  24: <rustc_middle::ty::context::TyCtxt>::create_global_ctxt::<core::option::Option<rustc_interface::queries::Linker>, rustc_interface::passes::create_and_enter_global_ctxt<core::option::Option<rustc_interface::queries::Linker>, rustc_driver_impl::run_compiler::{closure#0}::{closure#1}>::{closure#2}>
  25: rustc_interface::passes::create_and_enter_global_ctxt::<core::option::Option<rustc_interface::queries::Linker>, rustc_driver_impl::run_compiler::{closure#0}::{closure#1}>
  26: std::panicking::catch_unwind::<(), core::panic::unwind_safe::AssertUnwindSafe<rustc_interface::interface::run_compiler<(), rustc_driver_impl::run_compiler::{closure#0}>::{closure#2}::{closure#0}>>
  27: rustc_interface::interface::run_compiler::<(), rustc_driver_impl::run_compiler::{closure#0}>::{closure#2}
  28: rustc_span::create_session_globals_then::<(), rustc_interface::util::run_in_thread_with_globals<rustc_interface::util::run_in_thread_pool_with_globals<rustc_interface::interface::run_compiler<(), rustc_driver_impl::run_compiler::{closure#0}>::{closure#2}, ()>::{closure#0}, ()>::{closure#0}::{closure#0}::{closure#0}>
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
---
note: please make sure that you have updated to the latest nightly

note: rustc 1.100.0-nightly (e5ea755c6 2026-08-21) running on aarch64-unknown-linux-gnu

note: compiler flags: -Z simulate-remapped-rust-src-base=/rustc/FAKE_PREFIX -Z translate-remapped-path-to-local-path=no -Z ignore-directory-in-diagnostics-source-blocks=/cargo -Z ignore-directory-in-diagnostics-source-blocks=/checkout/vendor -C opt-level=1 -Z dump-mir=maybe_uninit -Z validate-mir -Z lint-mir -Z dump-mir-exclude-pass-number -Z mir-include-spans=false --crate-type rlib -Z mir-opt-level=4 -Z mir-enable-passes=+ReorderBasicBlocks,+ReorderLocals -Z dump-mir-dir=/checkout/obj/build/aarch64-unknown-linux-gnu/test/mir-opt/dataflow -C prefer-dynamic -C rpath -C debuginfo=0 -Z dump-mir=main -Z dump-mir-dataflow

query stack during panic:
#0 [mir_borrowck] borrow-checking `main`
#1 [analysis] running analysis passes on crate `dataflow`
end of query stack

------------------------------------------

error: compilation failed!
status: exit status: 101
command: env -u RUSTC_LOG_COLOR RUSTC_ICE="0" RUST_BACKTRACE="short" "/checkout/obj/build/aarch64-unknown-linux-gnu/stage2/bin/rustc" "/checkout/tests/mir-opt/dataflow.rs" "-Zsimulate-remapped-rust-src-base=/rustc/FAKE_PREFIX" "-Ztranslate-remapped-path-to-local-path=no" "-Z" "ignore-directory-in-diagnostics-source-blocks=/cargo" "-Z" "ignore-directory-in-diagnostics-source-blocks=/checkout/vendor" "--sysroot" "/checkout/obj/build/aarch64-unknown-linux-gnu/stage2" "--target=aarch64-unknown-linux-gnu" "--check-cfg" "cfg(test,FALSE)" "-O" "-Copt-level=1" "-Zdump-mir=maybe_uninit" "-Zvalidate-mir" "-Zlint-mir" "-Zdump-mir-exclude-pass-number" "-Zmir-include-spans=false" "--crate-type=rlib" "-Zmir-opt-level=4" "-Zmir-enable-passes=+ReorderBasicBlocks,+ReorderLocals" "-Zdump-mir-dir=/checkout/obj/build/aarch64-unknown-linux-gnu/test/mir-opt/dataflow" "--emit" "mir" "-C" "prefer-dynamic" "--out-dir" "/checkout/obj/build/aarch64-unknown-linux-gnu/test/mir-opt/dataflow" "-A" "internal_features" "-A" "incomplete_features" "-A" "unused_parens" "-A" "unused_braces" "-Crpath" "-Cdebuginfo=0" "-Z" "dump-mir=main" "-Z" "dump-mir-dataflow"
stdout: none
--- stderr -------------------------------

thread 'rustc' (186454) panicked at compiler/rustc_mir_dataflow/src/framework/fmt.rs:251:52:
index out of bounds: the len is 1 but the index is 1
stack backtrace:
   0: __rustc::rust_begin_unwind
   1: core::panicking::panic_fmt
   2: core::panicking::panic_bounds_check
   3: <rustc_mir_dataflow::move_paths::MovePathIndex as rustc_mir_dataflow::framework::fmt::DebugWithContext<rustc_mir_dataflow::impls::initialized::MaybeUninitializedPlaces>>::fmt_with
   4: <core::fmt::builders::DebugList>::entry
   5: <core::fmt::builders::DebugSet>::entries::<rustc_mir_dataflow::framework::fmt::DebugWithAdapter<rustc_mir_dataflow::move_paths::MovePathIndex, rustc_mir_dataflow::impls::initialized::MaybeUninitializedPlaces>, core::iter::adapters::map::Map<rustc_index::bit_set::BitIter<rustc_mir_dataflow::move_paths::MovePathIndex>, <rustc_index::bit_set::DenseBitSet<rustc_mir_dataflow::move_paths::MovePathIndex, rustc_index::bit_set::ArrayStorage<2>> as rustc_mir_dataflow::framework::fmt::DebugWithContext<rustc_mir_dataflow::impls::initialized::MaybeUninitializedPlaces>>::fmt_with::{closure#0}>>
   6: <&rustc_index::bit_set::MixedBitSet<rustc_mir_dataflow::move_paths::MovePathIndex> as rustc_mir_dataflow::framework::fmt::DebugWithContext<rustc_mir_dataflow::impls::initialized::MaybeUninitializedPlaces>>::fmt_with
   7: core::fmt::write
   8: alloc::fmt::format::format_inner
   9: <rustc_mir_dataflow::framework::graphviz::BlockFormatter<rustc_mir_dataflow::impls::initialized::MaybeUninitializedPlaces>>::write_row::<alloc::vec::Vec<u8>, <rustc_mir_dataflow::framework::graphviz::BlockFormatter<rustc_mir_dataflow::impls::initialized::MaybeUninitializedPlaces>>::write_row_with_full_state<alloc::vec::Vec<u8>>::{closure#0}>
  10: <rustc_mir_dataflow::framework::graphviz::Formatter<rustc_mir_dataflow::impls::initialized::MaybeUninitializedPlaces> as rustc_graphviz::Labeller>::node_label
  11: rustc_graphviz::render_opts::<rustc_middle::mir::BasicBlock, rustc_mir_dataflow::framework::graphviz::CfgEdge, rustc_mir_dataflow::framework::graphviz::Formatter<rustc_mir_dataflow::impls::initialized::MaybeUninitializedPlaces>, alloc::vec::Vec<u8>>
  12: rustc_mir_dataflow::framework::graphviz::write_graphviz_results::<rustc_mir_dataflow::impls::initialized::MaybeUninitializedPlaces>
  13: <rustc_mir_dataflow::impls::initialized::MaybeUninitializedPlaces as rustc_mir_dataflow::framework::Analysis>::iterate_to_fixpoint
  14: rustc_borrowck::borrowck_check_region_constraints
  15: <rustc_borrowck::root_cx::BorrowCheckRootCtxt>::do_mir_borrowck
  16: rustc_borrowck::mir_borrowck
      [... omitted 2 frames ...]
  17: <rustc_middle::ty::context::TyCtxt>::par_hir_body_owners::<rustc_interface::passes::run_required_analyses::{closure#2}::{closure#0}>::{closure#0}
  18: std::panicking::catch_unwind::<(), core::panic::unwind_safe::AssertUnwindSafe<rustc_data_structures::sync::parallel::par_for_each_in<&rustc_span::def_id::LocalDefId, &[rustc_span::def_id::LocalDefId], <rustc_middle::ty::context::TyCtxt>::par_hir_body_owners<rustc_interface::passes::run_required_analyses::{closure#2}::{closure#0}>::{closure#0}>::{closure#0}::{closure#1}::{closure#0}>>
  19: rustc_data_structures::sync::parallel::par_for_each_in::<&rustc_span::def_id::LocalDefId, &[rustc_span::def_id::LocalDefId], <rustc_middle::ty::context::TyCtxt>::par_hir_body_owners<rustc_interface::passes::run_required_analyses::{closure#2}::{closure#0}>::{closure#0}>
  20: <rustc_session::session::Session>::time::<(), rustc_interface::passes::run_required_analyses::{closure#2}>
  21: rustc_interface::passes::analysis
      [... omitted 2 frames ...]
  22: std::panicking::catch_unwind::<core::option::Option<rustc_interface::queries::Linker>, core::panic::unwind_safe::AssertUnwindSafe<rustc_interface::passes::create_and_enter_global_ctxt<core::option::Option<rustc_interface::queries::Linker>, rustc_driver_impl::run_compiler::{closure#0}::{closure#1}>::{closure#2}::{closure#0}>>
  23: <std::thread::local::LocalKey<core::cell::Cell<*const ()>>>::with::<rustc_middle::ty::context::tls::enter_context<<rustc_middle::ty::context::GlobalCtxt>::enter<rustc_interface::passes::create_and_enter_global_ctxt<core::option::Option<rustc_interface::queries::Linker>, rustc_driver_impl::run_compiler::{closure#0}::{closure#1}>::{closure#2}, core::option::Option<rustc_interface::queries::Linker>>::{closure#1}, core::option::Option<rustc_interface::queries::Linker>>::{closure#0}, core::option::Option<rustc_interface::queries::Linker>>
  24: <rustc_middle::ty::context::TyCtxt>::create_global_ctxt::<core::option::Option<rustc_interface::queries::Linker>, rustc_interface::passes::create_and_enter_global_ctxt<core::option::Option<rustc_interface::queries::Linker>, rustc_driver_impl::run_compiler::{closure#0}::{closure#1}>::{closure#2}>
  25: rustc_interface::passes::create_and_enter_global_ctxt::<core::option::Option<rustc_interface::queries::Linker>, rustc_driver_impl::run_compiler::{closure#0}::{closure#1}>
  26: std::panicking::catch_unwind::<(), core::panic::unwind_safe::AssertUnwindSafe<rustc_interface::interface::run_compiler<(), rustc_driver_impl::run_compiler::{closure#0}>::{closure#2}::{closure#0}>>
  27: rustc_interface::interface::run_compiler::<(), rustc_driver_impl::run_compiler::{closure#0}>::{closure#2}
  28: rustc_span::create_session_globals_then::<(), rustc_interface::util::run_in_thread_with_globals<rustc_interface::util::run_in_thread_pool_with_globals<rustc_interface::interface::run_compiler<(), rustc_driver_impl::run_compiler::{closure#0}>::{closure#2}, ()>::{closure#0}, ()>::{closure#0}::{closure#0}::{closure#0}>
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
---
note: please make sure that you have updated to the latest nightly

note: rustc 1.100.0-nightly (e5ea755c6 2026-08-21) running on aarch64-unknown-linux-gnu

note: compiler flags: -Z simulate-remapped-rust-src-base=/rustc/FAKE_PREFIX -Z translate-remapped-path-to-local-path=no -Z ignore-directory-in-diagnostics-source-blocks=/cargo -Z ignore-directory-in-diagnostics-source-blocks=/checkout/vendor -C opt-level=1 -Z dump-mir=maybe_uninit -Z validate-mir -Z lint-mir -Z dump-mir-exclude-pass-number -Z mir-include-spans=false --crate-type rlib -Z mir-opt-level=4 -Z mir-enable-passes=+ReorderBasicBlocks,+ReorderLocals -Z dump-mir-dir=/checkout/obj/build/aarch64-unknown-linux-gnu/test/mir-opt/dataflow -C prefer-dynamic -C rpath -C debuginfo=0 -Z dump-mir=main -Z dump-mir-dataflow

query stack during panic:
#0 [mir_borrowck] borrow-checking `main`
#1 [analysis] running analysis passes on crate `dataflow`
end of query stack
------------------------------------------

---- [mir-opt] tests/mir-opt/dataflow.rs stdout end ----

@rust-timer

Copy link
Copy Markdown
Collaborator

Finished benchmarking commit (477da19): comparison URL.

Overall result: ❌✅ regressions and improvements - please read:

Benchmarking means the PR may be perf-sensitive. It's automatically marked not fit for rolling up. Overriding is possible but disadvised: it risks changing compiler perf.

Next, please: If you can, justify the regressions found in this try perf run in writing along with @rustbot label: +perf-regression-triaged. If not, fix the regressions and do another perf run. Neutral or positive results will clear the label automatically.

@bors rollup=never rustc-perf
@rustbot label: -S-waiting-on-perf +perf-regression

Instruction count

Our most reliable metric. Used to determine the overall result above. However, even this metric can be noisy.

mean range count
Regressions ❌
(primary)
0.8% [0.1%, 2.3%] 22
Regressions ❌
(secondary)
1.0% [0.2%, 2.3%] 22
Improvements ✅
(primary)
-0.2% [-0.2%, -0.1%] 16
Improvements ✅
(secondary)
-0.4% [-0.9%, -0.1%] 29
All ❌✅ (primary) 0.4% [-0.2%, 2.3%] 38

Max RSS (memory usage)

Results (primary -1.0%, secondary -2.6%)

A less reliable metric. May be of interest, but not used to determine the overall result above.

mean range count
Regressions ❌
(primary)
0.5% [0.5%, 0.5%] 1
Regressions ❌
(secondary)
- - 0
Improvements ✅
(primary)
-2.5% [-2.5%, -2.5%] 1
Improvements ✅
(secondary)
-2.6% [-2.6%, -2.6%] 1
All ❌✅ (primary) -1.0% [-2.5%, 0.5%] 2

Cycles

Results (primary 2.2%, secondary 4.4%)

A less reliable metric. May be of interest, but not used to determine the overall result above.

mean range count
Regressions ❌
(primary)
2.2% [2.2%, 2.2%] 1
Regressions ❌
(secondary)
7.8% [6.4%, 9.1%] 2
Improvements ✅
(primary)
- - 0
Improvements ✅
(secondary)
-2.2% [-2.2%, -2.2%] 1
All ❌✅ (primary) 2.2% [2.2%, 2.2%] 1

Binary size

Results (primary 0.1%, secondary 0.1%)

A less reliable metric. May be of interest, but not used to determine the overall result above.

mean range count
Regressions ❌
(primary)
0.1% [0.0%, 0.2%] 97
Regressions ❌
(secondary)
0.1% [0.0%, 0.2%] 66
Improvements ✅
(primary)
-0.0% [-0.0%, -0.0%] 3
Improvements ✅
(secondary)
- - 0
All ❌✅ (primary) 0.1% [-0.0%, 0.2%] 100

Bootstrap: 457.604s -> 455.214s (-0.52%)
Artifact size: 398.89 MiB -> 399.04 MiB (0.04%)

@rustbot rustbot added perf-regression Performance regression. and removed S-waiting-on-perf Status: Waiting on a perf run to be completed. labels Aug 21, 2026
@panstromek

Copy link
Copy Markdown
Contributor Author

Turns out there's a very subtle bug in this, the failing test is a symptom, but I'm surprised that something else didn't fail much more loudly.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

perf-regression Performance regression. S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants