Skip to content

feat: implement transaction processor crate - #22

Open
bmuddha wants to merge 3 commits into
keeperfrom
processor
Open

feat: implement transaction processor crate#22
bmuddha wants to merge 3 commits into
keeperfrom
processor

Conversation

@bmuddha

@bmuddha bmuddha commented Jun 5, 2026

Copy link
Copy Markdown
Collaborator

What changed

Added the magicblock-processor crate, which schedules and executes transactions
over the keeper.

Why

Inbound transactions need conflict-aware scheduling that runs disjoint account
sets concurrently while reading and writing accounts through keeper/accountsdb.

Closes #11.

Impact

  • A sequencer fans conflict-free transactions across a pool of SVM executors and
    serializes the rest.
  • Conflicts are tracked per Pubkey with a write bit plus a per-executor
    occupancy bitset (MAX_EXECUTORS).
  • Adds a simulation path that runs against current state on owned account copies
    and returns the execution record without committing.

Reviewer notes

A quiescence barrier drains in-flight work for consistent snapshots at superblock
seals and during replay — the main concurrency-correctness surface.

Follow-up

engine wires this sequencer to durable state upstack.

@coderabbitai

coderabbitai Bot commented Aug 7, 2026

Copy link
Copy Markdown

Review Change Stack

Warning

Review limit reached

@bmuddha, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 59 minutes

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: fa295e4f-a5a3-4050-81ed-796efa00363f

📥 Commits

Reviewing files that changed from the base of the PR and between aa3abc8 and d6b29ab.

📒 Files selected for processing (1)
  • Cargo.toml
📝 Walkthrough

Walkthrough

The new processor crate adds parallel SVM execution, account-lock scheduling, block finalization, isolated simulation, metrics, error handling, and end-to-end tests. It is registered in the workspace and documents its execution model.

Changes

Transaction processor

Layer / File(s) Summary
Runtime contracts and SVM execution
Cargo.toml, processor/Cargo.toml, processor/src/lib.rs, processor/src/error.rs, processor/src/metrics.rs, processor/src/callback.rs, processor/src/svm.rs
Adds the workspace crate, processor message and error types, metrics, account callbacks, and block-aware SVM execution.
Executor worker lifecycle
processor/src/executor.rs
Adds worker-thread executors that process transaction batches and block transitions, report completion, and persist results with replay-mode handling.
Scheduling, locks, and block coordination
processor/src/sequencer/*, processor/README.md
Adds executor-pool management, account-lock coordination, blocked-transaction retry, barriers, draining, shutdown, hash chaining, block finalization, and related tests and documentation.
Isolated transaction simulation
processor/src/simulator.rs, processor/README.md
Adds a dedicated simulator worker for non-persistent transaction execution and documents simulation behavior.
End-to-end processor validation
processor/src/tests.rs
Adds tests for execution and simulation, block hashes, CPI loading, sysvar transitions, replay mode, failures, and contention workloads.

Estimated code review effort: 5 (Critical) | ~90 minutes

Sequence Diagram(s)

sequenceDiagram
  participant Client
  participant Sequencer
  participant Executor
  participant SvmContext
  participant Keeper

  Client->>Sequencer: submit transactions
  Sequencer->>Executor: dispatch conflict-free batch
  Executor->>SvmContext: execute transaction batch
  SvmContext->>Keeper: load and persist account state
  Executor->>Sequencer: report completed batch
  Sequencer->>Executor: release locks and dispatch blocked work
Loading

Possibly related PRs

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly identifies the primary change: implementing the transaction processor crate.
Description check ✅ Passed The description accurately covers the processor crate, scheduling, execution, conflict handling, simulation, and quiescence barriers.
Linked Issues check ✅ Passed The changes satisfy issue [#11] by adding scheduling, parallel executor dispatch, conflict sequencing, ordering, and keeper-backed state integration.
Out of Scope Changes check ✅ Passed The README, metrics, simulator, tests, and processor modules all support the transaction processor objectives in issue [#11].
Docstring Coverage ✅ Passed Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch processor

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@bmuddha

bmuddha commented Aug 7, 2026

Copy link
Copy Markdown
Collaborator Author

@CodeRabbit review

@coderabbitai

coderabbitai Bot commented Aug 7, 2026

Copy link
Copy Markdown
✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 3

🧹 Nitpick comments (1)
processor/src/tests.rs (1)

51-66: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low value

Document the channel-capacity coupling of the prefill tests.

unspawned lets tests fill the execution channel before the sequencer runs. The channel capacity is 1024 (processor/src/sequencer/mod.rs Line 90). The prefill tests send 128 and 512 transactions. If the capacity is lowered below a test's transaction count, Harness::execute blocks forever and the test hangs instead of failing. Add a note here so the constraint stays visible.

📝 Proposed doc note
     /// This lets tests fill the execution channel before the sequencer can
     /// consume from it, forcing contention resolution to happen from a backlog.
+    ///
+    /// Callers must keep their prefill count below the sequencer's execution
+    /// channel capacity. A larger prefill blocks `execute` and hangs the test.
     async fn unspawned(replay: bool) -> (Self, Sequencer) {
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@processor/src/tests.rs` around lines 51 - 66, Add a documentation note to
Harness::unspawned stating that prefill tests must remain at or below the
execution channel capacity of 1024, because exceeding it causes Harness::execute
to block while the sequencer is unspawned. Mention that the existing 128- and
512-transaction tests depend on this constraint.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@processor/src/sequencer/locks.rs`:
- Around line 77-78: Update the rustdoc for the `release` method to state that
it releases every account lock read from `executor.locks`, replacing the stale
`held` reference while preserving the documented behavior.
- Around line 60-73: Remove each zero-count account from executor.locks during
the rollback loop in LockTable’s release path, while preserving lock.contend and
unlock behavior for remaining counts; update processor/src/sequencer/tests.rs
lines 138-138 to assert blocked.locks.get(&a) is None. Also make
AccountLock::unlock clear WRITE_BIT only when that executor actually owns it.

In `@processor/src/simulator.rs`:
- Around line 51-65: Update the worker loop in run so self.rx.recv() explicitly
exits when the channel returns None, preserving message handling for Some(msg)
and the documented behavior that channel closure ends the worker.

---

Nitpick comments:
In `@processor/src/tests.rs`:
- Around line 51-66: Add a documentation note to Harness::unspawned stating that
prefill tests must remain at or below the execution channel capacity of 1024,
because exceeding it causes Harness::execute to block while the sequencer is
unspawned. Mention that the existing 128- and 512-transaction tests depend on
this constraint.
🪄 Autofix

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: 7fd7928a-706f-4d11-bafe-0a0c5ca26191

📥 Commits

Reviewing files that changed from the base of the PR and between e5f37e1 and ac70027.

📒 Files selected for processing (15)
  • Cargo.toml
  • processor/Cargo.toml
  • processor/README.md
  • processor/src/callback.rs
  • processor/src/error.rs
  • processor/src/executor.rs
  • processor/src/lib.rs
  • processor/src/metrics.rs
  • processor/src/sequencer/locks.rs
  • processor/src/sequencer/mod.rs
  • processor/src/sequencer/pool.rs
  • processor/src/sequencer/tests.rs
  • processor/src/simulator.rs
  • processor/src/svm.rs
  • processor/src/tests.rs

Comment thread processor/src/sequencer/locks.rs
Comment thread processor/src/sequencer/locks.rs Outdated
Comment on lines +77 to +78
/// Releases every account lock recorded in `held` for `executor`.
pub(super) fn release(&mut self, executor: &mut ExecutorHandle) {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win

Fix the stale parameter name in the doc comment.

The doc refers to held, but the function takes only executor. The locks are read from executor.locks.

📝 Proposed doc fix
-    /// Releases every account lock recorded in `held` for `executor`.
+    /// Releases every account lock recorded in `executor.locks`.
     pub(super) fn release(&mut self, executor: &mut ExecutorHandle) {

As per path instructions: "Check docs and rustdoc for factual consistency with the code."

📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
/// Releases every account lock recorded in `held` for `executor`.
pub(super) fn release(&mut self, executor: &mut ExecutorHandle) {
/// Releases every account lock recorded in `executor.locks`.
pub(super) fn release(&mut self, executor: &mut ExecutorHandle) {
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@processor/src/sequencer/locks.rs` around lines 77 - 78, Update the rustdoc
for the `release` method to state that it releases every account lock read from
`executor.locks`, replacing the stale `held` reference while preserving the
documented behavior.

Source: Path instructions

Comment thread processor/src/simulator.rs
@bmuddha

bmuddha commented Aug 10, 2026

Copy link
Copy Markdown
Collaborator Author

@CodeRabbit review

@coderabbitai

coderabbitai Bot commented Aug 10, 2026

Copy link
Copy Markdown
✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

coderabbit Trigger coderabbit review

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Implement parallel transaction processor

1 participant