feat: adds top level engine crate for global orchestration

[bmuddha]

What changed

Added the top-level magicblock-engine crate that wires keeper state,
transaction processing, block pacing, recovery, and MagicRoot account operations
behind the consumer-facing Engine handle.

Why

Embedding services need one entry point for opening durable state, reconstructing
it after recovery, submitting work, and coordinating shutdown.

Part of #4.

Closes #29.

Impact

• Engine::account exposes committed create, update, patch, and delete
  operations; Engine::transaction supports execute, schedule, and simulation.
• Instruction slices and Message values are signed with the engine authority
  and latest blockhash; sanitized views and encoded transactions are accepted
  without re-signing.
• Startup restores keeper state, replays retained entries after a snapshot rewind,
  and starts the sequencer with internal or external block pacing.
• Coordinated shutdown rejects new transactions, drains execution, and flushes
  durable or externally mirrored state according to the pacing source.

Reviewer notes

Replay quiesces at each sealed superblock and compares the reconstructed account
checksum with the recorded seal. Divergence returns
ReplayError::StateMismatch.

Follow-up

The replication crate uses the external pacer and replay paths to build followers
upstack.

Summary by CodeRabbit

• New Features

  • Added a durable execution engine for account management, transaction processing, simulation, scheduling, and coordinated shutdown.
  • Added account creation, updates, patching, deletion, and Magic Root instruction support.
  • Added internal and externally controlled block pacing.
  • Added transaction composition with signing and validation.
  • Added recovery, replay, state validation, and restart support.
  • Added an optional test harness for engine workflows.

• Documentation

  • Added engine usage and lifecycle documentation.
  • Updated account mutation examples and clarified patching, slot, and finalization behavior.

[bmuddha]

Warning

This pull request is not mergeable via GitHub because a downstack PR is open. Once all requirements are satisfied, merge this PR as a stack on Graphite.
Learn more

• feat: protect authoritative accounts from overwrites #37
• feat: add state replicator crate on top of the engine #25
• feat: adds top level engine crate for global orchestration #24 👈 (View in Graphite)
• feat: added internal built-in magic root program #23
• feat: implement transaction processor crate #22
• feat: implements keeper crate - state orchestrator #17
• feat: implement superblock based ledger crate #15
• feat: add optimized accountsdb crate #14
• feat: add nucleus engine crate #18
• feat: add svm crate #21
• feat: add program runtime crate #20
• feat: add transaction context crate #19
• feat: forked transaction-view for increased limits #33
• feat: add customized solana-account crate #13
• chore: import upstream Solana runtime crates #31
• dev

This stack of pull requests is managed by Graphite. Learn more about stacking.

[coderabbitai]

Warning

Review limit reached

@bmuddha, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 59 minutes

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: 25a6f151-6704-4eb8-8510-258f83b5c321

📥 Commits

Reviewing files that changed from the base of the PR and between 5d3d9ee and 6298cd8.

📒 Files selected for processing (9)

• Cargo.toml
• engine/Cargo.toml
• engine/README.md
• engine/src/accessor.rs
• engine/src/error.rs
• engine/src/lib.rs
• engine/src/pacemaker.rs
• engine/tests/recovery.rs
• engine/tests/security.rs

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[coderabbitai]

Actionable comments posted: 11

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@engine/README.md`:
- Around line 41-42: Update the internal pacing description in the README to say
that it appends the reset marker at the current slot, matching the behavior of
BlockTicker::new and PaceMaker::spawn. Leave the surrounding volatile-account
and pacemaker task description unchanged.

In `@engine/src/accessor.rs`:
- Around line 79-81: Add a dedicated `EngineError::ShuttingDown` variant in
`engine/src/error.rs` with the existing shutdown message, then update both
`execute` and `schedule` in the accessors to return that typed variant instead
of constructing a string error. Ensure both shutdown checks use the same variant
so callers can match it directly.
- Around line 86-90: Update the public execute method around the time::timeout
call to make timeout semantics explicit: return a distinct timeout error that
callers can recognize and use to query the transaction signature, while
preserving the submitted transaction’s continued processing. Ensure timeout
handling is distinguishable from receive and execution errors rather than
converting all errors to strings.
- Around line 103-111: Update Accessor::simulate to check
self.engine.terminating before sending the SimulatorMessage::Transaction,
matching the guards in execute and schedule. Return the established shutdown
error immediately when termination has begun; otherwise preserve the existing
channel send and response handling.

In `@engine/src/lib.rs`:
- Around line 166-208: Ensure locally owned ShutdownManager instances terminate
spawned services on all error paths. In engine/src/lib.rs lines 166-208, scope
the replay body so the temporary engine is dropped, capture its result, then
signal the LedgerReplayer and await shutdown. In engine/src/testkit.rs lines
86-88, match the Engine::new result and await shutdown termination before
returning an error; preserve successful initialization behavior.
- Around line 146-155: Update the OwnedBlockstoreEntry::Superblock replay path
so a checksum mismatch cannot leave expected.id persisted as the sealed
superblock: either validate observed against expected before calling
accounts().set_superblock, or restore the previous sealed id before returning
ReplayError::StateMismatch. Preserve the existing barrier, sync, and mismatch
error behavior.

In `@engine/src/pacemaker.rs`:
- Around line 53-61: Validate that BlockstoreParams::blocktime is non-zero
before it reaches BlockTicker::new, preferably by using a non-zero duration type
or rejecting Duration::ZERO during configuration parsing. Preserve the existing
ticker initialization only for valid positive blocktimes, preventing
time::interval from receiving zero.
- Around line 129-158: Update the Pacer::run shutdown loop to acquire the next
boundary through a cancellable next_block operation, then execute handle for
that acquired block outside tokio::select! so shutdown cannot interrupt its
internal awaits or skip finalize_superblock. Replace the current pace-based flow
while preserving normal shutdown and error propagation behavior.

In `@engine/src/testkit.rs`:
- Around line 86-88: Update the Engine::new call in the test harness to handle
its error explicitly: if construction fails, call shutdown.terminate().await
before propagating the original error. Preserve the existing successful path
that obtains the current slot from the created engine.

In `@engine/src/transaction.rs`:
- Around line 90-121: Update magicblock to reject compiled messages whose
header.num_required_signatures exceeds the Engine authority’s single-signature
model. Validate this immediately after v1::Message::try_compile and return an
appropriate transaction error before constructing VersionedTransaction, ensuring
external-signer instructions cannot produce malformed data.

In `@engine/tests/recovery.rs`:
- Around line 58-59: Wrap the TestEngine::with(dirs, authority).await call in
the replay_aborts_on_checksum_mismatch test with the same timeout mechanism used
at Lines 90-95, preserving the existing reopen behavior while ensuring hangs
fail fast. Keep the timeout comment aligned with the implementation.

🪄 Autofix

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: 0b5637b3-1416-4d7a-ae44-c9aebbf23379

📥 Commits

Reviewing files that changed from the base of the PR and between 35c9834 and 5d3d9ee.

📒 Files selected for processing (15)

• Cargo.toml
• README.md
• engine/Cargo.toml
• engine/README.md
• engine/src/accessor.rs
• engine/src/error.rs
• engine/src/lib.rs
• engine/src/pacemaker.rs
• engine/src/testkit.rs
• engine/src/transaction.rs
• engine/tests/accounts.rs
• engine/tests/builtins.rs
• engine/tests/recovery.rs
• engine/tests/security.rs
• engine/tests/transactions.rs