Skip to content

feat(snapshot): supply/value conservation invariants on import (#125 follow-up)#126

Merged
On1x merged 2 commits into
masterfrom
feat/snapshot-supply-invariant
Jul 14, 2026
Merged

feat(snapshot): supply/value conservation invariants on import (#125 follow-up)#126
On1x merged 2 commits into
masterfrom
feat/snapshot-supply-invariant

Conversation

@chiliec

@chiliec chiliec commented Jul 14, 2026

Copy link
Copy Markdown
Member

Follow-up to #125. Adds the deferred export-side completeness detector for snapshot import, and fixes a stale field that blocked the consensus-sim harness build.

Why

The #125 incident class is export-side incompleteness: a serving node serializes state with an account missing, the download matches its checksum, count-reconciliation passes (the recorded counts derive from the same short arrays), import succeeds, and the node wedges the moment a canonical block references the absent account (out_of_range "viz-social-bot"). Neither the checksum nor the count check can see this. A value invariant can: a missing balance/stake holder drops the summed total below the independently-tracked dgp supply.

What

plugins/snapshot/plugin.cpp — two post-import checks in load_snapshot():

  • SHARES: dgp.total_vesting_shares == Σ account.vesting_shares. Delegation objects only redistribute already-counted vests → excluded.
  • TOKEN: dgp.current_supply == Σ all liquid/locked TOKEN pools (account balance + reserved, escrow balance + pending_fee, invite balance, validator pending reward, vesting/reward/committee funds).

Both are strict equalities that raise the existing retryable FC_ASSERT on mismatch (→ retry another trusted peer), consistent with the other completeness checks. Per-component subtotals are logged unconditionally so any future mismatch is attributable to a specific pool.

tests/consensus_sim/harness/simulated_node.cpp: read signed_block::validator instead of the stale ->witness (Steem-ism). The harness had never compiled against current VIZ headers; this unblocks the suite so the is_wedged test from #125 can run.

Verification

Built on a real Linux/Boost env (Docker, Dockerfile-production). Ran the new vizd against six real mainnet snapshots (blocks 81334800–81620400): both invariants reconcile delta=0 to the satoshi at every height, including the height-varying validator pending-reward term. The wedge_predicate unit test (from #125) also builds and passes 8/8 with the harness fix.

Note: with the harness building, the full consensus suite surfaces pre-existing ASan/UBSan findings in the chain/serialization layer (unrelated to this PR; CI does not build BUILD_CONSENSUS_TESTS). Tracked separately.

chiliec added 2 commits July 14, 2026 14:42
simulated_node::recent_blocks read `b->witness` from a signed_block, a
leftover Steem/Golos field name. VIZ's block_header exposes the producer
as `validator` (block_header.hpp:18), so the consensus_sim harness failed
to compile against current headers — the suite had never been built in a
real environment (CI does not build BUILD_CONSENSUS_TESTS, local builds
are blocked by Boost 1.90). Fixing this unblocks the harness build so the
is_wedged truth-table test (added in #125) can actually run.
Adds two post-import checks to load_snapshot(), the deferred follow-up
from #125 (the incident class: an export-side incomplete snapshot with a
missing account passes the checksum and count reconciliation, then wedges
the node when a canonical block references the absent account).

- SHARES: dgp.total_vesting_shares == sum of every account's own
  vesting_shares. Delegation objects redistribute already-counted vests
  and are intentionally excluded.
- TOKEN: dgp.current_supply == sum of every liquid/locked TOKEN pool
  (account balance + reserved, escrow balance + pending fee, invite
  balance, validator pending reward, vesting/reward/committee funds).

Both are strict equalities: an account missing from the export that held
stake or a balance drops the sum below the independently-tracked dgp
total, so a short export no longer reconciles. On mismatch they raise the
existing retryable FC_ASSERT (retry another trusted peer). Per-component
subtotals are logged unconditionally for diagnosis.

Verified satoshi-exact (delta=0) against six real mainnet snapshots
spanning blocks 81334800-81620400, including the height-varying validator
pending-reward term.

@On1x On1x left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approve. This lands the export-side value invariant deferred from #125 (review finding #4) — the real detector for the incident class.

Verified the accounting is complete, not just delta=0 on the six tested heights:

  • TOKEN pools — checked every asset(TOKEN_SYMBOL) field in the codebase. The sum covers all real pools: account.balance + reserved_balance, escrow.token_balance + pending_fee, invite.balance, validator.pending_stakeholder_reward, and the three dgp funds. The fields left out are correctly left out:
    • account.current_bid — the bid TOKEN is held in reserved_balance (database.cpp:4704-4705 reserved_balance -= current_bid; balance += current_bid); summing it would double-count.
    • committee_request.remain_payout_amount — the TOKEN stays in committee_fund until paid; payout decrements both together (database.cpp:3245/3250). It's a claim schedule, not a holding.
    • account_offer_price / last_bid / subaccount_offer_price — price/record fields, no TOKEN held.
    • paid_subscription.amount — a price (share_type), paid by direct transfer each period (database.cpp:3080), no escrow pool.
  • SHARESΣ account.vesting_shares == dgp.total_vesting_shares is correct: delegation doesn't mint vests, it only moves voting power (delegated_/received_vesting_shares are separate fields; vesting_shares is the account's own minted stake), so the delegation objects must be excluded to avoid double-counting. The satoshi-exact match on real mainnet snapshots (which have active delegations) confirms this.
  • Harness fixsigned_block::validator (not the Steem-ism ->witness) is correct for current VIZ headers; unblocks the is_wedged test from #125.

Both are strict, retryable FC_ASSERTs on the snapshot-import path only (not block processing), with per-component subtotal logging for attribution — good.

Forward-looking note (non-blocking): these sums are now a maintenance coupling — any future hardfork that introduces a new TOKEN/VESTS pool must be added here or valid snapshots will start failing the strict equality. Worth a comment near the pool list pointing back to the supply-accounting sites, and ideally folding the eventual database::validate_invariants() definition into the same source of truth.

@On1x On1x merged commit 4be2954 into master Jul 14, 2026
1 check passed
On1x added a commit that referenced this pull request Jul 14, 2026
…126)

Brings the wedged-behind-network watchdog (#125) and the snapshot
post-import completeness/supply invariants (#126) onto the PM branch.

The #125 checks (count reconciliation, referential integrity, singletons)
and the #126 SHARES conservation check are PM-agnostic and stay fatal.

The #126 TOKEN supply invariant is extended for PM: PM is zero-sum, so a
bet / liquidity / lazy-deposit / commit-escrow / dispute-fee / oracle-
insurance / leverage-collateral moves TOKEN out of account.balance into a
PM object field. Those pools are now summed (minimal non-overlapping set,
duplicates such as bets_sum/reserves, empty-provider lazy LP, and
deposit.principal excluded).

Because this PM accounting is not yet satoshi-validated against real PM
snapshots, the TOKEN check is LOG-ONLY on this branch (per-component delta
logged, import NOT failed) until reconciled to delta==0 and re-armed as a
fatal FC_ASSERT. Tracked in #127 with a TODO(pm) at the check.

Conflicts: tests/consensus_sim/CMakeLists.txt (kept both test_pm_lifecycle
and test_wedge_predicate scenarios).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants