feat(snapshot): supply/value conservation invariants on import (#125 follow-up)#126
Conversation
simulated_node::recent_blocks read `b->witness` from a signed_block, a leftover Steem/Golos field name. VIZ's block_header exposes the producer as `validator` (block_header.hpp:18), so the consensus_sim harness failed to compile against current headers — the suite had never been built in a real environment (CI does not build BUILD_CONSENSUS_TESTS, local builds are blocked by Boost 1.90). Fixing this unblocks the harness build so the is_wedged truth-table test (added in #125) can actually run.
Adds two post-import checks to load_snapshot(), the deferred follow-up from #125 (the incident class: an export-side incomplete snapshot with a missing account passes the checksum and count reconciliation, then wedges the node when a canonical block references the absent account). - SHARES: dgp.total_vesting_shares == sum of every account's own vesting_shares. Delegation objects redistribute already-counted vests and are intentionally excluded. - TOKEN: dgp.current_supply == sum of every liquid/locked TOKEN pool (account balance + reserved, escrow balance + pending fee, invite balance, validator pending reward, vesting/reward/committee funds). Both are strict equalities: an account missing from the export that held stake or a balance drops the sum below the independently-tracked dgp total, so a short export no longer reconciles. On mismatch they raise the existing retryable FC_ASSERT (retry another trusted peer). Per-component subtotals are logged unconditionally for diagnosis. Verified satoshi-exact (delta=0) against six real mainnet snapshots spanning blocks 81334800-81620400, including the height-varying validator pending-reward term.
On1x
left a comment
There was a problem hiding this comment.
Approve. This lands the export-side value invariant deferred from #125 (review finding #4) — the real detector for the incident class.
Verified the accounting is complete, not just delta=0 on the six tested heights:
- TOKEN pools — checked every
asset(TOKEN_SYMBOL)field in the codebase. The sum covers all real pools:account.balance+reserved_balance,escrow.token_balance+pending_fee,invite.balance,validator.pending_stakeholder_reward, and the three dgp funds. The fields left out are correctly left out:account.current_bid— the bid TOKEN is held inreserved_balance(database.cpp:4704-4705reserved_balance -= current_bid; balance += current_bid); summing it would double-count.committee_request.remain_payout_amount— the TOKEN stays incommittee_funduntil paid; payout decrements both together (database.cpp:3245/3250). It's a claim schedule, not a holding.account_offer_price/last_bid/subaccount_offer_price— price/record fields, no TOKEN held.paid_subscription.amount— a price (share_type), paid by direct transfer each period (database.cpp:3080), no escrow pool.
- SHARES —
Σ account.vesting_shares == dgp.total_vesting_sharesis correct: delegation doesn't mint vests, it only moves voting power (delegated_/received_vesting_sharesare separate fields;vesting_sharesis the account's own minted stake), so the delegation objects must be excluded to avoid double-counting. The satoshi-exact match on real mainnet snapshots (which have active delegations) confirms this. - Harness fix —
signed_block::validator(not the Steem-ism->witness) is correct for current VIZ headers; unblocks theis_wedgedtest from #125.
Both are strict, retryable FC_ASSERTs on the snapshot-import path only (not block processing), with per-component subtotal logging for attribution — good.
Forward-looking note (non-blocking): these sums are now a maintenance coupling — any future hardfork that introduces a new TOKEN/VESTS pool must be added here or valid snapshots will start failing the strict equality. Worth a comment near the pool list pointing back to the supply-accounting sites, and ideally folding the eventual database::validate_invariants() definition into the same source of truth.
…126) Brings the wedged-behind-network watchdog (#125) and the snapshot post-import completeness/supply invariants (#126) onto the PM branch. The #125 checks (count reconciliation, referential integrity, singletons) and the #126 SHARES conservation check are PM-agnostic and stay fatal. The #126 TOKEN supply invariant is extended for PM: PM is zero-sum, so a bet / liquidity / lazy-deposit / commit-escrow / dispute-fee / oracle- insurance / leverage-collateral moves TOKEN out of account.balance into a PM object field. Those pools are now summed (minimal non-overlapping set, duplicates such as bets_sum/reserves, empty-provider lazy LP, and deposit.principal excluded). Because this PM accounting is not yet satoshi-validated against real PM snapshots, the TOKEN check is LOG-ONLY on this branch (per-component delta logged, import NOT failed) until reconciled to delta==0 and re-armed as a fatal FC_ASSERT. Tracked in #127 with a TODO(pm) at the check. Conflicts: tests/consensus_sim/CMakeLists.txt (kept both test_pm_lifecycle and test_wedge_predicate scenarios).
Follow-up to #125. Adds the deferred export-side completeness detector for snapshot import, and fixes a stale field that blocked the consensus-sim harness build.
Why
The #125 incident class is export-side incompleteness: a serving node serializes state with an account missing, the download matches its checksum, count-reconciliation passes (the recorded counts derive from the same short arrays), import succeeds, and the node wedges the moment a canonical block references the absent account (
out_of_range "viz-social-bot"). Neither the checksum nor the count check can see this. A value invariant can: a missing balance/stake holder drops the summed total below the independently-tracked dgp supply.What
plugins/snapshot/plugin.cpp— two post-import checks inload_snapshot():dgp.total_vesting_shares == Σ account.vesting_shares. Delegation objects only redistribute already-counted vests → excluded.dgp.current_supply == Σall liquid/locked TOKEN pools (account balance + reserved, escrow balance + pending_fee, invite balance, validator pending reward, vesting/reward/committee funds).Both are strict equalities that raise the existing retryable
FC_ASSERTon mismatch (→ retry another trusted peer), consistent with the other completeness checks. Per-component subtotals are logged unconditionally so any future mismatch is attributable to a specific pool.tests/consensus_sim/harness/simulated_node.cpp: readsigned_block::validatorinstead of the stale->witness(Steem-ism). The harness had never compiled against current VIZ headers; this unblocks the suite so theis_wedgedtest from #125 can run.Verification
Built on a real Linux/Boost env (Docker,
Dockerfile-production). Ran the new vizd against six real mainnet snapshots (blocks 81334800–81620400): both invariants reconcile delta=0 to the satoshi at every height, including the height-varying validator pending-reward term. Thewedge_predicateunit test (from #125) also builds and passes 8/8 with the harness fix.