WIP: Parallel Correctness#34779
Draft
def- wants to merge 4 commits into
Draft
Conversation
def-
force-pushed
the
pr-parallel-correctness
branch
from
6419e00 to
c453158
Compare
def-
force-pushed
the
pr-parallel-correctness
branch
from
c453158 to
3fc961f
Compare
`parse_oid` parsed OIDs as `i32` and reinterpreted them as `u32`, which rejected valid OIDs in `2147483648..=4294967295`. For example `'4294967295'::oid` errored even though `4294967295::oid` succeeded, and `COPY` into such a column failed with "invalid input syntax for type oid". PostgreSQL accepts the union of the `u32` and `i32` ranges (negatives are reinterpreted), so parse the full `u32` range directly and fall back to `i32` for negative inputs. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
def-
force-pushed
the
pr-parallel-correctness
branch
4 times, most recently
from
de0f65d to
c0fe080
Compare
def-
force-pushed
the
pr-parallel-correctness
branch
from
c0fe080 to
f41f5a7
Compare
Make correctness mode verify much more than quiesced base tables: - Create shadow objects per table (a default index plus SELECT *, count(*), and key-less projection materialized views, pinned to the quickstart cluster) and require them to agree with the table exactly at a single timestamp, covering dataflow maintenance, arrangements, and rehydration after restarts. DROP TABLE cascades to them. - Let reads race writes: tables track a versioned history of committed states, the verifying SELECT no longer takes the table lock and must match one state between the versions sampled before and after the read. Writes with unknown outcomes (connection died during COMMIT, statement failure on an autocommit session) fork the tracked state into both candidates until a quiesced verification resolves them. - Add a harness-managed unique key column (pw-key) so UPDATE and DELETE can use replayable random predicates over it instead of whole-table statements, and repeat rows within insert statements so the key-less shadow view holds true duplicates, making multiplicity bugs visible. - Verify the SUBSCRIBE change stream on the fetch threads (previously a no-op in correctness mode): per-timestamp batches finalized by progress messages must walk monotonically through the tracked versions and multiplicities must never go negative. - Compare at full precision (exact Decimal for numerics, float4 via 32-bit collapse, microseconds kept) and re-enable date, timestamp, and interval columns by reading them back as text, which also fixes timestamptz tables never being verified (years past 9999 made every read hit the client-side parse-failure skip). - Always assign the read and fetch action lists to the first two threads in correctness mode so the verifiers exist for every seed. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
def-
force-pushed
the
pr-parallel-correctness
branch
from
50f9c1f to
4789844
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Status: Blocked by:
SUBSCRIBEwrong results: stale rows in change streamthread 'coordinator' panicked at src/adapter/src/util.rs:263:23: dataflow creation cannot fail: CollectionMissing(User(628))