Skip to content

TiFlash may repeatedly reuse stale read-index cache after resolving locks #10985

Description

@windtalker

Bug Report

1. Minimal reproduce step (Required)

  1. Run an MPP / remote cop read against TiFlash where the target region has a secondary lock visible in TiFlash local LockCF.
  2. TiFlash gets a read index for the region and waits to that index successfully.
  3. During RegionTable::resolveLocksAndWriteRegion, TiFlash still finds the lock in local LockCF and returns a lock error to the caller.
  4. The lock resolver resolves the lock on the TiKV leader, but the resolve-lock raft log has not been applied by the TiFlash learner yet.
  5. The next read retry reuses the previous read-index cache for the same region / read ts, so TiFlash does not issue a fresh read-index request and does not wait to the newer resolve-lock raft log.
  6. TiFlash may see the same local lock again and repeat the retry loop.

2. What did you expect to see? (Required)

After TiFlash finds a local LockCF lock after a successful wait-index, the cached read index for that region should be invalidated. The next retry should fetch a fresh read index and wait until TiFlash has applied the raft log that resolves the lock, unless the lock can be bypassed by the caller.

3. What did you see instead (Required)

TiFlash can reuse the stale cached read index after reporting the lock. This may cause repeated remote cop retries / lock errors for the same region and read ts, because the retry keeps reading before the resolve-lock raft log is applied on the TiFlash learner.

4. What is your TiFlash version? (Required)

Development branch. The issue was found while investigating the read-index cache behavior on TiFlash master / release-8.5 materialized-view development branches.

Metadata

Metadata

Assignees

No one assigned

    Labels

    type/bugThe issue is confirmed as a bug.

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions