Skip to content

Fix GB12/GB13 when the regional indicator count is already known - #175

Merged
Manishearth merged 2 commits into
unicode-rs:masterfrom
cometkim:fix-ri-counting
Jul 28, 2026
Merged

Fix GB12/GB13 when the regional indicator count is already known#175
Manishearth merged 2 commits into
unicode-rs:masterfrom
cometkim:fix-ri-counting

Conversation

@cometkim

Copy link
Copy Markdown
Contributor

A GraphemeCursor driven forward in chunks splits a flag in two:

let s = "a\u{1F1E6}\u{1F1E6}"; // 'a' followed by two regional indicators

// s.graphemes(true)                    -> ["a", "🇦🇦"] boundaries [1, 9]
// GraphemeCursor over the whole string -> boundaries [1, 9]
// GraphemeCursor fed in chunks         -> boundaries [1, 5, 9] <-- wrong

Reproduces on 1.13.3. Both extended and legacy mode, and every chunk size.

ris_count has two readers with incompatible expectations. next_boundary maintains it as "the number of RIS immediately preceding offset", which is what the field's doc comment says. handle_regional instead treats it as an accumulator for the backward walk it is about to perform. It seeds ris_count from the field and then increments it for each RIS it walks over.

That is correct when handle_regional is resuming a walk it started itself across an earlier chunk. It is wrong when the field was populated by forward iteration, because the codepoints it walks back over are the very ones already counted, so they are counted twice and an odd run reads as even.

Failing test first, then the fix commit follows.

@cometkim

Copy link
Copy Markdown
Contributor Author

I just noticed that the same issue was reported already in #174. This PR should fix it.

@Manishearth

Copy link
Copy Markdown
Member

Clippy failure unrelated

@Manishearth
Manishearth merged commit 5dfedef into unicode-rs:master Jul 28, 2026
3 of 4 checks passed
@cometkim
cometkim deleted the fix-ri-counting branch July 28, 2026 15:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

GraphemeCursor::next_boundary reports an incorrect boundary for Regional Indicator sequences across chunks

2 participants