Skip to content

Fix Unicode Tokens Loading - #3886

Merged
domenukk merged 2 commits into
mainfrom
tokens
Aug 11, 2026
Merged

Fix Unicode Tokens Loading#3886
domenukk merged 2 commits into
mainfrom
tokens

Conversation

@domenukk

Copy link
Copy Markdown
Member

Description

Fixes #3884

Checklist

  • I have run ./scripts/precommit.sh and addressed all comments

@domenukk
domenukk merged commit 96b85b7 into main Aug 11, 2026
30 checks passed
@domenukk
domenukk deleted the tokens branch August 11, 2026 11:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Tokens::add_from_file rejects valid dictionary lines containing multi-byte UTF-8 characters

1 participant