Fix the preview of text files containing BOM for utf8#39669
Merged
jvillafanez merged 3 commits intomasterfrom Feb 15, 2022
Merged
Fix the preview of text files containing BOM for utf8#39669jvillafanez merged 3 commits intomasterfrom
jvillafanez merged 3 commits intomasterfrom
Conversation
Adjusted range so the BOM isn't detected as an arabic char. Latin is also added as part of possible fonts, not just if there isn't any match
|
Thanks for opening this pull request! The maintainers of this repository would appreciate it if you would create a changelog item based on your changes. |
jvillafanez
commented
Jan 10, 2022
| ['range' => [0xfb1d, 0xfb4f], 'script' => 'Hebrew'], // some unicode chars aren't assigned | ||
| ['range' => [0xfb50, 0xfdff], 'script' => 'Arabic'], | ||
| ['range' => [0xfe70, 0xfeff], 'script' => 'Arabic'], | ||
| ['range' => [0xfe70, 0xfefc], 'script' => 'Arabic'], |
Member
Author
There was a problem hiding this comment.
For reference: https://en.wikipedia.org/wiki/Arabic_Presentation_Forms-B
11 tasks
|
We should check manually if the code now works with all BOMs |
Member
Author
|
Added unit tests to verify the text analyzer process the BOM correctly. |
AlexAndBear
approved these changes
Jan 14, 2022
AlexAndBear
reviewed
Jan 14, 2022
jvillafanez
force-pushed
the
fix_bom_preview
branch
from
79658a8 to
2e19041
Compare
|
Kudos, SonarCloud Quality Gate passed!
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adjusted range so the BOM isn't detected as an arabic char.
Latin is also added as part of possible fonts, not just if there isn't
any match
Description
BOM was detected as arabic char. Alternative fix to #39644
Related Issue
#39645
Motivation and Context
How Has This Been Tested?
Screenshots (if appropriate):
Types of changes
Checklist: