[JENA-2186] Write U+FFFD as Unicode escape

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Done
• Affects Version/s: Jena 4.2.0
• Fix Version/s: Jena 4.3.0
• Assignee: Andy Seaborne
• Reporter: Andy Seaborne
• Created: 21/Oct/21 19:26
• Updated: 09/Dec/21 10:21
• Resolved: 25/Oct/21 09:23

Description

U+FFFD (Unicode replacement character) arises when there is an encoding mismatch between the input bytes and UTF-8 (see the wikipedia article).

The tokenizer for Turtle/N-Triple etc raises a warning when a literal U+FFFD is encountered to notify users/applications of potential problems.

The tokenizer does not warn if it is written intentionally in the input stream as \uFFFD (6 characters).

The write should this unicode escape form so charcater FFFD is written and read in again without warning.

 

Issue Links

• is related to: JENA-2179 TDB throws Unicode Replacement Character exception while fetching data
• links to: GitHub Pull Request #1090
• links to: GitHub Pull Request #1093