[OAK-7279] segment-tar update from java 7 to java 8 may break persisted names using invalid characters

• Type: Improvement
• Status: Open
• Priority: Minor
• Resolution: Unresolved
• Component/s: segment-tar
• Labels: tech-debt
• Assignee: Unassigned
• Reporter: Julian Reschke
• Created: 22/Feb/18 15:29
• Updated: 10/Dec/18 09:23

Description

segment-tar relies on String.getBytes() when persisting strings such as item names.

The problem is that the behavior for this has been changed in Java 8 with respect to invalid strings (here: null characters and unpaired surrogates).

In Java 7, these would roundtrip, as Java was using the so-called "modified UTF-8" encoding (see https://docs.oracle.com/javase/6/docs/api/java/io/DataInput.html#modified-utf-8). This will produce byte sequence that are not valid UTF-8.

Java 7 will read them back, but Java 8 will map the non-conforming byte sequences to the Unicode replacement character. Note that in particular, multiple child entries might get identical names as a consequence.

I'm not sure about the severity of this, and whether something needs to be done about it. AFAIC, this is another good reason to reject invalid strings as early as possible in the stack.

cc mduerig

Issue Links

• relates to: OAK-5506 reject item names with unpaired surrogates early