[NUTCH-1807] avoid methods relying on system-specific default locale / charset

• Type: Bug
• Status: Open
• Priority: Minor
• Resolution: Unresolved
• Affects Version/s: 1.8, 2.2.1
• Fix Version/s: 1.21
• Assignee: Unassigned
• Reporter: Sebastian Nagel
• Created: 29/Jun/14 20:26
• Updated: 30/Mar/24 17:19

Description

Many methods in Java (and libraries) used to convert Strings, Numbers, Dates rely on the system-specific default locale / character set. This may cause strange behaviour and errors impossible to reproduce on other systems, see thetaphi's blog post, and discussions in NUTCH-1693 and NUTCH-1554.

A search with the forbidden-apis client shows 120 calls of such methods in trunk (without test classes):

# compile Nutch before check: all tested class files
# are then located in build/ (including plugins)
% CLASSPATH=`find build/ -name '*.jar' | tr '\n' ':'`
% java -jar forbiddenapis-1.5.1.jar -d build/ -c $CLASSPATH \
      -b jdk-unsafe-1.8 -b commons-io-unsafe-2.4

It is also possible to integrate the check into the ant build (to avoid that "forbidden" calls slip into the code again).

Issue Links

• relates to: NUTCH-2254 Charset issues when using -addBinaryContent and -base64 options
• relates to: NUTCH-2815 Add Spotbugs target to build and address detected "bugs"
• requires: NUTCH-2264 Check Forbidden APIs at Build