[SOLR-10966] Add workaround for Hadoop-Common 2.7.2 incompatibility with Java 9

• Type: Bug
• Status: Resolved
• Priority: Critical
• Resolution: Fixed
• Affects Version/s: 6.6
• Fix Version/s: 6.6.1, 6.7, 7.0
• Component/s: Hadoop Integration, hdfs
• Assignee: Uwe Schindler
• Reporter: Uwe Schindler
• Created: 28/Jun/17 14:43
• Updated: 08/Jun/19 15:22
• Resolved: 20/Aug/17 11:01

Description

I did some testing to work around HADOOP-14586 and found a temporary solution. All tests pass with Java 9 build 175 (HDFS, Hadoop Auth / Kerberos).

This is a temporary workaround until we can upgrade Hadoop, see SOLR-10951

The trick here is a hack: The Hadoop Shell class tries to parse java.version system property, which is simply "9" on the Java 9 GA / release candidate. It contains no dots and is shorter than 3 characters. Hadoop tries to get the substring(0,3) and fails with an IndexOutOfBoundsException in clinit. To work around this, we do the following on early Solr startup / test startup (in a static analyzer, like we do for logging initialization):

• set java.version system property to "1.9"
• initialize the Shell class in Hadoop
• restore the old value of java.version

The whole thing is done in a doPrivileged. I ran some tests on Policeman Jenkins, everything works. The hack is only done, if we detect Java 9.

Issue Links

• is broken by: HADOOP-14586 StringIndexOutOfBoundsException breaks org.apache.hadoop.util.Shell on 2.7.x with Java 9
• is superceded by: SOLR-11261 Update to Hadoop 2.7.4
• is superceded by: SOLR-10951 Update Hadoop dependencies to 2.8.1, so Solr works with Java 9