[TRAFODION-2528] Classpath spec for hadoop-hdfs-*.jar needs to be changed in Trafodion env

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: any
• Fix Version/s: 2.1-incubating
• Component/s: sql-general
• Assignee: Sandhya Sundaresan
• Reporter: Sandhya Sundaresan
• Created: 09/Mar/17 08:14
• Updated: 05/Sep/18 21:33
• Resolved: 10/Mar/17 02:29

Description

Certain HDP installations - eg HBase version 1.1.2.2.3.4.7-4 does not have the directory "/usr/hdp/current/hadoop-client/client/.
Currently this is how the hadoop JARs are picked up for HDP.
export HADOOP_JAR_FILES="/usr/hdp/current/hadoop-client/client/hadoop-hdfs-*.jar"

SO in some installations this JAR isn't foudn and calls to HDFS interface fail with this kind of an error :

• • • ERROR[8448] Unable to access Hbase interface. Call to ExpHbaseInterface::addToHFile returned error HBASE_ADD_TO_HFILE_ERROR(-713). Cause: java.io.IOException: No FileSystem for scheme: hdfs
      org.apache.hadoop.fs.FileSystem.getFileSystemClass(FileSystem.java:2644)
      org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2651)
      org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:92)
      org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2687)
      org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2669)
      org.apache.hadoop.fs.FileSystem.get(FileSystem.java:371)
      org.apache.hadoop.fs.FileSystem.get(FileSystem.java:170)
      org.trafodion.sql.HBulkLoadClient.doCreateHFile(HBulkLoadClient.java:182)
      org.trafodion.sql.HBulkLoadClient.addToHFile(HBulkLoadClient.java:226).

• • • ERROR[1081] Loading of index TRAFODION.SEABASE.I failed unexpectedly.

Need to change the path to the hadoop-hdfs.jar file to be :
export HADOOP_JAR_FILES="/usr/hdp/current/hadoop-hdfs-client/hadoop-hdfs-*.jar

This si related to changes made in this 2 JIRAs. At some point, the directory current/hadoop-client was dropped from the distribution.

https://issues.apache.org/jira/browse/HDFS-9815
https://issues.apache.org/jira/browse/HDFS-6200

Issue Links

• links to: GitHub Pull Request #1002