Uploaded image for project: 'Apache Trafodion'
  1. Apache Trafodion
  2. TRAFODION-2528

Classpath spec for hadoop-hdfs-*.jar needs to be changed in Trafodion env

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: any
    • Fix Version/s: 2.1-incubating
    • Component/s: sql-general
    • Labels:
      None

      Description

      Certain HDP installations - eg HBase version 1.1.2.2.3.4.7-4 does not have the directory "/usr/hdp/current/hadoop-client/client/.
      Currently this is how the hadoop JARs are picked up for HDP.
      export HADOOP_JAR_FILES="/usr/hdp/current/hadoop-client/client/hadoop-hdfs-*.jar"

      SO in some installations this JAR isn't foudn and calls to HDFS interface fail with this kind of an error :

          • ERROR[8448] Unable to access Hbase interface. Call to ExpHbaseInterface::addToHFile returned error HBASE_ADD_TO_HFILE_ERROR(-713). Cause: java.io.IOException: No FileSystem for scheme: hdfs
            org.apache.hadoop.fs.FileSystem.getFileSystemClass(FileSystem.java:2644)
            org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2651)
            org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:92)
            org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2687)
            org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2669)
            org.apache.hadoop.fs.FileSystem.get(FileSystem.java:371)
            org.apache.hadoop.fs.FileSystem.get(FileSystem.java:170)
            org.trafodion.sql.HBulkLoadClient.doCreateHFile(HBulkLoadClient.java:182)
            org.trafodion.sql.HBulkLoadClient.addToHFile(HBulkLoadClient.java:226).
          • ERROR[1081] Loading of index TRAFODION.SEABASE.I failed unexpectedly.

      Need to change the path to the hadoop-hdfs.jar file to be :
      export HADOOP_JAR_FILES="/usr/hdp/current/hadoop-hdfs-client/hadoop-hdfs-*.jar

      This si related to changes made in this 2 JIRAs. At some point, the directory current/hadoop-client was dropped from the distribution.

      https://issues.apache.org/jira/browse/HDFS-9815
      https://issues.apache.org/jira/browse/HDFS-6200

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                sandhya Sandhya Sundaresan
                Reporter:
                sandhya Sandhya Sundaresan
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: