Hadoop Common
  1. Hadoop Common
  2. HADOOP-6923

Native Libraries do not load if a different platform signature is returned from org.apache.hadoop.util.PlatformName

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.20.0, 0.20.1, 0.20.2
    • Fix Version/s: 0.20.3
    • Component/s: native
    • Labels:
      None
    • Environment:

      SLES 10, IBM Java 6, Apache Hadoop 0.20.x

    • Hadoop Flags:
      Reviewed

      Description

      The bin/hadoop script has an environment variable called JAVA_PLATFORM which is set to to the results returned by org.apache.hadoop.util.PlatformName . These results are sometimes unique to the JRE being used. Although the value returned for 64 Bit Sun/Oracle Java and 64 Bit IBM Java is the same, it is different for the corresponding 32 Bit JREs.
      The issue is that the value returned is used in creating the path to the native libraries on disk, i.e $

      {HADOOP_COMMON_HOME}

      /lib/native/$

      {JAVA_PLATFORM}

      Since the path on disk is fixed with the Sun JRE value /lib/native/Linux-i386-32 it therefore fails when it attempts to load the native libraries with the value returned with 32 Bit IBM Java, /lib/native/Linux-x86-32

      1. HADOOP-6923.patch
        0.6 kB
        Stephen Watt

        Issue Links

          Activity

          Stephen Watt created issue -
          Hide
          Stephen Watt added a comment -

          Since the native libraries work regardless of JRE signature, the patches checks for the IBM 32 bit JRE signature and if found sets the JAVA_PLATFORM environment variable to the same signature of the Oracle/Sun JRE

          Show
          Stephen Watt added a comment - Since the native libraries work regardless of JRE signature, the patches checks for the IBM 32 bit JRE signature and if found sets the JAVA_PLATFORM environment variable to the same signature of the Oracle/Sun JRE
          Stephen Watt made changes -
          Field Original Value New Value
          Attachment HADOOP-6923.patch [ 12452966 ]
          Stephen Watt made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Hide
          Greg Roelofs added a comment -

          Other than the reported version numbers, isn't this identical to HADOOP-6895?

          Show
          Greg Roelofs added a comment - Other than the reported version numbers, isn't this identical to HADOOP-6895 ?
          Hide
          Stephen Watt added a comment -

          Identical Problem, slightly different resolutions. HADOOP-6895 is for Hadoop 0.21 where the code has been moved out of the bin/hadoop script into the bin/hadoop-config.sh file, so the 6895 patch modifies the hadoop-config.sh file, whereas this patch (for 0.20.3) modifies the bin/hadoop file.

          Show
          Stephen Watt added a comment - Identical Problem, slightly different resolutions. HADOOP-6895 is for Hadoop 0.21 where the code has been moved out of the bin/hadoop script into the bin/hadoop-config.sh file, so the 6895 patch modifies the hadoop-config.sh file, whereas this patch (for 0.20.3) modifies the bin/hadoop file.
          Hide
          Greg Roelofs added a comment -

          Gotcha, thanks. Might be good to link the two issues, though.

          Show
          Greg Roelofs added a comment - Gotcha, thanks. Might be good to link the two issues, though.
          Stephen Watt made changes -
          Link This issue relates to HADOOP-6895 [ HADOOP-6895 ]
          Hide
          Stephen Watt added a comment -

          Hudson doesn't seem to have kicked off a QA Build. Cancelling Patch and moving it back to Patch Available to see if that will start the Build and Test.

          Show
          Stephen Watt added a comment - Hudson doesn't seem to have kicked off a QA Build. Cancelling Patch and moving it back to Patch Available to see if that will start the Build and Test.
          Stephen Watt made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Stephen Watt made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12452966/HADOOP-6923.patch
          against trunk revision 995285.

          +1 @author. The patch does not contain any @author tags.

          -1 tests included. The patch doesn't appear to include any new or modified tests.
          Please justify why no new tests are needed for this patch.
          Also please list what manual steps were performed to verify this patch.

          -1 patch. The patch command could not apply the patch.

          Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/679/console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12452966/HADOOP-6923.patch against trunk revision 995285. +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. -1 patch. The patch command could not apply the patch. Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/679/console This message is automatically generated.
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12452966/HADOOP-6923.patch
          against trunk revision 1031422.

          +1 @author. The patch does not contain any @author tags.

          -1 tests included. The patch doesn't appear to include any new or modified tests.
          Please justify why no new tests are needed for this patch.
          Also please list what manual steps were performed to verify this patch.

          -1 patch. The patch command could not apply the patch.

          Console output: https://hudson.apache.org/hudson/job/PreCommit-HADOOP-Build/22//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12452966/HADOOP-6923.patch against trunk revision 1031422. +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. -1 patch. The patch command could not apply the patch. Console output: https://hudson.apache.org/hudson/job/PreCommit-HADOOP-Build/22//console This message is automatically generated.
          Owen O'Malley made changes -
          Fix Version/s 0.20.3 [ 12314812 ]
          Hide
          Todd Lipcon added a comment -

          +1. Committed to branch-20 for 0.20.4

          Show
          Todd Lipcon added a comment - +1. Committed to branch-20 for 0.20.4
          Todd Lipcon made changes -
          Status Patch Available [ 10002 ] Resolved [ 5 ]
          Hadoop Flags [Reviewed]
          Fix Version/s 0.20.4 [ 12316036 ]
          Resolution Fixed [ 1 ]
          Todd Lipcon made changes -
          Assignee Stephen Watt [ wattsteve ]
          Hide
          Stephen Watt added a comment -

          @Owen (or @Todd) - Why is this going into 0.20.4 and not 0.20.3?

          Show
          Stephen Watt added a comment - @Owen (or @Todd) - Why is this going into 0.20.4 and not 0.20.3?
          Owen O'Malley made changes -
          Fix Version/s 0.20.3 [ 12314812 ]
          Fix Version/s 0.20.4 [ 12316036 ]
          Hide
          Owen O'Malley added a comment -

          It ended up going into 0.20.3 since I needed to fix another minor glitch in the tests.

          Show
          Owen O'Malley added a comment - It ended up going into 0.20.3 since I needed to fix another minor glitch in the tests.

            People

            • Assignee:
              Stephen Watt
              Reporter:
              Stephen Watt
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development