Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-8781

hadoop-config.sh should add JAVA_LIBRARY_PATH to LD_LIBRARY_PATH

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.0.2-alpha
    • Fix Version/s: 1.2.0, 2.0.2-alpha
    • Component/s: scripts
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      Snappy SO fails to load properly if LD_LIBRARY_PATH does not include the path where snappy SO is. This is observed in setups that don't have an independent snappy installation (not installed by Hadoop)

      1. HADOOP-8781.patch
        0.7 kB
        Alejandro Abdelnur
      2. HADOOP-8781.patch
        0.6 kB
        Alejandro Abdelnur
      3. HADOOP-8781-branch1.patch
        0.4 kB
        Alejandro Abdelnur
      4. HADOOP-8781-branch1.patch
        0.4 kB
        Alejandro Abdelnur

        Issue Links

          Activity

          Hide
          tucu00 Alejandro Abdelnur added a comment -

          tested in pseudo cluster

          Show
          tucu00 Alejandro Abdelnur added a comment - tested in pseudo cluster
          Hide
          tucu00 Alejandro Abdelnur added a comment -

          patches for trunk/branch-2 and branch-1.

          Show
          tucu00 Alejandro Abdelnur added a comment - patches for trunk/branch-2 and branch-1.
          Hide
          rvs Roman Shaposhnik added a comment -

          Looks good to me. One small nit – I'd add this line to an already existing if statement that tests for non-emptiness of JAVA_LIBRARY_PATH a few lines down.

          Show
          rvs Roman Shaposhnik added a comment - Looks good to me. One small nit – I'd add this line to an already existing if statement that tests for non-emptiness of JAVA_LIBRARY_PATH a few lines down.
          Hide
          tucu00 Alejandro Abdelnur added a comment -

          with Roman's corrections

          Show
          tucu00 Alejandro Abdelnur added a comment - with Roman's corrections
          Hide
          hadoopqa Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12544545/HADOOP-8781.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          -1 tests included. The patch doesn't appear to include any new or modified tests.
          Please justify why no new tests are needed for this patch.
          Also please list what manual steps were performed to verify this patch.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed unit tests in hadoop-common-project/hadoop-common.

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/1431//testReport/
          Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/1431//console

          This message is automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12544545/HADOOP-8781.patch against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 javadoc. The javadoc tool did not generate any warning messages. +1 eclipse:eclipse. The patch built with eclipse:eclipse. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in hadoop-common-project/hadoop-common. +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/1431//testReport/ Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/1431//console This message is automatically generated.
          Hide
          ahmed.radwan Ahmed Radwan added a comment -

          lgtm +1

          Show
          ahmed.radwan Ahmed Radwan added a comment - lgtm +1
          Hide
          rvs Roman Shaposhnik added a comment -

          +1

          Show
          rvs Roman Shaposhnik added a comment - +1
          Hide
          tlipcon Todd Lipcon added a comment -

          +1, lgtm

          Show
          tlipcon Todd Lipcon added a comment - +1, lgtm
          Hide
          tucu00 Alejandro Abdelnur added a comment -

          committed to trunk, branch-1 and branch-2.

          Show
          tucu00 Alejandro Abdelnur added a comment - committed to trunk, branch-1 and branch-2.
          Hide
          hudson Hudson added a comment -

          Integrated in Hadoop-Mapreduce-trunk-Commit #2738 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/2738/)
          HADOOP-8781. hadoop-config.sh should add JAVA_LIBRARY_PATH to LD_LIBRARY_PATH. (tucu) (Revision 1383142)

          Result = FAILURE
          tucu : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1383142
          Files :

          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/bin/hadoop-config.sh
          Show
          hudson Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk-Commit #2738 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/2738/ ) HADOOP-8781 . hadoop-config.sh should add JAVA_LIBRARY_PATH to LD_LIBRARY_PATH. (tucu) (Revision 1383142) Result = FAILURE tucu : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1383142 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/bin/hadoop-config.sh
          Hide
          hudson Hudson added a comment -

          Integrated in Hadoop-Common-trunk-Commit #2714 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/2714/)
          HADOOP-8781. hadoop-config.sh should add JAVA_LIBRARY_PATH to LD_LIBRARY_PATH. (tucu) (Revision 1383142)

          Result = SUCCESS
          tucu : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1383142
          Files :

          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/bin/hadoop-config.sh
          Show
          hudson Hudson added a comment - Integrated in Hadoop-Common-trunk-Commit #2714 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/2714/ ) HADOOP-8781 . hadoop-config.sh should add JAVA_LIBRARY_PATH to LD_LIBRARY_PATH. (tucu) (Revision 1383142) Result = SUCCESS tucu : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1383142 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/bin/hadoop-config.sh
          Hide
          hudson Hudson added a comment -

          Integrated in Hadoop-Hdfs-trunk-Commit #2777 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/2777/)
          HADOOP-8781. hadoop-config.sh should add JAVA_LIBRARY_PATH to LD_LIBRARY_PATH. (tucu) (Revision 1383142)

          Result = SUCCESS
          tucu : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1383142
          Files :

          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/bin/hadoop-config.sh
          Show
          hudson Hudson added a comment - Integrated in Hadoop-Hdfs-trunk-Commit #2777 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/2777/ ) HADOOP-8781 . hadoop-config.sh should add JAVA_LIBRARY_PATH to LD_LIBRARY_PATH. (tucu) (Revision 1383142) Result = SUCCESS tucu : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1383142 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/bin/hadoop-config.sh
          Hide
          cmccabe Colin P. McCabe added a comment -

          thanks for fixing this, tucu. Unfortunately the patch has a bug if your LD_LIBRARY_PATH has spaces in it.

          It should be

          +  export LD_LIBRARY_PATH="$LD_LIBRARY_PATH:$JAVA_LIBRARY_PATH"
          

          rather than

          +  export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$JAVA_LIBRARY_PATH
          

          follow-up change?

          Show
          cmccabe Colin P. McCabe added a comment - thanks for fixing this, tucu. Unfortunately the patch has a bug if your LD_LIBRARY_PATH has spaces in it. It should be + export LD_LIBRARY_PATH= "$LD_LIBRARY_PATH:$JAVA_LIBRARY_PATH" rather than + export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$JAVA_LIBRARY_PATH follow-up change?
          Hide
          hudson Hudson added a comment -

          Integrated in Hadoop-Hdfs-trunk #1162 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1162/)
          HADOOP-8781. hadoop-config.sh should add JAVA_LIBRARY_PATH to LD_LIBRARY_PATH. (tucu) (Revision 1383142)

          Result = FAILURE
          tucu : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1383142
          Files :

          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/bin/hadoop-config.sh
          Show
          hudson Hudson added a comment - Integrated in Hadoop-Hdfs-trunk #1162 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1162/ ) HADOOP-8781 . hadoop-config.sh should add JAVA_LIBRARY_PATH to LD_LIBRARY_PATH. (tucu) (Revision 1383142) Result = FAILURE tucu : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1383142 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/bin/hadoop-config.sh
          Hide
          hudson Hudson added a comment -

          Integrated in Hadoop-Mapreduce-trunk #1193 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1193/)
          HADOOP-8781. hadoop-config.sh should add JAVA_LIBRARY_PATH to LD_LIBRARY_PATH. (tucu) (Revision 1383142)

          Result = SUCCESS
          tucu : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1383142
          Files :

          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/bin/hadoop-config.sh
          Show
          hudson Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk #1193 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1193/ ) HADOOP-8781 . hadoop-config.sh should add JAVA_LIBRARY_PATH to LD_LIBRARY_PATH. (tucu) (Revision 1383142) Result = SUCCESS tucu : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1383142 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/bin/hadoop-config.sh
          Hide
          aw Allen Wittenauer added a comment -

          Be aware that this change will likely have side-effects for non-Java code.

          Show
          aw Allen Wittenauer added a comment - Be aware that this change will likely have side-effects for non-Java code.
          Hide
          aw Allen Wittenauer added a comment -

          (and yes, I'd -1 this if anyone outside your hallway was given a chance to review stuff before it got committed.)

          Show
          aw Allen Wittenauer added a comment - (and yes, I'd -1 this if anyone outside your hallway was given a chance to review stuff before it got committed.)
          Hide
          tlipcon Todd Lipcon added a comment -

          Be aware that this change will likely have side-effects for non-Java code.

          Maybe you can elaborate what the negative side effects would be?

          Show
          tlipcon Todd Lipcon added a comment - Be aware that this change will likely have side-effects for non-Java code. Maybe you can elaborate what the negative side effects would be?

            People

            • Assignee:
              tucu00 Alejandro Abdelnur
              Reporter:
              tucu00 Alejandro Abdelnur
            • Votes:
              0 Vote for this issue
              Watchers:
              9 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development