Hadoop Common
  1. Hadoop Common
  2. HADOOP-8781

hadoop-config.sh should add JAVA_LIBRARY_PATH to LD_LIBRARY_PATH

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 2.0.2-alpha
    • Fix Version/s: 1.2.0, 2.0.2-alpha
    • Component/s: scripts
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      Snappy SO fails to load properly if LD_LIBRARY_PATH does not include the path where snappy SO is. This is observed in setups that don't have an independent snappy installation (not installed by Hadoop)

      1. HADOOP-8781-branch1.patch
        0.4 kB
        Alejandro Abdelnur
      2. HADOOP-8781-branch1.patch
        0.4 kB
        Alejandro Abdelnur
      3. HADOOP-8781.patch
        0.6 kB
        Alejandro Abdelnur
      4. HADOOP-8781.patch
        0.7 kB
        Alejandro Abdelnur

        Issue Links

          Activity

          Hide
          Alejandro Abdelnur added a comment -

          tested in pseudo cluster

          Show
          Alejandro Abdelnur added a comment - tested in pseudo cluster
          Hide
          Alejandro Abdelnur added a comment -

          patches for trunk/branch-2 and branch-1.

          Show
          Alejandro Abdelnur added a comment - patches for trunk/branch-2 and branch-1.
          Hide
          Roman Shaposhnik added a comment -

          Looks good to me. One small nit – I'd add this line to an already existing if statement that tests for non-emptiness of JAVA_LIBRARY_PATH a few lines down.

          Show
          Roman Shaposhnik added a comment - Looks good to me. One small nit – I'd add this line to an already existing if statement that tests for non-emptiness of JAVA_LIBRARY_PATH a few lines down.
          Hide
          Alejandro Abdelnur added a comment -

          with Roman's corrections

          Show
          Alejandro Abdelnur added a comment - with Roman's corrections
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12544545/HADOOP-8781.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          -1 tests included. The patch doesn't appear to include any new or modified tests.
          Please justify why no new tests are needed for this patch.
          Also please list what manual steps were performed to verify this patch.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed unit tests in hadoop-common-project/hadoop-common.

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/1431//testReport/
          Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/1431//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12544545/HADOOP-8781.patch against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 javadoc. The javadoc tool did not generate any warning messages. +1 eclipse:eclipse. The patch built with eclipse:eclipse. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in hadoop-common-project/hadoop-common. +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/1431//testReport/ Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/1431//console This message is automatically generated.
          Hide
          Ahmed Radwan added a comment -

          lgtm +1

          Show
          Ahmed Radwan added a comment - lgtm +1
          Hide
          Roman Shaposhnik added a comment -

          +1

          Show
          Roman Shaposhnik added a comment - +1
          Hide
          Todd Lipcon added a comment -

          +1, lgtm

          Show
          Todd Lipcon added a comment - +1, lgtm
          Hide
          Alejandro Abdelnur added a comment -

          committed to trunk, branch-1 and branch-2.

          Show
          Alejandro Abdelnur added a comment - committed to trunk, branch-1 and branch-2.
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Mapreduce-trunk-Commit #2738 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/2738/)
          HADOOP-8781. hadoop-config.sh should add JAVA_LIBRARY_PATH to LD_LIBRARY_PATH. (tucu) (Revision 1383142)

          Result = FAILURE
          tucu : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1383142
          Files :

          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/bin/hadoop-config.sh
          Show
          Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk-Commit #2738 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/2738/ ) HADOOP-8781 . hadoop-config.sh should add JAVA_LIBRARY_PATH to LD_LIBRARY_PATH. (tucu) (Revision 1383142) Result = FAILURE tucu : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1383142 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/bin/hadoop-config.sh
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Common-trunk-Commit #2714 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/2714/)
          HADOOP-8781. hadoop-config.sh should add JAVA_LIBRARY_PATH to LD_LIBRARY_PATH. (tucu) (Revision 1383142)

          Result = SUCCESS
          tucu : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1383142
          Files :

          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/bin/hadoop-config.sh
          Show
          Hudson added a comment - Integrated in Hadoop-Common-trunk-Commit #2714 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/2714/ ) HADOOP-8781 . hadoop-config.sh should add JAVA_LIBRARY_PATH to LD_LIBRARY_PATH. (tucu) (Revision 1383142) Result = SUCCESS tucu : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1383142 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/bin/hadoop-config.sh
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-trunk-Commit #2777 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/2777/)
          HADOOP-8781. hadoop-config.sh should add JAVA_LIBRARY_PATH to LD_LIBRARY_PATH. (tucu) (Revision 1383142)

          Result = SUCCESS
          tucu : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1383142
          Files :

          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/bin/hadoop-config.sh
          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-trunk-Commit #2777 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/2777/ ) HADOOP-8781 . hadoop-config.sh should add JAVA_LIBRARY_PATH to LD_LIBRARY_PATH. (tucu) (Revision 1383142) Result = SUCCESS tucu : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1383142 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/bin/hadoop-config.sh
          Hide
          Colin Patrick McCabe added a comment -

          thanks for fixing this, tucu. Unfortunately the patch has a bug if your LD_LIBRARY_PATH has spaces in it.

          It should be

          +  export LD_LIBRARY_PATH="$LD_LIBRARY_PATH:$JAVA_LIBRARY_PATH"
          

          rather than

          +  export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$JAVA_LIBRARY_PATH
          

          follow-up change?

          Show
          Colin Patrick McCabe added a comment - thanks for fixing this, tucu. Unfortunately the patch has a bug if your LD_LIBRARY_PATH has spaces in it. It should be + export LD_LIBRARY_PATH= "$LD_LIBRARY_PATH:$JAVA_LIBRARY_PATH" rather than + export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$JAVA_LIBRARY_PATH follow-up change?
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-trunk #1162 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1162/)
          HADOOP-8781. hadoop-config.sh should add JAVA_LIBRARY_PATH to LD_LIBRARY_PATH. (tucu) (Revision 1383142)

          Result = FAILURE
          tucu : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1383142
          Files :

          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/bin/hadoop-config.sh
          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-trunk #1162 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1162/ ) HADOOP-8781 . hadoop-config.sh should add JAVA_LIBRARY_PATH to LD_LIBRARY_PATH. (tucu) (Revision 1383142) Result = FAILURE tucu : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1383142 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/bin/hadoop-config.sh
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Mapreduce-trunk #1193 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1193/)
          HADOOP-8781. hadoop-config.sh should add JAVA_LIBRARY_PATH to LD_LIBRARY_PATH. (tucu) (Revision 1383142)

          Result = SUCCESS
          tucu : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1383142
          Files :

          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/bin/hadoop-config.sh
          Show
          Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk #1193 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1193/ ) HADOOP-8781 . hadoop-config.sh should add JAVA_LIBRARY_PATH to LD_LIBRARY_PATH. (tucu) (Revision 1383142) Result = SUCCESS tucu : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1383142 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/bin/hadoop-config.sh
          Hide
          Allen Wittenauer added a comment -

          Be aware that this change will likely have side-effects for non-Java code.

          Show
          Allen Wittenauer added a comment - Be aware that this change will likely have side-effects for non-Java code.
          Hide
          Allen Wittenauer added a comment -

          (and yes, I'd -1 this if anyone outside your hallway was given a chance to review stuff before it got committed.)

          Show
          Allen Wittenauer added a comment - (and yes, I'd -1 this if anyone outside your hallway was given a chance to review stuff before it got committed.)
          Hide
          Todd Lipcon added a comment -

          Be aware that this change will likely have side-effects for non-Java code.

          Maybe you can elaborate what the negative side effects would be?

          Show
          Todd Lipcon added a comment - Be aware that this change will likely have side-effects for non-Java code. Maybe you can elaborate what the negative side effects would be?

            People

            • Assignee:
              Alejandro Abdelnur
              Reporter:
              Alejandro Abdelnur
            • Votes:
              0 Vote for this issue
              Watchers:
              9 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development