Hadoop Common
  1. Hadoop Common
  2. HADOOP-8580

ant compile-native fails with automake version 1.11.3

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 1.2.0, 0.22.0
    • Fix Version/s: 1.2.0
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      The following:

      ant -d -v -DskipTests -Dcompile.native=true clean compile-native
      

      works with GNU automake version 1.11.1, but fails with automake version 1.11.3.

      Relevant lines of failure seem to be these:

      [exec] make[1]: Leaving directory `/tmp/hadoop-common/build/native/Linux-amd64-64'
           [exec] Current OS is Linux
           [exec] Executing 'sh' with arguments:
           [exec] '/tmp/hadoop-common/build/native/Linux-amd64-64/libtool'
           [exec] '--mode=install'
           [exec] 'cp'
           [exec] '/tmp/hadoop-common/build/native/Linux-amd64-64/libhadoop.la'
           [exec] '/tmp/hadoop-common/build/native/Linux-amd64-64/lib'
           [exec] 
           [exec] The ' characters around the executable and arguments are
           [exec] not part of the command.
           [exec] /tmp/hadoop-common/build/native/Linux-amd64-64/libtool: 3212: /tmp/hadoop-common/build/native/Linux-amd64-64/libtool: install_prog+=cp: not found
           [exec] /tmp/hadoop-common/build/native/Linux-amd64-64/libtool: 3232: /tmp/hadoop-common/build/native/Linux-amd64-64/libtool: files+= /tmp/hadoop-common/build/native/Linux-amd64-64/libhadoop.la: not found
           [exec] libtool: install: you must specify an install program
           [exec] libtool: install: Try `libtool --help --mode=install' for more information.
        [antcall] Exiting /tmp/hadoop-common/build.xml.
      
      BUILD FAILED
      

      Created transcript showing entire output of the above ant command for both automake 1.11.1 (successful) versus 1.11.3 (failed) here:

      https://gist.github.com/3078988

      1. hadoop-8580-branch-1.gs.patch
        0.6 kB
        Gera Shegalov
      2. HADOOP-8580-branch-1.patch
        0.5 kB
        Chuan Liu

        Activity

        Eugene Koontz created issue -
        Hide
        Eugene Koontz added a comment -

        Actually there are many other configure tools involved besides automake. From my /usr/local/ here are a set of working versions:

        https://gist.github.com/3079069

        and this is a corresponding set of failing versions from my /usr/:

        https://gist.github.com/3079113

        Show
        Eugene Koontz added a comment - Actually there are many other configure tools involved besides automake. From my /usr/local/ here are a set of working versions: https://gist.github.com/3079069 and this is a corresponding set of failing versions from my /usr/: https://gist.github.com/3079113
        Hide
        Chuan Liu added a comment -

        libtool is supposed to only work with 'bash' as its header defines:

        #! /bin/bash
        

        In Hadoop build.xml, we use 'sh' to execute 'libtool'. On many systems, 'sh' is linked to 'bash'. On Ubuntu, 'sh' is linked to 'dash'. Using 'dash' to run 'libtool' results this build error. The patch modifies build.xml to involve 'bash' directly.

        Show
        Chuan Liu added a comment - libtool is supposed to only work with 'bash' as its header defines: #! /bin/bash In Hadoop build.xml, we use 'sh' to execute 'libtool'. On many systems, 'sh' is linked to 'bash'. On Ubuntu, 'sh' is linked to 'dash'. Using 'dash' to run 'libtool' results this build error. The patch modifies build.xml to involve 'bash' directly.
        Chuan Liu made changes -
        Field Original Value New Value
        Attachment HADOOP-8580-branch-1.patch [ 12535942 ]
        Hide
        Gera Shegalov added a comment -

        libtool already references bash, so it can be used as exec, and no need to worry about the shell.

        Show
        Gera Shegalov added a comment - libtool already references bash, so it can be used as exec, and no need to worry about the shell.
        Gera Shegalov made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Affects Version/s 0.22.0 [ 12314296 ]
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12535942/HADOOP-8580-branch-1.patch
        against trunk revision .

        -1 patch. The patch command could not apply the patch.

        Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/1235//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12535942/HADOOP-8580-branch-1.patch against trunk revision . -1 patch. The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/1235//console This message is automatically generated.
        Hide
        Gera Shegalov added a comment -

        slightly different patch: exec libtool directly.

        Show
        Gera Shegalov added a comment - slightly different patch: exec libtool directly.
        Gera Shegalov made changes -
        Attachment hadoop-8580-branch-1.gs.patch [ 12538528 ]
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12538528/hadoop-8580-branch-1.gs.patch
        against trunk revision .

        -1 patch. The patch command could not apply the patch.

        Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/1236//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12538528/hadoop-8580-branch-1.gs.patch against trunk revision . -1 patch. The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/1236//console This message is automatically generated.
        Chuan Liu made changes -
        Status Patch Available [ 10002 ] Open [ 1 ]
        Hide
        Chuan Liu added a comment -

        Gera, I think this is a 1.0 problem. "Submit patch" is only used for trunk.

        Show
        Chuan Liu added a comment - Gera, I think this is a 1.0 problem. "Submit patch" is only used for trunk.
        Hide
        Gera Shegalov added a comment -

        Chuan, I would like to clarify whether you request a patch for the trunk.

        Show
        Gera Shegalov added a comment - Chuan, I would like to clarify whether you request a patch for the trunk.
        Hide
        Chuan Liu added a comment -

        No. I just want to point out this problem does not affect trunk because trunk has moved to use Maven. "Submit patch" triggered Hadoop QA to apply the patch to trunk, and hence all the -1's.

        Show
        Chuan Liu added a comment - No. I just want to point out this problem does not affect trunk because trunk has moved to use Maven. "Submit patch" triggered Hadoop QA to apply the patch to trunk, and hence all the -1's.
        Hide
        Arpit Agarwal added a comment -

        +1 (non-committer) for the second patch.

        Applying this patch allowed me to build native on Ubuntu 12.04. The workaround I had been using was to symlink /bin/sh to /bin/bash.

        Show
        Arpit Agarwal added a comment - +1 (non-committer) for the second patch. Applying this patch allowed me to build native on Ubuntu 12.04. The workaround I had been using was to symlink /bin/sh to /bin/bash.
        Hide
        Chuan Liu added a comment -

        +1 as non-committer as well.

        Show
        Chuan Liu added a comment - +1 as non-committer as well.
        Hide
        Giridharan Kesavan added a comment -

        patch looks good, verified on ubuntu and centos. +1

        Show
        Giridharan Kesavan added a comment - patch looks good, verified on ubuntu and centos. +1
        Hide
        Suresh Srinivas added a comment -

        I committed the patch to branch-1.

        Thank you Gera for the patch.

        Show
        Suresh Srinivas added a comment - I committed the patch to branch-1. Thank you Gera for the patch.
        Suresh Srinivas made changes -
        Status Open [ 1 ] Resolved [ 5 ]
        Hadoop Flags Reviewed [ 10343 ]
        Fix Version/s 1.2.0 [ 12321659 ]
        Resolution Fixed [ 1 ]
        Hide
        Matt Foley added a comment -

        Closed upon release of Hadoop 1.2.0.

        Show
        Matt Foley added a comment - Closed upon release of Hadoop 1.2.0.
        Matt Foley made changes -
        Status Resolved [ 5 ] Closed [ 6 ]

          People

          • Assignee:
            Unassigned
            Reporter:
            Eugene Koontz
          • Votes:
            0 Vote for this issue
            Watchers:
            10 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development