Hadoop Common
  1. Hadoop Common
  2. HADOOP-7276

Hadoop native builds fail on ARM due to -m32

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.21.0
    • Fix Version/s: 0.23.0
    • Component/s: native
    • Labels:
      None
    • Environment:
    • Hadoop Flags:
      Reviewed

      Description

      The native build fails on machine targets where gcc does not support -m32. This is any target other than x86, SPARC, RS/6000, or PowerPC, such as ARM.

      $ ant -Dcompile.native=true
      ...
      [exec] make all-am
      [exec] make[1]: Entering directory
      `/home/trobinson/dev/hadoop-common/build/native/Linux-arm-32'
      [exec] /bin/bash ./libtool --tag=CC --mode=compile gcc
      -DHAVE_CONFIG_H -I. -I/home/trobinson/dev/hadoop-common/src/native
      -I/usr/lib/jvm/java-6-openjdk/include
      -I/usr/lib/jvm/java-6-openjdk/include/linux
      -I/home/trobinson/dev/hadoop-common/src/native/src
      -Isrc/org/apache/hadoop/io/compress/zlib
      -Isrc/org/apache/hadoop/security -Isrc/org/apache/hadoop/io/nativeio/
      -g -Wall -fPIC -O2 -m32 -g -O2 -MT ZlibCompressor.lo -MD -MP -MF
      .deps/ZlibCompressor.Tpo -c -o ZlibCompressor.lo `test -f
      'src/org/apache/hadoop/io/compress/zlib/ZlibCompressor.c' || echo
      '/home/trobinson/dev/hadoop-common/src/native/'`src/org/apache/hadoop/io/compress/zlib/ZlibCompressor.c
      [exec] libtool: compile: gcc -DHAVE_CONFIG_H -I.
      -I/home/trobinson/dev/hadoop-common/src/native
      -I/usr/lib/jvm/java-6-openjdk/include
      -I/usr/lib/jvm/java-6-openjdk/include/linux
      -I/home/trobinson/dev/hadoop-common/src/native/src
      -Isrc/org/apache/hadoop/io/compress/zlib
      -Isrc/org/apache/hadoop/security -Isrc/org/apache/hadoop/io/nativeio/
      -g -Wall -fPIC -O2 -m32 -g -O2 -MT ZlibCompressor.lo -MD -MP -MF
      .deps/ZlibCompressor.Tpo -c
      /home/trobinson/dev/hadoop-common/src/native/src/org/apache/hadoop/io/compress/zlib/ZlibCompressor.c
      -fPIC -DPIC -o .libs/ZlibCompressor.o
      [exec] make[1]: Leaving directory
      `/home/trobinson/dev/hadoop-common/build/native/Linux-arm-32'
      [exec] cc1: error: unrecognized command line option "-m32"
      [exec] make[1]: *** [ZlibCompressor.lo] Error 1
      [exec] make: *** [all] Error 2

        Issue Links

          Activity

          Hide
          Trevor Robinson added a comment -

          This patch introduces an AM_CONDITIONAL called SPECIFY_DATA_MODEL that is disabled when host_cpu starts with "arm" (and is easily extensible for other CPUs). -m$(JVM_DATA_MODEL) is only added to AM_CFLAGS and AM_LDFLAGS when SPECIFY_DATA_MODEL is true.

          Show
          Trevor Robinson added a comment - This patch introduces an AM_CONDITIONAL called SPECIFY_DATA_MODEL that is disabled when host_cpu starts with "arm" (and is easily extensible for other CPUs). -m$(JVM_DATA_MODEL) is only added to AM_CFLAGS and AM_LDFLAGS when SPECIFY_DATA_MODEL is true.
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12478844/hadoop-common-arm.patch
          against trunk revision 1102123.

          +1 @author. The patch does not contain any @author tags.

          -1 tests included. The patch doesn't appear to include any new or modified tests.
          Please justify why no new tests are needed for this patch.
          Also please list what manual steps were performed to verify this patch.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          -1 findbugs. The patch appears to introduce 1 new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed core unit tests.

          +1 contrib tests. The patch passed contrib unit tests.

          +1 system test framework. The patch passed system test framework compile.

          Test results: https://builds.apache.org/hudson/job/PreCommit-HADOOP-Build/439//testReport/
          Findbugs warnings: https://builds.apache.org/hudson/job/PreCommit-HADOOP-Build/439//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
          Console output: https://builds.apache.org/hudson/job/PreCommit-HADOOP-Build/439//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12478844/hadoop-common-arm.patch against trunk revision 1102123. +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. -1 findbugs. The patch appears to introduce 1 new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed core unit tests. +1 contrib tests. The patch passed contrib unit tests. +1 system test framework. The patch passed system test framework compile. Test results: https://builds.apache.org/hudson/job/PreCommit-HADOOP-Build/439//testReport/ Findbugs warnings: https://builds.apache.org/hudson/job/PreCommit-HADOOP-Build/439//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/hudson/job/PreCommit-HADOOP-Build/439//console This message is automatically generated.
          Hide
          Trevor Robinson added a comment -

          No tests included because this change just fixes a build failure. Manually verified that x86-64 builds unchanged (-m64 is properly specified) and that ARM now builds (-m32 is not specified).

          Findbugs issue seems to be a configuration issue unrelated to change. From console output:

          [exec] findbugs:
          [exec] [mkdir] Created dir: /grid/0/hudson/hudson-slave/workspace/PreCommit-HADOOP-Build/trunk/build/test/findbugs
          [exec] [findbugs] Executing findbugs from ant task
          [exec] [findbugs] Running FindBugs...
          [exec] [findbugs] The following classes needed for analysis were missing:
          [exec] [findbugs] com.sun.javadoc.Doclet
          [exec] [findbugs] com.sun.javadoc.DocErrorReporter
          [exec] [findbugs] com.sun.javadoc.AnnotationTypeDoc
          [exec] [findbugs] com.sun.javadoc.RootDoc
          [exec] [findbugs] com.sun.javadoc.MethodDoc
          [exec] [findbugs] com.sun.javadoc.Doc
          [exec] [findbugs] com.sun.javadoc.PackageDoc
          [exec] [findbugs] com.sun.javadoc.LanguageVersion
          [exec] [findbugs] com.sun.javadoc.AnnotationDesc
          [exec] [findbugs] com.sun.javadoc.ConstructorDoc
          [exec] [findbugs] com.sun.javadoc.FieldDoc
          [exec] [findbugs] com.sun.javadoc.ProgramElementDoc
          [exec] [findbugs] com.sun.javadoc.ClassDoc
          [exec] [findbugs] com.sun.tools.doclets.standard.Standard
          [exec] [findbugs] Warnings generated: 1
          [exec] [findbugs] Missing classes: 15
          [exec] [findbugs] Calculating exit code...
          [exec] [findbugs] Setting 'missing class' flag (2)
          [exec] [findbugs] Setting 'bugs found' flag (1)
          [exec] [findbugs] Exit code set to: 3
          [exec] [findbugs] Classes needed for analysis were missing
          [exec] [findbugs] Output saved to /grid/0/hudson/hudson-slave/workspace/PreCommit-HADOOP-Build/trunk/build/test/findbugs/hadoop-findbugs-report.xml
          [exec] [findbugs] Java Result: 3

          Also, the precommit build queue (https://builds.apache.org/hudson/job/PreCommit-HADOOP-Build/) seems to be hanging multiple recent jobs at "Recording test results". They're still running at >1hr.

          Would a committer please review the change and let me know if I need to resubmit it?

          Show
          Trevor Robinson added a comment - No tests included because this change just fixes a build failure. Manually verified that x86-64 builds unchanged (-m64 is properly specified) and that ARM now builds (-m32 is not specified). Findbugs issue seems to be a configuration issue unrelated to change. From console output: [exec] findbugs: [exec] [mkdir] Created dir: /grid/0/hudson/hudson-slave/workspace/PreCommit-HADOOP-Build/trunk/build/test/findbugs [exec] [findbugs] Executing findbugs from ant task [exec] [findbugs] Running FindBugs... [exec] [findbugs] The following classes needed for analysis were missing: [exec] [findbugs] com.sun.javadoc.Doclet [exec] [findbugs] com.sun.javadoc.DocErrorReporter [exec] [findbugs] com.sun.javadoc.AnnotationTypeDoc [exec] [findbugs] com.sun.javadoc.RootDoc [exec] [findbugs] com.sun.javadoc.MethodDoc [exec] [findbugs] com.sun.javadoc.Doc [exec] [findbugs] com.sun.javadoc.PackageDoc [exec] [findbugs] com.sun.javadoc.LanguageVersion [exec] [findbugs] com.sun.javadoc.AnnotationDesc [exec] [findbugs] com.sun.javadoc.ConstructorDoc [exec] [findbugs] com.sun.javadoc.FieldDoc [exec] [findbugs] com.sun.javadoc.ProgramElementDoc [exec] [findbugs] com.sun.javadoc.ClassDoc [exec] [findbugs] com.sun.tools.doclets.standard.Standard [exec] [findbugs] Warnings generated: 1 [exec] [findbugs] Missing classes: 15 [exec] [findbugs] Calculating exit code... [exec] [findbugs] Setting 'missing class' flag (2) [exec] [findbugs] Setting 'bugs found' flag (1) [exec] [findbugs] Exit code set to: 3 [exec] [findbugs] Classes needed for analysis were missing [exec] [findbugs] Output saved to /grid/0/hudson/hudson-slave/workspace/PreCommit-HADOOP-Build/trunk/build/test/findbugs/hadoop-findbugs-report.xml [exec] [findbugs] Java Result: 3 Also, the precommit build queue ( https://builds.apache.org/hudson/job/PreCommit-HADOOP-Build/ ) seems to be hanging multiple recent jobs at "Recording test results". They're still running at >1hr. Would a committer please review the change and let me know if I need to resubmit it?
          Hide
          Eli Collins added a comment -

          +1 looks good. I also verified the x64 native build still works.

          Show
          Eli Collins added a comment - +1 looks good. I also verified the x64 native build still works.
          Hide
          Eli Collins added a comment -

          I don't see the findbugs warning locally. Don't think a test is needed since this to get the code compiling.

               [exec] 
               [exec] -1 overall.  
               [exec] 
               [exec]     +1 @author.  The patch does not contain any @author tags.
               [exec] 
               [exec]     -1 tests included.  The patch doesn't appear to include any new or modified tests.
               [exec]                         Please justify why no new tests are needed for this patch.
               [exec]                         Also please list what manual steps were performed to verify this patch.
               [exec] 
               [exec]     +1 javadoc.  The javadoc tool did not generate any warning messages.
               [exec] 
               [exec]     +1 javac.  The applied patch does not increase the total number of javac compiler warnings.
               [exec] 
               [exec]     +1 findbugs.  The patch does not introduce any new Findbugs (version 1.3.9) warnings.
               [exec] 
               [exec]     +1 release audit.  The applied patch does not increase the total number of release audit warnings.
               [exec] 
               [exec]     +1 system test framework.  The patch passed system test framework compile.
               [exec] 
          
          Show
          Eli Collins added a comment - I don't see the findbugs warning locally. Don't think a test is needed since this to get the code compiling. [exec] [exec] -1 overall. [exec] [exec] +1 @author. The patch does not contain any @author tags. [exec] [exec] -1 tests included. The patch doesn't appear to include any new or modified tests. [exec] Please justify why no new tests are needed for this patch. [exec] Also please list what manual steps were performed to verify this patch. [exec] [exec] +1 javadoc. The javadoc tool did not generate any warning messages. [exec] [exec] +1 javac. The applied patch does not increase the total number of javac compiler warnings. [exec] [exec] +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. [exec] [exec] +1 release audit. The applied patch does not increase the total number of release audit warnings. [exec] [exec] +1 system test framework. The patch passed system test framework compile. [exec]
          Hide
          Eli Collins added a comment -

          I've committed this to trunk. Thanks Trevor!

          Show
          Eli Collins added a comment - I've committed this to trunk. Thanks Trevor!
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Common-trunk-Commit #627 (See https://builds.apache.org/hudson/job/Hadoop-Common-trunk-Commit/627/)
          HADOOP-7276. Hadoop native builds fail on ARM due to -m32. Contributed by Trevor Robinson

          eli : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1128475
          Files :

          • /hadoop/common/trunk/CHANGES.txt
          • /hadoop/common/trunk/src/native/configure.ac
          • /hadoop/common/trunk/src/native/Makefile.am
          • /hadoop/common/trunk/src/native/src/org/apache/hadoop/io/compress/zlib/Makefile.am
          • /hadoop/common/trunk/src/native/lib/Makefile.am
          Show
          Hudson added a comment - Integrated in Hadoop-Common-trunk-Commit #627 (See https://builds.apache.org/hudson/job/Hadoop-Common-trunk-Commit/627/ ) HADOOP-7276 . Hadoop native builds fail on ARM due to -m32. Contributed by Trevor Robinson eli : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1128475 Files : /hadoop/common/trunk/CHANGES.txt /hadoop/common/trunk/src/native/configure.ac /hadoop/common/trunk/src/native/Makefile.am /hadoop/common/trunk/src/native/src/org/apache/hadoop/io/compress/zlib/Makefile.am /hadoop/common/trunk/src/native/lib/Makefile.am
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Common-trunk #702 (See https://builds.apache.org/hudson/job/Hadoop-Common-trunk/702/)
          HADOOP-7276. Hadoop native builds fail on ARM due to -m32. Contributed by Trevor Robinson

          eli : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1128475
          Files :

          • /hadoop/common/trunk/CHANGES.txt
          • /hadoop/common/trunk/src/native/configure.ac
          • /hadoop/common/trunk/src/native/Makefile.am
          • /hadoop/common/trunk/src/native/src/org/apache/hadoop/io/compress/zlib/Makefile.am
          • /hadoop/common/trunk/src/native/lib/Makefile.am
          Show
          Hudson added a comment - Integrated in Hadoop-Common-trunk #702 (See https://builds.apache.org/hudson/job/Hadoop-Common-trunk/702/ ) HADOOP-7276 . Hadoop native builds fail on ARM due to -m32. Contributed by Trevor Robinson eli : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1128475 Files : /hadoop/common/trunk/CHANGES.txt /hadoop/common/trunk/src/native/configure.ac /hadoop/common/trunk/src/native/Makefile.am /hadoop/common/trunk/src/native/src/org/apache/hadoop/io/compress/zlib/Makefile.am /hadoop/common/trunk/src/native/lib/Makefile.am

            People

            • Assignee:
              Trevor Robinson
              Reporter:
              Trevor Robinson
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development