Hadoop HDFS
  1. Hadoop HDFS
  2. HDFS-1920

libhdfs does not build for ARM processors

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.21.0
    • Fix Version/s: 0.23.0
    • Component/s: libhdfs
    • Labels:
      None
    • Environment:
    • Hadoop Flags:
      Reviewed

      Description

      $ ant compile -Dcompile.native=true -Dcompile.c++=1 -Dlibhdfs=1 -Dfusedfs=1
      ...
      create-libhdfs-configure:
      ...
      [exec] configure: error: Unsupported CPU architecture "armv7l"

      Once the CPU arch check is fixed in src/c++/libhdfs/m4/apsupport.m4, then next issue is -m32:

      $ ant compile -Dcompile.native=true -Dcompile.c++=1 -Dlibhdfs=1 -Dfusedfs=1
      ...

      compile-c++-libhdfs:
      [exec] /bin/bash ./libtool --tag=CC --mode=compile gcc -DPACKAGE_NAME=\"libhdfs\" -DPACKAGE_TARNAME=\"libhdfs\" -DPACKAGE_VERSION=\"0.1.0\" -DPACKAGE_STRING=\"libhdfs\ 0.1.0\" -DPACKAGE_BUGREPORT=\"omalley@apache.org\" -DPACKAGE_URL=\"\" -DPACKAGE=\"libhdfs\" -DVERSION=\"0.1.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -Dsize_t=unsigned\ int -Dconst=/**/ -Dvolatile=/**/ -I. -I/home/trobinson/dev/hadoop-hdfs/src/c+/libhdfs -g -O2 -DOS_LINUX -DDSO_DLFCN -DCPU=\"arm\" -m32 -I/usr/lib/jvm/java-6-openjdk/include -I/usr/lib/jvm/java-6-openjdk/include/arm -Wall -Wstrict-prototypes -MT hdfs.lo -MD -MP -MF .deps/hdfs.Tpo -c -o hdfs.lo /home/trobinson/dev/hadoop-hdfs/src/c+/libhdfs/hdfs.c
      [exec] make: Warning: File `.deps/hdfs_write.Po' has modification time 2.1 s in the future
      [exec] libtool: compile: gcc -DPACKAGE_NAME=\"libhdfs\" -DPACKAGE_TARNAME=\"libhdfs\" -DPACKAGE_VERSION=\"0.1.0\" "-DPACKAGE_STRING=\"libhdfs 0.1.0\"" -DPACKAGE_BUGREPORT=\"omalley@apache.org\" -DPACKAGE_URL=\"\" -DPACKAGE=\"libhdfs\" -DVERSION=\"0.1.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" "-Dsize_t=unsigned int" "-Dconst=/*/" "-Dvolatile=/*/" -I. -I/home/trobinson/dev/hadoop-hdfs/src/c+/libhdfs -g -O2 -DOS_LINUX -DDSO_DLFCN -DCPU=\"arm\" -m32 -I/usr/lib/jvm/java-6-openjdk/include -I/usr/lib/jvm/java-6-openjdk/include/arm -Wall -Wstrict-prototypes -MT hdfs.lo -MD -MP -MF .deps/hdfs.Tpo -c /home/trobinson/dev/hadoop-hdfs/src/c+/libhdfs/hdfs.c -fPIC -DPIC -o .libs/hdfs.o
      [exec] cc1: error: unrecognized command line option "-m32"
      [exec] make: *** [hdfs.lo] Error 1

      Here, gcc does not support -m32 for the ARM target, so -m$

      {JVM_ARCH}

      must be omitted from CFLAGS and LDFLAGS.

        Issue Links

          Activity

          Trevor Robinson created issue -
          Trevor Robinson made changes -
          Field Original Value New Value
          Link This issue is related to HADOOP-7276 [ HADOOP-7276 ]
          Hide
          Trevor Robinson added a comment -

          This patch adds a case for ARM to src/c++/libhdfs/m4/apsupport.m4 and omits -m$

          {JVM_ARCH}

          from CFLAGS and LDFLAGS if $host_cpu starts with "arm".

          Show
          Trevor Robinson added a comment - This patch adds a case for ARM to src/c++/libhdfs/m4/apsupport.m4 and omits -m$ {JVM_ARCH} from CFLAGS and LDFLAGS if $host_cpu starts with "arm".
          Trevor Robinson made changes -
          Attachment hadoop-hdfs-arm.patch [ 12478901 ]
          Trevor Robinson made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12478901/hadoop-hdfs-arm.patch
          against trunk revision 1102153.

          +1 @author. The patch does not contain any @author tags.

          -1 tests included. The patch doesn't appear to include any new or modified tests.
          Please justify why no new tests are needed for this patch.
          Also please list what manual steps were performed to verify this patch.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 core tests. The patch failed these core unit tests:
          org.apache.hadoop.cli.TestHDFSCLI
          org.apache.hadoop.hdfs.TestDFSShell
          org.apache.hadoop.hdfs.TestDFSStorageStateRecovery
          org.apache.hadoop.hdfs.TestFileConcurrentReader
          org.apache.hadoop.tools.TestJMXGet

          +1 contrib tests. The patch passed contrib unit tests.

          +1 system test framework. The patch passed system test framework compile.

          Test results: https://builds.apache.org/hudson/job/PreCommit-HDFS-Build/501//testReport/
          Findbugs warnings: https://builds.apache.org/hudson/job/PreCommit-HDFS-Build/501//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
          Console output: https://builds.apache.org/hudson/job/PreCommit-HDFS-Build/501//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12478901/hadoop-hdfs-arm.patch against trunk revision 1102153. +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these core unit tests: org.apache.hadoop.cli.TestHDFSCLI org.apache.hadoop.hdfs.TestDFSShell org.apache.hadoop.hdfs.TestDFSStorageStateRecovery org.apache.hadoop.hdfs.TestFileConcurrentReader org.apache.hadoop.tools.TestJMXGet +1 contrib tests. The patch passed contrib unit tests. +1 system test framework. The patch passed system test framework compile. Test results: https://builds.apache.org/hudson/job/PreCommit-HDFS-Build/501//testReport/ Findbugs warnings: https://builds.apache.org/hudson/job/PreCommit-HDFS-Build/501//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/hudson/job/PreCommit-HDFS-Build/501//console This message is automatically generated.
          Hide
          Trevor Robinson added a comment -

          No tests included because this change just fixes a build failure. Manually verified that x86-64 builds unchanged (-m64 is properly specified) and that ARM now builds (-m32 is not specified).

          Core unit test failures are existing and unrelated issues. This change only affects libhdfs.

          Would a committer please review the change?

          Show
          Trevor Robinson added a comment - No tests included because this change just fixes a build failure. Manually verified that x86-64 builds unchanged (-m64 is properly specified) and that ARM now builds (-m32 is not specified). Core unit test failures are existing and unrelated issues. This change only affects libhdfs. Would a committer please review the change?
          Hide
          Eli Collins added a comment -

          +1 looks good. (I also verified that x64 libhdfs still builds/runs).

          Show
          Eli Collins added a comment - +1 looks good. (I also verified that x64 libhdfs still builds/runs).
          Eli Collins made changes -
          Assignee Trevor Robinson [ scurrilous ]
          Hide
          Eli Collins added a comment -

          I've committed this to trunk. Thanks Trevor!

          Show
          Eli Collins added a comment - I've committed this to trunk. Thanks Trevor!
          Eli Collins made changes -
          Status Patch Available [ 10002 ] Resolved [ 5 ]
          Hadoop Flags [Reviewed]
          Fix Version/s 0.23.0 [ 12315571 ]
          Resolution Fixed [ 1 ]
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-trunk-Commit #693 (See https://builds.apache.org/hudson/job/Hadoop-Hdfs-trunk-Commit/693/)

          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-trunk-Commit #693 (See https://builds.apache.org/hudson/job/Hadoop-Hdfs-trunk-Commit/693/ )
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-trunk #680 (See https://builds.apache.org/hudson/job/Hadoop-Hdfs-trunk/680/)
          HDFS-1920. libhdfs does not build for ARM processors. Contributed by Trevor Robinson

          eli : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1128473
          Files :

          • /hadoop/hdfs/trunk/src/c++/libhdfs/configure.ac
          • /hadoop/hdfs/trunk/CHANGES.txt
          • /hadoop/hdfs/trunk/src/c++/libhdfs/m4/apsupport.m4
          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-trunk #680 (See https://builds.apache.org/hudson/job/Hadoop-Hdfs-trunk/680/ ) HDFS-1920 . libhdfs does not build for ARM processors. Contributed by Trevor Robinson eli : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1128473 Files : /hadoop/hdfs/trunk/src/c++/libhdfs/configure.ac /hadoop/hdfs/trunk/CHANGES.txt /hadoop/hdfs/trunk/src/c++/libhdfs/m4/apsupport.m4
          Arun C Murthy made changes -
          Status Resolved [ 5 ] Closed [ 6 ]
          Trevor Robinson made changes -
          Link This issue is related to HDFS-3383 [ HDFS-3383 ]
          Trevor Robinson made changes -
          Link This issue relates to HADOOP-8538 [ HADOOP-8538 ]
          Transition Time In Source Status Execution Times Last Executer Last Execution Date
          Open Open Patch Available Patch Available
          20h 26m 1 Trevor Robinson 12/May/11 19:54
          Patch Available Patch Available Resolved Resolved
          15d 2h 52m 1 Eli Collins 27/May/11 22:46
          Resolved Resolved Closed Closed
          171d 3h 6m 1 Arun C Murthy 15/Nov/11 00:53

            People

            • Assignee:
              Trevor Robinson
              Reporter:
              Trevor Robinson
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development