Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-3344

libhdfs: always builds 32bit, even when x86_64 Java used

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.20.0
    • Component/s: build
    • Labels:
      None
    • Environment:

      x86_64 linux, x86_64 Java installed

    • Hadoop Flags:
      Incompatible change, Reviewed
    • Release Note:
      Changed build procedure for libhdfs to build correctly for different platforms. Build instructions are in the Jira item.

      Description

      The makefile for libhdfs is hard-coded to compile 32bit libraries. It should perhaps compile dependent on which Java is set.

      The relevant lines are:

      LDFLAGS = -L$(JAVA_HOME)/jre/lib/$(OS_ARCH)/server -ljvm -shared -m32 -Wl,-x
      CPPFLAGS = -m32 -I$(JAVA_HOME)/include -I$(JAVA_HOME)/include/$(PLATFORM)

      $OS_ARCH can be (e.g.) amd64 if you're using a 64bit java on the x86_64 platform. So while gcc will try to link against the correct libjvm.so, it will fail because libhdfs is to be built 32bit (because of -m32)

           [exec] /usr/bin/ld: skipping incompatible /usr/java64/latest/jre/lib/amd64/server/libjvm.so when searching for -ljvm
           [exec] /usr/bin/ld: cannot find -ljvm
           [exec] collect2: ld returned 1 exit status
           [exec] make: *** [/root/def/hadoop-0.16.3/build/libhdfs/libhdfs.so.1] Error 1
      

      The solution should be to specify -m32 or -m64 depending on the os.arch detected.

      There are 3 cases to check:

      • 32bit OS, 32bit java => libhdfs should be built 32bit, specify -m32
      • 64bit OS, 32bit java => libhdfs should be built 32bit, specify -m32
      • 64bit OS, 64bit java => libhdfs should be built 64bit, specify -m64

        Attachments

        1. HADOOP-3344-v2.patch
          3.00 MB
          Giridharan Kesavan
        2. HADOOP-3344.v3.patch
          1.18 MB
          Giridharan Kesavan
        3. HADOOP-3344.v1.patch
          25 kB
          Craig Macdonald
        4. HADOOP-3344.v0.patch
          15 kB
          Craig Macdonald
        5. HADOOP-3344.patch
          1.18 MB
          Giridharan Kesavan
        6. HADOOP-3344.patch
          1.18 MB
          Giridharan Kesavan
        7. HADOOP-3344.patch
          1.18 MB
          Giridharan Kesavan

          Issue Links

            Activity

              People

              • Assignee:
                gkesavan Giridharan Kesavan
                Reporter:
                craigm Craig Macdonald
              • Votes:
                0 Vote for this issue
                Watchers:
                5 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: