Details

    • Type: Bug Bug
    • Status: Open
    • Priority: Major Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None

      Description

      Recently, I had a chance to talk to a Hadoop user, who complained there's no 64-bit native lib in Hadaoop releases, and it was user unfriendly to make them download all the dependenies to build 64-bit themselves.

      Hence I checked the recent two releases, 2.2 and 2.3, whose native lib are both ELF 32-bit LSB shared objects. I'm not aware of the reason why we don't release 64-bit, but I'd like to open the ticket to tackle this issue given we didn't before.

        Issue Links

          Activity

          Hide
          Roman Shaposhnik added a comment -

          I would be very cautious about setting expectations for having native libraries released as part of the tarball. In fact, I'd rather have them dropped completely. Otherwise you have to guarantee that you build on the lowest common denominator of the Linux OSes people are using and you essentially pin down your build environment. Something that I'm not sure all future Hadoop RMs are willing to do.

          Show
          Roman Shaposhnik added a comment - I would be very cautious about setting expectations for having native libraries released as part of the tarball. In fact, I'd rather have them dropped completely. Otherwise you have to guarantee that you build on the lowest common denominator of the Linux OSes people are using and you essentially pin down your build environment. Something that I'm not sure all future Hadoop RMs are willing to do.
          Hide
          André Kelpe added a comment -

          Right now hadoop ships with one set of native libs which are for linux 32bit. All I propose is switching to linux 64bit. That will cover almost all use cases and you don't have to build multiple tarballs. Make the one version you ship useful for your biggest user group makes sense to me.

          Show
          André Kelpe added a comment - Right now hadoop ships with one set of native libs which are for linux 32bit. All I propose is switching to linux 64bit. That will cover almost all use cases and you don't have to build multiple tarballs. Make the one version you ship useful for your biggest user group makes sense to me.
          Hide
          Alejandro Abdelnur added a comment -

          your build process will have to span across all target OSes you want native libs for to build them, then you'll have namespace them per OS version, etc etc. you'll end up duplicating most of what bigtop does.

          Show
          Alejandro Abdelnur added a comment - your build process will have to span across all target OSes you want native libs for to build them, then you'll have namespace them per OS version, etc etc. you'll end up duplicating most of what bigtop does.
          Hide
          André Kelpe added a comment -

          This is a duplicate of https://issues.apache.org/jira/browse/HADOOP-9911

          Alejandro: Fair enough, that you don't want to build multiple tarballs, but wouldn't it make sense to build the one and only tarball for 64bit linux? After all that is the deployment target of hadoop anyway, so having that instead of the 32bit version makes a lot more sense to me.

          Show
          André Kelpe added a comment - This is a duplicate of https://issues.apache.org/jira/browse/HADOOP-9911 Alejandro: Fair enough, that you don't want to build multiple tarballs, but wouldn't it make sense to build the one and only tarball for 64bit linux? After all that is the deployment target of hadoop anyway, so having that instead of the 32bit version makes a lot more sense to me.
          Hide
          Alejandro Abdelnur added a comment -

          we don't build native libs for the different versions of linux. furthermore, 2 different releases may have native libs for different versions of linux.

          if we want to address this, then we should build a tarball for each linux version.

          what i would do is refer your user to bigtop, they provide packages for the different linux distros with the corresponding native libs.

          i think the bug we have in the current release is having native libs, we should not (unless we do what i suggest at the beginning of this comment

          Show
          Alejandro Abdelnur added a comment - we don't build native libs for the different versions of linux. furthermore, 2 different releases may have native libs for different versions of linux. if we want to address this, then we should build a tarball for each linux version. what i would do is refer your user to bigtop, they provide packages for the different linux distros with the corresponding native libs. i think the bug we have in the current release is having native libs, we should not (unless we do what i suggest at the beginning of this comment

            People

            • Assignee:
              Unassigned
              Reporter:
              Zhijie Shen
            • Votes:
              0 Vote for this issue
              Watchers:
              10 Start watching this issue

              Dates

              • Created:
                Updated:

                Development