[HADOOP-8037] Binary tarball does not preserve platform info for native builds, and RPMs fail to provide needed symlinks for libhadoop.so

• Type: Bug
• Status: Closed
• Priority: Blocker
• Resolution: Fixed
• Affects Version/s: 1.0.1
• Fix Version/s: 1.0.1
• Component/s: build
• Target Version/s: 1.0.1
• Hadoop Flags: Incompatible change

Release Note

This fix is marked "incompatible" only because it changes the bin-tarball directory structure to be consistent with the source tarball directory structure. The source tarball is unchanged. RPMs and DEBs now use an intermediate bin-tarball with an "${os.arch}" tag (like the packages themselves). The un-tagged bin-tarball is now multi-platform and retains the structure of the source tarball; it is in fact generated by target "tar", not by target "binary". Finally, in the 64-bit RPMs and DEBs, the native libs go in the "lib64" directory instead of "lib".

• Assignee: Matthew Foley
• Reporter: Matthew Foley
• Created: 08/Feb/12 08:30
• Updated: 18/Mar/12 07:38
• Resolved: 14/Feb/12 06:23

Description

The source tarball uses "package" ant target, which includes both sets of native builds (32 and 64 bit libraries), under subdirectories that are named for the supported platform, so you can tell what they are.

The binary tarball uses the "bin-package" ant target, which projects both sets of native builds into a single directory, stripping out the platform names from the directory paths. Since the native built libraries have identical names, only one of each survives the process. Afterward, there is no way to know whether they are intended for 32 or 64 bit environments.

It seems to be done this way as a step toward building the rpm and deb artifacts. But the rpms and debs are self-identifying as to the platform they were built for, and contain only one set of libs each, while the binary tarball isn't. The binary tarball should have the same platform-specific subdirectories that the full tarball does; but this means that the rpm and deb builds have to be more careful about include/exclude specs for what goes into those artifacts.

Issue Links

• relates to: HADOOP-8181 Review trunk and v0.23 build specs for HADOOP-8037 and HADOOP-8090