[MAPREDUCE-6397] MAPREDUCE makes many endian-dependent assumptions

• Type: Sub-task
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 2.7.0
• Component/s: build

Environment

Any big-endian platform

• Assignee: Alan Burlison
• Reporter: Alan Burlison
• Created: 14/Jun/15 20:56
• Updated: 10/Dec/15 14:19

Description

MAPREDUCE native code contains multiple uses of the bswap and bswap64 assembler functions (from hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-nativetask/src/main/native/src/lib/primitives.h).

primitives.h contains neither a sparc implementation of bswap nor a platform-independent C fallback implementation. In addition, byte swaps are nearly always made without checking if the platform is big or little endian, the assumption hard-coded throughout the source seems to be that the platform is little-endian. This most likely means that the MAPREDUCE native is currently code non-portable to big-endian platforms. The code needs to be examined carefully to determine which byte swaps are correct on all platforms and which are endian-dependent.

Issue Links

• relates to: MAPREDUCE-6417 MapReduceClient's primitives.h is toxic and should be extirpated
• relates to: HADOOP-11505 Various native parts use bswap incorrectly and unportably