[MAPREDUCE-6980] Add support of ARM64 hardware crc instructions for nativetask

• Type: Improvement
• Status: Patch Available
• Priority: Minor
• Resolution: Unresolved
• Affects Version/s: 3.0.0-alpha3
• Component/s: nativetask, performance

Environment

ARM64 platform with CRC extension

• Assignee: Unassigned
• Reporter: Jun He
• Created: 11/Oct/17 05:08
• Updated: 17/Nov/17 09:41

Description

CRC instructions are introduced since ARM64V8 ISA. With this extension Arm supports CRC32(0x04C11DB7) and CRC32C(0x1EDC6F41) checksum hardware computation.
This patch enables support on such arm64 platform, similiar as HADOOP-11660.
Benchmark is done on a A57 platform using nttest (hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-nativetask/target/native/test/nttest), and the result is as following:
Before (sw crc):
[ RUN ] Perf.CRC
17/10/10 03:05:18 INFO CRC time: 0.10112s size: 100.000M speed: 988.92M/s
17/10/10 03:05:18 INFO CRC32C time: 0.10043s size: 100.000M speed: 995.74M/s
[ OK ] Perf.CRC (216 ms)

After (hw crc):
[ RUN ] Perf.CRC
17/10/11 05:06:27 INFO CRC time: 0.03173s size: 100.000M speed: 3151.14M/s
17/10/11 05:06:27 INFO CRC32C time: 0.03146s size: 100.000M speed: 3178.97M/s
[ OK ] Perf.CRC (77 ms)