Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-6980

Add support of ARM64 hardware crc instructions for nativetask

Add voteVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Patch Available
    • Minor
    • Resolution: Unresolved
    • 3.0.0-alpha3
    • None
    • nativetask, performance
    • None
    • ARM64 platform with CRC extension

    Description

      CRC instructions are introduced since ARM64V8 ISA. With this extension Arm supports CRC32(0x04C11DB7) and CRC32C(0x1EDC6F41) checksum hardware computation.
      This patch enables support on such arm64 platform, similiar as HADOOP-11660.
      Benchmark is done on a A57 platform using nttest (hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-nativetask/target/native/test/nttest), and the result is as following:
      Before (sw crc):
      [ RUN ] Perf.CRC
      17/10/10 03:05:18 INFO CRC time: 0.10112s size: 100.000M speed: 988.92M/s
      17/10/10 03:05:18 INFO CRC32C time: 0.10043s size: 100.000M speed: 995.74M/s
      [ OK ] Perf.CRC (216 ms)

      After (hw crc):
      [ RUN ] Perf.CRC
      17/10/11 05:06:27 INFO CRC time: 0.03173s size: 100.000M speed: 3151.14M/s
      17/10/11 05:06:27 INFO CRC32C time: 0.03146s size: 100.000M speed: 3178.97M/s
      [ OK ] Perf.CRC (77 ms)

      Attachments

        1. MAPREDUCE-6980.0.patch
          13 kB
          Jun He

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            Unassigned Unassigned
            jun.he Jun He

            Dates

              Created:
              Updated:

              Slack

                Issue deployment