Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-7446

Implement CRC32C native code using SSE4.2 instructions

VotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 0.23.0
    • Fix Version/s: 0.23.0
    • Component/s: native, performance
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      Once HADOOP-7445 is implemented, we can get further performance improvements by implementing CRC32C using the hardware support available in SSE4.2. This support should be dynamically enabled based on CPU feature flags, and of course should be ifdeffed properly so that it doesn't break the build on architectures/platforms where it's not available.

        Attachments

        1. hadoop-7446.txt
          8 kB
          Todd Lipcon
        2. hadoop-7446.txt
          7 kB
          Todd Lipcon
        3. hadoop-7446.txt
          7 kB
          Todd Lipcon
        4. pipelined_on_todds_patch.txt
          13 kB
          Kihwal Lee
        5. pipelined_with_todds_patch.txt
          18 kB
          Kihwal Lee
        6. crc-pipeline-fix.txt
          4 kB
          Todd Lipcon
        7. pipelined-crc.patch.txt
          18 kB
          Kihwal Lee

        Issue Links

          Activity

            People

            • Assignee:
              tlipcon Todd Lipcon
              Reporter:
              tlipcon Todd Lipcon

              Dates

              • Created:
                Updated:
                Resolved:

                Issue deployment