Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-7446

Implement CRC32C native code using SSE4.2 instructions

    Details

    • Type: Improvement
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 0.23.0
    • Fix Version/s: 0.23.0
    • Component/s: native, performance
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      Once HADOOP-7445 is implemented, we can get further performance improvements by implementing CRC32C using the hardware support available in SSE4.2. This support should be dynamically enabled based on CPU feature flags, and of course should be ifdeffed properly so that it doesn't break the build on architectures/platforms where it's not available.

        Attachments

        1. crc-pipeline-fix.txt
          4 kB
          Todd Lipcon
        2. hadoop-7446.txt
          7 kB
          Todd Lipcon
        3. hadoop-7446.txt
          7 kB
          Todd Lipcon
        4. hadoop-7446.txt
          8 kB
          Todd Lipcon
        5. pipelined_on_todds_patch.txt
          13 kB
          Kihwal Lee
        6. pipelined_with_todds_patch.txt
          18 kB
          Kihwal Lee
        7. pipelined-crc.patch.txt
          18 kB
          Kihwal Lee

          Issue Links

            Activity

              People

              • Assignee:
                tlipcon Todd Lipcon
                Reporter:
                tlipcon Todd Lipcon
              • Votes:
                0 Vote for this issue
                Watchers:
                16 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: