Hadoop Common
  1. Hadoop Common
  2. HADOOP-7446

Implement CRC32C native code using SSE4.2 instructions

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.23.0
    • Fix Version/s: 0.23.0
    • Component/s: native, performance
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      Once HADOOP-7445 is implemented, we can get further performance improvements by implementing CRC32C using the hardware support available in SSE4.2. This support should be dynamically enabled based on CPU feature flags, and of course should be ifdeffed properly so that it doesn't break the build on architectures/platforms where it's not available.

      1. pipelined-crc.patch.txt
        18 kB
        Kihwal Lee
      2. pipelined_with_todds_patch.txt
        18 kB
        Kihwal Lee
      3. pipelined_on_todds_patch.txt
        13 kB
        Kihwal Lee
      4. hadoop-7446.txt
        8 kB
        Todd Lipcon
      5. hadoop-7446.txt
        7 kB
        Todd Lipcon
      6. hadoop-7446.txt
        7 kB
        Todd Lipcon
      7. crc-pipeline-fix.txt
        4 kB
        Todd Lipcon

        Issue Links

          Activity

          No work has yet been logged on this issue.

            People

            • Assignee:
              Todd Lipcon
              Reporter:
              Todd Lipcon
            • Votes:
              0 Vote for this issue
              Watchers:
              15 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development