Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-3528

Use native CRC32 in DFS write path

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Closed
    • Major
    • Resolution: Fixed
    • 2.0.0-alpha
    • 2.6.0
    • None

    Description

      HDFS-2080 improved the CPU efficiency of the read path by using native SSE-enabled code for CRC verification. Benchmarks of the write path show that it's often CPU bound by checksums as well, so we should make the same improvement there.

      Attachments

        Issue Links

          Activity

            People

              james.thomas James Thomas
              tlipcon Todd Lipcon
              Votes:
              0 Vote for this issue
              Watchers:
              25 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: