Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-14405

Fix performance regression due to incorrect use of DataChecksum

    Details

    • Hadoop Flags:
      Reviewed
    • Flags:
      Patch

      Description

      Recently I have upgraded my Hadoop version from 2.6 to 3.0, and I find that the write performance decreased by 13%. After some days comparative analysis, It's seems introduced by HADOOP-10865.

      Since James Thomas have done the work that native checksum can run against byte[] arrays instead of just against byte buffers, we may use native method preferential because it runs faster than others.

      Tsz Wo Nicholas Sze and Masatake Iwasaki could you take a look at this to see if it make bad effect on your benchmark test? Todd Lipcon could you help to see if I have make mistakes in this patch?
      thanks!

        Attachments

          Activity

            People

            • Assignee:
              GeLiXin LiXin Ge
              Reporter:
              GeLiXin LiXin Ge
            • Votes:
              0 Vote for this issue
              Watchers:
              11 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: