Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-14405

Fix performance regression due to incorrect use of DataChecksum

    XMLWordPrintableJSON

    Details

    • Hadoop Flags:
      Reviewed
    • Flags:
      Patch

      Description

      Recently I have upgraded my Hadoop version from 2.6 to 3.0, and I find that the write performance decreased by 13%. After some days comparative analysis, It's seems introduced by HADOOP-10865.

      Since James Thomas have done the work that native checksum can run against byte[] arrays instead of just against byte buffers, we may use native method preferential because it runs faster than others.

      Tsz-wo Sze and Masatake Iwasaki could you take a look at this to see if it make bad effect on your benchmark test? Todd Lipcon could you help to see if I have make mistakes in this patch?
      thanks!

        Attachments

        1. HDFS-11765.patch
          2 kB
          LiXin Ge

          Issue Links

            Activity

              People

              • Assignee:
                GeLiXin LiXin Ge
                Reporter:
                GeLiXin LiXin Ge
              • Votes:
                0 Vote for this issue
                Watchers:
                11 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: