Hadoop Common
  1. Hadoop Common
  2. HADOOP-8617

backport pure Java CRC32 calculator changes to branch-1

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 1.1.0
    • Component/s: performance
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      Multiple efforts have been made gradually to improve the CRC performance in Hadoop. This JIRA is to back port these changes to branch-1, which include HADOOP-6166, HADOOP-6148, HADOOP-7333.

      The related HDFS and MAPREDUCE patches are uploaded to their original JIRAs HDFS-496 and MAPREDUCE-782.

        Issue Links

          Activity

          Hide
          Brandon Li added a comment -

          I ran teragen/terasort/teravalidate tests with the branch-1 patches from this JIRA and that of HDFS-496 and MAPREDUCE-782.

          Here is the result of test-patch with all the patches:

          BUILD SUCCESSFUL
          Total time: 4 minutes 17 seconds
          
          -1 overall.  
          
              +1 @author.  The patch does not contain any @author tags.
          
              +1 tests included.  The patch appears to include 3 new or modified tests.
          
              +1 javadoc.  The javadoc tool did not generate any warning messages.
          
              +1 javac.  The applied patch does not increase the total number of javac compiler warnings.
          
              -1 findbugs.  The patch appears to introduce 174 new Findbugs (version 2.0.0) warnings.

          The findbug warnings are not introduced by these patches.

          Show
          Brandon Li added a comment - I ran teragen/terasort/teravalidate tests with the branch-1 patches from this JIRA and that of HDFS-496 and MAPREDUCE-782 . Here is the result of test-patch with all the patches: BUILD SUCCESSFUL Total time: 4 minutes 17 seconds -1 overall. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. -1 findbugs. The patch appears to introduce 174 new Findbugs (version 2.0.0) warnings. The findbug warnings are not introduced by these patches.
          Hide
          Tsz Wo Nicholas Sze added a comment -

          +1 patch looks good.

          Show
          Tsz Wo Nicholas Sze added a comment - +1 patch looks good.
          Hide
          Tsz Wo Nicholas Sze added a comment -

          I have committed this. Thanks, Brandon!

          Show
          Tsz Wo Nicholas Sze added a comment - I have committed this. Thanks, Brandon!
          Hide
          Arun C Murthy added a comment -

          I merged this to branch-1.1 too.

          Show
          Arun C Murthy added a comment - I merged this to branch-1.1 too.
          Hide
          Matt Foley added a comment -

          Closed upon release of Hadoop-1.1.0.

          Show
          Matt Foley added a comment - Closed upon release of Hadoop-1.1.0.

            People

            • Assignee:
              Brandon Li
              Reporter:
              Brandon Li
            • Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development