Details

    • Hadoop Flags:
      Reviewed

      Description

      Common now has a pure java CRC32 implementation which is more efficient than java.util.zip.CRC32. This issue is to make use of it.

      1. HDFS-496.branch-1.patch
        2 kB
        Brandon Li
      2. hdfs-496.txt
        3 kB
        Todd Lipcon
      3. hdfs-496.txt
        2 kB
        Todd Lipcon

        Issue Links

          Activity

          Hide
          dhruba borthakur added a comment -

          For the records. the PureJavaCrc32 computes the same CRC value as the current one. So, this patch doe snot change HDFS data format. Can you pl link this with the one in the common project, because that JIRA has the performance numbers.

          Show
          dhruba borthakur added a comment - For the records. the PureJavaCrc32 computes the same CRC value as the current one. So, this patch doe snot change HDFS data format. Can you pl link this with the one in the common project, because that JIRA has the performance numbers.
          Hide
          Scott Carey added a comment -

          HADOOP-6148 contains the benchmarks and info about the change from java.uti.zip.CRC32 to the custom pure Java one.

          Show
          Scott Carey added a comment - HADOOP-6148 contains the benchmarks and info about the change from java.uti.zip.CRC32 to the custom pure Java one.
          Hide
          Tsz Wo Nicholas Sze added a comment -

          +1 patch looks good.

          Show
          Tsz Wo Nicholas Sze added a comment - +1 patch looks good.
          Hide
          Todd Lipcon added a comment -

          Anyone mind committing this? It's a trivial patch and well covered by the tests (and reviewers) in HADOOP-6148

          Show
          Todd Lipcon added a comment - Anyone mind committing this? It's a trivial patch and well covered by the tests (and reviewers) in HADOOP-6148
          Hide
          Tsz Wo Nicholas Sze added a comment -

          Need passing Hudson (or you may run test-patch and unit tests locally). Resubmitting ...

          Show
          Tsz Wo Nicholas Sze added a comment - Need passing Hudson (or you may run test-patch and unit tests locally). Resubmitting ...
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12414150/hdfs-496.txt
          against trunk revision 798233.

          +1 @author. The patch does not contain any @author tags.

          -1 tests included. The patch doesn't appear to include any new or modified tests.
          Please justify why no new tests are needed for this patch.
          Also please list what manual steps were performed to verify this patch.

          -1 patch. The patch command could not apply the patch.

          Console output: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-vesta.apache.org/31/console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12414150/hdfs-496.txt against trunk revision 798233. +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. -1 patch. The patch command could not apply the patch. Console output: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-vesta.apache.org/31/console This message is automatically generated.
          Hide
          Todd Lipcon added a comment -

          Rebased patch against trunk

          Show
          Todd Lipcon added a comment - Rebased patch against trunk
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12414771/hdfs-496.txt
          against trunk revision 799480.

          +1 @author. The patch does not contain any @author tags.

          -1 tests included. The patch doesn't appear to include any new or modified tests.
          Please justify why no new tests are needed for this patch.
          Also please list what manual steps were performed to verify this patch.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 findbugs. The patch does not introduce any new Findbugs warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed core unit tests.

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-vesta.apache.org/36/testReport/
          Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-vesta.apache.org/36/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
          Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-vesta.apache.org/36/artifact/trunk/build/test/checkstyle-errors.html
          Console output: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-vesta.apache.org/36/console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12414771/hdfs-496.txt against trunk revision 799480. +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed core unit tests. +1 contrib tests. The patch passed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-vesta.apache.org/36/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-vesta.apache.org/36/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-vesta.apache.org/36/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-vesta.apache.org/36/console This message is automatically generated.
          Hide
          Tsz Wo Nicholas Sze added a comment -

          No new unit test is needed since the changes are very simple.

          I have committed this. Thanks, Todd!

          Show
          Tsz Wo Nicholas Sze added a comment - No new unit test is needed since the changes are very simple. I have committed this. Thanks, Todd!
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-trunk #35 (See http://hudson.zones.apache.org/hudson/job/Hadoop-Hdfs-trunk/35/)
          . Use PureJavaCrc32 in HDFS. Contributed by Todd Lipcon

          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-trunk #35 (See http://hudson.zones.apache.org/hudson/job/Hadoop-Hdfs-trunk/35/ ) . Use PureJavaCrc32 in HDFS. Contributed by Todd Lipcon
          Hide
          Tsz Wo Nicholas Sze added a comment -

          I have committed the backport patch to branch-1. Thanks, Brandon!

          Show
          Tsz Wo Nicholas Sze added a comment - I have committed the backport patch to branch-1. Thanks, Brandon!
          Hide
          Arun C Murthy added a comment -

          I just merged this to branch-1.1 too.

          Show
          Arun C Murthy added a comment - I just merged this to branch-1.1 too.
          Hide
          Tsz Wo Nicholas Sze added a comment -

          Merged this to branch-1-win.

          Show
          Tsz Wo Nicholas Sze added a comment - Merged this to branch-1-win.

            People

            • Assignee:
              Todd Lipcon
              Reporter:
              Todd Lipcon
            • Votes:
              0 Vote for this issue
              Watchers:
              11 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development