Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-4715

Backport HDFS-3577 and other related WebHDFS issue to branch-1

Details

    • Bug
    • Status: Closed
    • Major
    • Resolution: Fixed
    • None
    • 1.2.0
    • webhdfs
    • None
    • Reviewed

    Description

      The related JIRAs are HDFS-3577, HDFS-3318, and HDFS-3788. Backporting them can fix some WebHDFS performance issues in branch-1.

      Attachments

        1. HDFS-4751.1.patch
          15 kB
          Mark Wagner
        2. HDFS-4751.2.patch
          15 kB
          Tsz-wo Sze

        Issue Links

          Activity

            szetszwo Tsz-wo Sze added a comment -

            The patch (HDFS-4549.2.patch) looks different from the current trunk code. For backporting, we would try to backport the code exactly so that it is easier to maintain the code. Could you make the backport patch the same as the original patches? Some other comments:

            • The import org.apache.log4j.helpers.BoundedFIFO is not used.
            • the parameter in update(..) should be the read length but not the read value.
              +    return update(getInputStream().read())
              
            • the filelength in udpate(..) may be null
            szetszwo Tsz-wo Sze added a comment - The patch ( HDFS-4549 .2.patch) looks different from the current trunk code. For backporting, we would try to backport the code exactly so that it is easier to maintain the code. Could you make the backport patch the same as the original patches? Some other comments: The import org.apache.log4j.helpers.BoundedFIFO is not used. the parameter in update(..) should be the read length but not the read value. + return update(getInputStream().read()) the filelength in udpate(..) may be null
            mwagner Mark Wagner added a comment -

            Addressed your points, and got things as close to trunk as possible.

            mwagner Mark Wagner added a comment - Addressed your points, and got things as close to trunk as possible.
            hadoopqa Hadoop QA added a comment -

            -1 overall. Here are the results of testing the latest attachment
            http://issues.apache.org/jira/secure/attachment/12579452/HDFS-4751.1.patch
            against trunk revision .

            -1 patch. The patch command could not apply the patch.

            Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/4278//console

            This message is automatically generated.

            hadoopqa Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12579452/HDFS-4751.1.patch against trunk revision . -1 patch . The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/4278//console This message is automatically generated.
            szetszwo Tsz-wo Sze added a comment -

            Thanks Mark, patch looks good. +1

            Two minor problems:

            • There is an additional System.out.println("length of response: " + e.length) in OpenEntity.
            • In ByteRangeInputStream, the update(..) header splits into two lines. It should be
              +  private int update(final int n) throws IOException {
              -      throws IOException {
              

              instead of

               +  private int update(final int n)
                      throws IOException {
              

            Since both are very minor and I have changed them in my machine, let me post a patch.

            szetszwo Tsz-wo Sze added a comment - Thanks Mark, patch looks good. +1 Two minor problems: There is an additional System.out.println("length of response: " + e.length) in OpenEntity. In ByteRangeInputStream, the update(..) header splits into two lines. It should be + private int update( final int n) throws IOException { - throws IOException { instead of + private int update( final int n) throws IOException { Since both are very minor and I have changed them in my machine, let me post a patch.
            szetszwo Tsz-wo Sze added a comment -

            HDFS-4751.2.patch: addresses above two minor problems.

            szetszwo Tsz-wo Sze added a comment - HDFS-4751 .2.patch: addresses above two minor problems.
            szetszwo Tsz-wo Sze added a comment -

            Mark, since Jenkins won't be able to test the patch, could you run "ant test-patch" and "ant test-core" and then post the results?

            szetszwo Tsz-wo Sze added a comment - Mark, since Jenkins won't be able to test the patch, could you run "ant test-patch" and "ant test-core" and then post the results?
            hadoopqa Hadoop QA added a comment -

            -1 overall. Here are the results of testing the latest attachment
            http://issues.apache.org/jira/secure/attachment/12579855/HDFS-4751.2.patch
            against trunk revision .

            -1 patch. The patch command could not apply the patch.

            Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/4283//console

            This message is automatically generated.

            hadoopqa Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12579855/HDFS-4751.2.patch against trunk revision . -1 patch . The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/4283//console This message is automatically generated.
            hadoopqa Hadoop QA added a comment -

            -1 overall. Here are the results of testing the latest attachment
            http://issues.apache.org/jira/secure/attachment/12579858/HDFS-4751.2.patch
            against trunk revision .

            -1 patch. The patch command could not apply the patch.

            Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/4284//console

            This message is automatically generated.

            hadoopqa Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12579858/HDFS-4751.2.patch against trunk revision . -1 patch . The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/4284//console This message is automatically generated.
            szetszwo Tsz-wo Sze added a comment -

            test-patch results:

                 [exec] -1 overall.  
                 [exec] 
                 [exec]     +1 @author.  The patch does not contain any @author tags.
                 [exec] 
                 [exec]     +1 tests included.  The patch appears to include 8 new or modified tests.
                 [exec] 
                 [exec]     +1 javadoc.  The javadoc tool did not generate any warning messages.
                 [exec] 
                 [exec]     +1 javac.  The applied patch does not increase the total number of javac compiler warnings.
                 [exec] 
                 [exec]     -1 findbugs.  The patch appears to introduce 17 new Findbugs (version 1.3.9) warnings.
            

            The findbugs warnings were not introduced by the patch.

            szetszwo Tsz-wo Sze added a comment - test-patch results: [exec] -1 overall. [exec] [exec] +1 @author. The patch does not contain any @author tags. [exec] [exec] +1 tests included. The patch appears to include 8 new or modified tests. [exec] [exec] +1 javadoc. The javadoc tool did not generate any warning messages. [exec] [exec] +1 javac. The applied patch does not increase the total number of javac compiler warnings. [exec] [exec] -1 findbugs. The patch appears to introduce 17 new Findbugs (version 1.3.9) warnings. The findbugs warnings were not introduced by the patch.
            szetszwo Tsz-wo Sze added a comment -

            All unit tests passed in my machine except TestCLI. It is not related; see HADOOP-9492

            szetszwo Tsz-wo Sze added a comment - All unit tests passed in my machine except TestCLI. It is not related; see HADOOP-9492
            szetszwo Tsz-wo Sze added a comment -

            I have committed this. Thanks, Mark!

            szetszwo Tsz-wo Sze added a comment - I have committed this. Thanks, Mark!
            mattf Matthew Foley added a comment -

            Closed upon release of Hadoop 1.2.0.

            mattf Matthew Foley added a comment - Closed upon release of Hadoop 1.2.0.

            People

              mwagner Mark Wagner
              szetszwo Tsz-wo Sze
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: