Details

    • Improvement
    • Status: Closed
    • Major
    • Resolution: Fixed
    • 2.5.1
    • 2.7.0
    • hdfs-client
    • None
    • Reviewed

    Description

      DFSInputStream.buffersize is not used anywhere in the stream.

      Attachments

        1. HDFS-7336.patch
          7 kB
          Milan Desai
        2. HDFS-7336.patch
          2 kB
          Milan Desai

        Activity

          This means that even though we pass buffersize as a parameter into the contructors of DFSInputStream it does not determine the size of the buffer.
          I think we can live with the default buffer, so I just propose to remove the member.
          Another way is to incorporate buffersize into block reader. Not sure if it helps anything though.

          shv Konstantin Shvachko added a comment - This means that even though we pass buffersize as a parameter into the contructors of DFSInputStream it does not determine the size of the buffer. I think we can live with the default buffer, so I just propose to remove the member. Another way is to incorporate buffersize into block reader. Not sure if it helps anything though.

          And there is an unused import of AtomicLong.

          shv Konstantin Shvachko added a comment - And there is an unused import of AtomicLong.
          milandesai Milan Desai added a comment -

          Removed buffersize parameter from DFSInputStream and DFSClient constructor/method signatures and fixed side effects.

          milandesai Milan Desai added a comment - Removed buffersize parameter from DFSInputStream and DFSClient constructor/method signatures and fixed side effects.
          hadoopqa Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12679618/HDFS-7336.patch
          against trunk revision bc80251.

          -1 patch. The patch command could not apply the patch.

          Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/8663//console

          This message is automatically generated.

          hadoopqa Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12679618/HDFS-7336.patch against trunk revision bc80251. -1 patch . The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/8663//console This message is automatically generated.
          milandesai Milan Desai added a comment -

          Signature changes no longer as deep.

          milandesai Milan Desai added a comment - Signature changes no longer as deep.
          hadoopqa Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12679694/HDFS-7336.patch
          against trunk revision 8549fa5.

          +1 @author. The patch does not contain any @author tags.

          -1 tests included. The patch doesn't appear to include any new or modified tests.
          Please justify why no new tests are needed for this patch.
          Also please list what manual steps were performed to verify this patch.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. There were no new javadoc warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 2.0.3) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 core tests. The patch failed these unit tests in hadoop-hdfs-project/hadoop-hdfs:

          org.apache.hadoop.hdfs.server.namenode.TestCheckpoint

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/8672//testReport/
          Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/8672//console

          This message is automatically generated.

          hadoopqa Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12679694/HDFS-7336.patch against trunk revision 8549fa5. +1 @author . The patch does not contain any @author tags. -1 tests included . The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . There were no new javadoc warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 2.0.3) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. -1 core tests . The patch failed these unit tests in hadoop-hdfs-project/hadoop-hdfs: org.apache.hadoop.hdfs.server.namenode.TestCheckpoint +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/8672//testReport/ Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/8672//console This message is automatically generated.

          +1
          TestCheckpoint failure is related to HDFS-7361.

          shv Konstantin Shvachko added a comment - +1 TestCheckpoint failure is related to HDFS-7361 .

          I jusr committed this. Thank you Milan.

          shv Konstantin Shvachko added a comment - I jusr committed this. Thank you Milan.
          hudson Hudson added a comment -

          SUCCESS: Integrated in Hadoop-trunk-Commit #6466 (See https://builds.apache.org/job/Hadoop-trunk-Commit/6466/)
          HDFS-7336. Unused member DFSInputStream.buffersize. Contributed by Milan Desai. (shv: rev 80d7d183cd4052d6e6d412ff6588d26471c85d6d)

          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSInputStream.java
          • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSClient.java
          hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-trunk-Commit #6466 (See https://builds.apache.org/job/Hadoop-trunk-Commit/6466/ ) HDFS-7336 . Unused member DFSInputStream.buffersize. Contributed by Milan Desai. (shv: rev 80d7d183cd4052d6e6d412ff6588d26471c85d6d) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSInputStream.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSClient.java
          hudson Hudson added a comment -

          SUCCESS: Integrated in Hadoop-Yarn-trunk #735 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/735/)
          HDFS-7336. Unused member DFSInputStream.buffersize. Contributed by Milan Desai. (shv: rev 80d7d183cd4052d6e6d412ff6588d26471c85d6d)

          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSClient.java
          • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSInputStream.java
          hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-Yarn-trunk #735 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/735/ ) HDFS-7336 . Unused member DFSInputStream.buffersize. Contributed by Milan Desai. (shv: rev 80d7d183cd4052d6e6d412ff6588d26471c85d6d) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSClient.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSInputStream.java
          hudson Hudson added a comment -

          SUCCESS: Integrated in Hadoop-Hdfs-trunk #1925 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1925/)
          HDFS-7336. Unused member DFSInputStream.buffersize. Contributed by Milan Desai. (shv: rev 80d7d183cd4052d6e6d412ff6588d26471c85d6d)

          • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSInputStream.java
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSClient.java
          hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-Hdfs-trunk #1925 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1925/ ) HDFS-7336 . Unused member DFSInputStream.buffersize. Contributed by Milan Desai. (shv: rev 80d7d183cd4052d6e6d412ff6588d26471c85d6d) hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSInputStream.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSClient.java
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Mapreduce-trunk #1949 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1949/)
          HDFS-7336. Unused member DFSInputStream.buffersize. Contributed by Milan Desai. (shv: rev 80d7d183cd4052d6e6d412ff6588d26471c85d6d)

          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSInputStream.java
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSClient.java
          • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk #1949 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1949/ ) HDFS-7336 . Unused member DFSInputStream.buffersize. Contributed by Milan Desai. (shv: rev 80d7d183cd4052d6e6d412ff6588d26471c85d6d) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSInputStream.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSClient.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt

          People

            milandesai Milan Desai
            shv Konstantin Shvachko
            Votes:
            0 Vote for this issue
            Watchers:
            Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment