Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-8076

Code cleanup for DFSInputStream: use offset instead of LocatedBlock when possible

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.8.0, 3.0.0-alpha1
    • Component/s: None
    • Labels:
      None

      Description

      This JIRA aims to refactor the signatures fetchBlockByteRange and actualGetFromOneDataNode. Instead of taking a LocatedBlock, I think they should just take the starting offset of that block, since they'll later call getBlockAt to refresh the location anyway. I think we should make it clear so the callers are not surprised if the finally used LocatedBlock is not the one they passed in.

        Activity

        Hide
        hadoopqa Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12723682/HDFS-8076-000.patch
        against trunk revision d27e924.

        +1 @author. The patch does not contain any @author tags.

        -1 tests included. The patch doesn't appear to include any new or modified tests.
        Please justify why no new tests are needed for this patch.
        Also please list what manual steps were performed to verify this patch.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 javadoc. There were no new javadoc warning messages.

        +1 eclipse:eclipse. The patch built with eclipse:eclipse.

        +1 findbugs. The patch does not introduce any new Findbugs (version 2.0.3) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        -1 core tests. The patch failed these unit tests in hadoop-hdfs-project/hadoop-hdfs:

        org.apache.hadoop.hdfs.server.namenode.TestDiskspaceQuotaUpdate

        Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/10194//testReport/
        Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/10194//console

        This message is automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12723682/HDFS-8076-000.patch against trunk revision d27e924. +1 @author . The patch does not contain any @author tags. -1 tests included . The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . There were no new javadoc warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 2.0.3) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. -1 core tests . The patch failed these unit tests in hadoop-hdfs-project/hadoop-hdfs: org.apache.hadoop.hdfs.server.namenode.TestDiskspaceQuotaUpdate Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/10194//testReport/ Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/10194//console This message is automatically generated.
        Hide
        zhz Zhe Zhang added a comment -

        TestDiskspaceQuotaUpdate is unrelated to the patch and passes locally.

        Show
        zhz Zhe Zhang added a comment - TestDiskspaceQuotaUpdate is unrelated to the patch and passes locally.
        Hide
        andrew.wang Andrew Wang added a comment -

        +1 LGTM, will commit shortly

        Show
        andrew.wang Andrew Wang added a comment - +1 LGTM, will commit shortly
        Hide
        zhz Zhe Zhang added a comment -

        Thanks Andrew for the review!

        Show
        zhz Zhe Zhang added a comment - Thanks Andrew for the review!
        Hide
        andrew.wang Andrew Wang added a comment -

        Committed to trunk and branch-2, thanks Zhe for the contribution!

        Show
        andrew.wang Andrew Wang added a comment - Committed to trunk and branch-2, thanks Zhe for the contribution!
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-trunk-Commit #7539 (See https://builds.apache.org/job/Hadoop-trunk-Commit/7539/)
        HDFS-8076. Code cleanup for DFSInputStream: use offset instead of LocatedBlock when possible. Contributed by Zhe Zhang. (wang: rev a42bb1cd915abe5dc33eda3c01e8c74c64f35748)

        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSInputStream.java
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-trunk-Commit #7539 (See https://builds.apache.org/job/Hadoop-trunk-Commit/7539/ ) HDFS-8076 . Code cleanup for DFSInputStream: use offset instead of LocatedBlock when possible. Contributed by Zhe Zhang. (wang: rev a42bb1cd915abe5dc33eda3c01e8c74c64f35748) hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSInputStream.java
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #158 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/158/)
        HDFS-8076. Code cleanup for DFSInputStream: use offset instead of LocatedBlock when possible. Contributed by Zhe Zhang. (wang: rev a42bb1cd915abe5dc33eda3c01e8c74c64f35748)

        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSInputStream.java
        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #158 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/158/ ) HDFS-8076 . Code cleanup for DFSInputStream: use offset instead of LocatedBlock when possible. Contributed by Zhe Zhang. (wang: rev a42bb1cd915abe5dc33eda3c01e8c74c64f35748) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSInputStream.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Hdfs-trunk #2090 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2090/)
        HDFS-8076. Code cleanup for DFSInputStream: use offset instead of LocatedBlock when possible. Contributed by Zhe Zhang. (wang: rev a42bb1cd915abe5dc33eda3c01e8c74c64f35748)

        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSInputStream.java
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk #2090 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2090/ ) HDFS-8076 . Code cleanup for DFSInputStream: use offset instead of LocatedBlock when possible. Contributed by Zhe Zhang. (wang: rev a42bb1cd915abe5dc33eda3c01e8c74c64f35748) hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSInputStream.java
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #149 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/149/)
        HDFS-8076. Code cleanup for DFSInputStream: use offset instead of LocatedBlock when possible. Contributed by Zhe Zhang. (wang: rev a42bb1cd915abe5dc33eda3c01e8c74c64f35748)

        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSInputStream.java
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #149 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/149/ ) HDFS-8076 . Code cleanup for DFSInputStream: use offset instead of LocatedBlock when possible. Contributed by Zhe Zhang. (wang: rev a42bb1cd915abe5dc33eda3c01e8c74c64f35748) hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSInputStream.java
        Hide
        hudson Hudson added a comment -

        SUCCESS: Integrated in Hadoop-Yarn-trunk #892 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/892/)
        HDFS-8076. Code cleanup for DFSInputStream: use offset instead of LocatedBlock when possible. Contributed by Zhe Zhang. (wang: rev a42bb1cd915abe5dc33eda3c01e8c74c64f35748)

        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSInputStream.java
        Show
        hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-Yarn-trunk #892 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/892/ ) HDFS-8076 . Code cleanup for DFSInputStream: use offset instead of LocatedBlock when possible. Contributed by Zhe Zhang. (wang: rev a42bb1cd915abe5dc33eda3c01e8c74c64f35748) hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSInputStream.java
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #159 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/159/)
        HDFS-8076. Code cleanup for DFSInputStream: use offset instead of LocatedBlock when possible. Contributed by Zhe Zhang. (wang: rev a42bb1cd915abe5dc33eda3c01e8c74c64f35748)

        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSInputStream.java
        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #159 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/159/ ) HDFS-8076 . Code cleanup for DFSInputStream: use offset instead of LocatedBlock when possible. Contributed by Zhe Zhang. (wang: rev a42bb1cd915abe5dc33eda3c01e8c74c64f35748) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSInputStream.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Mapreduce-trunk #2108 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2108/)
        HDFS-8076. Code cleanup for DFSInputStream: use offset instead of LocatedBlock when possible. Contributed by Zhe Zhang. (wang: rev a42bb1cd915abe5dc33eda3c01e8c74c64f35748)

        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSInputStream.java
        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk #2108 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2108/ ) HDFS-8076 . Code cleanup for DFSInputStream: use offset instead of LocatedBlock when possible. Contributed by Zhe Zhang. (wang: rev a42bb1cd915abe5dc33eda3c01e8c74c64f35748) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSInputStream.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt

          People

          • Assignee:
            zhz Zhe Zhang
            Reporter:
            zhz Zhe Zhang
          • Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development