Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-7831

Fix the starting index and end condition of the loop in FileDiffList.findEarlierSnapshotBlocks()

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.7.0
    • Fix Version/s: 2.7.0
    • Component/s: None
    • Labels:
      None
    • Target Version/s:
    • Hadoop Flags:
      Reviewed

      Description

      Currently the loop in FileDiffList.findEarlierSnapshotBlocks() starts from insertPoint + 1. It should start from insertPoint - 1. As noted in Jing's comment

      1. HDFS-7831-01.patch
        0.9 kB
        Konstantin Shvachko

        Issue Links

          Activity

          Hide
          shv Konstantin Shvachko added a comment - - edited

          binarySearch(diffs, snapshotId) returns i == -insertPoint-1, where insertPoint is the index of the first element greater than the key. So if the snapshotId is not found then we can start searching backwards from insertPoint-1 == -i-2.
          Current code start with -i == insertPoint+1 and unnecessarily checks two extra elements.

          Show
          shv Konstantin Shvachko added a comment - - edited binarySearch(diffs, snapshotId) returns i == -insertPoint-1 , where insertPoint is the index of the first element greater than the key. So if the snapshotId is not found then we can start searching backwards from insertPoint-1 == -i-2 . Current code start with -i == insertPoint+1 and unnecessarily checks two extra elements.
          Hide
          shv Konstantin Shvachko added a comment -

          Fixed the starting index for the loop. Also we do not need to check that i < diffs.size(), because now it always is.
          This should be treated as an optimization, so there are no additional test cases.

          Show
          shv Konstantin Shvachko added a comment - Fixed the starting index for the loop. Also we do not need to check that i < diffs.size() , because now it always is. This should be treated as an optimization, so there are no additional test cases.
          Hide
          hadoopqa Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12700372/HDFS-7831-01.patch
          against trunk revision b610c68.

          +1 @author. The patch does not contain any @author tags.

          -1 tests included. The patch doesn't appear to include any new or modified tests.
          Please justify why no new tests are needed for this patch.
          Also please list what manual steps were performed to verify this patch.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. There were no new javadoc warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 2.0.3) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 core tests. The patch failed these unit tests in hadoop-hdfs-project/hadoop-hdfs:

          org.apache.hadoop.hdfs.server.namenode.ha.TestFailureToReadEdits

          Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/9656//testReport/
          Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/9656//console

          This message is automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12700372/HDFS-7831-01.patch against trunk revision b610c68. +1 @author . The patch does not contain any @author tags. -1 tests included . The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . There were no new javadoc warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 2.0.3) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. -1 core tests . The patch failed these unit tests in hadoop-hdfs-project/hadoop-hdfs: org.apache.hadoop.hdfs.server.namenode.ha.TestFailureToReadEdits Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/9656//testReport/ Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/9656//console This message is automatically generated.
          Hide
          jingzhao Jing Zhao added a comment -

          Thanks for the fix, Konstantin Shvachko. +1. I will commit it shortly.

          Show
          jingzhao Jing Zhao added a comment - Thanks for the fix, Konstantin Shvachko . +1. I will commit it shortly.
          Hide
          jingzhao Jing Zhao added a comment -

          I've committed this to trunk and branch-2.

          Show
          jingzhao Jing Zhao added a comment - I've committed this to trunk and branch-2.
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-trunk-Commit #7189 (See https://builds.apache.org/job/Hadoop-trunk-Commit/7189/)
          HDFS-7831. Fix the starting index and end condition of the loop in FileDiffList.findEarlierSnapshotBlocks(). Contributed by Konstantin Shvachko. (jing9: rev 73bcfa99af61e5202f030510db8954c17cba43cc)

          • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/snapshot/FileDiffList.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-trunk-Commit #7189 (See https://builds.apache.org/job/Hadoop-trunk-Commit/7189/ ) HDFS-7831 . Fix the starting index and end condition of the loop in FileDiffList.findEarlierSnapshotBlocks(). Contributed by Konstantin Shvachko. (jing9: rev 73bcfa99af61e5202f030510db8954c17cba43cc) hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/snapshot/FileDiffList.java
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #115 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/115/)
          HDFS-7831. Fix the starting index and end condition of the loop in FileDiffList.findEarlierSnapshotBlocks(). Contributed by Konstantin Shvachko. (jing9: rev 73bcfa99af61e5202f030510db8954c17cba43cc)

          • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/snapshot/FileDiffList.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #115 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/115/ ) HDFS-7831 . Fix the starting index and end condition of the loop in FileDiffList.findEarlierSnapshotBlocks(). Contributed by Konstantin Shvachko. (jing9: rev 73bcfa99af61e5202f030510db8954c17cba43cc) hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/snapshot/FileDiffList.java
          Hide
          hudson Hudson added a comment -

          SUCCESS: Integrated in Hadoop-Yarn-trunk #849 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/849/)
          HDFS-7831. Fix the starting index and end condition of the loop in FileDiffList.findEarlierSnapshotBlocks(). Contributed by Konstantin Shvachko. (jing9: rev 73bcfa99af61e5202f030510db8954c17cba43cc)

          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/snapshot/FileDiffList.java
          • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          Show
          hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-Yarn-trunk #849 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/849/ ) HDFS-7831 . Fix the starting index and end condition of the loop in FileDiffList.findEarlierSnapshotBlocks(). Contributed by Konstantin Shvachko. (jing9: rev 73bcfa99af61e5202f030510db8954c17cba43cc) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/snapshot/FileDiffList.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Hdfs-trunk #2047 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2047/)
          HDFS-7831. Fix the starting index and end condition of the loop in FileDiffList.findEarlierSnapshotBlocks(). Contributed by Konstantin Shvachko. (jing9: rev 73bcfa99af61e5202f030510db8954c17cba43cc)

          • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/snapshot/FileDiffList.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk #2047 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2047/ ) HDFS-7831 . Fix the starting index and end condition of the loop in FileDiffList.findEarlierSnapshotBlocks(). Contributed by Konstantin Shvachko. (jing9: rev 73bcfa99af61e5202f030510db8954c17cba43cc) hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/snapshot/FileDiffList.java
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #106 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/106/)
          HDFS-7831. Fix the starting index and end condition of the loop in FileDiffList.findEarlierSnapshotBlocks(). Contributed by Konstantin Shvachko. (jing9: rev 73bcfa99af61e5202f030510db8954c17cba43cc)

          • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/snapshot/FileDiffList.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #106 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/106/ ) HDFS-7831 . Fix the starting index and end condition of the loop in FileDiffList.findEarlierSnapshotBlocks(). Contributed by Konstantin Shvachko. (jing9: rev 73bcfa99af61e5202f030510db8954c17cba43cc) hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/snapshot/FileDiffList.java
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #115 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/115/)
          HDFS-7831. Fix the starting index and end condition of the loop in FileDiffList.findEarlierSnapshotBlocks(). Contributed by Konstantin Shvachko. (jing9: rev 73bcfa99af61e5202f030510db8954c17cba43cc)

          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/snapshot/FileDiffList.java
          • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #115 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/115/ ) HDFS-7831 . Fix the starting index and end condition of the loop in FileDiffList.findEarlierSnapshotBlocks(). Contributed by Konstantin Shvachko. (jing9: rev 73bcfa99af61e5202f030510db8954c17cba43cc) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/snapshot/FileDiffList.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Mapreduce-trunk #2065 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2065/)
          HDFS-7831. Fix the starting index and end condition of the loop in FileDiffList.findEarlierSnapshotBlocks(). Contributed by Konstantin Shvachko. (jing9: rev 73bcfa99af61e5202f030510db8954c17cba43cc)

          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/snapshot/FileDiffList.java
          • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk #2065 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2065/ ) HDFS-7831 . Fix the starting index and end condition of the loop in FileDiffList.findEarlierSnapshotBlocks(). Contributed by Konstantin Shvachko. (jing9: rev 73bcfa99af61e5202f030510db8954c17cba43cc) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/snapshot/FileDiffList.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          Hide
          sjlee0 Sangjin Lee added a comment -

          Removing the 2.6.1-candidate label as this is not applicable to 2.6.

          Show
          sjlee0 Sangjin Lee added a comment - Removing the 2.6.1-candidate label as this is not applicable to 2.6.
          Hide
          vinodkv Vinod Kumar Vavilapalli added a comment -

          Re-adding 2.6.1-candidate label for tracking. Will remove it once 2.6.1 is done.

          Show
          vinodkv Vinod Kumar Vavilapalli added a comment - Re-adding 2.6.1-candidate label for tracking. Will remove it once 2.6.1 is done.
          Hide
          vinodkv Vinod Kumar Vavilapalli added a comment -

          About to make an RC for 2.6.1. Dropping the 2.6.1 label as it doesn't apply to 2.6 per Sangjin's comment above.

          Show
          vinodkv Vinod Kumar Vavilapalli added a comment - About to make an RC for 2.6.1. Dropping the 2.6.1 label as it doesn't apply to 2.6 per Sangjin's comment above.

            People

            • Assignee:
              shv Konstantin Shvachko
              Reporter:
              shv Konstantin Shvachko
            • Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development