Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-8269

getBlockLocations() does not resolve the .reserved path and generates incorrect edit logs when updating the atime

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Blocker
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.8.0, 2.7.1, 3.0.0-alpha1
    • Component/s: None
    • Labels:
      None
    • Target Version/s:
    • Hadoop Flags:
      Reviewed

      Description

      When FSNamesystem#getBlockLocations updates the access time of the INode, it uses the path passed from the client, which generates incorrect edit logs entries:

        <RECORD>
          <OPCODE>OP_TIMES</OPCODE>
          <DATA>
            <TXID>5085</TXID>
            <LENGTH>0</LENGTH>
            <PATH>/.reserved/.inodes/18230</PATH>
            <MTIME>-1</MTIME>
            <ATIME>1429908236392</ATIME>
          </DATA>
        </RECORD>
      

      Note that the NN does not resolve the /.reserved path when processing the edit log, therefore it eventually leads to a NPE when loading the edit logs.

      1. HDFS-8269.000.patch
        12 kB
        Haohui Mai
      2. HDFS-8269.001.patch
        15 kB
        Haohui Mai
      3. HDFS-8269.002.patch
        14 kB
        Haohui Mai
      4. HDFS-8269.003.patch
        15 kB
        Haohui Mai

        Issue Links

          Activity

          Hide
          jingzhao Jing Zhao added a comment -

          Thanks for the fix, Haohui. The patch looks good to me. Some comments:

          1. Looks like GetBlockLocationsResult#iip can now be replaced by a boolean indicating whether to update access time.
          2. If the given srcArg is a reserved path (suppose it is mapped to /foo/bar), and if between the read and write lock the original /foo/bar is deleted and another file is named as /foo/bar, then the following code will update the access time of the wrong file.
            +        // Resolve the path again and update the atime only when the file
            +        // exists
            +        final INodesInPath iip = dir.getINodesInPath(src, true);
            
          3. Maybe we can also add an extra test testGetBlockLocationsRacingWithRename to catch the above case?
          Show
          jingzhao Jing Zhao added a comment - Thanks for the fix, Haohui. The patch looks good to me. Some comments: Looks like GetBlockLocationsResult#iip can now be replaced by a boolean indicating whether to update access time. If the given srcArg is a reserved path (suppose it is mapped to /foo/bar), and if between the read and write lock the original /foo/bar is deleted and another file is named as /foo/bar, then the following code will update the access time of the wrong file. + // Resolve the path again and update the atime only when the file + // exists + final INodesInPath iip = dir.getINodesInPath(src, true ); Maybe we can also add an extra test testGetBlockLocationsRacingWithRename to catch the above case?
          Hide
          hadoopqa Hadoop QA added a comment -



          -1 overall



          Vote Subsystem Runtime Comment
          0 pre-patch 14m 37s Pre-patch trunk compilation is healthy.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 tests included 0m 0s The patch appears to include 2 new or modified test files.
          +1 whitespace 0m 0s The patch has no lines that end in whitespace.
          +1 javac 7m 30s There were no new javac warning messages.
          +1 javadoc 9m 36s There were no new javadoc warning messages.
          +1 release audit 0m 23s The applied patch does not increase the total number of release audit warnings.
          -1 checkstyle 6m 57s The applied patch generated 1 additional checkstyle issues.
          +1 install 1m 33s mvn install still works.
          +1 eclipse:eclipse 0m 32s The patch built with eclipse:eclipse.
          +1 findbugs 3m 4s The patch does not introduce any new Findbugs (version 2.0.3) warnings.
          +1 native 3m 13s Pre-build of native portion
          -1 hdfs tests 184m 53s Tests failed in hadoop-hdfs.
              232m 23s  



          Reason Tests
          Failed unit tests hadoop.hdfs.server.namenode.TestFsck
            hadoop.hdfs.TestEncryptionZones
            hadoop.hdfs.server.blockmanagement.TestDatanodeManager
            hadoop.hdfs.server.namenode.TestFileTruncate
            hadoop.hdfs.TestReservedRawPaths
            hadoop.hdfs.TestEncryptionZonesWithKMS



          Subsystem Report/Notes
          Patch URL http://issues.apache.org/jira/secure/attachment/12728511/HDFS-8269.000.patch
          Optional Tests javadoc javac unit findbugs checkstyle
          git revision trunk / 32cd2c8
          checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/10418/artifact/patchprocess/checkstyle-result-diff.txt
          hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/10418/artifact/patchprocess/testrun_hadoop-hdfs.txt
          Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/10418/testReport/
          Java 1.7.0_55
          uname Linux asf900.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/10418/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 14m 37s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 2 new or modified test files. +1 whitespace 0m 0s The patch has no lines that end in whitespace. +1 javac 7m 30s There were no new javac warning messages. +1 javadoc 9m 36s There were no new javadoc warning messages. +1 release audit 0m 23s The applied patch does not increase the total number of release audit warnings. -1 checkstyle 6m 57s The applied patch generated 1 additional checkstyle issues. +1 install 1m 33s mvn install still works. +1 eclipse:eclipse 0m 32s The patch built with eclipse:eclipse. +1 findbugs 3m 4s The patch does not introduce any new Findbugs (version 2.0.3) warnings. +1 native 3m 13s Pre-build of native portion -1 hdfs tests 184m 53s Tests failed in hadoop-hdfs.     232m 23s   Reason Tests Failed unit tests hadoop.hdfs.server.namenode.TestFsck   hadoop.hdfs.TestEncryptionZones   hadoop.hdfs.server.blockmanagement.TestDatanodeManager   hadoop.hdfs.server.namenode.TestFileTruncate   hadoop.hdfs.TestReservedRawPaths   hadoop.hdfs.TestEncryptionZonesWithKMS Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12728511/HDFS-8269.000.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / 32cd2c8 checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/10418/artifact/patchprocess/checkstyle-result-diff.txt hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/10418/artifact/patchprocess/testrun_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/10418/testReport/ Java 1.7.0_55 uname Linux asf900.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-HDFS-Build/10418/console This message was automatically generated.
          Hide
          hadoopqa Hadoop QA added a comment -



          -1 overall



          Vote Subsystem Runtime Comment
          0 pre-patch 15m 7s Pre-patch trunk compilation is healthy.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 tests included 0m 0s The patch appears to include 2 new or modified test files.
          +1 whitespace 0m 0s The patch has no lines that end in whitespace.
          +1 javac 7m 39s There were no new javac warning messages.
          +1 javadoc 9m 35s There were no new javadoc warning messages.
          +1 release audit 0m 23s The applied patch does not increase the total number of release audit warnings.
          -1 checkstyle 5m 35s The applied patch generated 1 additional checkstyle issues.
          +1 install 1m 39s mvn install still works.
          +1 eclipse:eclipse 0m 33s The patch built with eclipse:eclipse.
          +1 findbugs 3m 5s The patch does not introduce any new Findbugs (version 2.0.3) warnings.
          +1 native 3m 14s Pre-build of native portion
          -1 hdfs tests 188m 53s Tests failed in hadoop-hdfs.
              236m 5s  



          Reason Tests
          Failed unit tests hadoop.hdfs.server.datanode.TestDataNodeMetrics
            hadoop.hdfs.TestReservedRawPaths
            hadoop.hdfs.server.namenode.ha.TestPipelinesFailover
            hadoop.hdfs.TestEncryptionZones
            hadoop.hdfs.server.namenode.TestFileTruncate
            hadoop.hdfs.server.namenode.TestFsck
            hadoop.hdfs.TestEncryptionZonesWithKMS
          Timed out tests org.apache.hadoop.hdfs.server.blockmanagement.TestDatanodeManager



          Subsystem Report/Notes
          Patch URL http://issues.apache.org/jira/secure/attachment/12728673/HDFS-8269.001.patch
          Optional Tests javadoc javac unit findbugs checkstyle
          git revision trunk / feb68cb
          checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/10421/artifact/patchprocess/checkstyle-result-diff.txt
          hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/10421/artifact/patchprocess/testrun_hadoop-hdfs.txt
          Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/10421/testReport/
          Java 1.7.0_55
          uname Linux asf908.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/10421/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 15m 7s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 2 new or modified test files. +1 whitespace 0m 0s The patch has no lines that end in whitespace. +1 javac 7m 39s There were no new javac warning messages. +1 javadoc 9m 35s There were no new javadoc warning messages. +1 release audit 0m 23s The applied patch does not increase the total number of release audit warnings. -1 checkstyle 5m 35s The applied patch generated 1 additional checkstyle issues. +1 install 1m 39s mvn install still works. +1 eclipse:eclipse 0m 33s The patch built with eclipse:eclipse. +1 findbugs 3m 5s The patch does not introduce any new Findbugs (version 2.0.3) warnings. +1 native 3m 14s Pre-build of native portion -1 hdfs tests 188m 53s Tests failed in hadoop-hdfs.     236m 5s   Reason Tests Failed unit tests hadoop.hdfs.server.datanode.TestDataNodeMetrics   hadoop.hdfs.TestReservedRawPaths   hadoop.hdfs.server.namenode.ha.TestPipelinesFailover   hadoop.hdfs.TestEncryptionZones   hadoop.hdfs.server.namenode.TestFileTruncate   hadoop.hdfs.server.namenode.TestFsck   hadoop.hdfs.TestEncryptionZonesWithKMS Timed out tests org.apache.hadoop.hdfs.server.blockmanagement.TestDatanodeManager Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12728673/HDFS-8269.001.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / feb68cb checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/10421/artifact/patchprocess/checkstyle-result-diff.txt hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/10421/artifact/patchprocess/testrun_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/10421/testReport/ Java 1.7.0_55 uname Linux asf908.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-HDFS-Build/10421/console This message was automatically generated.
          Hide
          jingzhao Jing Zhao added a comment -

          The 001 patch looks good to me. But among the failed tests, TestEncryptionZones, TestEncryptionZonesWithKMS, TestFsck, and TestReservedRawPaths may be related. The failures can be reproduced in my local machine and they all passed without the patch.

          Show
          jingzhao Jing Zhao added a comment - The 001 patch looks good to me. But among the failed tests, TestEncryptionZones, TestEncryptionZonesWithKMS, TestFsck, and TestReservedRawPaths may be related. The failures can be reproduced in my local machine and they all passed without the patch.
          Hide
          hadoopqa Hadoop QA added a comment -



          -1 overall



          Vote Subsystem Runtime Comment
          0 pre-patch 14m 55s Pre-patch trunk compilation is healthy.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 tests included 0m 0s The patch appears to include 2 new or modified test files.
          +1 whitespace 0m 0s The patch has no lines that end in whitespace.
          +1 javac 7m 40s There were no new javac warning messages.
          +1 javadoc 9m 42s There were no new javadoc warning messages.
          +1 release audit 0m 22s The applied patch does not increase the total number of release audit warnings.
          -1 checkstyle 4m 2s The applied patch generated 1 additional checkstyle issues.
          +1 install 1m 34s mvn install still works.
          +1 eclipse:eclipse 0m 34s The patch built with eclipse:eclipse.
          +1 findbugs 3m 12s The patch does not introduce any new Findbugs (version 2.0.3) warnings.
          +1 native 3m 19s Pre-build of native portion
          -1 hdfs tests 163m 53s Tests failed in hadoop-hdfs.
              209m 18s  



          Reason Tests
          Failed unit tests hadoop.hdfs.TestFileCreationClient
            hadoop.hdfs.server.namenode.TestFsck
          Timed out tests org.apache.hadoop.hdfs.server.mover.TestMover



          Subsystem Report/Notes
          Patch URL http://issues.apache.org/jira/secure/attachment/12728915/HDFS-8269.002.patch
          Optional Tests javadoc javac unit findbugs checkstyle
          git revision trunk / bc1bd7e
          checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/10434/artifact/patchprocess/checkstyle-result-diff.txt
          hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/10434/artifact/patchprocess/testrun_hadoop-hdfs.txt
          Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/10434/testReport/
          Java 1.7.0_55
          uname Linux asf903.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/10434/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 14m 55s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 2 new or modified test files. +1 whitespace 0m 0s The patch has no lines that end in whitespace. +1 javac 7m 40s There were no new javac warning messages. +1 javadoc 9m 42s There were no new javadoc warning messages. +1 release audit 0m 22s The applied patch does not increase the total number of release audit warnings. -1 checkstyle 4m 2s The applied patch generated 1 additional checkstyle issues. +1 install 1m 34s mvn install still works. +1 eclipse:eclipse 0m 34s The patch built with eclipse:eclipse. +1 findbugs 3m 12s The patch does not introduce any new Findbugs (version 2.0.3) warnings. +1 native 3m 19s Pre-build of native portion -1 hdfs tests 163m 53s Tests failed in hadoop-hdfs.     209m 18s   Reason Tests Failed unit tests hadoop.hdfs.TestFileCreationClient   hadoop.hdfs.server.namenode.TestFsck Timed out tests org.apache.hadoop.hdfs.server.mover.TestMover Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12728915/HDFS-8269.002.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / bc1bd7e checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/10434/artifact/patchprocess/checkstyle-result-diff.txt hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/10434/artifact/patchprocess/testrun_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/10434/testReport/ Java 1.7.0_55 uname Linux asf903.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-HDFS-Build/10434/console This message was automatically generated.
          Hide
          hadoopqa Hadoop QA added a comment -



          -1 overall



          Vote Subsystem Runtime Comment
          0 pre-patch 14m 36s Pre-patch trunk compilation is healthy.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 tests included 0m 0s The patch appears to include 2 new or modified test files.
          +1 whitespace 0m 0s The patch has no lines that end in whitespace.
          +1 javac 7m 26s There were no new javac warning messages.
          +1 javadoc 9m 35s There were no new javadoc warning messages.
          +1 release audit 0m 22s The applied patch does not increase the total number of release audit warnings.
          -1 checkstyle 7m 29s The applied patch generated 1 additional checkstyle issues.
          +1 install 1m 34s mvn install still works.
          +1 eclipse:eclipse 0m 32s The patch built with eclipse:eclipse.
          +1 findbugs 3m 3s The patch does not introduce any new Findbugs (version 2.0.3) warnings.
          +1 native 3m 16s Pre-build of native portion
          -1 hdfs tests 165m 26s Tests failed in hadoop-hdfs.
              213m 25s  



          Reason Tests
          Failed unit tests hadoop.hdfs.server.namenode.ha.TestPipelinesFailover



          Subsystem Report/Notes
          Patch URL http://issues.apache.org/jira/secure/attachment/12729005/HDFS-8269.003.patch
          Optional Tests javadoc javac unit findbugs checkstyle
          git revision trunk / 439614b
          checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/10447/artifact/patchprocess/checkstyle-result-diff.txt
          hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/10447/artifact/patchprocess/testrun_hadoop-hdfs.txt
          Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/10447/testReport/
          Java 1.7.0_55
          uname Linux asf902.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/10447/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 14m 36s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 2 new or modified test files. +1 whitespace 0m 0s The patch has no lines that end in whitespace. +1 javac 7m 26s There were no new javac warning messages. +1 javadoc 9m 35s There were no new javadoc warning messages. +1 release audit 0m 22s The applied patch does not increase the total number of release audit warnings. -1 checkstyle 7m 29s The applied patch generated 1 additional checkstyle issues. +1 install 1m 34s mvn install still works. +1 eclipse:eclipse 0m 32s The patch built with eclipse:eclipse. +1 findbugs 3m 3s The patch does not introduce any new Findbugs (version 2.0.3) warnings. +1 native 3m 16s Pre-build of native portion -1 hdfs tests 165m 26s Tests failed in hadoop-hdfs.     213m 25s   Reason Tests Failed unit tests hadoop.hdfs.server.namenode.ha.TestPipelinesFailover Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12729005/HDFS-8269.003.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / 439614b checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/10447/artifact/patchprocess/checkstyle-result-diff.txt hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/10447/artifact/patchprocess/testrun_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/10447/testReport/ Java 1.7.0_55 uname Linux asf902.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-HDFS-Build/10447/console This message was automatically generated.
          Hide
          jingzhao Jing Zhao added a comment -

          The 003 patch looks pretty good to me. +1. The failed test TestPipelinesFailover should be unrelated.

          Show
          jingzhao Jing Zhao added a comment - The 003 patch looks pretty good to me. +1. The failed test TestPipelinesFailover should be unrelated.
          Hide
          wheat9 Haohui Mai added a comment -

          I've committed the patch to trunk, branch-2 and branch-2.7. Thanks Jing for the reviews.

          Show
          wheat9 Haohui Mai added a comment - I've committed the patch to trunk, branch-2 and branch-2.7. Thanks Jing for the reviews.
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-trunk-Commit #7700 (See https://builds.apache.org/job/Hadoop-trunk-Commit/7700/)
          HDFS-8269. getBlockLocations() does not resolve the .reserved path and generates incorrect edit logs when updating the atime. Contributed by Haohui Mai. (wheat9: rev 3dd6395bb2448e5b178a51c864e3c9a3d12e8bc9)

          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestGetBlockLocations.java
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NamenodeFsck.java
          • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestFsck.java
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-trunk-Commit #7700 (See https://builds.apache.org/job/Hadoop-trunk-Commit/7700/ ) HDFS-8269 . getBlockLocations() does not resolve the .reserved path and generates incorrect edit logs when updating the atime. Contributed by Haohui Mai. (wheat9: rev 3dd6395bb2448e5b178a51c864e3c9a3d12e8bc9) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestGetBlockLocations.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NamenodeFsck.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestFsck.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Hdfs-trunk #2111 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2111/)
          HDFS-8269. getBlockLocations() does not resolve the .reserved path and generates incorrect edit logs when updating the atime. Contributed by Haohui Mai. (wheat9: rev 3dd6395bb2448e5b178a51c864e3c9a3d12e8bc9)

          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestFsck.java
          • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NamenodeFsck.java
          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestGetBlockLocations.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk #2111 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2111/ ) HDFS-8269 . getBlockLocations() does not resolve the .reserved path and generates incorrect edit logs when updating the atime. Contributed by Haohui Mai. (wheat9: rev 3dd6395bb2448e5b178a51c864e3c9a3d12e8bc9) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestFsck.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NamenodeFsck.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestGetBlockLocations.java
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #179 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/179/)
          HDFS-8269. getBlockLocations() does not resolve the .reserved path and generates incorrect edit logs when updating the atime. Contributed by Haohui Mai. (wheat9: rev 3dd6395bb2448e5b178a51c864e3c9a3d12e8bc9)

          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestGetBlockLocations.java
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NamenodeFsck.java
          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestFsck.java
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
          • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #179 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/179/ ) HDFS-8269 . getBlockLocations() does not resolve the .reserved path and generates incorrect edit logs when updating the atime. Contributed by Haohui Mai. (wheat9: rev 3dd6395bb2448e5b178a51c864e3c9a3d12e8bc9) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestGetBlockLocations.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NamenodeFsck.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestFsck.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #170 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/170/)
          HDFS-8269. getBlockLocations() does not resolve the .reserved path and generates incorrect edit logs when updating the atime. Contributed by Haohui Mai. (wheat9: rev 3dd6395bb2448e5b178a51c864e3c9a3d12e8bc9)

          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestGetBlockLocations.java
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NamenodeFsck.java
          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestFsck.java
          • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #170 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/170/ ) HDFS-8269 . getBlockLocations() does not resolve the .reserved path and generates incorrect edit logs when updating the atime. Contributed by Haohui Mai. (wheat9: rev 3dd6395bb2448e5b178a51c864e3c9a3d12e8bc9) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestGetBlockLocations.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NamenodeFsck.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestFsck.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Yarn-trunk #913 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/913/)
          HDFS-8269. getBlockLocations() does not resolve the .reserved path and generates incorrect edit logs when updating the atime. Contributed by Haohui Mai. (wheat9: rev 3dd6395bb2448e5b178a51c864e3c9a3d12e8bc9)

          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestGetBlockLocations.java
          • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestFsck.java
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NamenodeFsck.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk #913 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/913/ ) HDFS-8269 . getBlockLocations() does not resolve the .reserved path and generates incorrect edit logs when updating the atime. Contributed by Haohui Mai. (wheat9: rev 3dd6395bb2448e5b178a51c864e3c9a3d12e8bc9) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestGetBlockLocations.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestFsck.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NamenodeFsck.java
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #180 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/180/)
          HDFS-8269. getBlockLocations() does not resolve the .reserved path and generates incorrect edit logs when updating the atime. Contributed by Haohui Mai. (wheat9: rev 3dd6395bb2448e5b178a51c864e3c9a3d12e8bc9)

          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestFsck.java
          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestGetBlockLocations.java
          • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NamenodeFsck.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #180 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/180/ ) HDFS-8269 . getBlockLocations() does not resolve the .reserved path and generates incorrect edit logs when updating the atime. Contributed by Haohui Mai. (wheat9: rev 3dd6395bb2448e5b178a51c864e3c9a3d12e8bc9) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestFsck.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestGetBlockLocations.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NamenodeFsck.java
          Hide
          hudson Hudson added a comment -

          SUCCESS: Integrated in Hadoop-Mapreduce-trunk #2129 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2129/)
          HDFS-8269. getBlockLocations() does not resolve the .reserved path and generates incorrect edit logs when updating the atime. Contributed by Haohui Mai. (wheat9: rev 3dd6395bb2448e5b178a51c864e3c9a3d12e8bc9)

          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestGetBlockLocations.java
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NamenodeFsck.java
          • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestFsck.java
          Show
          hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-Mapreduce-trunk #2129 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2129/ ) HDFS-8269 . getBlockLocations() does not resolve the .reserved path and generates incorrect edit logs when updating the atime. Contributed by Haohui Mai. (wheat9: rev 3dd6395bb2448e5b178a51c864e3c9a3d12e8bc9) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestGetBlockLocations.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NamenodeFsck.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestFsck.java

            People

            • Assignee:
              wheat9 Haohui Mai
              Reporter:
              yeshavora Yesha Vora
            • Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development