Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-8577

Avoid retrying to recover lease on a file which does not exist

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.8.0, 3.0.0-alpha1
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      1. Avoid retrying to recover lease on a file which does not exist

      recoverLease got exception:
      java.io.FileNotFoundException: File does not exist: /hello_hi
              at org.apache.hadoop.hdfs.server.namenode.INodeFile.valueOf(INodeFile.java:71)
      
      Retrying in 5000 ms...
      Retry #1
      recoverLease got exception:
      java.io.FileNotFoundException: File does not exist: /hello_hi
              at org.apache.hadoop.hdfs.server.namenode.INodeFile.valueOf(INodeFile.java:71)
      
      

      2. Avoid populating huge stack trace for each retry for recovering lease on a file , being displayed on CLI .

      1. HDFS-8577.1.patch
        2 kB
        J.Andreina
      2. HDFS-8577.2.patch
        3 kB
        J.Andreina

        Activity

        Hide
        andreina J.Andreina added a comment -

        Attaching an initial patch.
        Please review.

        Show
        andreina J.Andreina added a comment - Attaching an initial patch. Please review.
        Hide
        hadoopqa Hadoop QA added a comment -



        -1 overall



        Vote Subsystem Runtime Comment
        0 pre-patch 14m 45s Pre-patch trunk compilation is healthy.
        +1 @author 0m 0s The patch does not contain any @author tags.
        -1 tests included 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
        +1 javac 7m 33s There were no new javac warning messages.
        +1 javadoc 9m 34s There were no new javadoc warning messages.
        +1 release audit 0m 25s The applied patch does not increase the total number of release audit warnings.
        +1 checkstyle 0m 0s There were no new checkstyle issues.
        +1 whitespace 0m 0s The patch has no lines that end in whitespace.
        +1 install 1m 35s mvn install still works.
        +1 eclipse:eclipse 0m 33s The patch built with eclipse:eclipse.
        +1 findbugs 0m 0s The patch does not introduce any new Findbugs (version ) warnings.
            34m 28s  



        Subsystem Report/Notes
        Patch URL http://issues.apache.org/jira/secure/attachment/12739592/HDFS-8577.1.patch
        Optional Tests javadoc javac unit findbugs checkstyle
        git revision trunk / 4c5da9b
        Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/11358/testReport/
        Java 1.7.0_55
        uname Linux asf901.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/11358/console

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 14m 45s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. -1 tests included 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac 7m 33s There were no new javac warning messages. +1 javadoc 9m 34s There were no new javadoc warning messages. +1 release audit 0m 25s The applied patch does not increase the total number of release audit warnings. +1 checkstyle 0m 0s There were no new checkstyle issues. +1 whitespace 0m 0s The patch has no lines that end in whitespace. +1 install 1m 35s mvn install still works. +1 eclipse:eclipse 0m 33s The patch built with eclipse:eclipse. +1 findbugs 0m 0s The patch does not introduce any new Findbugs (version ) warnings.     34m 28s   Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12739592/HDFS-8577.1.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / 4c5da9b Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/11358/testReport/ Java 1.7.0_55 uname Linux asf901.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-HDFS-Build/11358/console This message was automatically generated.
        Hide
        andreina J.Andreina added a comment -

        Updated the patch and added a testcase.
        Please review.

        Show
        andreina J.Andreina added a comment - Updated the patch and added a testcase. Please review.
        Hide
        hadoopqa Hadoop QA added a comment -



        -1 overall



        Vote Subsystem Runtime Comment
        -1 pre-patch 15m 2s Findbugs (version ) appears to be broken on trunk.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 tests included 0m 0s The patch appears to include 1 new or modified test files.
        +1 javac 7m 30s There were no new javac warning messages.
        +1 javadoc 9m 37s There were no new javadoc warning messages.
        +1 release audit 0m 23s The applied patch does not increase the total number of release audit warnings.
        +1 checkstyle 0m 36s There were no new checkstyle issues.
        +1 whitespace 0m 0s The patch has no lines that end in whitespace.
        +1 install 1m 37s mvn install still works.
        +1 eclipse:eclipse 0m 33s The patch built with eclipse:eclipse.
        -1 findbugs 3m 18s The patch appears to introduce 1 new Findbugs (version 3.0.0) warnings.
        +1 native 3m 13s Pre-build of native portion
        -1 hdfs tests 117m 13s Tests failed in hadoop-hdfs.
            159m 5s  



        Reason Tests
        FindBugs module:hadoop-hdfs
        Failed unit tests hadoop.hdfs.server.namenode.ha.TestStandbyIsHot
        Timed out tests org.apache.hadoop.hdfs.server.namenode.TestCacheDirectives



        Subsystem Report/Notes
        Patch URL http://issues.apache.org/jira/secure/attachment/12742748/HDFS-8577.2.patch
        Optional Tests javadoc javac unit findbugs checkstyle
        git revision trunk / d3797f9
        Findbugs warnings https://builds.apache.org/job/PreCommit-HDFS-Build/11547/artifact/patchprocess/newPatchFindbugsWarningshadoop-hdfs.html
        hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/11547/artifact/patchprocess/testrun_hadoop-hdfs.txt
        Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/11547/testReport/
        Java 1.7.0_55
        uname Linux asf905.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/11547/console

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment -1 pre-patch 15m 2s Findbugs (version ) appears to be broken on trunk. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 1 new or modified test files. +1 javac 7m 30s There were no new javac warning messages. +1 javadoc 9m 37s There were no new javadoc warning messages. +1 release audit 0m 23s The applied patch does not increase the total number of release audit warnings. +1 checkstyle 0m 36s There were no new checkstyle issues. +1 whitespace 0m 0s The patch has no lines that end in whitespace. +1 install 1m 37s mvn install still works. +1 eclipse:eclipse 0m 33s The patch built with eclipse:eclipse. -1 findbugs 3m 18s The patch appears to introduce 1 new Findbugs (version 3.0.0) warnings. +1 native 3m 13s Pre-build of native portion -1 hdfs tests 117m 13s Tests failed in hadoop-hdfs.     159m 5s   Reason Tests FindBugs module:hadoop-hdfs Failed unit tests hadoop.hdfs.server.namenode.ha.TestStandbyIsHot Timed out tests org.apache.hadoop.hdfs.server.namenode.TestCacheDirectives Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12742748/HDFS-8577.2.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / d3797f9 Findbugs warnings https://builds.apache.org/job/PreCommit-HDFS-Build/11547/artifact/patchprocess/newPatchFindbugsWarningshadoop-hdfs.html hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/11547/artifact/patchprocess/testrun_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/11547/testReport/ Java 1.7.0_55 uname Linux asf905.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-HDFS-Build/11547/console This message was automatically generated.
        Hide
        vinayrpet Vinayakumar B added a comment -

        Latest patch LGTM. +1.
        Will commit shortly.

        Show
        vinayrpet Vinayakumar B added a comment - Latest patch LGTM. +1. Will commit shortly.
        Hide
        vinayrpet Vinayakumar B added a comment -

        Committed to trunk and branch-2.
        Thanks for the contribution J.Andreina.

        Show
        vinayrpet Vinayakumar B added a comment - Committed to trunk and branch-2. Thanks for the contribution J.Andreina .
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-trunk-Commit #8112 (See https://builds.apache.org/job/Hadoop-trunk-Commit/8112/)
        HDFS-8577. Avoid retrying to recover lease on a file which does not exist (Contributed by J.Andreina) (vinayakumarb: rev 2eae130ab9edd318c82503c2306f610f2b5a3e51)

        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/DebugAdmin.java
        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/tools/TestDebugAdmin.java
        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-trunk-Commit #8112 (See https://builds.apache.org/job/Hadoop-trunk-Commit/8112/ ) HDFS-8577 . Avoid retrying to recover lease on a file which does not exist (Contributed by J.Andreina) (vinayakumarb: rev 2eae130ab9edd318c82503c2306f610f2b5a3e51) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/DebugAdmin.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/tools/TestDebugAdmin.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #246 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/246/)
        HDFS-8577. Avoid retrying to recover lease on a file which does not exist (Contributed by J.Andreina) (vinayakumarb: rev 2eae130ab9edd318c82503c2306f610f2b5a3e51)

        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/DebugAdmin.java
        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/tools/TestDebugAdmin.java
        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #246 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/246/ ) HDFS-8577 . Avoid retrying to recover lease on a file which does not exist (Contributed by J.Andreina) (vinayakumarb: rev 2eae130ab9edd318c82503c2306f610f2b5a3e51) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/DebugAdmin.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/tools/TestDebugAdmin.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        Hide
        hudson Hudson added a comment -

        SUCCESS: Integrated in Hadoop-Yarn-trunk #976 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/976/)
        HDFS-8577. Avoid retrying to recover lease on a file which does not exist (Contributed by J.Andreina) (vinayakumarb: rev 2eae130ab9edd318c82503c2306f610f2b5a3e51)

        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/DebugAdmin.java
        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/tools/TestDebugAdmin.java
        Show
        hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-Yarn-trunk #976 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/976/ ) HDFS-8577 . Avoid retrying to recover lease on a file which does not exist (Contributed by J.Andreina) (vinayakumarb: rev 2eae130ab9edd318c82503c2306f610f2b5a3e51) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/DebugAdmin.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/tools/TestDebugAdmin.java
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #244 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/244/)
        HDFS-8577. Avoid retrying to recover lease on a file which does not exist (Contributed by J.Andreina) (vinayakumarb: rev 2eae130ab9edd318c82503c2306f610f2b5a3e51)

        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/tools/TestDebugAdmin.java
        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/DebugAdmin.java
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #244 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/244/ ) HDFS-8577 . Avoid retrying to recover lease on a file which does not exist (Contributed by J.Andreina) (vinayakumarb: rev 2eae130ab9edd318c82503c2306f610f2b5a3e51) hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/tools/TestDebugAdmin.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/DebugAdmin.java
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #234 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/234/)
        HDFS-8577. Avoid retrying to recover lease on a file which does not exist (Contributed by J.Andreina) (vinayakumarb: rev 2eae130ab9edd318c82503c2306f610f2b5a3e51)

        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/DebugAdmin.java
        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/tools/TestDebugAdmin.java
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #234 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/234/ ) HDFS-8577 . Avoid retrying to recover lease on a file which does not exist (Contributed by J.Andreina) (vinayakumarb: rev 2eae130ab9edd318c82503c2306f610f2b5a3e51) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/DebugAdmin.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/tools/TestDebugAdmin.java
        Hide
        hudson Hudson added a comment -

        SUCCESS: Integrated in Hadoop-Hdfs-trunk #2173 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2173/)
        HDFS-8577. Avoid retrying to recover lease on a file which does not exist (Contributed by J.Andreina) (vinayakumarb: rev 2eae130ab9edd318c82503c2306f610f2b5a3e51)

        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/DebugAdmin.java
        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/tools/TestDebugAdmin.java
        Show
        hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-Hdfs-trunk #2173 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2173/ ) HDFS-8577 . Avoid retrying to recover lease on a file which does not exist (Contributed by J.Andreina) (vinayakumarb: rev 2eae130ab9edd318c82503c2306f610f2b5a3e51) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/DebugAdmin.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/tools/TestDebugAdmin.java
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Mapreduce-trunk #2192 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2192/)
        HDFS-8577. Avoid retrying to recover lease on a file which does not exist (Contributed by J.Andreina) (vinayakumarb: rev 2eae130ab9edd318c82503c2306f610f2b5a3e51)

        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/DebugAdmin.java
        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/tools/TestDebugAdmin.java
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk #2192 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2192/ ) HDFS-8577 . Avoid retrying to recover lease on a file which does not exist (Contributed by J.Andreina) (vinayakumarb: rev 2eae130ab9edd318c82503c2306f610f2b5a3e51) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/DebugAdmin.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/tools/TestDebugAdmin.java

          People

          • Assignee:
            andreina J.Andreina
            Reporter:
            andreina J.Andreina
          • Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development