Hadoop HDFS
  1. Hadoop HDFS
  2. HDFS-2803

Adding logging to LeaseRenewer for better lease expiration triage.

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Minor Minor
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.24.0, 0.23.1
    • Component/s: namenode
    • Labels:
    • Target Version/s:
    • Hadoop Flags:
      Reviewed

      Description

      It will be helpful to add some logging to LeaseRenewer when the daemon is terminated (Info level),
      and when the lease is renewed (Debug level). Since lacking logging, it is hard to know
      if a DFS client doesn't renew the lease because it hangs, or the lease renewer daemon is gone somehow.

      1. hdfs-2803.txt
        3 kB
        Jimmy Xiang

        Activity

        Hide
        Eli Collins added a comment -

        This will help debug HBase issues.

        Show
        Eli Collins added a comment - This will help debug HBase issues.
        Hide
        Todd Lipcon added a comment -

        I think DEBUG level is more appropriate for both. Otherwise any time we run a dfs command like dfs -put, we'll end up with an INFO level message.

        Show
        Todd Lipcon added a comment - I think DEBUG level is more appropriate for both. Otherwise any time we run a dfs command like dfs -put, we'll end up with an INFO level message.
        Hide
        Jimmy Xiang added a comment -

        Added some debug logging to track down HBase lease expiration issue

        Show
        Jimmy Xiang added a comment - Added some debug logging to track down HBase lease expiration issue
        Hide
        Todd Lipcon added a comment -

        +1 pending Hudson results.

        Show
        Todd Lipcon added a comment - +1 pending Hudson results.
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12511062/hdfs-2803.txt
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        -1 tests included. The patch doesn't appear to include any new or modified tests.
        Please justify why no new tests are needed for this patch.
        Also please list what manual steps were performed to verify this patch.

        -1 javadoc. The javadoc tool appears to have generated 21 warning messages.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 eclipse:eclipse. The patch built with eclipse:eclipse.

        -1 findbugs. The patch appears to introduce 1 new Findbugs (version 1.3.9) warnings.

        -1 release audit. The applied patch generated 1 release audit warnings (more than the trunk's current 0 warnings).

        +1 core tests. The patch passed unit tests in .

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/1781//testReport/
        Release audit warnings: https://builds.apache.org/job/PreCommit-HDFS-Build/1781//artifact/trunk/patchprocess/patchReleaseAuditProblems.txt
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HDFS-Build/1781//artifact/trunk/hadoop-hdfs-project/patchprocess/newPatchFindbugsWarningshadoop-hdfs.html
        Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/1781//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12511062/hdfs-2803.txt against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. -1 javadoc. The javadoc tool appears to have generated 21 warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 eclipse:eclipse. The patch built with eclipse:eclipse. -1 findbugs. The patch appears to introduce 1 new Findbugs (version 1.3.9) warnings. -1 release audit. The applied patch generated 1 release audit warnings (more than the trunk's current 0 warnings). +1 core tests. The patch passed unit tests in . +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/1781//testReport/ Release audit warnings: https://builds.apache.org/job/PreCommit-HDFS-Build/1781//artifact/trunk/patchprocess/patchReleaseAuditProblems.txt Findbugs warnings: https://builds.apache.org/job/PreCommit-HDFS-Build/1781//artifact/trunk/hadoop-hdfs-project/patchprocess/newPatchFindbugsWarningshadoop-hdfs.html Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/1781//console This message is automatically generated.
        Hide
        Todd Lipcon added a comment -

        The above hudson warnings look spurious. Committed to 23 and trunk. Thanks, Jimmy!

        Show
        Todd Lipcon added a comment - The above hudson warnings look spurious. Committed to 23 and trunk. Thanks, Jimmy!
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Common-0.23-Commit #389 (See https://builds.apache.org/job/Hadoop-Common-0.23-Commit/389/)
        HDFS-2803. Add logging to LeaseRenewer for better lease expiration debugging. Contributed by Jimmy Xiang.

        todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1233452
        Files :

        • /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/LeaseRenewer.java
        Show
        Hudson added a comment - Integrated in Hadoop-Common-0.23-Commit #389 (See https://builds.apache.org/job/Hadoop-Common-0.23-Commit/389/ ) HDFS-2803 . Add logging to LeaseRenewer for better lease expiration debugging. Contributed by Jimmy Xiang. todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1233452 Files : /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/LeaseRenewer.java
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Common-trunk-Commit #1556 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/1556/)
        HDFS-2803. Add logging to LeaseRenewer for better lease expiration debugging. Contributed by Jimmy Xiang.

        todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1233453
        Files :

        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/LeaseRenewer.java
        Show
        Hudson added a comment - Integrated in Hadoop-Common-trunk-Commit #1556 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/1556/ ) HDFS-2803 . Add logging to LeaseRenewer for better lease expiration debugging. Contributed by Jimmy Xiang. todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1233453 Files : /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/LeaseRenewer.java
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Hdfs-0.23-Commit #379 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Commit/379/)
        HDFS-2803. Add logging to LeaseRenewer for better lease expiration debugging. Contributed by Jimmy Xiang.

        todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1233452
        Files :

        • /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/LeaseRenewer.java
        Show
        Hudson added a comment - Integrated in Hadoop-Hdfs-0.23-Commit #379 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Commit/379/ ) HDFS-2803 . Add logging to LeaseRenewer for better lease expiration debugging. Contributed by Jimmy Xiang. todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1233452 Files : /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/LeaseRenewer.java
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Hdfs-trunk-Commit #1629 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/1629/)
        HDFS-2803. Add logging to LeaseRenewer for better lease expiration debugging. Contributed by Jimmy Xiang.

        todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1233453
        Files :

        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/LeaseRenewer.java
        Show
        Hudson added a comment - Integrated in Hadoop-Hdfs-trunk-Commit #1629 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/1629/ ) HDFS-2803 . Add logging to LeaseRenewer for better lease expiration debugging. Contributed by Jimmy Xiang. todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1233453 Files : /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/LeaseRenewer.java
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Mapreduce-trunk-Commit #1574 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/1574/)
        HDFS-2803. Add logging to LeaseRenewer for better lease expiration debugging. Contributed by Jimmy Xiang.

        todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1233453
        Files :

        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/LeaseRenewer.java
        Show
        Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk-Commit #1574 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/1574/ ) HDFS-2803 . Add logging to LeaseRenewer for better lease expiration debugging. Contributed by Jimmy Xiang. todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1233453 Files : /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/LeaseRenewer.java
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Mapreduce-0.23-Commit #402 (See https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Commit/402/)
        HDFS-2803. Add logging to LeaseRenewer for better lease expiration debugging. Contributed by Jimmy Xiang.

        todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1233452
        Files :

        • /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/LeaseRenewer.java
        Show
        Hudson added a comment - Integrated in Hadoop-Mapreduce-0.23-Commit #402 (See https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Commit/402/ ) HDFS-2803 . Add logging to LeaseRenewer for better lease expiration debugging. Contributed by Jimmy Xiang. todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1233452 Files : /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/LeaseRenewer.java
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Hdfs-trunk #931 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/931/)
        HDFS-2803. Add logging to LeaseRenewer for better lease expiration debugging. Contributed by Jimmy Xiang.

        todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1233453
        Files :

        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/LeaseRenewer.java
        Show
        Hudson added a comment - Integrated in Hadoop-Hdfs-trunk #931 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/931/ ) HDFS-2803 . Add logging to LeaseRenewer for better lease expiration debugging. Contributed by Jimmy Xiang. todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1233453 Files : /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/LeaseRenewer.java
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Hdfs-0.23-Build #144 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/144/)
        HDFS-2803. Add logging to LeaseRenewer for better lease expiration debugging. Contributed by Jimmy Xiang.

        todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1233452
        Files :

        • /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/LeaseRenewer.java
        Show
        Hudson added a comment - Integrated in Hadoop-Hdfs-0.23-Build #144 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/144/ ) HDFS-2803 . Add logging to LeaseRenewer for better lease expiration debugging. Contributed by Jimmy Xiang. todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1233452 Files : /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/LeaseRenewer.java
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Mapreduce-0.23-Build #166 (See https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Build/166/)
        HDFS-2803. Add logging to LeaseRenewer for better lease expiration debugging. Contributed by Jimmy Xiang.

        todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1233452
        Files :

        • /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/LeaseRenewer.java
        Show
        Hudson added a comment - Integrated in Hadoop-Mapreduce-0.23-Build #166 (See https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Build/166/ ) HDFS-2803 . Add logging to LeaseRenewer for better lease expiration debugging. Contributed by Jimmy Xiang. todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1233452 Files : /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/LeaseRenewer.java
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Mapreduce-trunk #964 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/964/)
        HDFS-2803. Add logging to LeaseRenewer for better lease expiration debugging. Contributed by Jimmy Xiang.

        todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1233453
        Files :

        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/LeaseRenewer.java
        Show
        Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk #964 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/964/ ) HDFS-2803 . Add logging to LeaseRenewer for better lease expiration debugging. Contributed by Jimmy Xiang. todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1233453 Files : /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/LeaseRenewer.java

          People

          • Assignee:
            Jimmy Xiang
            Reporter:
            Jimmy Xiang
          • Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development