Hadoop HDFS
  1. Hadoop HDFS
  2. HDFS-3032

Lease renewer tries forever even if renewal is not possible

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.23.0, 0.24.0, 0.23.1
    • Fix Version/s: 2.0.0-alpha
    • Component/s: hdfs-client
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      When LeaseRenewer gets an IOException while attempting to renew for a client, it retries after sleeping 500ms. If the exception is caused by a condition that will never change, it keeps talking to the name node until the DFSClient object is closed or aborted. With the FileSystem cache, a DFSClient can stay alive for very long time. We've seen the cases in which node managers and long living jobs flooding name node with this type of calls.

      The current proposal is to abort the client when RemoteException is caught during renewal. LeaseRenewer already does abort on all clients when it sees a SocketTimeoutException.

      1. hdfs-3032.patch.txt
        6 kB
        Kihwal Lee
      2. hdfs-3032.patch.txt
        8 kB
        Kihwal Lee
      3. hdfs-3032.patch.txt
        7 kB
        Kihwal Lee
      4. h3032_20120205.patch
        13 kB
        Tsz Wo Nicholas Sze
      5. hdfs-3032.patch.txt
        13 kB
        Kihwal Lee

        Activity

        Hide
        Kihwal Lee added a comment -

        > The current proposal is to abort the client when RemoteException is caught during renewal.

        DFSClient.abort() calls abort() on all output streams and clientRunning is set to false. LeaseRenewer thread will stop trying renewing for the client. The thread will return if the client list becomes empty.

        One fatal case we observed involed an expired token. The token had expired and couldn't be renewed due to a different issue. This made any further RPC calls return InvalidToken from the name node, which in turn made LeaseRenewer repeat this behavior.

        Among the exceptions wrapped in RemoteException, SafeModeException might be okay for retry. All other exceptions from name node seem fatal for lease renewal.

        Show
        Kihwal Lee added a comment - > The current proposal is to abort the client when RemoteException is caught during renewal. DFSClient.abort() calls abort() on all output streams and clientRunning is set to false. LeaseRenewer thread will stop trying renewing for the client. The thread will return if the client list becomes empty. One fatal case we observed involed an expired token. The token had expired and couldn't be renewed due to a different issue. This made any further RPC calls return InvalidToken from the name node, which in turn made LeaseRenewer repeat this behavior. Among the exceptions wrapped in RemoteException, SafeModeException might be okay for retry. All other exceptions from name node seem fatal for lease renewal.
        Hide
        Kihwal Lee added a comment -

        The attached patch modifies DFSClient.renewLease() to handle RemoteException. Unless it's SafeModeException, it aborts all writes. A test case was added to make sure existing reads and new writes are unaffected and only the existing writes fail. Without this change, DFSClient may not be freed even if DistributedFileSystem disappears. This is because LeaseRenewer keeps a reference to the DFSClient until the map is cleared of all writes.

        Show
        Kihwal Lee added a comment - The attached patch modifies DFSClient.renewLease() to handle RemoteException. Unless it's SafeModeException, it aborts all writes. A test case was added to make sure existing reads and new writes are unaffected and only the existing writes fail. Without this change, DFSClient may not be freed even if DistributedFileSystem disappears. This is because LeaseRenewer keeps a reference to the DFSClient until the map is cleared of all writes.
        Hide
        Hadoop QA added a comment -

        +1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12516909/hdfs-3032.patch.txt
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 5 new or modified tests.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 eclipse:eclipse. The patch built with eclipse:eclipse.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 core tests. The patch passed unit tests in .

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/1942//testReport/
        Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/1942//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - +1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12516909/hdfs-3032.patch.txt against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 5 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 eclipse:eclipse. The patch built with eclipse:eclipse. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in . +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/1942//testReport/ Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/1942//console This message is automatically generated.
        Hide
        Tsz Wo Nicholas Sze added a comment -
        • It is better to try a few times, say 5, before exiting.
        • SafeModeException should be caught separatedly, i.e.
                try {
                  ...
                } catch (SafeModeException sme) {
                  // Let the renewer thread retry if in safe mode
                  throw sme;
                } catch (RemoteException re) {
                  ...
                }
          
        Show
        Tsz Wo Nicholas Sze added a comment - It is better to try a few times, say 5, before exiting. SafeModeException should be caught separatedly, i.e. try { ... } catch (SafeModeException sme) { // Let the renewer thread retry if in safe mode throw sme; } catch (RemoteException re) { ... }
        Hide
        Tsz Wo Nicholas Sze added a comment -

        > It is better to try a few times, say 5, before exiting.

        On a second thought, it should retry for at least 1 minute, which is the value of HdfsConstants.LEASE_SOFTLIMIT_PERIOD.

        Show
        Tsz Wo Nicholas Sze added a comment - > It is better to try a few times, say 5, before exiting. On a second thought, it should retry for at least 1 minute, which is the value of HdfsConstants.LEASE_SOFTLIMIT_PERIOD.
        Hide
        Amol Kekre added a comment -

        Nicholas, eta on when we can commit this?

        Show
        Amol Kekre added a comment - Nicholas, eta on when we can commit this?
        Hide
        Kihwal Lee added a comment -

        Thanks for the review Nicholas. In that case, we can add the retry limit in LeaseRenewer where IOException is caught and retried forever. After all, it doesn't make sense to renew after HdfsConstants.LEASE_SOFTLIMIT_PERIOD has passed.

        I will upload a new patch soon. I am adding a test case for the limited retry right now.

        Show
        Kihwal Lee added a comment - Thanks for the review Nicholas. In that case, we can add the retry limit in LeaseRenewer where IOException is caught and retried forever. After all, it doesn't make sense to renew after HdfsConstants.LEASE_SOFTLIMIT_PERIOD has passed. I will upload a new patch soon. I am adding a test case for the limited retry right now.
        Hide
        Tsz Wo Nicholas Sze added a comment -

        Hi Kihwal, I think we may simply change LeaseRenewer to retry up to a time limit as below. I made the limit to 2*HdfsConstants.LEASE_SOFTLIMIT_PERIOD since HdfsConstants.LEASE_SOFTLIMIT_PERIOD is only one minute. What do you think?

        Index: hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/LeaseRenewer.java
        ===================================================================
        --- hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/LeaseRenewer.java	(revision 1297199)
        +++ hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/LeaseRenewer.java	(working copy)
        @@ -430,7 +430,8 @@
             for(long lastRenewed = System.currentTimeMillis();
                 clientsRunning() && !Thread.interrupted();
                 Thread.sleep(getSleepPeriod())) {
        -      if (System.currentTimeMillis() - lastRenewed >= getRenewalTime()) {
        +      final long diff = System.currentTimeMillis() - lastRenewed;
        +      if (diff >= getRenewalTime()) {
                 try {
                   renew();
                   if (LOG.isDebugEnabled()) {
        @@ -438,19 +439,19 @@
                         + " with renew id " + id + " executed");
                   }
                   lastRenewed = System.currentTimeMillis();
        -        } catch (SocketTimeoutException ie) {
        +        } catch (IOException ie) {
        +          final boolean abort = diff > 2*HdfsConstants.LEASE_SOFTLIMIT_PERIOD;
                   LOG.warn("Failed to renew lease for " + clientsString() + " for "
        -              + (getRenewalTime()/1000) + " seconds.  Aborting ...", ie);
        -          synchronized (this) {
        -            for(DFSClient c : dfsclients) {
        -              c.abort();
        +              + (getRenewalTime()/1000) + " seconds.  "
        +              + (abort? "Aborting ...": "Will retry shortly ..."), ie);
        +          if (abort) {
        +            synchronized (this) {
        +              for(DFSClient c : dfsclients) {
        +                c.abort();
        +              }
                     }
        +            break;
                   }
        -          break;
        -        } catch (IOException ie) {
        -          LOG.warn("Failed to renew lease for " + clientsString() + " for "
        -              + (getRenewalTime()/1000) + " seconds.  Will retry shortly ...",
        -              ie);
                 }
               }
        
        Show
        Tsz Wo Nicholas Sze added a comment - Hi Kihwal, I think we may simply change LeaseRenewer to retry up to a time limit as below. I made the limit to 2*HdfsConstants.LEASE_SOFTLIMIT_PERIOD since HdfsConstants.LEASE_SOFTLIMIT_PERIOD is only one minute. What do you think? Index: hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/LeaseRenewer.java =================================================================== --- hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/LeaseRenewer.java (revision 1297199) +++ hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/LeaseRenewer.java (working copy) @@ -430,7 +430,8 @@ for ( long lastRenewed = System .currentTimeMillis(); clientsRunning() && ! Thread .interrupted(); Thread .sleep(getSleepPeriod())) { - if ( System .currentTimeMillis() - lastRenewed >= getRenewalTime()) { + final long diff = System .currentTimeMillis() - lastRenewed; + if (diff >= getRenewalTime()) { try { renew(); if (LOG.isDebugEnabled()) { @@ -438,19 +439,19 @@ + " with renew id " + id + " executed" ); } lastRenewed = System .currentTimeMillis(); - } catch (SocketTimeoutException ie) { + } catch (IOException ie) { + final boolean abort = diff > 2*HdfsConstants.LEASE_SOFTLIMIT_PERIOD; LOG.warn( "Failed to renew lease for " + clientsString() + " for " - + (getRenewalTime()/1000) + " seconds. Aborting ..." , ie); - synchronized ( this ) { - for (DFSClient c : dfsclients) { - c.abort(); + + (getRenewalTime()/1000) + " seconds. " + + (abort? "Aborting ..." : "Will retry shortly ..." ), ie); + if (abort) { + synchronized ( this ) { + for (DFSClient c : dfsclients) { + c.abort(); + } } + break ; } - break ; - } catch (IOException ie) { - LOG.warn( "Failed to renew lease for " + clientsString() + " for " - + (getRenewalTime()/1000) + " seconds. Will retry shortly ..." , - ie); } }
        Hide
        Tsz Wo Nicholas Sze added a comment -

        Hi Amol, the current patch won't work since the lease renewer stops right after the first exception but the exception may be caused by some temporary network problems. I hope that we can fix this today.

        Show
        Tsz Wo Nicholas Sze added a comment - Hi Amol, the current patch won't work since the lease renewer stops right after the first exception but the exception may be caused by some temporary network problems. I hope that we can fix this today.
        Hide
        Amol Kekre added a comment -

        Agreed, Kihwal is on it. Thks for the update

        Show
        Amol Kekre added a comment - Agreed, Kihwal is on it. Thks for the update
        Hide
        Tsz Wo Nicholas Sze added a comment -

        The line below

        +              + (getRenewalTime()/1000) + " seconds.  "
        

        should be

        +              + (diff/1000) + " seconds.  "
        

        It is an existing bug in the warning message.

        Show
        Tsz Wo Nicholas Sze added a comment - The line below + + (getRenewalTime()/1000) + " seconds. " should be + + (diff/1000) + " seconds. " It is an existing bug in the warning message.
        Hide
        Kihwal Lee added a comment -

        Hi Kihwal, I think we may simply change LeaseRenewer to retry up to a time limit as below. I made the limit to 2*HdfsConstants.LEASE_SOFTLIMIT_PERIOD since HdfsConstants.LEASE_SOFTLIMIT_PERIOD is only one minute. What do you think?

        I walked down that path too, but soon realized that it would abort all clients. So my alternative aproach is to do it at individual client. Since the renewal is attempted every LEASE_SOFTLIMIT_PERIOD/2, we can be sure that leases are expired after LEASE_SOFTLIMIT_PERIOD.

        I will upload my patch in a moment.

        Show
        Kihwal Lee added a comment - Hi Kihwal, I think we may simply change LeaseRenewer to retry up to a time limit as below. I made the limit to 2*HdfsConstants.LEASE_SOFTLIMIT_PERIOD since HdfsConstants.LEASE_SOFTLIMIT_PERIOD is only one minute. What do you think? I walked down that path too, but soon realized that it would abort all clients. So my alternative aproach is to do it at individual client. Since the renewal is attempted every LEASE_SOFTLIMIT_PERIOD/2, we can be sure that leases are expired after LEASE_SOFTLIMIT_PERIOD. I will upload my patch in a moment.
        Hide
        Kihwal Lee added a comment -

        A new patch add a check for lease expiry in DFSClient.renewLease().

        Show
        Kihwal Lee added a comment - A new patch add a check for lease expiry in DFSClient.renewLease().
        Hide
        Kihwal Lee added a comment -

        Since the renewal is attempted every LEASE_SOFTLIMIT_PERIOD/2, we can be sure that leases are expired after LEASE_SOFTLIMIT_PERIOD

        This statement is ambiguous. What I meant to say is that checking against LEASE_SOFTLIMIT_PERIOD should be sufficient because there will only be a few milliseconds of error. By the time the the renewer thread retries, the lease will have expired.

        Show
        Kihwal Lee added a comment - Since the renewal is attempted every LEASE_SOFTLIMIT_PERIOD/2, we can be sure that leases are expired after LEASE_SOFTLIMIT_PERIOD This statement is ambiguous. What I meant to say is that checking against LEASE_SOFTLIMIT_PERIOD should be sufficient because there will only be a few milliseconds of error. By the time the the renewer thread retries, the lease will have expired.
        Hide
        Tsz Wo Nicholas Sze added a comment -

        I think it should retry even for RemoteException since RemoteException only indicates the exception originated from the server side but it may still a transient problem, e.g. rpc server is very busy and start dropping connection.

        Show
        Tsz Wo Nicholas Sze added a comment - I think it should retry even for RemoteException since RemoteException only indicates the exception originated from the server side but it may still a transient problem, e.g. rpc server is very busy and start dropping connection.
        Hide
        Hadoop QA added a comment -

        +1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12517132/hdfs-3032.patch.txt
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 5 new or modified tests.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 eclipse:eclipse. The patch built with eclipse:eclipse.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 core tests. The patch passed unit tests in .

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/1954//testReport/
        Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/1954//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - +1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12517132/hdfs-3032.patch.txt against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 5 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 eclipse:eclipse. The patch built with eclipse:eclipse. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in . +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/1954//testReport/ Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/1954//console This message is automatically generated.
        Hide
        Kihwal Lee added a comment -

        I think it should retry even for RemoteException since RemoteException only indicates the exception originated from the server side but it may still a transient problem, e.g. rpc server is very busy and start dropping connection.

        There are RemoteExceptions caused by permanent failure conditions. But since we are limiting retries, I think it is safe to let the renewer retry as you suggested.

        Show
        Kihwal Lee added a comment - I think it should retry even for RemoteException since RemoteException only indicates the exception originated from the server side but it may still a transient problem, e.g. rpc server is very busy and start dropping connection. There are RemoteExceptions caused by permanent failure conditions. But since we are limiting retries, I think it is safe to let the renewer retry as you suggested.
        Hide
        Kihwal Lee added a comment -

        The new patch makes LeaseRenewer retry on all IOException types including SafeModeException and RemoteException.

        Show
        Kihwal Lee added a comment - The new patch makes LeaseRenewer retry on all IOException types including SafeModeException and RemoteException.
        Hide
        Tsz Wo Nicholas Sze added a comment -

        h3032_20120205.patch: based on Kihwal's patch and

        • removes catching RemoteException,
        • fixes some warnings messages.
        Show
        Tsz Wo Nicholas Sze added a comment - h3032_20120205.patch: based on Kihwal's patch and removes catching RemoteException, fixes some warnings messages.
        Hide
        Kihwal Lee added a comment -

        Hold on. The test needs an update.

        Show
        Kihwal Lee added a comment - Hold on. The test needs an update.
        Hide
        Kihwal Lee added a comment -

        I've updated the test and verified it working on the version of trunk w/o HA. The patch applies to the current version of trunk, but I had hard time building it.

        Show
        Kihwal Lee added a comment - I've updated the test and verified it working on the version of trunk w/o HA. The patch applies to the current version of trunk, but I had hard time building it.
        Hide
        Tsz Wo Nicholas Sze added a comment -

        Hi Kihwal, the new test failed.

        Running org.apache.hadoop.hdfs.TestLease
        Tests run: 3, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 7.708 sec <<< FAILURE!
        
        Results :
        
        Tests in error: 
          testLeaseAbort(org.apache.hadoop.hdfs.TestLease): Your token is worthless
        
        Show
        Tsz Wo Nicholas Sze added a comment - Hi Kihwal, the new test failed. Running org.apache.hadoop.hdfs.TestLease Tests run: 3, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 7.708 sec <<< FAILURE! Results : Tests in error: testLeaseAbort(org.apache.hadoop.hdfs.TestLease): Your token is worthless
        Hide
        Kihwal Lee added a comment -

        I just applied the patch against trunk rev 1297315 and did "mvn install -Dtest=TestLease" five times. They all passed.

        Show
        Kihwal Lee added a comment - I just applied the patch against trunk rev 1297315 and did "mvn install -Dtest=TestLease" five times. They all passed.
        Hide
        Tsz Wo Nicholas Sze added a comment -

        I just have tried again and it passed. My fault.

        +1 patch look good. Thanks for working hard on this!

        Show
        Tsz Wo Nicholas Sze added a comment - I just have tried again and it passed. My fault. +1 patch look good. Thanks for working hard on this!
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12517167/hdfs-3032.patch.txt
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 5 new or modified tests.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 eclipse:eclipse. The patch built with eclipse:eclipse.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        -1 core tests. The patch failed these unit tests:
        org.apache.hadoop.hdfs.server.namenode.TestListCorruptFileBlocks

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/1957//testReport/
        Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/1957//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12517167/hdfs-3032.patch.txt against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 5 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 eclipse:eclipse. The patch built with eclipse:eclipse. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: org.apache.hadoop.hdfs.server.namenode.TestListCorruptFileBlocks +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/1957//testReport/ Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/1957//console This message is automatically generated.
        Hide
        Kihwal Lee added a comment -
        -1 core tests. The patch failed these unit tests:
        org.apache.hadoop.hdfs.server.namenode.TestListCorruptFileBlocks
        

        I cannot reproduce the failure. I successfully ran the test case with and without the patch.

        Show
        Kihwal Lee added a comment - -1 core tests. The patch failed these unit tests: org.apache.hadoop.hdfs.server.namenode.TestListCorruptFileBlocks I cannot reproduce the failure. I successfully ran the test case with and without the patch.
        Hide
        Kihwal Lee added a comment -

        It seems BlockPoolSliceScanner had trouble running on the build machine. Even if it was reproducible, it is not related to this patch. I will keep an eye on it and if I see more failures of this case, I will file a jira.

        Show
        Kihwal Lee added a comment - It seems BlockPoolSliceScanner had trouble running on the build machine. Even if it was reproducible, it is not related to this patch. I will keep an eye on it and if I see more failures of this case, I will file a jira.
        Hide
        Tsz Wo Nicholas Sze added a comment -

        > ... it is not related to this patch. ...

        I agree that it is not related.

        Show
        Tsz Wo Nicholas Sze added a comment - > ... it is not related to this patch. ... I agree that it is not related.
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Hdfs-trunk-Commit #1914 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/1914/)
        HDFS-3032. Change DFSClient.renewLease() so that it only retries up to the lease soft-limit. Contributed by Kihwal Lee (Revision 1297328)

        Result = SUCCESS
        szetszwo : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1297328
        Files :

        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSClient.java
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/LeaseRenewer.java
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestLease.java
        Show
        Hudson added a comment - Integrated in Hadoop-Hdfs-trunk-Commit #1914 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/1914/ ) HDFS-3032 . Change DFSClient.renewLease() so that it only retries up to the lease soft-limit. Contributed by Kihwal Lee (Revision 1297328) Result = SUCCESS szetszwo : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1297328 Files : /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSClient.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/LeaseRenewer.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestLease.java
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Common-trunk-Commit #1840 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/1840/)
        HDFS-3032. Change DFSClient.renewLease() so that it only retries up to the lease soft-limit. Contributed by Kihwal Lee (Revision 1297328)

        Result = SUCCESS
        szetszwo : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1297328
        Files :

        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSClient.java
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/LeaseRenewer.java
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestLease.java
        Show
        Hudson added a comment - Integrated in Hadoop-Common-trunk-Commit #1840 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/1840/ ) HDFS-3032 . Change DFSClient.renewLease() so that it only retries up to the lease soft-limit. Contributed by Kihwal Lee (Revision 1297328) Result = SUCCESS szetszwo : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1297328 Files : /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSClient.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/LeaseRenewer.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestLease.java
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Mapreduce-trunk-Commit #1847 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/1847/)
        HDFS-3032. Change DFSClient.renewLease() so that it only retries up to the lease soft-limit. Contributed by Kihwal Lee (Revision 1297328)

        Result = FAILURE
        szetszwo : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1297328
        Files :

        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSClient.java
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/LeaseRenewer.java
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestLease.java
        Show
        Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk-Commit #1847 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/1847/ ) HDFS-3032 . Change DFSClient.renewLease() so that it only retries up to the lease soft-limit. Contributed by Kihwal Lee (Revision 1297328) Result = FAILURE szetszwo : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1297328 Files : /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSClient.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/LeaseRenewer.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestLease.java
        Hide
        Tsz Wo Nicholas Sze added a comment -

        I have committed this to trunk, 0.23 and 0.23.2. Thanks Kihwal!

        Show
        Tsz Wo Nicholas Sze added a comment - I have committed this to trunk, 0.23 and 0.23.2. Thanks Kihwal!
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Hdfs-0.23-Commit #634 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Commit/634/)
        svn merge -c 1297328 from trunk for HDFS-3032. (Revision 1297334)

        Result = SUCCESS
        szetszwo : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1297334
        Files :

        • /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs
        • /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java
        • /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSClient.java
        • /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/LeaseRenewer.java
        • /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestLease.java
        Show
        Hudson added a comment - Integrated in Hadoop-Hdfs-0.23-Commit #634 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Commit/634/ ) svn merge -c 1297328 from trunk for HDFS-3032 . (Revision 1297334) Result = SUCCESS szetszwo : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1297334 Files : /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSClient.java /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/LeaseRenewer.java /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestLease.java
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Common-0.23-Commit #644 (See https://builds.apache.org/job/Hadoop-Common-0.23-Commit/644/)
        svn merge -c 1297328 from trunk for HDFS-3032. (Revision 1297334)

        Result = SUCCESS
        szetszwo : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1297334
        Files :

        • /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs
        • /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java
        • /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSClient.java
        • /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/LeaseRenewer.java
        • /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestLease.java
        Show
        Hudson added a comment - Integrated in Hadoop-Common-0.23-Commit #644 (See https://builds.apache.org/job/Hadoop-Common-0.23-Commit/644/ ) svn merge -c 1297328 from trunk for HDFS-3032 . (Revision 1297334) Result = SUCCESS szetszwo : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1297334 Files : /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSClient.java /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/LeaseRenewer.java /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestLease.java
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Mapreduce-0.23-Commit #645 (See https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Commit/645/)
        svn merge -c 1297328 from trunk for HDFS-3032. (Revision 1297334)

        Result = FAILURE
        szetszwo : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1297334
        Files :

        • /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs
        • /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java
        • /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSClient.java
        • /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/LeaseRenewer.java
        • /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestLease.java
        Show
        Hudson added a comment - Integrated in Hadoop-Mapreduce-0.23-Commit #645 (See https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Commit/645/ ) svn merge -c 1297328 from trunk for HDFS-3032 . (Revision 1297334) Result = FAILURE szetszwo : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1297334 Files : /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSClient.java /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/LeaseRenewer.java /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestLease.java
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Hdfs-0.23-Build #189 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/189/)
        svn merge -c 1297328 from trunk for HDFS-3032. (Revision 1297334)

        Result = SUCCESS
        szetszwo : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1297334
        Files :

        • /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs
        • /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java
        • /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSClient.java
        • /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/LeaseRenewer.java
        • /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestLease.java
        Show
        Hudson added a comment - Integrated in Hadoop-Hdfs-0.23-Build #189 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/189/ ) svn merge -c 1297328 from trunk for HDFS-3032 . (Revision 1297334) Result = SUCCESS szetszwo : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1297334 Files : /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSClient.java /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/LeaseRenewer.java /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestLease.java
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Hdfs-trunk #976 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/976/)
        HDFS-3032. Change DFSClient.renewLease() so that it only retries up to the lease soft-limit. Contributed by Kihwal Lee (Revision 1297328)

        Result = SUCCESS
        szetszwo : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1297328
        Files :

        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSClient.java
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/LeaseRenewer.java
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestLease.java
        Show
        Hudson added a comment - Integrated in Hadoop-Hdfs-trunk #976 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/976/ ) HDFS-3032 . Change DFSClient.renewLease() so that it only retries up to the lease soft-limit. Contributed by Kihwal Lee (Revision 1297328) Result = SUCCESS szetszwo : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1297328 Files : /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSClient.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/LeaseRenewer.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestLease.java
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Mapreduce-0.23-Build #217 (See https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Build/217/)
        svn merge -c 1297328 from trunk for HDFS-3032. (Revision 1297334)

        Result = FAILURE
        szetszwo : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1297334
        Files :

        • /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs
        • /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java
        • /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSClient.java
        • /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/LeaseRenewer.java
        • /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestLease.java
        Show
        Hudson added a comment - Integrated in Hadoop-Mapreduce-0.23-Build #217 (See https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Build/217/ ) svn merge -c 1297328 from trunk for HDFS-3032 . (Revision 1297334) Result = FAILURE szetszwo : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1297334 Files : /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSClient.java /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/LeaseRenewer.java /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestLease.java
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Mapreduce-trunk #1011 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1011/)
        HDFS-3032. Change DFSClient.renewLease() so that it only retries up to the lease soft-limit. Contributed by Kihwal Lee (Revision 1297328)

        Result = FAILURE
        szetszwo : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1297328
        Files :

        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSClient.java
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/LeaseRenewer.java
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestLease.java
        Show
        Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk #1011 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1011/ ) HDFS-3032 . Change DFSClient.renewLease() so that it only retries up to the lease soft-limit. Contributed by Kihwal Lee (Revision 1297328) Result = FAILURE szetszwo : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1297328 Files : /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSClient.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/LeaseRenewer.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestLease.java

          People

          • Assignee:
            Kihwal Lee
            Reporter:
            Kihwal Lee
          • Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development