Hadoop Common
  1. Hadoop Common
  2. HADOOP-6221

RPC Client operations cannot be interrupted

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Minor Minor
    • Resolution: Fixed
    • Affects Version/s: 0.21.0
    • Fix Version/s: 2.7.0
    • Component/s: ipc
    • Labels:
      None
    • Target Version/s:

      Description

      RPC.waitForProxy swallows any attempts to interrupt it while waiting for a proxy; this makes it hard to shutdown a service that you are starting; you have to wait for the timeouts.

      There are only 4-5 places in the code that use either of the two overloaded methods, removing the catch and changing the signature should not be too painful, unless anyone is using the method outside the hadoop codebase.

      1. HADOOP-6221-008.patch
        8 kB
        Steve Loughran
      2. HADOOP-6221-007.patch
        8 kB
        Steve Loughran
      3. HADOOP-6221.patch
        5 kB
        steve_l
      4. HADOOP-6221.patch
        7 kB
        steve_l
      5. HADOOP-6221.patch
        8 kB
        steve_l
      6. HADOOP-6221.patch
        8 kB
        steve_l
      7. HADOOP-6221.patch
        7 kB
        Steve Loughran
      8. HADOOP-6221.patch
        8 kB
        Vinayakumar B

        Issue Links

          Activity

          steve_l created issue -
          steve_l made changes -
          Field Original Value New Value
          Assignee Steve Loughran [ steve_l ]
          Hide
          steve_l added a comment -

          Tests show this problem is harder than it appears, even with improved Interrupt support during the sleep(), you can't interrupt the client while it is actually waiting to connect, that gets swallowed somewhere too. The test case shows this; a thread that tries to wait for 20s for a connection, but which is interrupted in another thread never sees the interrupt, instead after 20s it times out and throws the ConnectionRefused fault. What I'd like would be the interrupt acknowledged with the inner exception nested, and for the operation to fail as soon as the thread is interrupted.

          Show
          steve_l added a comment - Tests show this problem is harder than it appears, even with improved Interrupt support during the sleep(), you can't interrupt the client while it is actually waiting to connect, that gets swallowed somewhere too. The test case shows this; a thread that tries to wait for 20s for a connection, but which is interrupted in another thread never sees the interrupt, instead after 20s it times out and throws the ConnectionRefused fault. What I'd like would be the interrupt acknowledged with the inner exception nested, and for the operation to fail as soon as the thread is interrupted.
          steve_l made changes -
          Attachment HADOOP-6221.patch [ 12428916 ]
          Hide
          steve_l added a comment -

          Incidentally, this patch converts the InterruptedException to an InterruptedIOException, therefore the signature of the methods do not change, only their behaviour when interrupted. Programs not interrupting threads that were trying to connect to an unresponsive remote server will not see any change in method signature or behaviour.

          Show
          steve_l added a comment - Incidentally, this patch converts the InterruptedException to an InterruptedIOException, therefore the signature of the methods do not change, only their behaviour when interrupted. Programs not interrupting threads that were trying to connect to an unresponsive remote server will not see any change in method signature or behaviour.
          Hide
          steve_l added a comment -

          the interrupt is being swallowed in Client.call()), which blocks until a call completes or times out

               while (!call.done) {
                  try {
                    call.wait();                           // wait for the result
                  } catch (InterruptedException ie) {
                    // save the fact that we were interrupted
                    interrupted = true;
                  }
                }
          
                if (interrupted) {
                  // set the interrupt flag now that we are done waiting
                  Thread.currentThread().interrupt();
                }
          

          This means that the client thread is blocked until whichever thread is actually doing the RPC. Client.waitForWork() also swallows interrupts.

          This will get complex fast. I think a good tactic would be rather than trying to make the old RPC stack interruptible, focus on making Avro something that you can interrupt, so that going forward you can interrupt client programs trying to talk to unresponsive servers.

          Show
          steve_l added a comment - the interrupt is being swallowed in Client.call()) , which blocks until a call completes or times out while (!call.done) { try { call.wait(); // wait for the result } catch (InterruptedException ie) { // save the fact that we were interrupted interrupted = true ; } } if (interrupted) { // set the interrupt flag now that we are done waiting Thread .currentThread().interrupt(); } This means that the client thread is blocked until whichever thread is actually doing the RPC. Client.waitForWork() also swallows interrupts. This will get complex fast. I think a good tactic would be rather than trying to make the old RPC stack interruptible, focus on making Avro something that you can interrupt, so that going forward you can interrupt client programs trying to talk to unresponsive servers.
          Hide
          steve_l added a comment -

          Retitling this to cover the entire underlying problem: you cannot interrupt a Client if the far end isn't responding.

          Show
          steve_l added a comment - Retitling this to cover the entire underlying problem: you cannot interrupt a Client if the far end isn't responding.
          steve_l made changes -
          Summary RPC.WaitForProxy swallows InterruptedException Make it possible to interrupt RPC Client operations
          steve_l made changes -
          Link This issue relates to HADOOP-3457 [ HADOOP-3457 ]
          Hide
          steve_l added a comment -

          This issue benefits from the HADOOP-3457 patch, to the extent that to be able to interrupt RPC operations client-side properly, you need the HADOOP-3457 patch applied.

          I propose merging, as this patch provides the test

          Show
          steve_l added a comment - This issue benefits from the HADOOP-3457 patch, to the extent that to be able to interrupt RPC operations client-side properly, you need the HADOOP-3457 patch applied. I propose merging, as this patch provides the test
          Hide
          steve_l added a comment -

          Updated patch incorporating HADOOP-3457 with the tests passing. There are still no guarantees that the underlying IPC classes can be interrupted but now

          1. the sleeps between attempts to communicate can be interrupted
          2. InterruptedIOExceptions get passed up, so can be filtered on in client code (including tests)
          Show
          steve_l added a comment - Updated patch incorporating HADOOP-3457 with the tests passing. There are still no guarantees that the underlying IPC classes can be interrupted but now the sleeps between attempts to communicate can be interrupted InterruptedIOExceptions get passed up, so can be filtered on in client code (including tests)
          steve_l made changes -
          Attachment HADOOP-6221.patch [ 12431866 ]
          steve_l made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Affects Version/s 0.21.0 [ 12313563 ]
          Fix Version/s 0.22.0 [ 12314296 ]
          steve_l made changes -
          Link This issue incorporates HADOOP-3457 [ HADOOP-3457 ]
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12431866/HADOOP-6221.patch
          against trunk revision 904339.

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 5 new or modified tests.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 findbugs. The patch does not introduce any new Findbugs warnings.

          -1 release audit. The applied patch generated 2 release audit warnings (more than the trunk's current 1 warnings).

          +1 core tests. The patch passed core unit tests.

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/304/testReport/
          Release audit warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/304/artifact/trunk/patchprocess/releaseAuditDiffWarnings.txt
          Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/304/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
          Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/304/artifact/trunk/build/test/checkstyle-errors.html
          Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/304/console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12431866/HADOOP-6221.patch against trunk revision 904339. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 5 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. -1 release audit. The applied patch generated 2 release audit warnings (more than the trunk's current 1 warnings). +1 core tests. The patch passed core unit tests. +1 contrib tests. The patch passed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/304/testReport/ Release audit warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/304/artifact/trunk/patchprocess/releaseAuditDiffWarnings.txt Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/304/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/304/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/304/console This message is automatically generated.
          steve_l made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Hide
          steve_l added a comment -

          Updated patch with the ASF header in the test class, which is the probable cause of RAT being unhappy

          Show
          steve_l added a comment - Updated patch with the ASF header in the test class, which is the probable cause of RAT being unhappy
          steve_l made changes -
          Attachment HADOOP-6221.patch [ 12434950 ]
          steve_l made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12434950/HADOOP-6221.patch
          against trunk revision 906388.

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 5 new or modified tests.

          -1 patch. The patch command could not apply the patch.

          Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/330/console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12434950/HADOOP-6221.patch against trunk revision 906388. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 5 new or modified tests. -1 patch. The patch command could not apply the patch. Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/330/console This message is automatically generated.
          steve_l made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Hide
          steve_l added a comment -

          patch in sync w/ SVN_HEAD

          Show
          steve_l added a comment - patch in sync w/ SVN_HEAD
          steve_l made changes -
          Attachment HADOOP-6221.patch [ 12434971 ]
          steve_l made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Hide
          Hadoop QA added a comment -

          +1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12434971/HADOOP-6221.patch
          against trunk revision 906388.

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 5 new or modified tests.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 findbugs. The patch does not introduce any new Findbugs warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed core unit tests.

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/331/testReport/
          Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/331/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
          Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/331/artifact/trunk/build/test/checkstyle-errors.html
          Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/331/console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - +1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12434971/HADOOP-6221.patch against trunk revision 906388. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 5 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed core unit tests. +1 contrib tests. The patch passed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/331/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/331/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/331/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/331/console This message is automatically generated.
          Gavin made changes -
          Reporter Steve Loughran [ steve_l ] Steve Loughran [ stevel@apache.org ]
          Gavin made changes -
          Assignee Steve Loughran [ steve_l ] Steve Loughran [ stevel@apache.org ]
          Nigel Daley made changes -
          Fix Version/s 0.22.0 [ 12314296 ]
          Hide
          Steve Loughran added a comment -

          sync up with HEAD

          Show
          Steve Loughran added a comment - sync up with HEAD
          Steve Loughran made changes -
          Attachment HADOOP-6221.patch [ 12468404 ]
          Hide
          Hadoop QA added a comment -

          +1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12468404/HADOOP-6221.patch
          against trunk revision 1058881.

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 2 new or modified tests.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed core unit tests.

          +1 contrib tests. The patch passed contrib unit tests.

          +1 system test framework. The patch passed system test framework compile.

          Test results: https://hudson.apache.org/hudson/job/PreCommit-HADOOP-Build/182//testReport/
          Findbugs warnings: https://hudson.apache.org/hudson/job/PreCommit-HADOOP-Build/182//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
          Console output: https://hudson.apache.org/hudson/job/PreCommit-HADOOP-Build/182//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - +1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12468404/HADOOP-6221.patch against trunk revision 1058881. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 2 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed core unit tests. +1 contrib tests. The patch passed contrib unit tests. +1 system test framework. The patch passed system test framework compile. Test results: https://hudson.apache.org/hudson/job/PreCommit-HADOOP-Build/182//testReport/ Findbugs warnings: https://hudson.apache.org/hudson/job/PreCommit-HADOOP-Build/182//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Console output: https://hudson.apache.org/hudson/job/PreCommit-HADOOP-Build/182//console This message is automatically generated.
          Steve Loughran made changes -
          Hadoop Flags [Incompatible change]
          Priority Major [ 3 ] Minor [ 4 ]
          Hide
          Hadoop QA added a comment -

          +1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12468404/HADOOP-6221.patch
          against trunk revision 1071364.

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 2 new or modified tests.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed core unit tests.

          +1 contrib tests. The patch passed contrib unit tests.

          +1 system test framework. The patch passed system test framework compile.

          Test results: https://hudson.apache.org/hudson/job/PreCommit-HADOOP-Build/288//testReport/
          Findbugs warnings: https://hudson.apache.org/hudson/job/PreCommit-HADOOP-Build/288//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
          Console output: https://hudson.apache.org/hudson/job/PreCommit-HADOOP-Build/288//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - +1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12468404/HADOOP-6221.patch against trunk revision 1071364. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 2 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed core unit tests. +1 contrib tests. The patch passed contrib unit tests. +1 system test framework. The patch passed system test framework compile. Test results: https://hudson.apache.org/hudson/job/PreCommit-HADOOP-Build/288//testReport/ Findbugs warnings: https://hudson.apache.org/hudson/job/PreCommit-HADOOP-Build/288//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Console output: https://hudson.apache.org/hudson/job/PreCommit-HADOOP-Build/288//console This message is automatically generated.
          Hide
          Steve Loughran added a comment -

          reworking for the various svn structures

          Show
          Steve Loughran added a comment - reworking for the various svn structures
          Steve Loughran made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Steve Loughran made changes -
          Status Open [ 1 ] In Progress [ 3 ]
          Steve Loughran made changes -
          Link This issue is duplicated by HADOOP-8641 [ HADOOP-8641 ]
          Hide
          Vinayakumar B added a comment -

          Attached the patch for trunk version.

          Show
          Vinayakumar B added a comment - Attached the patch for trunk version.
          Vinayakumar B made changes -
          Attachment HADOOP-6221.patch [ 12539301 ]
          Steve Loughran made changes -
          Link This issue is related to HADOOP-9107 [ HADOOP-9107 ]
          Steve Loughran made changes -
          Status In Progress [ 3 ] Open [ 1 ]
          Steve Loughran made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Target Version/s 2.7.0 [ 12327583 ]
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12539301/HADOOP-6221.patch
          against trunk revision acf1e03.

          -1 patch. The patch command could not apply the patch.

          Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/5109//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12539301/HADOOP-6221.patch against trunk revision acf1e03. -1 patch . The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/5109//console This message is automatically generated.
          Steve Loughran made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Hide
          Steve Loughran added a comment -

          -007 patch. In sync with trunk.

          Show
          Steve Loughran added a comment - -007 patch. In sync with trunk.
          Steve Loughran made changes -
          Attachment HADOOP-6221-007.patch [ 12694440 ]
          Steve Loughran made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Steve Loughran made changes -
          Summary Make it possible to interrupt RPC Client operations RPC Client operations cannot be interrupted
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12694440/HADOOP-6221-007.patch
          against trunk revision 35f6496.

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 1 new or modified test files.

          -1 javac. The applied patch generated 1206 javac compiler warnings (more than the trunk's current 1204 warnings).

          +1 javadoc. There were no new javadoc warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 2.0.3) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed unit tests in hadoop-common-project/hadoop-common.

          Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/5475//testReport/
          Javac warnings: https://builds.apache.org/job/PreCommit-HADOOP-Build/5475//artifact/patchprocess/diffJavacWarnings.txt
          Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/5475//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12694440/HADOOP-6221-007.patch against trunk revision 35f6496. +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 1 new or modified test files. -1 javac . The applied patch generated 1206 javac compiler warnings (more than the trunk's current 1204 warnings). +1 javadoc . There were no new javadoc warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 2.0.3) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 core tests . The patch passed unit tests in hadoop-common-project/hadoop-common. Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/5475//testReport/ Javac warnings: https://builds.apache.org/job/PreCommit-HADOOP-Build/5475//artifact/patchprocess/diffJavacWarnings.txt Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/5475//console This message is automatically generated.
          Hide
          Steve Loughran added a comment -

          javac warnings about deprecation in junit. This patch is so old it isn't junit4 yet

          Show
          Steve Loughran added a comment - javac warnings about deprecation in junit. This patch is so old it isn't junit4 yet
          Steve Loughran made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Hide
          Steve Loughran added a comment -

          Patch -008. Modernised test and cleaner checks for underlying exception

          Show
          Steve Loughran added a comment - Patch -008. Modernised test and cleaner checks for underlying exception
          Steve Loughran made changes -
          Attachment HADOOP-6221-008.patch [ 12694520 ]
          Steve Loughran made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Hide
          Hadoop QA added a comment -

          +1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12694520/HADOOP-6221-008.patch
          against trunk revision 7b82c4a.

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 1 new or modified test files.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. There were no new javadoc warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 2.0.3) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed unit tests in hadoop-common-project/hadoop-common.

          Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/5479//testReport/
          Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/5479//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - +1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12694520/HADOOP-6221-008.patch against trunk revision 7b82c4a. +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 1 new or modified test files. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . There were no new javadoc warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 2.0.3) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 core tests . The patch passed unit tests in hadoop-common-project/hadoop-common. Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/5479//testReport/ Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/5479//console This message is automatically generated.
          Hide
          Chris Douglas added a comment -
          • At RPC.java:415, won't the following call to Thread.sleep() check if the interrupt flag is set? Is the explicit check required?
          • According to the docs, a thread blocked a a Selector should set the interrupt flag and return, rather than throwing InterruptedIOException, but I'm not sure the rest of the RPC stack is consistent. HDFS seems to use it this way, so this is arguably correct.

          AFAIK, the only way to forcibly interrupt some IO operations is to close the channel from outside the thread, but implementing that would be more invasive.

          +1 overall

          Show
          Chris Douglas added a comment - At RPC.java:415, won't the following call to Thread.sleep() check if the interrupt flag is set? Is the explicit check required? According to the docs , a thread blocked a a Selector should set the interrupt flag and return, rather than throwing InterruptedIOException , but I'm not sure the rest of the RPC stack is consistent. HDFS seems to use it this way, so this is arguably correct. AFAIK, the only way to forcibly interrupt some IO operations is to close the channel from outside the thread, but implementing that would be more invasive. +1 overall
          Steve Loughran made changes -
          Status Patch Available [ 10002 ] Resolved [ 5 ]
          Fix Version/s 2.7.0 [ 12327583 ]
          Resolution Fixed [ 1 ]
          Hide
          Hudson added a comment -

          FAILURE: Integrated in Hadoop-trunk-Commit #6935 (See https://builds.apache.org/job/Hadoop-trunk-Commit/6935/)
          HADOOP-6221 RPC Client operations cannot be interrupted (stevel) (stevel: rev 1f2b6956c2012a7d6ea7e7ba5116d3ad71c23d7e)

          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/ipc/TestRPCWaitForProxy.java
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/RPC.java
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/net/SocketIOWithTimeout.java
          • hadoop-common-project/hadoop-common/CHANGES.txt
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/Client.java
          Show
          Hudson added a comment - FAILURE: Integrated in Hadoop-trunk-Commit #6935 (See https://builds.apache.org/job/Hadoop-trunk-Commit/6935/ ) HADOOP-6221 RPC Client operations cannot be interrupted (stevel) (stevel: rev 1f2b6956c2012a7d6ea7e7ba5116d3ad71c23d7e) hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/ipc/TestRPCWaitForProxy.java hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/RPC.java hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/net/SocketIOWithTimeout.java hadoop-common-project/hadoop-common/CHANGES.txt hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/Client.java
          Hide
          Chris Douglas added a comment -

          Still curious about RPC.java:415. Is the check required?

          Show
          Chris Douglas added a comment - Still curious about RPC.java:415. Is the check required?
          Hide
          Hudson added a comment -

          FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #86 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/86/)
          HADOOP-6221 RPC Client operations cannot be interrupted (stevel) (stevel: rev 1f2b6956c2012a7d6ea7e7ba5116d3ad71c23d7e)

          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/RPC.java
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/net/SocketIOWithTimeout.java
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/Client.java
          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/ipc/TestRPCWaitForProxy.java
          • hadoop-common-project/hadoop-common/CHANGES.txt
          Show
          Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #86 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/86/ ) HADOOP-6221 RPC Client operations cannot be interrupted (stevel) (stevel: rev 1f2b6956c2012a7d6ea7e7ba5116d3ad71c23d7e) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/RPC.java hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/net/SocketIOWithTimeout.java hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/Client.java hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/ipc/TestRPCWaitForProxy.java hadoop-common-project/hadoop-common/CHANGES.txt
          Hide
          Hudson added a comment -

          FAILURE: Integrated in Hadoop-Yarn-trunk #820 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/820/)
          HADOOP-6221 RPC Client operations cannot be interrupted (stevel) (stevel: rev 1f2b6956c2012a7d6ea7e7ba5116d3ad71c23d7e)

          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/ipc/TestRPCWaitForProxy.java
          • hadoop-common-project/hadoop-common/CHANGES.txt
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/RPC.java
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/net/SocketIOWithTimeout.java
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/Client.java
          Show
          Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk #820 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/820/ ) HADOOP-6221 RPC Client operations cannot be interrupted (stevel) (stevel: rev 1f2b6956c2012a7d6ea7e7ba5116d3ad71c23d7e) hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/ipc/TestRPCWaitForProxy.java hadoop-common-project/hadoop-common/CHANGES.txt hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/RPC.java hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/net/SocketIOWithTimeout.java hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/Client.java
          Hide
          Hudson added a comment -

          FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #83 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/83/)
          HADOOP-6221 RPC Client operations cannot be interrupted (stevel) (stevel: rev 1f2b6956c2012a7d6ea7e7ba5116d3ad71c23d7e)

          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/ipc/TestRPCWaitForProxy.java
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/net/SocketIOWithTimeout.java
          • hadoop-common-project/hadoop-common/CHANGES.txt
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/RPC.java
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/Client.java
          Show
          Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #83 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/83/ ) HADOOP-6221 RPC Client operations cannot be interrupted (stevel) (stevel: rev 1f2b6956c2012a7d6ea7e7ba5116d3ad71c23d7e) hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/ipc/TestRPCWaitForProxy.java hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/net/SocketIOWithTimeout.java hadoop-common-project/hadoop-common/CHANGES.txt hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/RPC.java hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/Client.java
          Hide
          Hudson added a comment -

          FAILURE: Integrated in Hadoop-Mapreduce-trunk #2037 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2037/)
          HADOOP-6221 RPC Client operations cannot be interrupted (stevel) (stevel: rev 1f2b6956c2012a7d6ea7e7ba5116d3ad71c23d7e)

          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/net/SocketIOWithTimeout.java
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/RPC.java
          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/ipc/TestRPCWaitForProxy.java
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/Client.java
          • hadoop-common-project/hadoop-common/CHANGES.txt
          Show
          Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk #2037 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2037/ ) HADOOP-6221 RPC Client operations cannot be interrupted (stevel) (stevel: rev 1f2b6956c2012a7d6ea7e7ba5116d3ad71c23d7e) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/net/SocketIOWithTimeout.java hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/RPC.java hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/ipc/TestRPCWaitForProxy.java hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/Client.java hadoop-common-project/hadoop-common/CHANGES.txt
          Hide
          Hudson added a comment -

          SUCCESS: Integrated in Hadoop-Hdfs-trunk #2018 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2018/)
          HADOOP-6221 RPC Client operations cannot be interrupted (stevel) (stevel: rev 1f2b6956c2012a7d6ea7e7ba5116d3ad71c23d7e)

          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/RPC.java
          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/ipc/TestRPCWaitForProxy.java
          • hadoop-common-project/hadoop-common/CHANGES.txt
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/Client.java
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/net/SocketIOWithTimeout.java
          Show
          Hudson added a comment - SUCCESS: Integrated in Hadoop-Hdfs-trunk #2018 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2018/ ) HADOOP-6221 RPC Client operations cannot be interrupted (stevel) (stevel: rev 1f2b6956c2012a7d6ea7e7ba5116d3ad71c23d7e) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/RPC.java hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/ipc/TestRPCWaitForProxy.java hadoop-common-project/hadoop-common/CHANGES.txt hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/Client.java hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/net/SocketIOWithTimeout.java
          Hide
          Hudson added a comment -

          FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #87 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/87/)
          HADOOP-6221 RPC Client operations cannot be interrupted (stevel) (stevel: rev 1f2b6956c2012a7d6ea7e7ba5116d3ad71c23d7e)

          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/Client.java
          • hadoop-common-project/hadoop-common/CHANGES.txt
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/net/SocketIOWithTimeout.java
          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/ipc/TestRPCWaitForProxy.java
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/RPC.java
          Show
          Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #87 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/87/ ) HADOOP-6221 RPC Client operations cannot be interrupted (stevel) (stevel: rev 1f2b6956c2012a7d6ea7e7ba5116d3ad71c23d7e) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/Client.java hadoop-common-project/hadoop-common/CHANGES.txt hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/net/SocketIOWithTimeout.java hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/ipc/TestRPCWaitForProxy.java hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/RPC.java

            People

            • Assignee:
              Steve Loughran
              Reporter:
              Steve Loughran
            • Votes:
              0 Vote for this issue
              Watchers:
              11 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development