Hadoop Common
  1. Hadoop Common
  2. HADOOP-7140

IPC Reader threads do not stop when server stops

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Critical Critical
    • Resolution: Fixed
    • Affects Version/s: 0.22.0, 1-win, 1.3.0
    • Fix Version/s: 0.22.0, 1.3.0
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      After HADOOP-6713, the new IPC "Reader" threads are not properly stopped when the server shuts down. One repercussion of this is that conditions that are supposed to shut down a daemon no longer work (eg the TT doesn't shut itself down if it detects an incompatible build version)

      1. hadoop-7140.txt
        5 kB
        Todd Lipcon
      2. hadoop-7140.txt
        5 kB
        Todd Lipcon
      3. HADOOP-7140.branch-1-win.patch
        5 kB
        Ivan Mitic
      4. HADOOP-7140.branch-1.patch
        5 kB
        Ivan Mitic

        Issue Links

          Activity

          Transition Time In Source Status Execution Times Last Executer Last Execution Date
          Open Open Patch Available Patch Available
          15m 9s 1 Todd Lipcon 14/Feb/11 03:33
          Patch Available Patch Available Resolved Resolved
          1d 3h 59m 1 Todd Lipcon 15/Feb/11 07:33
          Resolved Resolved Closed Closed
          299d 22h 45m 1 Konstantin Shvachko 12/Dec/11 06:18
          Ivan Mitic made changes -
          Fix Version/s 1.3.0 [ 12324327 ]
          Hide
          Ivan Mitic added a comment -

          Patch committed to branch-1.

          Show
          Ivan Mitic added a comment - Patch committed to branch-1.
          Hide
          Ivan Mitic added a comment -

          Thanks Chris! Will commit to branch-1 shortly.

          Since we'll be merging branch-1 changes into branch-1-win, I'll wait for the merge to pick up this change.

          Show
          Ivan Mitic added a comment - Thanks Chris! Will commit to branch-1 shortly. Since we'll be merging branch-1 changes into branch-1-win, I'll wait for the merge to pick up this change.
          Hide
          Chris Nauroth added a comment -

          +1 for the backport patches. I ran TestRPC successfully with the backport.

          Show
          Chris Nauroth added a comment - +1 for the backport patches. I ran TestRPC successfully with the backport.
          Ivan Mitic made changes -
          Attachment HADOOP-7140.branch-1.patch [ 12588927 ]
          Attachment HADOOP-7140.branch-1-win.patch [ 12588928 ]
          Hide
          Ivan Mitic added a comment -

          Attaching the branch-1 and branch-1-win compatible patches.

          Show
          Ivan Mitic added a comment - Attaching the branch-1 and branch-1-win compatible patches.
          Ivan Mitic made changes -
          Affects Version/s 1-win [ 12320361 ]
          Affects Version/s 1.3.0 [ 12324327 ]
          Hide
          Ivan Mitic added a comment -

          Looks like we merged HADOOP-6713 to branch-1 but missed this Jira. I run into an issue where TT didn't shut itself down because of running IPC reader threads.

          Reopening for backport to branch-1 and branch-1-win.

          Show
          Ivan Mitic added a comment - Looks like we merged HADOOP-6713 to branch-1 but missed this Jira. I run into an issue where TT didn't shut itself down because of running IPC reader threads. Reopening for backport to branch-1 and branch-1-win.
          Konstantin Shvachko made changes -
          Status Resolved [ 5 ] Closed [ 6 ]
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Common-trunk #607 (See https://hudson.apache.org/hudson/job/Hadoop-Common-trunk/607/)
          HADOOP-7140. IPC Reader threads do not stop when server stops. Contributed by Todd Lipcon.

          Show
          Hudson added a comment - Integrated in Hadoop-Common-trunk #607 (See https://hudson.apache.org/hudson/job/Hadoop-Common-trunk/607/ ) HADOOP-7140 . IPC Reader threads do not stop when server stops. Contributed by Todd Lipcon.
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Common-22-branch #25 (See https://hudson.apache.org/hudson/job/Hadoop-Common-22-branch/25/)
          HADOOP-7140. IPC Reader threads do not stop when server stops. Contributed by Todd Lipcon.

          Show
          Hudson added a comment - Integrated in Hadoop-Common-22-branch #25 (See https://hudson.apache.org/hudson/job/Hadoop-Common-22-branch/25/ ) HADOOP-7140 . IPC Reader threads do not stop when server stops. Contributed by Todd Lipcon.
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Common-trunk-Commit #503 (See https://hudson.apache.org/hudson/job/Hadoop-Common-trunk-Commit/503/)
          HADOOP-7140. IPC Reader threads do not stop when server stops. Contributed by Todd Lipcon.

          Show
          Hudson added a comment - Integrated in Hadoop-Common-trunk-Commit #503 (See https://hudson.apache.org/hudson/job/Hadoop-Common-trunk-Commit/503/ ) HADOOP-7140 . IPC Reader threads do not stop when server stops. Contributed by Todd Lipcon.
          Todd Lipcon made changes -
          Status Patch Available [ 10002 ] Resolved [ 5 ]
          Hadoop Flags [Reviewed]
          Resolution Fixed [ 1 ]
          Hide
          Todd Lipcon added a comment -

          Committed to trunk and 22, thanks for reviewing, Tom.

          Show
          Todd Lipcon added a comment - Committed to trunk and 22, thanks for reviewing, Tom.
          Hide
          Tom White added a comment -

          +1

          Show
          Tom White added a comment - +1
          Hide
          Hadoop QA added a comment -

          +1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12471020/hadoop-7140.txt
          against trunk revision 1070021.

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 3 new or modified tests.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed core unit tests.

          +1 contrib tests. The patch passed contrib unit tests.

          +1 system test framework. The patch passed system test framework compile.

          Test results: https://hudson.apache.org/hudson/job/PreCommit-HADOOP-Build/235//testReport/
          Findbugs warnings: https://hudson.apache.org/hudson/job/PreCommit-HADOOP-Build/235//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
          Console output: https://hudson.apache.org/hudson/job/PreCommit-HADOOP-Build/235//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - +1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12471020/hadoop-7140.txt against trunk revision 1070021. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed core unit tests. +1 contrib tests. The patch passed contrib unit tests. +1 system test framework. The patch passed system test framework compile. Test results: https://hudson.apache.org/hudson/job/PreCommit-HADOOP-Build/235//testReport/ Findbugs warnings: https://hudson.apache.org/hudson/job/PreCommit-HADOOP-Build/235//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Console output: https://hudson.apache.org/hudson/job/PreCommit-HADOOP-Build/235//console This message is automatically generated.
          Todd Lipcon made changes -
          Attachment hadoop-7140.txt [ 12471020 ]
          Hide
          Todd Lipcon added a comment -

          New rev of the patch:

          • improves test assertions as suggested by Tom
          • adds a try{}finally{} in the test so as to avoid causing problems if the test fails

          Tom and I discussed his idea about interrupts offline - we decided that using interrupt() is a bit harsh since we don't want to catch a reader in the middle of a readAndProcess(). Using selector.wakeup is cleaner.

          Show
          Todd Lipcon added a comment - New rev of the patch: improves test assertions as suggested by Tom adds a try{}finally{} in the test so as to avoid causing problems if the test fails Tom and I discussed his idea about interrupts offline - we decided that using interrupt() is a bit harsh since we don't want to catch a reader in the middle of a readAndProcess(). Using selector.wakeup is cleaner.
          Hide
          Tom White added a comment -

          This looks fine, but I wonder if it would be simpler to act on the InterruptedException in Reader (rather than just log it as the code currently does) so the thread exits when the pool is shutdown?

          Minor nit: add messages to the test assertions so that the case of zero running threads can be more easily distinguished for the before and after case.

          Show
          Tom White added a comment - This looks fine, but I wonder if it would be simpler to act on the InterruptedException in Reader (rather than just log it as the code currently does) so the thread exits when the pool is shutdown? Minor nit: add messages to the test assertions so that the case of zero running threads can be more easily distinguished for the before and after case.
          Hide
          Todd Lipcon added a comment -

          This patch also fixes HADOOP-7132, since it was simple to fix while I was at it.

          Show
          Todd Lipcon added a comment - This patch also fixes HADOOP-7132 , since it was simple to fix while I was at it.
          Todd Lipcon made changes -
          Link This issue incorporates HADOOP-7132 [ HADOOP-7132 ]
          Hide
          Hadoop QA added a comment -

          +1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12470978/hadoop-7140.txt
          against trunk revision 1070021.

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 3 new or modified tests.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed core unit tests.

          +1 contrib tests. The patch passed contrib unit tests.

          +1 system test framework. The patch passed system test framework compile.

          Test results: https://hudson.apache.org/hudson/job/PreCommit-HADOOP-Build/234//testReport/
          Findbugs warnings: https://hudson.apache.org/hudson/job/PreCommit-HADOOP-Build/234//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
          Console output: https://hudson.apache.org/hudson/job/PreCommit-HADOOP-Build/234//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - +1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12470978/hadoop-7140.txt against trunk revision 1070021. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed core unit tests. +1 contrib tests. The patch passed contrib unit tests. +1 system test framework. The patch passed system test framework compile. Test results: https://hudson.apache.org/hudson/job/PreCommit-HADOOP-Build/234//testReport/ Findbugs warnings: https://hudson.apache.org/hudson/job/PreCommit-HADOOP-Build/234//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Console output: https://hudson.apache.org/hudson/job/PreCommit-HADOOP-Build/234//console This message is automatically generated.
          Todd Lipcon made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Todd Lipcon made changes -
          Field Original Value New Value
          Attachment hadoop-7140.txt [ 12470978 ]
          Todd Lipcon created issue -

            People

            • Assignee:
              Todd Lipcon
              Reporter:
              Todd Lipcon
            • Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development