Hive
  1. Hive
  2. HIVE-5901

Query cancel should stop running MR tasks

    Details

    • Type: Improvement Improvement
    • Status: Resolved
    • Priority: Minor Minor
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.13.0
    • Component/s: Query Processor
    • Labels:
      None

      Description

      Currently, query canceling does not stop running MR job immediately.

      1. HIVE-5901.1.patch.txt
        15 kB
        Navis
      2. HIVE-5901.2.patch.txt
        15 kB
        Navis
      3. HIVE-5901.3.patch.txt
        14 kB
        Navis
      4. HIVE-5901.4.patch.txt
        17 kB
        Navis
      5. HIVE-5901.5.patch.txt
        17 kB
        Navis
      6. HIVE-5901.6.patch.txt
        16 kB
        Thejas M Nair
      7. HIVE-5901.7.patch.txt
        18 kB
        Navis

        Issue Links

          Activity

          Transition Time In Source Status Execution Times Last Executer Last Execution Date
          Patch Available Patch Available Open Open
          90d 19h 35m 4 Thejas M Nair 26/Feb/14 03:56
          Open Open Patch Available Patch Available
          9m 12s 5 Thejas M Nair 26/Feb/14 03:56
          Patch Available Patch Available Resolved Resolved
          10d 7h 55m 1 Thejas M Nair 08/Mar/14 11:52
          Navis made changes -
          Link This issue is duplicated by HIVE-7198 [ HIVE-7198 ]
          Hide
          Ashutosh Chauhan added a comment -

          Has someone tested this with hadoop-2? Asking because AFAIK doing control-C on hive cli doesn't cancel MR tasks on hadoop-2 and since we use same api I would assume this would result in same problem for hadoop-2 even for this issue.

          Show
          Ashutosh Chauhan added a comment - Has someone tested this with hadoop-2? Asking because AFAIK doing control-C on hive cli doesn't cancel MR tasks on hadoop-2 and since we use same api I would assume this would result in same problem for hadoop-2 even for this issue.
          Thejas M Nair made changes -
          Fix Version/s 0.13.0 [ 12324986 ]
          Fix Version/s 0.14.0 [ 12326450 ]
          Hide
          Thejas M Nair added a comment -

          Thanks Harish, patch committed to 0.13 branch as well.

          Show
          Thejas M Nair added a comment - Thanks Harish, patch committed to 0.13 branch as well.
          Hide
          Harish Butani added a comment -

          +1 for 0.13

          Show
          Harish Butani added a comment - +1 for 0.13
          Thejas M Nair made changes -
          Status Patch Available [ 10002 ] Resolved [ 5 ]
          Fix Version/s 0.14.0 [ 12326450 ]
          Resolution Fixed [ 1 ]
          Hide
          Thejas M Nair added a comment -

          Patch committed to trunk. Thanks Navis!

          Harish Butani I think we should include this in 0.13 as well . It is a very useful feature. I had mentioned about this jira in the mailing list as well.

          Show
          Thejas M Nair added a comment - Patch committed to trunk. Thanks Navis! Harish Butani I think we should include this in 0.13 as well . It is a very useful feature. I had mentioned about this jira in the mailing list as well.
          Hide
          Hive QA added a comment -

          Overall: +1 all checks pass

          Here are the results of testing the latest attachment:
          https://issues.apache.org/jira/secure/attachment/12633277/HIVE-5901.7.patch.txt

          SUCCESS: +1 5373 tests passed

          Test results: http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/1656/testReport
          Console output: http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/1656/console

          Messages:

          Executing org.apache.hive.ptest.execution.PrepPhase
          Executing org.apache.hive.ptest.execution.ExecutionPhase
          Executing org.apache.hive.ptest.execution.ReportingPhase
          

          This message is automatically generated.

          ATTACHMENT ID: 12633277

          Show
          Hive QA added a comment - Overall : +1 all checks pass Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12633277/HIVE-5901.7.patch.txt SUCCESS: +1 5373 tests passed Test results: http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/1656/testReport Console output: http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/1656/console Messages: Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase This message is automatically generated. ATTACHMENT ID: 12633277
          Hide
          Thejas M Nair added a comment -

          Navis Can you please add this to the pre commit queue, if its not already there ?

          Show
          Thejas M Nair added a comment - Navis Can you please add this to the pre commit queue, if its not already there ?
          Hide
          Thejas M Nair added a comment -

          np, looks good. +1

          Show
          Thejas M Nair added a comment - np, looks good. +1
          Navis made changes -
          Attachment HIVE-5901.7.patch.txt [ 12633277 ]
          Hide
          Navis added a comment - - edited

          Comments are addressed after you've mentioned first but I've seemingly missed merging them in git. (Sorry again, I'm totally absent-minded theses days). And rebased to trunk

          Show
          Navis added a comment - - edited Comments are addressed after you've mentioned first but I've seemingly missed merging them in git. (Sorry again, I'm totally absent-minded theses days). And rebased to trunk
          Hide
          Thejas M Nair added a comment - - edited

          Navis I have added a comment on reviewboard on the latest update there.
          But there seems to be slight difference in the HIVE-5901.6.patch.txt patch here and latest one in reviewboard. In HIVE-5901.6.patch.txt the 'boolean shutdown' has been made volatile, which is not necessary as you pointed out with the changes to make the functions synchronized.

          Show
          Thejas M Nair added a comment - - edited Navis I have added a comment on reviewboard on the latest update there. But there seems to be slight difference in the HIVE-5901 .6.patch.txt patch here and latest one in reviewboard. In HIVE-5901 .6.patch.txt the 'boolean shutdown' has been made volatile, which is not necessary as you pointed out with the changes to make the functions synchronized.
          Hide
          Thejas M Nair added a comment -

          Navis Thanks for pointing that out! I didn't realize I was looking at an older patch file on reviewboard. I have added some comments.

          Show
          Thejas M Nair added a comment - Navis Thanks for pointing that out! I didn't realize I was looking at an older patch file on reviewboard. I have added some comments.
          Hide
          Navis added a comment -

          I always forget publishing review board changes. Sorry.

          Show
          Navis added a comment - I always forget publishing review board changes. Sorry.
          Thejas M Nair made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Thejas M Nair made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Thejas M Nair made changes -
          Attachment HIVE-5901.6.patch.txt [ 12631148 ]
          Hide
          Thejas M Nair added a comment -

          HIVE-5901.6.patch.txt - updating patch to address the comment I made (I figured you are probably not at work today ) Makes the shutdown boolean in DriverContext.java volatile.

          Show
          Thejas M Nair added a comment - HIVE-5901 .6.patch.txt - updating patch to address the comment I made (I figured you are probably not at work today ) Makes the shutdown boolean in DriverContext.java volatile.
          Hide
          Thejas M Nair added a comment -

          Looks good to me. I just have one minor comment on reviewboard.

          Show
          Thejas M Nair added a comment - Looks good to me. I just have one minor comment on reviewboard.
          Navis made changes -
          Link This issue relates to HIVE-4017 [ HIVE-4017 ]
          Hide
          Hive QA added a comment -

          Overall: +1 all checks pass

          Here are the results of testing the latest attachment:
          https://issues.apache.org/jira/secure/attachment/12627544/HIVE-5901.5.patch.txt

          SUCCESS: +1 5040 tests passed

          Test results: http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/1237/testReport
          Console output: http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/1237/console

          Messages:

          Executing org.apache.hive.ptest.execution.PrepPhase
          Executing org.apache.hive.ptest.execution.ExecutionPhase
          Executing org.apache.hive.ptest.execution.ReportingPhase
          

          This message is automatically generated.

          ATTACHMENT ID: 12627544

          Show
          Hive QA added a comment - Overall : +1 all checks pass Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12627544/HIVE-5901.5.patch.txt SUCCESS: +1 5040 tests passed Test results: http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/1237/testReport Console output: http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/1237/console Messages: Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase This message is automatically generated. ATTACHMENT ID: 12627544
          Navis made changes -
          Attachment HIVE-5901.5.patch.txt [ 12627544 ]
          Hide
          Navis added a comment -

          Rebased to trunk

          Show
          Navis added a comment - Rebased to trunk
          Hide
          Sergey Shelukhin added a comment -

          sorry, missed the response; lgtm

          Show
          Sergey Shelukhin added a comment - sorry, missed the response; lgtm
          Hide
          Hive QA added a comment -

          Overall: +1 all checks pass

          Here are the results of testing the latest attachment:
          https://issues.apache.org/jira/secure/attachment/12621020/HIVE-5901.4.patch.txt

          SUCCESS: +1 4873 tests passed

          Test results: http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/788/testReport
          Console output: http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/788/console

          Messages:

          Executing org.apache.hive.ptest.execution.PrepPhase
          Executing org.apache.hive.ptest.execution.ExecutionPhase
          Executing org.apache.hive.ptest.execution.ReportingPhase
          

          This message is automatically generated.

          ATTACHMENT ID: 12621020

          Show
          Hive QA added a comment - Overall : +1 all checks pass Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12621020/HIVE-5901.4.patch.txt SUCCESS: +1 4873 tests passed Test results: http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/788/testReport Console output: http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/788/console Messages: Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase This message is automatically generated. ATTACHMENT ID: 12621020
          Navis made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Navis made changes -
          Attachment HIVE-5901.4.patch.txt [ 12621020 ]
          Hide
          Navis added a comment -

          synchronize methods in DriverContext

          Show
          Navis added a comment - synchronize methods in DriverContext
          Navis made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Hide
          Hive QA added a comment -

          Overall: -1 at least one tests failed

          Here are the results of testing the latest attachment:
          https://issues.apache.org/jira/secure/attachment/12620812/HIVE-5901.3.patch.txt

          ERROR: -1 due to 1 failed/errored test(s), 4818 tests executed
          Failed tests:

          org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_parallel_orderby
          

          Test results: http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/771/testReport
          Console output: http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/771/console

          Messages:

          Executing org.apache.hive.ptest.execution.PrepPhase
          Executing org.apache.hive.ptest.execution.ExecutionPhase
          Executing org.apache.hive.ptest.execution.ReportingPhase
          Tests exited with: TestsFailedException: 1 tests failed
          

          This message is automatically generated.

          ATTACHMENT ID: 12620812

          Show
          Hive QA added a comment - Overall : -1 at least one tests failed Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12620812/HIVE-5901.3.patch.txt ERROR: -1 due to 1 failed/errored test(s), 4818 tests executed Failed tests: org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_parallel_orderby Test results: http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/771/testReport Console output: http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/771/console Messages: Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 1 tests failed This message is automatically generated. ATTACHMENT ID: 12620812
          Navis made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Navis made changes -
          Attachment HIVE-5901.3.patch.txt [ 12620812 ]
          Navis made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Hide
          Hive QA added a comment -

          Overall: +1 all checks pass

          Here are the results of testing the latest attachment:
          https://issues.apache.org/jira/secure/attachment/12616192/HIVE-5901.2.patch.txt

          SUCCESS: +1 4740 tests passed

          Test results: http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/479/testReport
          Console output: http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/479/console

          Messages:

          Executing org.apache.hive.ptest.execution.PrepPhase
          Executing org.apache.hive.ptest.execution.ExecutionPhase
          Executing org.apache.hive.ptest.execution.ReportingPhase
          

          This message is automatically generated.

          ATTACHMENT ID: 12616192

          Show
          Hive QA added a comment - Overall : +1 all checks pass Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12616192/HIVE-5901.2.patch.txt SUCCESS: +1 4740 tests passed Test results: http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/479/testReport Console output: http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/479/console Messages: Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase This message is automatically generated. ATTACHMENT ID: 12616192
          Navis made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Navis made changes -
          Attachment HIVE-5901.2.patch.txt [ 12616192 ]
          Hide
          Navis added a comment -

          Fixed test fails

          Show
          Navis added a comment - Fixed test fails
          Navis made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Hide
          Hive QA added a comment -

          Overall: -1 at least one tests failed

          Here are the results of testing the latest attachment:
          https://issues.apache.org/jira/secure/attachment/12616012/HIVE-5901.1.patch.txt

          ERROR: -1 due to 2 failed/errored test(s), 4740 tests executed
          Failed tests:

          org.apache.hive.jdbc.TestJdbcDriver2.testErrorDiag
          org.apache.hive.jdbc.TestJdbcDriver2.testErrorMessages
          

          Test results: http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/476/testReport
          Console output: http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/476/console

          Messages:

          Executing org.apache.hive.ptest.execution.PrepPhase
          Executing org.apache.hive.ptest.execution.ExecutionPhase
          Executing org.apache.hive.ptest.execution.ReportingPhase
          Tests exited with: TestsFailedException: 2 tests failed
          

          This message is automatically generated.

          ATTACHMENT ID: 12616012

          Show
          Hive QA added a comment - Overall : -1 at least one tests failed Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12616012/HIVE-5901.1.patch.txt ERROR: -1 due to 2 failed/errored test(s), 4740 tests executed Failed tests: org.apache.hive.jdbc.TestJdbcDriver2.testErrorDiag org.apache.hive.jdbc.TestJdbcDriver2.testErrorMessages Test results: http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/476/testReport Console output: http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/476/console Messages: Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 2 tests failed This message is automatically generated. ATTACHMENT ID: 12616012
          Navis made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Navis made changes -
          Attachment HIVE-5901.1.patch.txt [ 12616012 ]
          Navis made changes -
          Field Original Value New Value
          Remote Link This issue links to "review board (Web Link)" [ 13414 ]
          Navis created issue -

            People

            • Assignee:
              Navis
              Reporter:
              Navis
            • Votes:
              0 Vote for this issue
              Watchers:
              9 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development