Hive
  1. Hive
  2. HIVE-243

^C breaks out of running query, but not whole CLI

    Details

    • Type: Wish Wish
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.8.0
    • Fix Version/s: 0.8.0
    • Component/s: Query Processor
    • Labels:
      None
    • Hadoop Flags:
      Reviewed
    • Release Note:
      Added support to stop CLI commands with Ctrl+C

      Description

      It would be lovely if, when I know a query is bad, I could just ^C out of it. I can do that now, but the whole CLI quits.

      It'd be quite nice if it took an extra ^C to break the CLI, or if there was some control character to break out of a query without breaking out of the CLI.

      1. HIVE-243.patch
        22 kB
        Ning Zhang

        Issue Links

          Activity

          Transition Time In Source Status Execution Times Last Executer Last Execution Date
          Patch Available Patch Available Open Open
          15h 42m 1 Ning Zhang 20/Apr/11 18:55
          Open Open Patch Available Patch Available
          843d 19h 3m 2 George Djabarov 17/May/11 19:52
          Patch Available Patch Available Resolved Resolved
          4h 29m 1 Ning Zhang 18/May/11 00:21
          Resolved Resolved Closed Closed
          213d 34m 1 Carl Steinbach 16/Dec/11 23:56
          Carl Steinbach made changes -
          Status Resolved [ 5 ] Closed [ 6 ]
          Hide
          Hudson added a comment -

          Integrated in Hive-trunk-h0.21 #771 (See https://builds.apache.org/job/Hive-trunk-h0.21/771/)
          HIVE-2211. Fix a bug caused by HIVE-243 (Siying Dong via Ning Zhang)

          nzhang : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1134179
          Files :

          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java
          Show
          Hudson added a comment - Integrated in Hive-trunk-h0.21 #771 (See https://builds.apache.org/job/Hive-trunk-h0.21/771/ ) HIVE-2211 . Fix a bug caused by HIVE-243 (Siying Dong via Ning Zhang) nzhang : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1134179 Files : /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java
          Hide
          Hudson added a comment -

          Integrated in Hive-trunk-h0.21 #735 (See https://builds.apache.org/hudson/job/Hive-trunk-h0.21/735/)
          HIVE-243. ^C breaks out of running query, but not whole CLI (George Djabarov via Ning Zhang)

          nzhang : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1104625
          Files :

          • /hive/trunk/common/src/java/org/apache/hadoop/hive/common/HiveInterruptUtils.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/HadoopJobExecHelper.java
          • /hive/trunk/cli/src/java/org/apache/hadoop/hive/cli/CliDriver.java
          • /hive/trunk/common/src/java/org/apache/hadoop/hive/common/HiveInterruptCallback.java
          Show
          Hudson added a comment - Integrated in Hive-trunk-h0.21 #735 (See https://builds.apache.org/hudson/job/Hive-trunk-h0.21/735/ ) HIVE-243 . ^C breaks out of running query, but not whole CLI (George Djabarov via Ning Zhang) nzhang : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1104625 Files : /hive/trunk/common/src/java/org/apache/hadoop/hive/common/HiveInterruptUtils.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/HadoopJobExecHelper.java /hive/trunk/cli/src/java/org/apache/hadoop/hive/cli/CliDriver.java /hive/trunk/common/src/java/org/apache/hadoop/hive/common/HiveInterruptCallback.java
          Ning Zhang made changes -
          Assignee George Djabarov [ georgedj ]
          Ning Zhang made changes -
          Status Patch Available [ 10002 ] Resolved [ 5 ]
          Fix Version/s 0.8.0 [ 12316178 ]
          Resolution Fixed [ 1 ]
          Hide
          Ning Zhang added a comment -

          Add unit tests passed and just committed. Thanks George!

          Show
          Ning Zhang added a comment - Add unit tests passed and just committed. Thanks George!
          Ning Zhang made changes -
          Attachment HIVE-243.patch [ 12479531 ]
          Hide
          Ning Zhang added a comment -

          Upload a patch downloaded from George's review board (with Apache licenses added to the head of new files).

          Show
          Ning Zhang added a comment - Upload a patch downloaded from George's review board (with Apache licenses added to the head of new files).
          Hide
          jiraposter@reviews.apache.org added a comment -

          -----------------------------------------------------------
          This is an automatically generated e-mail. To reply, visit:
          https://reviews.apache.org/r/626/
          -----------------------------------------------------------

          (Updated 2011-05-17 18:56:20.310048)

          Review request for hive.

          Changes
          -------

          updating diff against the latest trunk

          Summary
          -------

          Fixed by adding INT signal hander that interrupts the CLI thread. The CLI thread gets InterruptedException and it stops the current command.

          This addresses bug HIVE-243.
          https://issues.apache.org/jira/browse/HIVE-243

          Diffs (updated)


          trunk/cli/src/java/org/apache/hadoop/hive/cli/CliDriver.java 1104456
          trunk/common/src/java/org/apache/hadoop/hive/common/HiveInterruptCallback.java PRE-CREATION
          trunk/common/src/java/org/apache/hadoop/hive/common/HiveInterruptUtils.java PRE-CREATION
          trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/HadoopJobExecHelper.java 1104456
          trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java 1104456

          Diff: https://reviews.apache.org/r/626/diff

          Testing
          -------

          Manual test Ctrl+C on bunch of command (show tables, select, etc)
          Tested interrupting the command on large data set in different stages of the processing (JDO, MapReduce, building the input summary).
          example queries:
          explain select count from xyz where ds = '2011-02-01';
          select count from xyz where ds = '2011-02-01';

          Thanks,

          georgedj

          Show
          jiraposter@reviews.apache.org added a comment - ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/626/ ----------------------------------------------------------- (Updated 2011-05-17 18:56:20.310048) Review request for hive. Changes ------- updating diff against the latest trunk Summary ------- Fixed by adding INT signal hander that interrupts the CLI thread. The CLI thread gets InterruptedException and it stops the current command. This addresses bug HIVE-243 . https://issues.apache.org/jira/browse/HIVE-243 Diffs (updated) trunk/cli/src/java/org/apache/hadoop/hive/cli/CliDriver.java 1104456 trunk/common/src/java/org/apache/hadoop/hive/common/HiveInterruptCallback.java PRE-CREATION trunk/common/src/java/org/apache/hadoop/hive/common/HiveInterruptUtils.java PRE-CREATION trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/HadoopJobExecHelper.java 1104456 trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java 1104456 Diff: https://reviews.apache.org/r/626/diff Testing ------- Manual test Ctrl+C on bunch of command (show tables, select, etc) Tested interrupting the command on large data set in different stages of the processing (JDO, MapReduce, building the input summary). example queries: explain select count from xyz where ds = '2011-02-01'; select count from xyz where ds = '2011-02-01'; Thanks, georgedj
          George Djabarov made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Hadoop Flags [Reviewed]
          Release Note Added support to stop CLI commands with Ctrl+C
          Hide
          George Djabarov added a comment -

          First Ctrl+C will print a message and try to stop the current command. Second Ctrl+C will kill the CLI jvm

          Show
          George Djabarov added a comment - First Ctrl+C will print a message and try to stop the current command. Second Ctrl+C will kill the CLI jvm
          Hide
          jiraposter@reviews.apache.org added a comment -

          -----------------------------------------------------------
          This is an automatically generated e-mail. To reply, visit:
          https://reviews.apache.org/r/626/#review670
          -----------------------------------------------------------

          trunk/cli/src/java/org/apache/hadoop/hive/cli/CliDriver.java
          <https://reviews.apache.org/r/626/#comment1337>

          Is this comment correct?

          trunk/common/src/java/org/apache/hadoop/hive/common/HiveInterruptCallback.java
          <https://reviews.apache.org/r/626/#comment1336>

          typo: interrup -> interrupt.

          trunk/metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java
          <https://reviews.apache.org/r/626/#comment1335>

          We shouldn't check interrupt here since ObjectStore should be run at the metastore server side (if Hive is running non-local mode for metastore). So ^C on CLI won't be detected here. Instead we should check it in HiveMetaStoreClient.java.

          Since interrupting JDO is quite complicated, I suggest removing the interrupt handling for JDO completely in this JIRA and file a separate JIRA for that.

          trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java
          <https://reviews.apache.org/r/626/#comment1334>

          typo: interrup -> interrupt

          trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java
          <https://reviews.apache.org/r/626/#comment1338>

          why these are removed?

          • Ning

          On 2011-05-10 21:55:56, georgedj wrote:

          -----------------------------------------------------------

          This is an automatically generated e-mail. To reply, visit:

          https://reviews.apache.org/r/626/

          -----------------------------------------------------------

          (Updated 2011-05-10 21:55:56)

          Review request for hive.

          Summary

          -------

          Fixed by adding INT signal hander that interrupts the CLI thread. The CLI thread gets InterruptedException and it stops the current command.

          This addresses bug HIVE-243.

          https://issues.apache.org/jira/browse/HIVE-243

          Diffs

          -----

          trunk/cli/src/java/org/apache/hadoop/hive/cli/CliDriver.java 1100312

          trunk/common/src/java/org/apache/hadoop/hive/common/HiveInterruptCallback.java PRE-CREATION

          trunk/common/src/java/org/apache/hadoop/hive/common/HiveInterruptUtils.java PRE-CREATION

          trunk/metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java 1100312

          trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/HadoopJobExecHelper.java 1100312

          trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java 1100312

          Diff: https://reviews.apache.org/r/626/diff

          Testing

          -------

          Manual test Ctrl+C on bunch of command (show tables, select, etc)

          Tested interrupting the command on large data set in different stages of the processing (JDO, MapReduce, building the input summary).

          example queries:

          explain select count from xyz where ds = '2011-02-01';

          select count from xyz where ds = '2011-02-01';

          Thanks,

          georgedj

          Show
          jiraposter@reviews.apache.org added a comment - ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/626/#review670 ----------------------------------------------------------- trunk/cli/src/java/org/apache/hadoop/hive/cli/CliDriver.java < https://reviews.apache.org/r/626/#comment1337 > Is this comment correct? trunk/common/src/java/org/apache/hadoop/hive/common/HiveInterruptCallback.java < https://reviews.apache.org/r/626/#comment1336 > typo: interrup -> interrupt. trunk/metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java < https://reviews.apache.org/r/626/#comment1335 > We shouldn't check interrupt here since ObjectStore should be run at the metastore server side (if Hive is running non-local mode for metastore). So ^C on CLI won't be detected here. Instead we should check it in HiveMetaStoreClient.java. Since interrupting JDO is quite complicated, I suggest removing the interrupt handling for JDO completely in this JIRA and file a separate JIRA for that. trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java < https://reviews.apache.org/r/626/#comment1334 > typo: interrup -> interrupt trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java < https://reviews.apache.org/r/626/#comment1338 > why these are removed? Ning On 2011-05-10 21:55:56, georgedj wrote: ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/626/ ----------------------------------------------------------- (Updated 2011-05-10 21:55:56) Review request for hive. Summary ------- Fixed by adding INT signal hander that interrupts the CLI thread. The CLI thread gets InterruptedException and it stops the current command. This addresses bug HIVE-243 . https://issues.apache.org/jira/browse/HIVE-243 Diffs ----- trunk/cli/src/java/org/apache/hadoop/hive/cli/CliDriver.java 1100312 trunk/common/src/java/org/apache/hadoop/hive/common/HiveInterruptCallback.java PRE-CREATION trunk/common/src/java/org/apache/hadoop/hive/common/HiveInterruptUtils.java PRE-CREATION trunk/metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java 1100312 trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/HadoopJobExecHelper.java 1100312 trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java 1100312 Diff: https://reviews.apache.org/r/626/diff Testing ------- Manual test Ctrl+C on bunch of command (show tables, select, etc) Tested interrupting the command on large data set in different stages of the processing (JDO, MapReduce, building the input summary). example queries: explain select count from xyz where ds = '2011-02-01'; select count from xyz where ds = '2011-02-01'; Thanks, georgedj
          Hide
          jiraposter@reviews.apache.org added a comment -

          -----------------------------------------------------------
          This is an automatically generated e-mail. To reply, visit:
          https://reviews.apache.org/r/626/
          -----------------------------------------------------------

          (Updated 2011-05-10 21:55:56.707055)

          Review request for hive.

          Changes
          -------

          Fixed some white space formatting

          Summary
          -------

          Fixed by adding INT signal hander that interrupts the CLI thread. The CLI thread gets InterruptedException and it stops the current command.

          This addresses bug HIVE-243.
          https://issues.apache.org/jira/browse/HIVE-243

          Diffs (updated)


          trunk/cli/src/java/org/apache/hadoop/hive/cli/CliDriver.java 1100312
          trunk/common/src/java/org/apache/hadoop/hive/common/HiveInterruptCallback.java PRE-CREATION
          trunk/common/src/java/org/apache/hadoop/hive/common/HiveInterruptUtils.java PRE-CREATION
          trunk/metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java 1100312
          trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/HadoopJobExecHelper.java 1100312
          trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java 1100312

          Diff: https://reviews.apache.org/r/626/diff

          Testing
          -------

          Manual test Ctrl+C on bunch of command (show tables, select, etc)
          Tested interrupting the command on large data set in different stages of the processing (JDO, MapReduce, building the input summary).
          example queries:
          explain select count from xyz where ds = '2011-02-01';
          select count from xyz where ds = '2011-02-01';

          Thanks,

          georgedj

          Show
          jiraposter@reviews.apache.org added a comment - ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/626/ ----------------------------------------------------------- (Updated 2011-05-10 21:55:56.707055) Review request for hive. Changes ------- Fixed some white space formatting Summary ------- Fixed by adding INT signal hander that interrupts the CLI thread. The CLI thread gets InterruptedException and it stops the current command. This addresses bug HIVE-243 . https://issues.apache.org/jira/browse/HIVE-243 Diffs (updated) trunk/cli/src/java/org/apache/hadoop/hive/cli/CliDriver.java 1100312 trunk/common/src/java/org/apache/hadoop/hive/common/HiveInterruptCallback.java PRE-CREATION trunk/common/src/java/org/apache/hadoop/hive/common/HiveInterruptUtils.java PRE-CREATION trunk/metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java 1100312 trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/HadoopJobExecHelper.java 1100312 trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java 1100312 Diff: https://reviews.apache.org/r/626/diff Testing ------- Manual test Ctrl+C on bunch of command (show tables, select, etc) Tested interrupting the command on large data set in different stages of the processing (JDO, MapReduce, building the input summary). example queries: explain select count from xyz where ds = '2011-02-01'; select count from xyz where ds = '2011-02-01'; Thanks, georgedj
          Hide
          jiraposter@reviews.apache.org added a comment -

          -----------------------------------------------------------
          This is an automatically generated e-mail. To reply, visit:
          https://reviews.apache.org/r/626/
          -----------------------------------------------------------

          (Updated 2011-05-10 21:33:17.693931)

          Review request for hive.

          Summary
          -------

          Fixed by adding INT signal hander that interrupts the CLI thread. The CLI thread gets InterruptedException and it stops the current command.

          This addresses bug HIVE-243.
          https://issues.apache.org/jira/browse/HIVE-243

          Diffs


          trunk/cli/src/java/org/apache/hadoop/hive/cli/CliDriver.java 1100312
          trunk/common/src/java/org/apache/hadoop/hive/common/HiveInterruptCallback.java PRE-CREATION
          trunk/common/src/java/org/apache/hadoop/hive/common/HiveInterruptUtils.java PRE-CREATION
          trunk/metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java 1100312
          trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/HadoopJobExecHelper.java 1100312
          trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java 1100312

          Diff: https://reviews.apache.org/r/626/diff

          Testing (updated)
          -------

          Manual test Ctrl+C on bunch of command (show tables, select, etc)
          Tested interrupting the command on large data set in different stages of the processing (JDO, MapReduce, building the input summary).
          example queries:
          explain select count from xyz where ds = '2011-02-01';
          select count from xyz where ds = '2011-02-01';

          Thanks,

          georgedj

          Show
          jiraposter@reviews.apache.org added a comment - ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/626/ ----------------------------------------------------------- (Updated 2011-05-10 21:33:17.693931) Review request for hive. Summary ------- Fixed by adding INT signal hander that interrupts the CLI thread. The CLI thread gets InterruptedException and it stops the current command. This addresses bug HIVE-243 . https://issues.apache.org/jira/browse/HIVE-243 Diffs trunk/cli/src/java/org/apache/hadoop/hive/cli/CliDriver.java 1100312 trunk/common/src/java/org/apache/hadoop/hive/common/HiveInterruptCallback.java PRE-CREATION trunk/common/src/java/org/apache/hadoop/hive/common/HiveInterruptUtils.java PRE-CREATION trunk/metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java 1100312 trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/HadoopJobExecHelper.java 1100312 trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java 1100312 Diff: https://reviews.apache.org/r/626/diff Testing (updated) ------- Manual test Ctrl+C on bunch of command (show tables, select, etc) Tested interrupting the command on large data set in different stages of the processing (JDO, MapReduce, building the input summary). example queries: explain select count from xyz where ds = '2011-02-01'; select count from xyz where ds = '2011-02-01'; Thanks, georgedj
          Hide
          jiraposter@reviews.apache.org added a comment -

          On 2011-04-20 17:51:59, Ning Zhang wrote:

          > trunk/cli/src/java/org/apache/hadoop/hive/cli/CliDriver.java, line 313

          > <https://reviews.apache.org/r/626/diff/1/?file=16075#file16075line313>

          >

          > Does this handle the case when a MapReduce job is launched and CLI got a CTL+C? Currently we have a JVM shutdownHook defined in HadoopJobExecHelper.java to kill the MR job before the JVM shut down (the old behavior when hitting a CTL+C).

          >

          > Here if we don't shut down the JVM, you should do the same thing to kill the running MR jobs.

          Sorry for the delayed response, I didn't get email notification for the comment. Let me try to hookup the running jobs kill to the ctrl+c handler. I don't think it will do it with the current code as it will prevent the shutdown hook of being called.

          • georgedj

          -----------------------------------------------------------
          This is an automatically generated e-mail. To reply, visit:
          https://reviews.apache.org/r/626/#review505
          -----------------------------------------------------------

          On 2011-05-10 21:25:03, georgedj wrote:

          -----------------------------------------------------------

          This is an automatically generated e-mail. To reply, visit:

          https://reviews.apache.org/r/626/

          -----------------------------------------------------------

          (Updated 2011-05-10 21:25:03)

          Review request for hive.

          Summary

          -------

          Fixed by adding INT signal hander that interrupts the CLI thread. The CLI thread gets InterruptedException and it stops the current command.

          This addresses bug HIVE-243.

          https://issues.apache.org/jira/browse/HIVE-243

          Diffs

          -----

          trunk/cli/src/java/org/apache/hadoop/hive/cli/CliDriver.java 1100312

          trunk/common/src/java/org/apache/hadoop/hive/common/HiveInterruptCallback.java PRE-CREATION

          trunk/common/src/java/org/apache/hadoop/hive/common/HiveInterruptUtils.java PRE-CREATION

          trunk/metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java 1100312

          trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/HadoopJobExecHelper.java 1100312

          trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java 1100312

          Diff: https://reviews.apache.org/r/626/diff

          Testing

          -------

          Manual test Ctrl+C on bunch of command (show tables, select, etc)

          Thanks,

          georgedj

          Show
          jiraposter@reviews.apache.org added a comment - On 2011-04-20 17:51:59, Ning Zhang wrote: > trunk/cli/src/java/org/apache/hadoop/hive/cli/CliDriver.java, line 313 > < https://reviews.apache.org/r/626/diff/1/?file=16075#file16075line313 > > > Does this handle the case when a MapReduce job is launched and CLI got a CTL+C? Currently we have a JVM shutdownHook defined in HadoopJobExecHelper.java to kill the MR job before the JVM shut down (the old behavior when hitting a CTL+C). > > Here if we don't shut down the JVM, you should do the same thing to kill the running MR jobs. Sorry for the delayed response, I didn't get email notification for the comment. Let me try to hookup the running jobs kill to the ctrl+c handler. I don't think it will do it with the current code as it will prevent the shutdown hook of being called. georgedj ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/626/#review505 ----------------------------------------------------------- On 2011-05-10 21:25:03, georgedj wrote: ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/626/ ----------------------------------------------------------- (Updated 2011-05-10 21:25:03) Review request for hive. Summary ------- Fixed by adding INT signal hander that interrupts the CLI thread. The CLI thread gets InterruptedException and it stops the current command. This addresses bug HIVE-243 . https://issues.apache.org/jira/browse/HIVE-243 Diffs ----- trunk/cli/src/java/org/apache/hadoop/hive/cli/CliDriver.java 1100312 trunk/common/src/java/org/apache/hadoop/hive/common/HiveInterruptCallback.java PRE-CREATION trunk/common/src/java/org/apache/hadoop/hive/common/HiveInterruptUtils.java PRE-CREATION trunk/metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java 1100312 trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/HadoopJobExecHelper.java 1100312 trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java 1100312 Diff: https://reviews.apache.org/r/626/diff Testing ------- Manual test Ctrl+C on bunch of command (show tables, select, etc) Thanks, georgedj
          Hide
          jiraposter@reviews.apache.org added a comment -

          -----------------------------------------------------------
          This is an automatically generated e-mail. To reply, visit:
          https://reviews.apache.org/r/626/
          -----------------------------------------------------------

          (Updated 2011-05-10 21:25:03.998477)

          Review request for hive.

          Changes
          -------

          • First ctrl+c will print a message and try to stop the current command. Second ctrl+c will kill the jvm
          • Added code that stops some of the JDO processing
          • Now the code can also stop generating input summary.
          • Fixed loop in Utilities.getInputSummary that hangs under some conditions

          Summary
          -------

          Fixed by adding INT signal hander that interrupts the CLI thread. The CLI thread gets InterruptedException and it stops the current command.

          This addresses bug HIVE-243.
          https://issues.apache.org/jira/browse/HIVE-243

          Diffs (updated)


          trunk/cli/src/java/org/apache/hadoop/hive/cli/CliDriver.java 1100312
          trunk/common/src/java/org/apache/hadoop/hive/common/HiveInterruptCallback.java PRE-CREATION
          trunk/common/src/java/org/apache/hadoop/hive/common/HiveInterruptUtils.java PRE-CREATION
          trunk/metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java 1100312
          trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/HadoopJobExecHelper.java 1100312
          trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java 1100312

          Diff: https://reviews.apache.org/r/626/diff

          Testing
          -------

          Manual test Ctrl+C on bunch of command (show tables, select, etc)

          Thanks,

          georgedj

          Show
          jiraposter@reviews.apache.org added a comment - ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/626/ ----------------------------------------------------------- (Updated 2011-05-10 21:25:03.998477) Review request for hive. Changes ------- First ctrl+c will print a message and try to stop the current command. Second ctrl+c will kill the jvm Added code that stops some of the JDO processing Now the code can also stop generating input summary. Fixed loop in Utilities.getInputSummary that hangs under some conditions Summary ------- Fixed by adding INT signal hander that interrupts the CLI thread. The CLI thread gets InterruptedException and it stops the current command. This addresses bug HIVE-243 . https://issues.apache.org/jira/browse/HIVE-243 Diffs (updated) trunk/cli/src/java/org/apache/hadoop/hive/cli/CliDriver.java 1100312 trunk/common/src/java/org/apache/hadoop/hive/common/HiveInterruptCallback.java PRE-CREATION trunk/common/src/java/org/apache/hadoop/hive/common/HiveInterruptUtils.java PRE-CREATION trunk/metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java 1100312 trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/HadoopJobExecHelper.java 1100312 trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java 1100312 Diff: https://reviews.apache.org/r/626/diff Testing ------- Manual test Ctrl+C on bunch of command (show tables, select, etc) Thanks, georgedj
          Hide
          jiraposter@reviews.apache.org added a comment -

          -----------------------------------------------------------
          This is an automatically generated e-mail. To reply, visit:
          https://reviews.apache.org/r/626/#review609
          -----------------------------------------------------------

          trunk/cli/src/java/org/apache/hadoop/hive/cli/CliDriver.java
          <https://reviews.apache.org/r/626/#comment1252>

          This change kills the launched MR job. however the following interrupt() only called to the current thread. This does not really behaves what it is expected to be when there are multiple threads running.

          For example when we are getting partitions from metastore (JDO may use its thread pool) and when getting HDFS summaries (we are also using thread pool). For these situations we probably need to interrupt all child threads.

          • Ning

          On 2011-04-28 20:54:48, georgedj wrote:

          -----------------------------------------------------------

          This is an automatically generated e-mail. To reply, visit:

          https://reviews.apache.org/r/626/

          -----------------------------------------------------------

          (Updated 2011-04-28 20:54:48)

          Review request for hive.

          Summary

          -------

          Fixed by adding INT signal hander that interrupts the CLI thread. The CLI thread gets InterruptedException and it stops the current command.

          This addresses bug HIVE-243.

          https://issues.apache.org/jira/browse/HIVE-243

          Diffs

          -----

          trunk/cli/src/java/org/apache/hadoop/hive/cli/CliDriver.java 1097569

          trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/HadoopJobExecHelper.java 1097569

          Diff: https://reviews.apache.org/r/626/diff

          Testing

          -------

          Manual test Ctrl+C on bunch of command (show tables, select, etc)

          Thanks,

          georgedj

          Show
          jiraposter@reviews.apache.org added a comment - ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/626/#review609 ----------------------------------------------------------- trunk/cli/src/java/org/apache/hadoop/hive/cli/CliDriver.java < https://reviews.apache.org/r/626/#comment1252 > This change kills the launched MR job. however the following interrupt() only called to the current thread. This does not really behaves what it is expected to be when there are multiple threads running. For example when we are getting partitions from metastore (JDO may use its thread pool) and when getting HDFS summaries (we are also using thread pool). For these situations we probably need to interrupt all child threads. Ning On 2011-04-28 20:54:48, georgedj wrote: ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/626/ ----------------------------------------------------------- (Updated 2011-04-28 20:54:48) Review request for hive. Summary ------- Fixed by adding INT signal hander that interrupts the CLI thread. The CLI thread gets InterruptedException and it stops the current command. This addresses bug HIVE-243 . https://issues.apache.org/jira/browse/HIVE-243 Diffs ----- trunk/cli/src/java/org/apache/hadoop/hive/cli/CliDriver.java 1097569 trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/HadoopJobExecHelper.java 1097569 Diff: https://reviews.apache.org/r/626/diff Testing ------- Manual test Ctrl+C on bunch of command (show tables, select, etc) Thanks, georgedj
          Hide
          jiraposter@reviews.apache.org added a comment -

          -----------------------------------------------------------
          This is an automatically generated e-mail. To reply, visit:
          https://reviews.apache.org/r/626/
          -----------------------------------------------------------

          (Updated 2011-04-28 20:54:48.288055)

          Review request for hive.

          Changes
          -------

          Added code to kill all running jobs before interrupting the current CLI thread.

          Summary
          -------

          Fixed by adding INT signal hander that interrupts the CLI thread. The CLI thread gets InterruptedException and it stops the current command.

          This addresses bug HIVE-243.
          https://issues.apache.org/jira/browse/HIVE-243

          Diffs (updated)


          trunk/cli/src/java/org/apache/hadoop/hive/cli/CliDriver.java 1097569
          trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/HadoopJobExecHelper.java 1097569

          Diff: https://reviews.apache.org/r/626/diff

          Testing
          -------

          Manual test Ctrl+C on bunch of command (show tables, select, etc)

          Thanks,

          georgedj

          Show
          jiraposter@reviews.apache.org added a comment - ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/626/ ----------------------------------------------------------- (Updated 2011-04-28 20:54:48.288055) Review request for hive. Changes ------- Added code to kill all running jobs before interrupting the current CLI thread. Summary ------- Fixed by adding INT signal hander that interrupts the CLI thread. The CLI thread gets InterruptedException and it stops the current command. This addresses bug HIVE-243 . https://issues.apache.org/jira/browse/HIVE-243 Diffs (updated) trunk/cli/src/java/org/apache/hadoop/hive/cli/CliDriver.java 1097569 trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/HadoopJobExecHelper.java 1097569 Diff: https://reviews.apache.org/r/626/diff Testing ------- Manual test Ctrl+C on bunch of command (show tables, select, etc) Thanks, georgedj
          Ning Zhang made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Hide
          Ning Zhang added a comment -

          @George, thanks for taking this! I've updated the review board with a comment. Mostly to make sure that this patch kills the MR job also when getting a CTL+C.

          Show
          Ning Zhang added a comment - @George, thanks for taking this! I've updated the review board with a comment. Mostly to make sure that this patch kills the MR job also when getting a CTL+C.
          Hide
          jiraposter@reviews.apache.org added a comment -

          -----------------------------------------------------------
          This is an automatically generated e-mail. To reply, visit:
          https://reviews.apache.org/r/626/#review505
          -----------------------------------------------------------

          trunk/cli/src/java/org/apache/hadoop/hive/cli/CliDriver.java
          <https://reviews.apache.org/r/626/#comment1030>

          Does this handle the case when a MapReduce job is launched and CLI got a CTL+C? Currently we have a JVM shutdownHook defined in HadoopJobExecHelper.java to kill the MR job before the JVM shut down (the old behavior when hitting a CTL+C).

          Here if we don't shut down the JVM, you should do the same thing to kill the running MR jobs.

          • Ning

          On 2011-04-20 16:49:05, georgedj wrote:

          -----------------------------------------------------------

          This is an automatically generated e-mail. To reply, visit:

          https://reviews.apache.org/r/626/

          -----------------------------------------------------------

          (Updated 2011-04-20 16:49:05)

          Review request for hive.

          Summary

          -------

          Fixed by adding INT signal hander that interrupts the CLI thread. The CLI thread gets InterruptedException and it stops the current command.

          This addresses bug HIVE-243.

          https://issues.apache.org/jira/browse/HIVE-243

          Diffs

          -----

          trunk/cli/src/java/org/apache/hadoop/hive/cli/CliDriver.java 1095439

          Diff: https://reviews.apache.org/r/626/diff

          Testing

          -------

          Manual test Ctrl+C on bunch of command (show tables, select, etc)

          Thanks,

          georgedj

          Show
          jiraposter@reviews.apache.org added a comment - ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/626/#review505 ----------------------------------------------------------- trunk/cli/src/java/org/apache/hadoop/hive/cli/CliDriver.java < https://reviews.apache.org/r/626/#comment1030 > Does this handle the case when a MapReduce job is launched and CLI got a CTL+C? Currently we have a JVM shutdownHook defined in HadoopJobExecHelper.java to kill the MR job before the JVM shut down (the old behavior when hitting a CTL+C). Here if we don't shut down the JVM, you should do the same thing to kill the running MR jobs. Ning On 2011-04-20 16:49:05, georgedj wrote: ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/626/ ----------------------------------------------------------- (Updated 2011-04-20 16:49:05) Review request for hive. Summary ------- Fixed by adding INT signal hander that interrupts the CLI thread. The CLI thread gets InterruptedException and it stops the current command. This addresses bug HIVE-243 . https://issues.apache.org/jira/browse/HIVE-243 Diffs ----- trunk/cli/src/java/org/apache/hadoop/hive/cli/CliDriver.java 1095439 Diff: https://reviews.apache.org/r/626/diff Testing ------- Manual test Ctrl+C on bunch of command (show tables, select, etc) Thanks, georgedj
          Hide
          jiraposter@reviews.apache.org added a comment -

          -----------------------------------------------------------
          This is an automatically generated e-mail. To reply, visit:
          https://reviews.apache.org/r/626/
          -----------------------------------------------------------

          (Updated 2011-04-20 16:49:05.429702)

          Review request for hive.

          Summary (updated)
          -------

          Fixed by adding INT signal hander that interrupts the CLI thread. The CLI thread gets InterruptedException and it stops the current command.

          This addresses bug HIVE-243.
          https://issues.apache.org/jira/browse/HIVE-243

          Diffs


          trunk/cli/src/java/org/apache/hadoop/hive/cli/CliDriver.java 1095439

          Diff: https://reviews.apache.org/r/626/diff

          Testing
          -------

          Manual test Ctrl+C on bunch of command (show tables, select, etc)

          Thanks,

          georgedj

          Show
          jiraposter@reviews.apache.org added a comment - ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/626/ ----------------------------------------------------------- (Updated 2011-04-20 16:49:05.429702) Review request for hive. Summary (updated) ------- Fixed by adding INT signal hander that interrupts the CLI thread. The CLI thread gets InterruptedException and it stops the current command. This addresses bug HIVE-243 . https://issues.apache.org/jira/browse/HIVE-243 Diffs trunk/cli/src/java/org/apache/hadoop/hive/cli/CliDriver.java 1095439 Diff: https://reviews.apache.org/r/626/diff Testing ------- Manual test Ctrl+C on bunch of command (show tables, select, etc) Thanks, georgedj
          Hide
          jiraposter@reviews.apache.org added a comment -

          -----------------------------------------------------------
          This is an automatically generated e-mail. To reply, visit:
          https://reviews.apache.org/r/626/
          -----------------------------------------------------------

          Review request for hive.

          Summary
          -------

          Fixed by adding INT signal hander that interrupts the CLI thread. The CLI thread gets InterruptedException and it stops the current command.

          This addresses bug HIVE-243.
          https://issues.apache.org/jira/browse/HIVE-243

          Diffs


          trunk/cli/src/java/org/apache/hadoop/hive/cli/CliDriver.java 1095439

          Diff: https://reviews.apache.org/r/626/diff

          Testing
          -------

          Manual test Ctrl+C on bunch of command (show tables, select, etc)

          Thanks,

          georgedj

          Show
          jiraposter@reviews.apache.org added a comment - ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/626/ ----------------------------------------------------------- Review request for hive. Summary ------- Fixed by adding INT signal hander that interrupts the CLI thread. The CLI thread gets InterruptedException and it stops the current command. This addresses bug HIVE-243 . https://issues.apache.org/jira/browse/HIVE-243 Diffs trunk/cli/src/java/org/apache/hadoop/hive/cli/CliDriver.java 1095439 Diff: https://reviews.apache.org/r/626/diff Testing ------- Manual test Ctrl+C on bunch of command (show tables, select, etc) Thanks, georgedj
          George Djabarov made changes -
          Affects Version/s 0.8.0 [ 12316178 ]
          Status Open [ 1 ] Patch Available [ 10002 ]
          Hide
          George Djabarov added a comment -

          When running interactive query through the CLI Ctrl+C will interrupt the query and go back to the hive prompt instead of terminating the process.

          Show
          George Djabarov added a comment - When running interactive query through the CLI Ctrl+C will interrupt the query and go back to the hive prompt instead of terminating the process.
          Hide
          George Djabarov added a comment -

          I have a patch that fixes this by handling the INT signal and interrupting the CLI thread. Most commands do behave when interrupted (with some error message).

          Show
          George Djabarov added a comment - I have a patch that fixes this by handling the INT signal and interrupting the CLI thread. Most commands do behave when interrupted (with some error message).
          Jerome Boulon made changes -
          Field Original Value New Value
          Link This issue relates to HIVE-1382 [ HIVE-1382 ]
          Hide
          Adam Kramer added a comment -

          Specifically, this is an issue because I may or may not have many variables "set" in my CLI instance. I routinely exit out of the CLI via ^C to kill a query, but then end up with (e.g.) the wrong number of reducers selected because it resets to the default.

          Show
          Adam Kramer added a comment - Specifically, this is an issue because I may or may not have many variables "set" in my CLI instance. I routinely exit out of the CLI via ^C to kill a query, but then end up with (e.g.) the wrong number of reducers selected because it resets to the default.
          Hide
          Adam Kramer added a comment -

          Also also, ^C when a query is half-entered should cancel the query in progress and return to the hive> prompt.

          Show
          Adam Kramer added a comment - Also also, ^C when a query is half-entered should cancel the query in progress and return to the hive> prompt.
          Hide
          Adam Kramer added a comment -

          A lot of scripts are currently being used to extend the CLI, for example to allow for tab-completion of table names. These scripts, however, take a long time to run--my version of hive takes about a minute to get me to the CLI. When I hit ^C because I want to stop a query, and the CLI quits, this is actually now a serious performance hit.

          Show
          Adam Kramer added a comment - A lot of scripts are currently being used to extend the CLI, for example to allow for tab-completion of table names. These scripts, however, take a long time to run--my version of hive takes about a minute to get me to the CLI. When I hit ^C because I want to stop a query, and the CLI quits, this is actually now a serious performance hit.
          Adam Kramer created issue -

            People

            • Assignee:
              George Djabarov
              Reporter:
              Adam Kramer
            • Votes:
              2 Vote for this issue
              Watchers:
              0 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development