Hive
  1. Hive
  2. HIVE-2264

Hive server is SHUTTING DOWN when invalid queries beeing executed.

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Blocker Blocker
    • Resolution: Fixed
    • Affects Version/s: 0.9.0
    • Fix Version/s: 0.11.0
    • Labels:
      None
    • Environment:

      SuSE-Linux-11

      Description

      When invalid query is beeing executed, Hive server is shutting down.

      "CREATE TABLE SAMPLETABLE(IP STRING , showtime BIGINT ) partitioned by (ds string,ipz int) ROW FORMAT DELIMITED FIELDS TERMINATED BY '\040'"
      
      "ALTER TABLE SAMPLETABLE add Partition(ds='sf') location '/user/hive/warehouse' Partition(ipz=100) location '/user/hive/warehouse'"
      
      1. HIVE-2264-2.patch
        7 kB
        Brock Noland
      2. HIVE-2264.D9489.1.patch
        6 kB
        Phabricator
      3. HIVE-2264.1.patch.txt
        7 kB
        Navis

        Issue Links

          Activity

          Rohith created issue -
          Rohith made changes -
          Field Original Value New Value
          Description When invalid query is beeing executed, Hive server is shutting down.

          {noformat}
          "CREATE TABLE SAMPLETABLE(IP STRING , showtime BIGINT ) partitioned by (ds string,ipz int) ROW FORMAT DELIMITED FIELDS TERMINATED BY '\040'"

          "ALTER TABLE SAMPLETABLE add Partition(ds='sf') location '/user/hive/warehouse' Partition(ipz=100) location '/user/hive/warehouse'"
          {noformat}

          When invalid query is beeing executed, Hive server is shutting down.

          {noformat}
          "CREATE TABLE SAMPLETABLE(IP STRING , showtime BIGINT ) partitioned by (ds string,ipz int) ROW FORMAT DELIMITED FIELDS TERMINATED BY '\040'"

          "ALTER TABLE SAMPLETABLE add Partition(ds='sf') location '/user/hive/warehouse' Partition(ipz=100) location '/user/hive/warehouse'"
          {noformat}
          Rohith made changes -
          Summary Hive server is SHUTTING DOWN when invalid/complex queries been used. Hive server is SHUTTING DOWN when invalid queries beeing executed.
          Hide
          Navis added a comment -
          Show
          Navis added a comment - https://reviews.apache.org/r/3025/
          Navis made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Affects Version/s 0.9.0 [ 12317742 ]
          Affects Version/s 0.7.0 [ 12315150 ]
          Affects Version/s 0.7.1 [ 12316336 ]
          Fix Version/s 0.9.0 [ 12317742 ]
          Navis made changes -
          Attachment HIVE-2264.1.patch.txt [ 12506236 ]
          Hide
          jiraposter@reviews.apache.org added a comment -

          -----------------------------------------------------------
          This is an automatically generated e-mail. To reply, visit:
          https://reviews.apache.org/r/3025/
          -----------------------------------------------------------

          Review request for hive, John Sichi and Carl Steinbach.

          Summary
          -------

          Currently, if Driver fails executing any of the task, it tries suicide by System.exit() which should not be happened in our use cases.

          This is simple hot-fix I've provided to my coworkers. Someone could do more elaborated work.

          This addresses bug HIVE-2264.
          https://issues.apache.org/jira/browse/HIVE-2264

          Diffs


          ql/src/java/org/apache/hadoop/hive/ql/Driver.java da02fa5
          ql/src/java/org/apache/hadoop/hive/ql/exec/ExecDriver.java f612119
          ql/src/java/org/apache/hadoop/hive/ql/exec/MapRedTask.java a3e40f7
          ql/src/java/org/apache/hadoop/hive/ql/exec/MapredLocalTask.java d2b12a7
          ql/src/java/org/apache/hadoop/hive/ql/exec/Task.java a4e59ca
          ql/src/test/queries/clientnegative/driver_suicide.q PRE-CREATION
          ql/src/test/results/clientnegative/driver_suicide.q.out PRE-CREATION

          Diff: https://reviews.apache.org/r/3025/diff

          Testing
          -------

          test added : driver_suicide.q (negative)

          Thanks,

          Navis

          Show
          jiraposter@reviews.apache.org added a comment - ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/3025/ ----------------------------------------------------------- Review request for hive, John Sichi and Carl Steinbach. Summary ------- Currently, if Driver fails executing any of the task, it tries suicide by System.exit() which should not be happened in our use cases. This is simple hot-fix I've provided to my coworkers. Someone could do more elaborated work. This addresses bug HIVE-2264 . https://issues.apache.org/jira/browse/HIVE-2264 Diffs ql/src/java/org/apache/hadoop/hive/ql/Driver.java da02fa5 ql/src/java/org/apache/hadoop/hive/ql/exec/ExecDriver.java f612119 ql/src/java/org/apache/hadoop/hive/ql/exec/MapRedTask.java a3e40f7 ql/src/java/org/apache/hadoop/hive/ql/exec/MapredLocalTask.java d2b12a7 ql/src/java/org/apache/hadoop/hive/ql/exec/Task.java a4e59ca ql/src/test/queries/clientnegative/driver_suicide.q PRE-CREATION ql/src/test/results/clientnegative/driver_suicide.q.out PRE-CREATION Diff: https://reviews.apache.org/r/3025/diff Testing ------- test added : driver_suicide.q (negative) Thanks, Navis
          John Sichi made changes -
          Assignee Navis [ navis ]
          Hide
          Ashutosh Chauhan added a comment -

          Unlinking from 0.9

          Show
          Ashutosh Chauhan added a comment - Unlinking from 0.9
          Ashutosh Chauhan made changes -
          Fix Version/s 0.9.0 [ 12317742 ]
          Hide
          Zhenxiao Luo added a comment -

          This patch is very useful. like the idea.
          Non-Committer +1.

          Show
          Zhenxiao Luo added a comment - This patch is very useful. like the idea. Non-Committer +1.
          Brock Noland made changes -
          Link This issue is related to HIVE-2017 [ HIVE-2017 ]
          Brock Noland made changes -
          Link This issue is related to HIVE-1872 [ HIVE-1872 ]
          Hide
          Brock Noland added a comment -

          Ran into this myself. +1.

          Show
          Brock Noland added a comment - Ran into this myself. +1.
          Hide
          Brock Noland added a comment -

          Hi,

          I rebased this patch on trunk (attached) and removed the commented out System.exit().

          Brock

          Show
          Brock Noland added a comment - Hi, I rebased this patch on trunk (attached) and removed the commented out System.exit(). Brock
          Brock Noland made changes -
          Attachment HIVE-2264-2.patch [ 12571214 ]
          Hide
          Brock Noland added a comment -

          Navis, you can use my rebased patch to update review board or if you don't have interest in this any longer, no worries, I'd be willing to take it up.

          Show
          Brock Noland added a comment - Navis, you can use my rebased patch to update review board or if you don't have interest in this any longer, no worries, I'd be willing to take it up.
          Hide
          Navis added a comment -

          I'm applying this for all internal hive releases and wish to be reviewed/applied into apache hive. But sadly, no committer seemed interested in it.

          It's already in patch-available status. Do you have more idea to be merged with this? Then I'll happily assign it to you.

          Show
          Navis added a comment - I'm applying this for all internal hive releases and wish to be reviewed/applied into apache hive. But sadly, no committer seemed interested in it. It's already in patch-available status. Do you have more idea to be merged with this? Then I'll happily assign it to you.
          Hide
          Brock Noland added a comment -

          Navis You have been running with this patch for quite a long time? In regards to getting it merged, I think the best we can do is update the review board item with the rebased patch. Another item that may bring it up in terms of visibility is linking it to HIVE-2935 as it's quite important for HS2.

          Show
          Brock Noland added a comment - Navis You have been running with this patch for quite a long time? In regards to getting it merged, I think the best we can do is update the review board item with the rebased patch. Another item that may bring it up in terms of visibility is linking it to HIVE-2935 as it's quite important for HS2.
          Brock Noland made changes -
          Link This issue is depended upon by HIVE-2935 [ HIVE-2935 ]
          Hide
          Brock Noland added a comment -

          I think we should mark this as a blocker as this can kill HS2 which can have dozens of jobs running in the cluster for clients.

          Show
          Brock Noland added a comment - I think we should mark this as a blocker as this can kill HS2 which can have dozens of jobs running in the cluster for clients.
          Brock Noland made changes -
          Fix Version/s 0.11.0 [ 12323587 ]
          Hide
          Edward Capriolo added a comment -

          Navis, if Brock rebased the patch you can commit. I am +1 as well.

          Show
          Edward Capriolo added a comment - Navis, if Brock rebased the patch you can commit. I am +1 as well.
          Hide
          Phabricator added a comment -

          navis requested code review of "HIVE-2264 [jira] Hive server is SHUTTING DOWN when invalid queries beeing executed.".

          Reviewers: JIRA

          HIVE-2264 Hive server is SHUTTING DOWN when invalid queries beeing executed

          When invalid query is beeing executed, Hive server is shutting down.

          "CREATE TABLE SAMPLETABLE(IP STRING , showtime BIGINT ) partitioned by (ds string,ipz int) ROW FORMAT DELIMITED FIELDS TERMINATED BY '\040'"

          "ALTER TABLE SAMPLETABLE add Partition(ds='sf') location '/user/hive/warehouse' Partition(ipz=100) location '/user/hive/warehouse'"

          TEST PLAN
          EMPTY

          REVISION DETAIL
          https://reviews.facebook.net/D9489

          AFFECTED FILES
          ql/src/java/org/apache/hadoop/hive/ql/Driver.java
          ql/src/java/org/apache/hadoop/hive/ql/exec/ExecDriver.java
          ql/src/java/org/apache/hadoop/hive/ql/exec/MapRedTask.java
          ql/src/java/org/apache/hadoop/hive/ql/exec/MapredLocalTask.java
          ql/src/java/org/apache/hadoop/hive/ql/exec/Task.java

          MANAGE HERALD RULES
          https://reviews.facebook.net/herald/view/differential/

          WHY DID I GET THIS EMAIL?
          https://reviews.facebook.net/herald/transcript/22683/

          To: JIRA, navis

          Show
          Phabricator added a comment - navis requested code review of " HIVE-2264 [jira] Hive server is SHUTTING DOWN when invalid queries beeing executed.". Reviewers: JIRA HIVE-2264 Hive server is SHUTTING DOWN when invalid queries beeing executed When invalid query is beeing executed, Hive server is shutting down. "CREATE TABLE SAMPLETABLE(IP STRING , showtime BIGINT ) partitioned by (ds string,ipz int) ROW FORMAT DELIMITED FIELDS TERMINATED BY '\040'" "ALTER TABLE SAMPLETABLE add Partition(ds='sf') location '/user/hive/warehouse' Partition(ipz=100) location '/user/hive/warehouse'" TEST PLAN EMPTY REVISION DETAIL https://reviews.facebook.net/D9489 AFFECTED FILES ql/src/java/org/apache/hadoop/hive/ql/Driver.java ql/src/java/org/apache/hadoop/hive/ql/exec/ExecDriver.java ql/src/java/org/apache/hadoop/hive/ql/exec/MapRedTask.java ql/src/java/org/apache/hadoop/hive/ql/exec/MapredLocalTask.java ql/src/java/org/apache/hadoop/hive/ql/exec/Task.java MANAGE HERALD RULES https://reviews.facebook.net/herald/view/differential/ WHY DID I GET THIS EMAIL? https://reviews.facebook.net/herald/transcript/22683/ To: JIRA, navis
          Phabricator made changes -
          Attachment HIVE-2264.D9489.1.patch [ 12574100 ]
          Hide
          Navis added a comment -

          Ok, let's make this committed. Rebased to trunk.

          Show
          Navis added a comment - Ok, let's make this committed. Rebased to trunk.
          Hide
          Brock Noland added a comment -

          Navis,

          Edward already +1'd the patch. Are you looking for a +1 on the rebased patch?

          Show
          Brock Noland added a comment - Navis, Edward already +1'd the patch. Are you looking for a +1 on the rebased patch?
          Hide
          Navis added a comment -
          Show
          Navis added a comment - I was intended to do that but I've reminded comments from Namit Jain 'dont commit our own patches' ( https://issues.apache.org/jira/browse/HIVE-3171?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13440920#comment-13440920 ).
          Hide
          Brock Noland added a comment -

          Marking as blocker for 0.11.

          Show
          Brock Noland added a comment - Marking as blocker for 0.11.
          Brock Noland made changes -
          Priority Critical [ 2 ] Blocker [ 1 ]
          Carl Steinbach made changes -
          Component/s HiveServer2 [ 12320408 ]
          Hide
          Carl Steinbach added a comment -

          @Navis: we relaxed that rule. You can commit your own patches as long as you get a +1 from another committer. You're good to go.

          Show
          Carl Steinbach added a comment - @Navis: we relaxed that rule. You can commit your own patches as long as you get a +1 from another committer. You're good to go.
          Hide
          Navis added a comment -

          Committed and was my first commit. Thanks to all.

          Show
          Navis added a comment - Committed and was my first commit. Thanks to all.
          Navis made changes -
          Status Patch Available [ 10002 ] Resolved [ 5 ]
          Resolution Fixed [ 1 ]
          Hide
          Phabricator added a comment -

          navis has abandoned the revision "HIVE-2264 [jira] Hive server is SHUTTING DOWN when invalid queries beeing executed.".

          Committed

          REVISION DETAIL
          https://reviews.facebook.net/D9489

          To: JIRA, navis

          Show
          Phabricator added a comment - navis has abandoned the revision " HIVE-2264 [jira] Hive server is SHUTTING DOWN when invalid queries beeing executed.". Committed REVISION DETAIL https://reviews.facebook.net/D9489 To: JIRA, navis
          Hide
          Brock Noland added a comment -

          Awesome! Thanks guys!

          Show
          Brock Noland added a comment - Awesome! Thanks guys!
          Hide
          Hudson added a comment -

          Integrated in Hive-trunk-h0.21 #2036 (See https://builds.apache.org/job/Hive-trunk-h0.21/2036/)
          HIVE-2264 Hive server is SHUTTING DOWN when invalid queries beeing executed (Revision 1462406)

          Result = FAILURE
          navis : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1462406
          Files :

          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/Driver.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/ExecDriver.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/MapRedTask.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/MapredLocalTask.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/Task.java
          Show
          Hudson added a comment - Integrated in Hive-trunk-h0.21 #2036 (See https://builds.apache.org/job/Hive-trunk-h0.21/2036/ ) HIVE-2264 Hive server is SHUTTING DOWN when invalid queries beeing executed (Revision 1462406) Result = FAILURE navis : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1462406 Files : /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/Driver.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/ExecDriver.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/MapRedTask.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/MapredLocalTask.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/Task.java
          Hide
          Hudson added a comment -

          Integrated in Hive-trunk-hadoop2 #138 (See https://builds.apache.org/job/Hive-trunk-hadoop2/138/)
          HIVE-2264 Hive server is SHUTTING DOWN when invalid queries beeing executed (Revision 1462406)

          Result = FAILURE
          navis : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1462406
          Files :

          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/Driver.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/ExecDriver.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/MapRedTask.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/MapredLocalTask.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/Task.java
          Show
          Hudson added a comment - Integrated in Hive-trunk-hadoop2 #138 (See https://builds.apache.org/job/Hive-trunk-hadoop2/138/ ) HIVE-2264 Hive server is SHUTTING DOWN when invalid queries beeing executed (Revision 1462406) Result = FAILURE navis : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1462406 Files : /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/Driver.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/ExecDriver.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/MapRedTask.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/MapredLocalTask.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/Task.java
          Owen O'Malley made changes -
          Status Resolved [ 5 ] Closed [ 6 ]

            People

            • Assignee:
              Navis
              Reporter:
              Rohith
            • Votes:
              1 Vote for this issue
              Watchers:
              11 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development