Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-5268

After canceling query on secure cluster coordinator node doesn't accept new connections

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Duplicate
    • Affects Version/s: Impala 2.9.0
    • Fix Version/s: Impala 2.9.0
    • Component/s: Distributed Exec
    • Labels:
      None
    • Epic Color:
      ghx-label-8

      Description

      After canceling a query on a secure cluster the coordinator node doesn't accept any new connections.

      Interestingly after killing all running impala-shell processes the coordinator node started accepting new connections.

      [##################                                                                                  ] 18%
      +--------------+--------+----------+----------+-------+------------+----------+---------------+----------------------------------------------------------+
      | Operator     | #Hosts | Avg Time | Max Time | #Rows | Est. #Rows | Peak Mem | Est. Peak Mem | Detail                                                   |
      +--------------+--------+----------+----------+-------+------------+----------+---------------+----------------------------------------------------------+
      | 03:AGGREGATE | 1      | 116.54us | 116.54us | 0     | 1          | 8.00 KB  | 10.00 MB      | FINALIZE                                                 |
      | 02:EXCHANGE  | 1      | 0ns      | 0ns      | 0     | 1          | 0 B      | 0 B           | UNPARTITIONED                                            |
      | 01:AGGREGATE | 7      | 0ns      | 0ns      | 0     | 1          | 8.12 MB  | 10.00 MB      |                                                          |
      | 00:SCAN HDFS | 7      | 31.89s   | 34.14s   | 0     | 1          | 78.66 MB | 88.00 MB      | scan_primitives_tpch_3tb.lineiten_parquet_un_partitioned |
      +--------------+--------+----------+----------+-------+------------+----------+---------------+----------------------------------------------------------+
      ^C Cancelling Query
      Failed to reconnect and close (try 1/3): Could not start SASL: Error in sasl_client_start (-1) SASL(-1): generic failure: GSSAPI Error: Unspecified GSS failure.  Minor code may provide more information (Ticket expired)
       Cancelling Query
      
      1. IMPALA-5268.stacks.out
        985 kB
        Matthew Jacobs
      2. impalad-stacks-full-30143.out
        673 kB
        Mostafa Mokhtar

        Issue Links

          Activity

          Hide
          mjacobs Matthew Jacobs added a comment -

          Downgrading for now because I wasn't able to reproduce the issue and can't find anything leading to what may have caused this (no logs are available, and the stacks don't appear to have anything suspicious). If this happens again we need to get logs sooner.

          Show
          mjacobs Matthew Jacobs added a comment - Downgrading for now because I wasn't able to reproduce the issue and can't find anything leading to what may have caused this (no logs are available, and the stacks don't appear to have anything suspicious). If this happens again we need to get logs sooner.
          Hide
          mjacobs Matthew Jacobs added a comment -

          Mostafa saw this again, I was able to capture the stacks but not sure what's causing this.

          Show
          mjacobs Matthew Jacobs added a comment - Mostafa saw this again, I was able to capture the stacks but not sure what's causing this.
          Hide
          dhecht Dan Hecht added a comment - - edited

          Matthew Jacobs/Sailesh Mukil, we believe this is a dup of IMPALA-5394, correct?

          Show
          dhecht Dan Hecht added a comment - - edited Matthew Jacobs / Sailesh Mukil , we believe this is a dup of IMPALA-5394 , correct?
          Hide
          mjacobs Matthew Jacobs added a comment -

          correct, I'll close this one since that describes the issue more precisely

          Show
          mjacobs Matthew Jacobs added a comment - correct, I'll close this one since that describes the issue more precisely

            People

            • Assignee:
              Unassigned
              Reporter:
              mmokhtar Mostafa Mokhtar
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development