Uploaded image for project: 'Apache HAWQ'
  1. Apache HAWQ
  2. HAWQ-1567

Unknown process holds the lock causes DROP TABLE hangs forever

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: Core
    • Labels:
      None

      Description

      On Hawq 2.2.0.0-incubating (Jun 2017), we meet several times that query is hanging for long time:

      1. 1. DROP TABLE hangs for tens of minutes, because it waits for AccessExclusiveLock.
      2. 2. BUT the lock is held by a ghost process ( not alive, and little message in log file is availabe to know what's up)

      A detailed context is pasted:

      postgres=# select procpid, sess_id, usesysid, xact_start, waiting, current_query from pg_stat_activity where current_query <> '<IDLE>';
      procpid | sess_id | usesysid | xact_start | waiting | current_query
      -----------------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
      91321 | 120242 | 328199 | 2017-11-28 14:45:52.631739+08 | t | drop table if exists ads_is_svc_rcv_approval_detail_df

      postgres=# select * from pg_locks where pid = 91321;
      locktype | database | relation | page | tuple | transactionid | classid | objid | objsubid | transaction | pid | mode | granted | mppsessionid | mppiswriter | gp_segment_id
      -----------------------------------------------------------------------------------------------------------------------------------------+--------------
      transactionid | | | | | 21867785 | | | | 21867785 | 91321 | ExclusiveLock | t | 120242 | f | -10000
      relation | 16510 | 2608 | | | | | | | 21867785 | 91321 | RowExclusiveLock | t | 120242 | f | -10000
      relation | 16510 | 1259 | | | | | | | 21867785 | 91321 | RowExclusiveLock | t | 120242 | f | -10000
      relation | 16510 | 3212612 | | | | | | | 21867785 | 91321 | AccessExclusiveLock | f | 120242 | f | -10000
      relation | 16510 | 1247 | | | | | | | 21867785 | 91321 | RowExclusiveLock | t | 120242 | f | -10000
      (5 rows)

      postgres=# select * from pg_locks where relation = 3212612;
      locktype | database | relation | page | tuple | transactionid | classid | objid | objsubid | transaction | pid | mode | granted | mppsessionid | mppiswriter | gp_segment_id
      -------------------------------------------------------------------------------------------------------------------------------------+--------------
      relation | 16510 | 3212612 | | | | | | | 21867785 | 91321 | AccessExclusiveLock | f | 120242 | f | -10000
      relation | 16510 | 3212612 | | | | | | | 0 | 107940 | AccessShareLock | t | 120553 | f | -10000
      (2 rows)

      postgres=# select * from pg_stat_activity where procpid = 107940;
      datid | datname | procpid | sess_id | usesysid | usename | current_query | waiting | query_start | backend_start | client_addr | client_port | application_name | xact_start | waiting_resource
      -------------------------------------------------------------------------------------------------------------------------------------------------------
      (0 rows)

      postgres=# select * from pg_locks where pid = 107940 or mppsessionid = 120553;
      locktype | database | relation | page | tuple | transactionid | classid | objid | objsubid | transaction | pid | mode | granted | mppsessionid | mppiswriter | gp_segment_id
      -------------------------------------------------------------------------------------------------------------------+--------------
      (0 rows)

      postgres=# select logtime, logpid, logmessage from hawq_toolkit.__hawq_log_master_latest where logsession = 'con120553';
      logtime | logpid | logmessage
      -------------------------------------------------------------------------------------
      2017-11-28 15:14:38.277254+08 | p107940 | clean up communication to resource manager now.
      2017-11-28 15:14:38.322206+08 | p107940 | generateResourceRefreshHeartBeat exits.
      2017-11-28 15:14:38.388077+08 | p107940 | APSARA uninit:DONE
      (3 rows)

      In the master log file, it is not easy to know who is accessing the relation.

      To Yi Jin and all: any suggestion on it? thanks a lot.

        Attachments

          Activity

            People

            • Assignee:
              rlei Radar Lei
              Reporter:
              kuien Kuien Liu
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated: