[HIVE-20511] REPL DUMP is leaking metastore connections

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 4.0.0
• Fix Version/s: 4.0.0-alpha-1
• Component/s: repl
• Labels: pull-request-available
• Target Version/s: 4.0.0
• Assignee: mahesh kumar behera
• Reporter: mahesh kumar behera
• Created: 06/Sep/18 15:39
• Updated: 17/Nov/22 08:54
• Resolved: 24/Sep/18 06:32

Description

With remote metastore, REPL DUMP leaking connections. Each repl dump task is leaking one connection due to the usage of stale hive object.

18/09/04 16:01:46 INFO ReplState: REPL::EVENT_DUMP: {"dbName":"*","eventId":"566","eventType":"EVENT_COMMIT_TXN","eventsDumpProgress":"1/0","dumpTime":1536076906}
18/09/04 16:01:46 INFO events.AbstractEventHandler: Processing#567 OPEN_TXN message : {"txnIds":null,"timestamp":1536076905,"fromTxnId":269,"toTxnId":269,"server":"thrift://metastore-service.warehouse-1536062326-s74h.svc.cluster.local:9083","servicePrincipal":""}
18/09/04 16:01:46 INFO ReplState: REPL::EVENT_DUMP: {"dbName":"*","eventId":"567","eventType":"EVENT_OPEN_TXN","eventsDumpProgress":"2/0","dumpTime":1536076906}
18/09/04 16:01:46 INFO metastore.HiveMetaStoreClient: Trying to connect to metastore with URI thrift://metastore-service.warehouse-1536062326-s74h.svc.cluster.local:9083
18/09/04 16:01:46 INFO metastore.HiveMetaStoreClient: Opened a connection to metastore, current connections: 471
18/09/04 16:01:46 INFO metastore.HiveMetaStoreClient: Connected to metastore.
18/09/04 16:01:46 INFO metastore.RetryingMetaStoreClient: RetryingMetaStoreClient proxy=class org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient ugi=hive (auth:SIMPLE) retries=24 delay=5 lifetime=0
18/09/04 16:01:46 INFO ReplState: REPL::END: {"dbName":"*","dumpType":"INCREMENTAL","actualNumEvents":2,"dumpEndTime":1536076906,"dumpDir":"/user/hive/repl/e45bde27-74dc-45cd-9823-400a8fc1aea3","lastReplId":"567"}
18/09/04 16:01:46 INFO repl.ReplDumpTask: Done dumping events, preparing to return /user/hive/repl/e45bde27-74dc-45cd-9823-400a8fc1aea3,567
18/09/04 16:01:46 INFO ql.Driver: Completed executing command(queryId=hive_20180904160145_30f9570a-44e0-4f3b-b961-1906d3972fc4); Time taken: 0.585 seconds
OK
18/09/04 16:01:46 INFO ql.Driver: OK
18/09/04 16:01:46 INFO lockmgr.DbTxnManager: Stopped heartbeat for query: hive_20180904160145_30f9570a-44e0-4f3b-b961-1906d3972fc4
18/09/04 16:01:46 INFO metastore.HiveMetaStoreClient: Trying to connect to metastore with URI thrift://metastore-service.warehouse-1536062326-s74h.svc.cluster.local:9083
18/09/04 16:01:46 INFO metastore.HiveMetaStoreClient: Opened a connection to metastore, current connections: 472
18/09/04 16:01:46 INFO metastore.HiveMetaStoreClient: Connected to metastore.

Issue Links

• links to: GitHub Pull Request #429
• links to: GitHub Pull Request #429