[PHOENIX-4489] HBase Connection leak in Phoenix MR Jobs

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 4.14.0, 5.0.0
• Assignee: Karan Mehta
• Reporter: Karan Mehta
• Created: 23/Dec/17 04:27
• Updated: 26/Jul/18 04:14
• Resolved: 24/Jan/18 10:44

Description

Phoenix MR jobs uses a custom class PhoenixInputFormat to determine the splits and the parallelism of the work. The class directly opens up a HBase connection, which is not closed after the usage. Independently running MR jobs should not have any concern, however jobs that run through Phoenix-Spark can cause leak issues if this is left unclosed (since those jobs run as a part of same JVM).

Apart from this, the connection should be instantiated with HBaseFactoryProvider.getHConnectionFactory() instead of the default one. It can be useful if a separate client is trying to run jobs and wants to provide a custom implementation of HConnection.

Josh Mahonin Any ideas?
James R. Taylor Vincent Poon Any concerns around this?

Issue Links

• is caused by: PHOENIX-3600 Core MapReduce classes don't provide location info
• is duplicated by: PHOENIX-4503 Phoenix-Spark plugin doesn't release zookeeper connections
• is duplicated by: PHOENIX-4631 PhoenixInputFormat should close connection after generateSplits()
• relates to: PHOENIX-4247 Phoenix/Spark/ZK connection