Uploaded image for project: 'Phoenix'
  1. Phoenix
  2. PHOENIX-4489

HBase Connection leak in Phoenix MR Jobs

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 4.14.0, 5.0.0
    • Labels:
      None

      Description

      Phoenix MR jobs uses a custom class PhoenixInputFormat to determine the splits and the parallelism of the work. The class directly opens up a HBase connection, which is not closed after the usage. Independently running MR jobs should not have any concern, however jobs that run through Phoenix-Spark can cause leak issues if this is left unclosed (since those jobs run as a part of same JVM).

      Apart from this, the connection should be instantiated with HBaseFactoryProvider.getHConnectionFactory() instead of the default one. It can be useful if a separate client is trying to run jobs and wants to provide a custom implementation of HConnection.

      Josh Mahonin Any ideas?
      James Taylor Vincent Poon Any concerns around this?

        Attachments

        1. PHOENIX-4489.001.patch
          8 kB
          Karan Mehta
        2. PHOENIX-4489.002.patch
          3 kB
          Karan Mehta
        3. PHOENIX-4489.4.x-HBase-0.98.001.patch
          3 kB
          Karan Mehta

          Issue Links

            Activity

              People

              • Assignee:
                karanmehta93 Karan Mehta
                Reporter:
                karanmehta93 Karan Mehta
              • Votes:
                0 Vote for this issue
                Watchers:
                8 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: