Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-16117

Fix Connection leak in mapred.TableOutputFormat

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 1.3.0, 1.2.2, 1.1.6, 2.0.0
    • 2.0.0
    • mapreduce
    • None
    • Incompatible change, Reviewed
    • Hide
      (This change will be irrelevant after HBASE-16774 lands).
      There is a subtle change with error handling when a connection is not able to connect to ZK. Attempts to create a connection when ZK is not up will now fail immediately instead of silently creating and then failing on a subsequent HBaseAdmin call.

      Show
      (This change will be irrelevant after HBASE-16774 lands). There is a subtle change with error handling when a connection is not able to connect to ZK. Attempts to create a connection when ZK is not up will now fail immediately instead of silently creating and then failing on a subsequent HBaseAdmin call.

    Description

      Spark seems to instantiate multiple instances of output formats within a single process. When mapred.TableOutputFormat (not mapreduce.TableOutputFormat) is used, this may cause connection leaks that slowly exhaust the cluster's zk connections.

      This patch fixes that.

      Attachments

        1. hbase-16117.part2.v6.patch
          10 kB
          Jonathan Hsieh
        2. hbase-16117-part2.v5.patch
          10 kB
          Jonathan Hsieh
        3. hbase-16117.branch-1.002.patch
          12 kB
          Jonathan Hsieh
        4. HBASE-16117.branch-1.001.patch
          13 kB
          Jonathan Hsieh
        5. hbase-16117.v3.branch-1.patch
          14 kB
          Jonathan Hsieh
        6. hbase-16117.v2.branch-1.patch
          14 kB
          Jonathan Hsieh
        7. hbase-16117.v4.patch
          13 kB
          Jonathan Hsieh
        8. hbase-16117.v3.patch
          12 kB
          Jonathan Hsieh
        9. hbase-16117.v2.patch
          13 kB
          Jonathan Hsieh
        10. hbase-16117.patch
          9 kB
          Jonathan Hsieh
        11. hbase-16117.branch-1.patch
          9 kB
          Jonathan Hsieh

        Issue Links

          is duplicated by

          Bug - A problem which impairs or prevents the functions of the product. HBASE-16017 HBase TableOutputFormat has connection leak in getRecordWriter

          • Major - Major loss of function.
          • Resolved
          is related to

          Improvement - An improvement or enhancement to an existing feature or task. HBASE-16774 [shell] Add coverage to TestShell when ZooKeeper is not reachable

          • Major - Major loss of function.
          • Resolved
          links to

          Web Link Review Board (branch-1)

          Activity

            People

              jmhsieh Jonathan Hsieh
              jmhsieh Jonathan Hsieh
              Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: