Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-20056

Performance optimization on MultiTableInputFormatBase#getSplits()

    XMLWordPrintableJSON

Details

    • Reviewed

    Description

      Currently this method iterates the List of scan objects to get splits and for each iteration it opens the HConnection object and closes it, which is heavy.

      It can be optimized such that a single Hconnection can be used to compute all the splits of for all the scan objects for their splits computation.

      This optimization will help in reducing the launch time for MR Job.

      Attachments

        1. HBASE-20056.branch-1.2.patch
          6 kB
          Yechao Chen

        Issue Links

          relates to

          Bug - A problem which impairs or prevents the functions of the product. HBASE-19245 MultiTableInputFormatBase#getSplits creates a Connection per Table

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved

          Activity

            People

              chenyechao Yechao Chen
              shivakumar.ss ShivaKumar SS
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: