Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-20056

Performance optimization on MultiTableInputFormatBase#getSplits()

    XMLWordPrintableJSON

    Details

    • Hadoop Flags:
      Reviewed

      Description

      Currently this method iterates the List of scan objects to get splits and for each iteration it opens the HConnection object and closes it, which is heavy.

      It can be optimized such that a single Hconnection can be used to compute all the splits of for all the scan objects for their splits computation.

      This optimization will help in reducing the launch time for MR Job.

        Attachments

        1. HBASE-20056.branch-1.2.patch
          6 kB
          Yechao Chen

          Issue Links

            Activity

              People

              • Assignee:
                chenyechao Yechao Chen
                Reporter:
                shivakumar.ss ShivaKumar SS
              • Votes:
                0 Vote for this issue
                Watchers:
                5 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: