Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-14796

Enhance the Gets in the connector

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.0.0
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed
    • Release Note:
      spark.hbase.bulkGetSize in HBaseSparkConf is for grouping bulkGet, and default value is 1000.

      Description

      Current the Spark-Module Spark SQL implementation gets records from HBase from the driver if there is something like the following found in the SQL.

      rowkey = 123

      The reason for this original was normal sql will not have many equal operations in a single where clause.

      Zhan, had brought up too points that have value.
      1. The SQL may be generated and may have many many equal statements in it so moving the work to an executor protects the driver from load
      2. In the correct implementation the drive is connecting to HBase and exceptions may cause trouble with the Spark application and not just with the a single task execution

        Attachments

        1. HBASE-14976.patch
          11 kB
          Zhan Zhang
        2. HBASE-14796-1.patch
          11 kB
          Zhan Zhang

          Issue Links

            Activity

              People

              • Assignee:
                zhanzhang Zhan Zhang
                Reporter:
                ted.m Theodore michael Malaska
              • Votes:
                0 Vote for this issue
                Watchers:
                7 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: