Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-15223

Make convertScanToString public for Spark

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • 1.3.0, 2.0.0
    • None
    • None
    • Reviewed

    Description

      One way to access HBase from Spark is to use newAPIHadoopRDD, which can take a TableInputFormat as class name. But we are not able to set a Scan object in there, for example to set a HBase filter.

      In MR, the public API TableMapReduceUtil.initTableMapperJob() or equivalent is used which can take a Scan object. But this call is not used in Spark conveniently.

      We need to make the TableMapReduceUtil.convertScanToString() public.
      So that a Scan object can be created, populated and then convert to the property and used by Spark. They are now package private.

      Attachments

        1. HBASE-15223-branch-1.patch
          5 kB
          Jerry He
        2. HBASE-15223-branch-1-v2.patch
          6 kB
          Jerry He
        3. HBASE-15223-master.patch
          5 kB
          Michael Stack
        4. HBASE-15223-master.patch
          5 kB
          Jerry He

        Activity

          People

            jinghe Jerry He
            jinghe Jerry He
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: