[HBASE-14789] Enhance the current spark-hbase connector

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 3.0.0, connector-1.0.0
• Component/s: hbase-connectors, spark
• Labels: spark

Release Note

New features in hbase-spark:
* native type support (short, int, long, float, double),
* support for Dataframe writes,
* avro support,
* catalog can be defined in json.

• Assignee: Zhan Zhang
• Reporter: Zhan Zhang
• Created: 09/Nov/15 22:41
• Updated: 27/Apr/19 01:26
• Resolved: 26/Apr/19 10:48

Description

This JIRA is to optimize the RDD construction in the current connector implementation.

Issue Links

• incorporates: HBASE-14801 Enhance the Spark-HBase connector catalog with json format
• incorporates: HBASE-15572 Adding optional timestamp semantics to HBase-Spark
• incorporates: HBASE-14795 Enhance the spark-hbase scan operations
• incorporates: HBASE-14796 Enhance the Gets in the connector

Sub-Tasks

• [hbase-spark] Enhance dataframe filters to handle naively encoded short, integer, long, float and double: Resolved
• Add avro support for spark hbase connector: Resolved
• Support Dataframe writer to the spark connector: Resolved
• Enable individual unit test in hbase-spark module: Resolved
• Enhance the Spark-HBase connector catalog with json format: Resolved
• Documentation for the usage of hbase dataframe user api (JSON, Avro, etc): Resolved
• Fix the null pointer in DynamicLogicExpressionSuite: Resolved