[HBASE-18570] use hbase-spark without HBaseContext runs into NPE

• Type: Improvement
• Status: In Progress
• Priority: Minor
• Resolution: Unresolved
• Affects Version/s: 1.2.0
• Assignee: Junegunn Choi
• Reporter: Yuexin Zhang
• Created: 11/Aug/17 09:26
• Updated: 3 days ago 05:20

Description

I recently run into the same issue as described in stackoverflow :

https://stackoverflow.com/questions/38865558/sparksql-dataframes-does-not-work-in-spark-shell-and-application#

If we don't explicitly initialize a HBaseContext and don't set hbase.use.hbase.context option to false, it will run into NPE at:

    val wrappedConf = new SerializableConfiguration(hbaseContext.config)

https://github.com/apache/hbase/blob/master/hbase-spark/src/main/scala/org/apache/hadoop/hbase/spark/DefaultSource.scala#L140

Should we safe guard with a NULL validation on hbaseContext?

Something like:

    //create or get latest HBaseContext
  val hbaseContext:HBaseContext = if (useHBaseContext && null != LatestHBaseContextCache.latest) {
    LatestHBaseContextCache.latest
  } else {
    val config = HBaseConfiguration.create()
    configResources.split(",").foreach( r => config.addResource(r))
    new HBaseContext(sqlContext.sparkContext, config)
  }

Or maybe it's better to make sure the HBaseContext is instantiated properly.

Issue Links

• links to: GitHub Pull Request #127