Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-24276

hbase spark connector doesn't support writing to table not in default namespace

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: connector-1.0.0
    • Fix Version/s: connector-1.0.1
    • Component/s: hbase-connectors, spark
    • Labels:
      None
    • Environment:
      • HBase 2.2.4
      • Hadoop 2.10.0
      • Spark 2.4.5

      Description

      Defining the following table catalog:

      val catalog = """{
        |"table": {"namespace": "ns1", "name": "test1"},
        |"rowkey": "id",
        |"columns": {
          |"id": {"cf": "rowkey", "col": "id", "type": "string"}, 
          |"x": {"cf": "d", "col": "xxx", "type": "int"}
        |}
      |}""".stripMargin
      

      Try to write some test data to ns1:test1 table using spark:

      val df = Seq(("abc", 1), ("def", 2)).toDF("id", "x")
      df.write.options(Map(HBaseTableCatalog.tableCatalog -> catalog, "hbase.spark.use.hbasecontext" -> "false", HBaseTableCatalog.newTable-> "5")).format("org.apache.hadoop.hbase.spark").save()
      

      After executing the code above, I found out that the test data was written to default:test1, rather than ns1:test1.

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              naitree Naitree Zhu
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: