Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-33164

HBase connector support ignore null value for partial update

    XMLWordPrintableJSON

Details

    Description

      Sometimes, user want to write data and ignore null value to achieve partial update. So i suggest adding an options: sink.ignore-null-value.

       

      CREATE TABLE hTable (
       rowkey STRING,
       cf1 ROW<q1 STRING, q2 STRING>,
       PRIMARY KEY (rowkey) NOT ENFORCED
      ) WITH (
       'connector' = 'hbase-2.2',
       'table-name' = 'default:test',
       'zookeeper.quorum' = 'localhost:2181',
       'sink.ignore-null-value' = 'true' -- default is false, true is enabled
      );
      
      INSERT INTO hTable VALUES('1', ROW('10', 'hello, world'));
      INSERT INTO hTable VALUES('1', ROW('30', CAST(NULL AS STRING))); -- null value to cf1.q2
      
      -- when sink.ignore-null-value is false
      // after first insert
      {rowkey: "1", "cf1": {q1: "10", q2: "hello, world"}} 
      // after second insert, cf1.q2 update to null
      {rowkey: "1", "cf1": {q1: "30", q2: "null"}} 
      
      
      -- when sink.ignore-null-value is true
      // after first insert 
      {rowkey: "1", "cf1": {q1: "10", q2: "hello, world"}}
      // after second insert, cf1.q2 is still the old value 
      {rowkey: "1", "cf1": {q1: "30", q2: "hello, world"}} 

       

       

      Attachments

        Activity

          People

            tanjialiang tanjialiang
            tanjialiang tanjialiang
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: