1. HBase
  2. HBASE-8753

Provide new delete flag which can delete all cells under a column-family which have designated timestamp


    • Type: New Feature New Feature
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.95.1
    • Fix Version/s: 0.98.0, 0.95.2
    • Component/s: Deletes, Scanners
    • Labels:
    • Hadoop Flags:


      In one of our production scenario (Xiaomi message search), multiple cells will be put in batch using a same timestamp with different column names under a specific column-family.

      And after some time these cells also need to be deleted in batch by given a specific timestamp. But the column names are parsed tokens which can be arbitrary words , so such batch delete is impossible without first retrieving all KVs from that CF and get the column name list which has KV with that given timestamp, and then issuing individual deleteColumn for each column in that column-list.

      Though it's possible to do such batch delete, its performance is poor, and customers also find their code is quite clumsy by first retrieving and populating the column list and then issuing a deleteColumn for each column in that column-list.

      This feature resolves this problem by introducing a new delete flag: DeleteFamilyVersion.

      1). When you need to delete all KVs under a column-family with a given timestamp, just call Delete.deleteFamilyVersion(cfName, timestamp); only a DeleteFamilyVersion type KV is put to HBase (like DeleteFamily / DeleteColumn / Delete) without read operation;

      2). Like other delete types, DeleteFamilyVersion takes effect in get/scan/flush/compact operations, the ScanDeleteTracker now parses out and uses DeleteFamilyVersion to prevent all KVs under the specific CF which has the same timestamp as the DeleteFamilyVersion KV to pop-up as part of a get/scan result (also in flush/compact).

      Our customers find this feature efficient, clean and easy-to-use since it does its work without knowing the exact column names list that needs to be deleted.

      This feature has been running smoothly for a couple of months in our production clusters.

      1. 8753-trunk-V2.patch
        27 kB
        Ted Yu
      2. 8753-trunk-v4.txt
        26 kB
        Ted Yu
      3. HBASE-8753-0.94-V0.patch
        14 kB
        Honghua Feng
      4. HBASE-8753-0.94-V1.patch
        14 kB
        Honghua Feng
      5. HBASE-8753-trunk-V0.patch
        15 kB
        Honghua Feng
      6. HBASE-8753-trunk-V1.patch
        15 kB
        Liang Xie
      7. HBASE-8753-trunk-V3.patch
        14 kB
        Honghua Feng


        Honghua Feng created issue -
        Honghua Feng made changes -
        Field Original Value New Value
        Attachment HBASE-8753-0.94-V0.patch [ 12588074 ]
        Honghua Feng made changes -
        Attachment HBASE-8753-trunk-V0.patch [ 12588509 ]
        Liang Xie made changes -
        Attachment HBASE-8753-trunk-V1.patch [ 12590771 ]
        Liang Xie made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Affects Version/s 0.95.1 [ 12324288 ]
        Liang Xie made changes -
        Component/s Scanners [ 12319662 ]
        Ted Yu made changes -
        Assignee Feng Honghua [ fenghh ]
        Ted Yu made changes -
        Attachment 8753-trunk-V2.patch [ 12591012 ]
        Honghua Feng made changes -
        Attachment HBASE-8753-0.94-V1.patch [ 12591187 ]
        Attachment HBASE-8753-trunk-V3.patch [ 12591188 ]
        Ted Yu made changes -
        Attachment 8753-trunk-v4.txt [ 12591210 ]
        Ted Yu made changes -
        Summary Provide new delete flag which can delete all cells under a column-family which have a same designated timestamp Provide new delete flag which can delete all cells under a column-family which have designated timestamp
        Ted Yu made changes -
        Hadoop Flags Reviewed [ 10343 ]
        Fix Version/s 0.98.0 [ 12323143 ]
        Fix Version/s 0.95.2 [ 12320040 ]
        stack made changes -
        Status Patch Available [ 10002 ] Resolved [ 5 ]
        Resolution Fixed [ 1 ]
        stack made changes -
        Status Resolved [ 5 ] Closed [ 6 ]


          • Assignee:
            Honghua Feng
            Honghua Feng
          • Votes:
            1 Vote for this issue
            11 Start watching this issue


            • Created: