HBase
  1. HBase
  2. HBASE-10322

Strip tags from KV while sending back to client on reads

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Blocker Blocker
    • Resolution: Fixed
    • Affects Version/s: 0.98.0
    • Fix Version/s: 0.98.0, 0.99.0
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed
    • Release Note:
      Hide
      Added a new config hbase.replication.rpc.codec, using which one can specify the codec to be used by the replication. This config don't have a default value. When it is not specified, the codec specified by hbase.client.rpc.codec config will be used for replication.(For which org.apache.hadoop.hbase.codec.KeyValueCodec is the default value)

      Note : When using tags directly or indirectly (ie. Usage of cell level visibility labels or per cell ACL) make sure to configure hbase.replication.rpc.codec with a Codec which supports shipping of tags. We have org.apache.hadoop.hbase.codec.KeyValueCodecWithTags and org.apache.hadoop.hbase.codec.CellCodecWithTags, which can handle tags.
      Show
      Added a new config hbase.replication.rpc.codec, using which one can specify the codec to be used by the replication. This config don't have a default value. When it is not specified, the codec specified by hbase.client.rpc.codec config will be used for replication.(For which org.apache.hadoop.hbase.codec.KeyValueCodec is the default value) Note : When using tags directly or indirectly (ie. Usage of cell level visibility labels or per cell ACL) make sure to configure hbase.replication.rpc.codec with a Codec which supports shipping of tags. We have org.apache.hadoop.hbase.codec.KeyValueCodecWithTags and org.apache.hadoop.hbase.codec.CellCodecWithTags, which can handle tags.

      Description

      Right now we have some inconsistency wrt sending back tags on read. We do this in scan when using Java client(Codec based cell block encoding). But during a Get operation or when a pure PB based Scan comes we are not sending back the tags. So any of the below fix we have to do
      1. Send back tags in missing cases also. But sending back visibility expression/ cell ACL is not correct.
      2. Don't send back tags in any case. This will a problem when a tool like ExportTool use the scan to export the table data. We will miss exporting the cell visibility/ACL.
      3. Send back tags based on some condition. It has to be per scan basis. Simplest way is pass some kind of attribute in Scan which says whether to send back tags or not. But believing some thing what scan specifies might not be correct IMO. Then comes the way of checking the user who is doing the scan. When a HBase super user doing the scan then only send back tags. So when a case comes like Export Tool's the execution should happen from a super user.

      So IMO we should go with #3.
      Patch coming soon.

      1. HBASE-10322_V6.patch
        61 kB
        Anoop Sam John
      2. HBASE-10322_V5.patch
        71 kB
        Anoop Sam John
      3. HBASE-10322_V4.patch
        70 kB
        Anoop Sam John
      4. HBASE-10322_V3.patch
        59 kB
        Anoop Sam John
      5. HBASE-10322_codec.patch
        3 kB
        ramkrishna.s.vasudevan
      6. HBASE-10322_V2.patch
        32 kB
        Anoop Sam John
      7. HBASE-10322.patch
        73 kB
        Anoop Sam John

        Issue Links

          Activity

          Anoop Sam John created issue -
          Anoop Sam John made changes -
          Field Original Value New Value
          Attachment HBASE-10322.patch [ 12622534 ]
          Anoop Sam John made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Anoop Sam John made changes -
          Attachment HBASE-10322.patch [ 12622534 ]
          Anoop Sam John made changes -
          Attachment HBASE-10322.patch [ 12622539 ]
          ramkrishna.s.vasudevan made changes -
          Link This issue relates to HBASE-10326 [ HBASE-10326 ]
          Anoop Sam John made changes -
          Attachment HBASE-10322_V2.patch [ 12623185 ]
          ramkrishna.s.vasudevan made changes -
          Attachment HBASE-10322_codec.patch [ 12623779 ]
          Anoop Sam John made changes -
          Attachment HBASE-10322_V3.patch [ 12624069 ]
          Anoop Sam John made changes -
          Attachment HBASE-10322_V4.patch [ 12624103 ]
          Anoop Sam John made changes -
          Attachment HBASE-10322_V5.patch [ 12624161 ]
          Anoop Sam John made changes -
          Attachment HBASE-10322_V6.patch [ 12624263 ]
          Anoop Sam John made changes -
          Status Patch Available [ 10002 ] Resolved [ 5 ]
          Hadoop Flags Reviewed [ 10343 ]
          Release Note Added a new config hbase.replication.rpc.codec, using which one can specify the codec to be used by the replication. This config don't have a default value. When it is not specified, the codec specified by hbase.client.rpc.codec config will be used for replication.(For which org.apache.hadoop.hbase.codec.KeyValueCodec is the default value)

          Note : When using tags directly or indirectly (ie. Usage of cell level visibility labels or per cell ACL) make sure to configure hbase.replication.rpc.codec with a Codec which supports shipping of tags. We have org.apache.hadoop.hbase.codec.KeyValueCodecWithTags and org.apache.hadoop.hbase.codec.CellCodecWithTags, which can handle tags.
          Resolution Fixed [ 1 ]
          Misty Stanley-Jones made changes -
          Link This issue incorporates HBASE-10398 [ HBASE-10398 ]
          Jerry He made changes -
          Link This issue relates to HBASE-12441 [ HBASE-12441 ]
          Jerry He made changes -
          Link This issue relates to HBASE-12470 [ HBASE-12470 ]
          Enis Soztutar made changes -
          Status Resolved [ 5 ] Closed [ 6 ]

            People

            • Assignee:
              Anoop Sam John
              Reporter:
              Anoop Sam John
            • Votes:
              0 Vote for this issue
              Watchers:
              10 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development