[HBASE-8496] Implement tags and the internals of how a tag should look like - ASF JIRA

XML

Word

Printable

JSON

Hadoop Flags:

Incompatible change, Reviewed
Release Note:

Hide
Tags are additional metadata to be added with the KVs.
To enable the tags to be persisted in the HFiles, V3 version of HFile should be used.
<property>
      <name>hfile.format.version</name>
      <value>3</value>
  </property>
The tags has the below format
<2 byte tag length><1 byte type code><tag>
where <type> is the type of the tag, <tag> is a byte[] that has the tag data.
To add Tags using Puts
Put.add(byte[] family, byte [] qualifier, byte [] value, Tag[] tag)
Put.add(byte[] family, byte[] qualifier, long ts, byte[] value, Tag[] tag)
Can be used.
Note that even after changing the version to V3, the no-tag case will also be working fine as in V2 format.

Show
Tags are additional metadata to be added with the KVs. To enable the tags to be persisted in the HFiles, V3 version of HFile should be used. <property>       <name>hfile.format.version</name>       <value>3</value>   </property> The tags has the below format <2 byte tag length><1 byte type code><tag> where <type> is the type of the tag, <tag> is a byte[] that has the tag data. To add Tags using Puts Put.add(byte[] family, byte [] qualifier, byte [] value, Tag[] tag) Put.add(byte[] family, byte[] qualifier, long ts, byte[] value, Tag[] tag) Can be used. Note that even after changing the version to V3, the no-tag case will also be working fine as in V2 format.

The intent of this JIRA comes from ~~HBASE-7897~~.
This would help us to decide on the structure and format of how the tags should look like.

Tag design.pdf
26/Jun/13 10:01
118 kB
ramkrishna.s.vasudevan
Tag design_updated.pdf
03/Aug/13 03:44
46 kB
ramkrishna.s.vasudevan
Tag_In_KV_Buffer_For_reference.patch
27/Jul/13 17:43
303 kB
ramkrishna.s.vasudevan
Performance_report.xlsx
19/Sep/13 10:16
28 kB
ramkrishna.s.vasudevan
HBASE-8496.patch
25/Jul/13 17:59
337 kB
ramkrishna.s.vasudevan
HBASE-8496_6.patch
21/Sep/13 09:09
582 kB
ramkrishna.s.vasudevan
HBASE-8496_5.patch
21/Sep/13 04:41
582 kB
ramkrishna.s.vasudevan
HBASE-8496_4.patch
20/Sep/13 08:24
579 kB
ramkrishna.s.vasudevan
HBASE-8496_3.patch
19/Sep/13 10:14
579 kB
ramkrishna.s.vasudevan
HBASE-8496_3.patch
19/Sep/13 10:33
579 kB
ramkrishna.s.vasudevan
HBASE-8496_3.patch
20/Sep/13 04:23
579 kB
ramkrishna.s.vasudevan
HBASE-8496_2.patch
27/Jul/13 17:44
336 kB
ramkrishna.s.vasudevan
Comparison.pdf
25/Jul/13 18:14
50 kB
ramkrishna.s.vasudevan

is related to

HBASE-7897 Add support for tags to Cell Interface

is required by

HBASE-6222 Add per-KeyValue Security

HBASE-7544 Transparent table/CF encryption

relates to

HBASE-9816 Address review comments in HBASE-8496

HBASE-9801 Change the default HFile version to V3

1.	Support Dictionary based Tag compression in HFiles	Closed	Anoop Sam John
2.	Add client side support for Tags	Resolved	ramkrishna.s.vasudevan
3.	Support Tags in ImportTSV tool	Open	ramkrishna.s.vasudevan
4.	Optimization in HFile V3 when no tags are present in a file	Closed	Anoop Sam John
5.	RPC side changes to have a different codec for server to client communication	Closed	ramkrishna.s.vasudevan
6.	Add Tag dictionary in WAL compression	Closed	Anoop Sam John
7.	HFileContext should adopt Builder pattern	Closed	ramkrishna.s.vasudevan
8.	HStore should use HFileContext for holding the state therein rather than duplicating it	Open	ramkrishna.s.vasudevan
9.	HFileContext defaults can lead to inconsistency with expectations of HFile block header sizing between readers and writers	Resolved	ramkrishna.s.vasudevan