[HIVE-8049] Transparent column level encryption using kms

• Type: Sub-task
• Status: Patch Available
• Priority: Major
• Resolution: Unresolved
• Assignee: Xiaomeng Huang
• Reporter: Xiaomeng Huang
• Created: 11/Sep/14 05:35
• Updated: 11/Dec/14 01:44

Description

This patch implement transparent column level encryption. Users don't need to set anything when they quey tables.

1. setup kms and set kms-acls.xml (e.g. user1 and root has permission to get key)

    <property>
       <name>hadoop.kms.acl.GET</name>
       <value>user1 root</value>
       <description>
         ACL for get-key-version and get-current-key operations.
       </description>
     </property>
   

2. set hive-site.xml

    <property>  
       <name>hadoop.security.key.provider.path</name>  
       <value>kms://http@localhost:16000/kms</value>  
    </property> 
   

3. create an encrypted table

   drop table student_column_encrypt;
   create table student_column_encrypt (s_key INT, s_name STRING, s_country STRING, s_age INT) ROW FORMAT SERDE 'org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe'
     WITH SERDEPROPERTIES ('column.encode.columns'='s_country,s_age', 'column.encode.classname'='org.apache.hadoop.hive.serde2.crypto.CryptoRewriter') 
     STORED AS TEXTFILE TBLPROPERTIES('hive.encrypt.keynames'='hive.k1');
   insert overwrite table student_column_encrypt 
   select 
     s_key, s_name, s_country, s_age
   from student;
                
   select * from student_column_encrypt; 
   

4. query table by different user, this is transparent to users. It is very convenient and don't need to set anything.

   [root@huang1 hive_data]# hive
   hive> select * from student_column_encrypt;       
   OK
   0	Armon	China	20
   1	Jack	USA	21
   2	Lucy	England	22
   3	Lily	France	23
   4	Yom	Spain	24
   Time taken: 0.759 seconds, Fetched: 5 row(s)
   
   [root@huang1 hive_data]# su user2
   [user2@huang1 hive_data]$ hive
   hive> select * from student_column_encrypt;
   OK
   0	Armon	dqyb188=	NULL
   1	Jack	YJez	NULL
   2	Lucy	cKqV1c8MTw==	NULL
   3	Lily	c7aT180H	NULL
   4	Yom	ZrST0MA=	NULL
   Time taken: 0.77 seconds, Fetched: 5 row(s)
   

Issue Links

• depends upon: HIVE-8749 Change Hadoop version on HIVE-8065 to 2.6-SNAPSHOT
• depends upon: HIVE-6329 Support column level encryption/decryption