Details
-
Improvement
-
Status: Open
-
Major
-
Resolution: Unresolved
-
1.4.6
-
None
Description
As many companies are using UDF to establish column level encryption during write time, Sqoop should support applying such a UDF during the write process. This would be an extension of the map-column-hive functionality, where the value of the parseColumnMapping would accept the UDF:
sqoop import --verbose --connect "jdbcconnectionstring" --username user --password password --hive-import --hive-database hiveschematest --map-column-hive "emptest.id=int,emptest.name=varchar(100),emptest.ssn=UDF_ENCRYPT()" -m 1
With this approach, the data does not have to be written to HDFS in the clear. This functionality can also be extended to other UDFs (naturally).