Uploaded image for project: 'Sqoop'
  1. Sqoop
  2. SQOOP-3281

Support for Hive UDFs on import

    XMLWordPrintableJSON

    Details

      Description

      As many companies are using UDF to establish column level encryption during write time, Sqoop should support applying such a UDF during the write process.  This would be an extension of the map-column-hive functionality, where the value of the parseColumnMapping would accept the UDF:

      sqoop import --verbose --connect "jdbcconnectionstring" --username user --password password --hive-import --hive-database hiveschematest --map-column-hive "emptest.id=int,emptest.name=varchar(100),emptest.ssn=UDF_ENCRYPT()" -m 1

      With this approach, the data does not have to be written to HDFS in the clear.  This functionality can also be extended to other UDFs (naturally).

       

       

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              scyphers C Scyphers
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated: