[SQOOP-2103] Not able define Decimal(n,p) data type in map-column-hive option

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.4.5
• Fix Version/s: 1.4.7
• Labels: map-column-hive,decimal,sqoop

Environment

Unix OS

• Assignee: Abraham Elmahrek
• Reporter: Anitha
• Created: 12/Feb/15 23:25
• Updated: 09/Dec/15 09:18
• Resolved: 25/Jun/15 03:25

Description

Not able to define any column as decimal(n,p) while sqooping data and creating hive table using --map-column-hive option

I tried different syntax (with single quotes,double quotes,backslash..) but the command errors out.

Below is the command and error:

time sqoop import -D oraoop.disabled=true --options-file /home/sqoopuser/configfiles/c360_db.config --query "SELECT DLRS.*,'C360' as SOURCE,TO_CHAR(current_timestamp,'YYYY-MM-DD HH24:MI:SS.FF') AS LOAD_TS FROM DEALERS DLRS WHERE \$CONDITIONS" --split-by DEALER_NUMBER --create-hive-table --hive-table "c360dev.dealers_test" --hive-import --map-column-hive DEALER_NUMBER="VARCHAR(20)",DEALER_NAME="VARCHAR(50)",GEO_CODE="VARCHAR(50)",REGIONCODE=INT,REGIONNAME="VARCHAR(30)",DISTRICTCODE="VARCHAR(4)",21_INDICATOR="VARCHAR(1)",ADDRESS_TYPE="VARCHAR(25)",LINEONE="VARCHAR(50)",CITY="VARCHAR(25)",STATE="VARCHAR(2)",ZIPCODE="VARCHAR(5)",EFFECTIVE_DATE=TIMESTAMP,TERMINATION_CODE="VARCHAR(1)",PHONE_NUMBER="VARCHAR(25)",ENTITY_NAME="VARCHAR(50)",DEALERGROUP_NAME="VARCHAR(50)",DEALERGROUP_1COUNT=SMALLINT,DEALERGROUP_2COUNT=SMALLINT,DEALER_CLASSCODE="DECIMAL(22,7)",REGIONAREA_NAME="VARCHAR(50)",REGIONAREA_SHORTNAME="VARCHAR(50)",REGIONAREA_ABBREVATION="VARCHAR(5)",REGIONAREA_AREACCODE=SMALLINT,REGIONAREA_REGIONCODE=SMALLINT,PMACODE=SMALLINT,MARKETCODE=INT,MARKETNAME="VARCHAR(50)",MARKETTYPE="VARCHAR(50)",DUAL_INDICATOR="VARCHAR(5)",DEALERSHIP_TYPE="VARCHAR(5)",CREATION_DATE_TIMESTAMP=TIMESTAMP,CREATED_BY="VARCHAR(20)",LAST_UPDATE_DATE=TIMESTAMP,LAST_UPDATED_BY="VARCHAR(20)",TXM_DEALER="VARCHAR(50)",TFS_ECONTRACTING="VARCHAR(50)",YEARS_OF_PRESS_AWARDS=SMALLINT,IMAGE_USA="VARCHAR(50)",IMAGE_USA_EFFECTIVE_DATE=TIMESTAMP,TERMINATION_DATE=TIMESTAMP,3_DEALERNAME="VARCHAR(50)",IMAGE_USA_COMMENT="VARCHAR(100)",SINGLE_COMBINED_INDICATOR="VARCHAR(10)",1_URLNAME="VARCHAR(25)",1_URLVALUE="VARCHAR(100)",3_URLNAME="VARCHAR(25)",3_URLVALUE="VARCHAR(100)",2_URLNAME="VARCHAR(25)",2_URLVALUE="VARCHAR(100)" --hive-delims-replacement " " --null-string '
N' --null-non-string '
N' --target-dir "/user/hive/warehouse/c360dev.db/dealers_test" --verbose > dealers_test.log 2>&1

real 0m0.876s
user 0m1.062s
sys 0m0.112s
[sqoopuser@*****09 ~]$ more dealers_test.log
Warning: /usr/lib/sqoop/../accumulo does not exist! Accumulo imports will fail.
Please set $ACCUMULO_HOME to the root of your Accumulo installation.
15/02/12 15:19:26 INFO sqoop.Sqoop: Running Sqoop version: 1.4.5-cdh5.3.1
15/02/12 15:19:26 DEBUG tool.BaseSqoopTool: Enabled debug logging.
15/02/12 15:19:26 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead.
15/02/12 15:19:26 DEBUG sqoop.Sqoop: 1
java.lang.ArrayIndexOutOfBoundsException: 1
at org.apache.sqoop.SqoopOptions.parseColumnMapping(SqoopOptions.java:1200)
at org.apache.sqoop.SqoopOptions.setMapColumnHive(SqoopOptions.java:1205)
at org.apache.sqoop.tool.BaseSqoopTool.applyHiveOptions(BaseSqoopTool.java:1112)
at org.apache.sqoop.tool.ImportTool.applyOptions(ImportTool.java:973)
at org.apache.sqoop.tool.SqoopTool.parseArguments(SqoopTool.java:435)
at org.apache.sqoop.Sqoop.run(Sqoop.java:131)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:179)
at org.apache.sqoop.Sqoop.runTool(Sqoop.java:218)
at org.apache.sqoop.Sqoop.runTool(Sqoop.java:227)
at org.apache.sqoop.Sqoop.main(Sqoop.java:236)
1

Issue Links

• is related to: SQOOP-2642 Document ability to specify commas in --map-column-hive option
• links to: RB