[SQOOP-3455] Sqoop job fails while importing to S3 as Parquet

• Type: Bug
• Status: Open
• Priority: Blocker
• Resolution: Unresolved
• Affects Version/s: 1.4.7
• Component/s: sqoop2-kite-connector
• Assignee: Unassigned
• Reporter: Kriti Jha
• Created: 02/Nov/19 06:08
• Updated: 19/May/20 08:18

Description

A Sqoop job to import data from a MySQL database into S3 fails on using --as-parquetfile with the error as shown below:

---

{{ERROR sqoop.Sqoop: Got exception running Sqoop: org.kitesdk.data.DatasetNotFoundException: Unknown dataset URI pattern: dataset:s3://sqoop-trial-bucket/sqoop-trial/trial
Check that JARs for s3 datasets are on the classpath
org.kitesdk.data.DatasetNotFoundException: Unknown dataset URI pattern: dataset:s3://}}sqoop-trial-bucket/sqoop-trial/trial Check that JARs for s3 datasets are on the classpath at org.kitesdk.data.spi.Registration.lookupDatasetUri(Registration.java:128) at org.kitesdk.data.Datasets.exists(Datasets.java:624) at org.kitesdk.data.Datasets.exists(Datasets.java:646) at org.apache.sqoop.mapreduce.ParquetJob.configureImportJob(ParquetJob.java:118) at org.apache.sqoop.mapreduce.DataDrivenImportJob.configureMapper(DataDrivenImportJob.java:132) at org.apache.sqoop.mapreduce.ImportJobBase.runImport(ImportJobBase.java:264) at org.apache.sqoop.manager.SqlManager.importTable(SqlManager.java:692) at org.apache.sqoop.manager.MySQLManager.importTable(MySQLManager.java:127) at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:520) at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:628) at org.apache.sqoop.Sqoop.run(Sqoop.java:147) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76) at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:183) at org.apache.sqoop.Sqoop.runTool(Sqoop.java:234) at org.apache.sqoop.Sqoop.runTool(Sqoop.java:243) at org.apache.sqoop.Sqoop.main(Sqoop.java:252)

---

{{}}

All the JARs for S3 are present in the classpath. Further, the same works on simply removing the argument --as-parquetfile, i.e. with any other format.

{{}}

Issue Links

• duplicates: SQOOP-3453 Kite sdk issue with sqoop version 1.4.7