Sqoop
  1. Sqoop
  2. SQOOP-483

Allow target dir to be set to a different name than table name for hive import

    Details

    • Type: Improvement Improvement
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 1.4.2
    • Component/s: None
    • Labels:
      None

      Description

      Currently, the table name (set by --table) has to be the same as the target dir (set by --target-dir) for hive import. But it would be nice if the target dir could be set to a different name than the table name.

      For example, we like to be able to run something like:

      sqoop import ... --table "foo" --target-dir "foo1" --hive-import
      sqoop import ... --table "foo" --target-dir "foo2" --hive-import
      

      Now this generates the following call stack:

      12/05/02 17:11:55 INFO hive.HiveImport: FAILED: Error in semantic analysis: Line 2:17 Invalid path 'hdfs://localhost/user/cheolsoo/foo': No files matching path hdfs://localhost/user/cheolsoo/foo 
      12/05/02 17:11:56 ERROR tool.ImportTool: Encountered IOException running import job: java.io.IOException: Hive exited with status 10 
      at com.cloudera.sqoop.hive.HiveImport.executeExternalHiveScript(HiveImport.java:349) 
      at com.cloudera.sqoop.hive.HiveImport.executeScript(HiveImport.java:299) 
      at com.cloudera.sqoop.hive.HiveImport.importTable(HiveImport.java:241) 
      at com.cloudera.sqoop.tool.ImportTool.importTable(ImportTool.java:394) 
      at com.cloudera.sqoop.tool.ImportTool.run(ImportTool.java:455) 
      at com.cloudera.sqoop.Sqoop.run(Sqoop.java:146) 
      at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65) 
      at com.cloudera.sqoop.Sqoop.runSqoop(Sqoop.java:182) 
      at com.cloudera.sqoop.Sqoop.runTool(Sqoop.java:221) 
      at com.cloudera.sqoop.Sqoop.runTool(Sqoop.java:230) 
      at com.cloudera.sqoop.Sqoop.main(Sqoop.java:239)
      
      1. SQOOP-483.patch
        8 kB
        Cheolsoo Park
      2. SQOOP-483-2.patch
        8 kB
        Cheolsoo Park

        Activity

        Cheolsoo Park created issue -
        Cheolsoo Park made changes -
        Field Original Value New Value
        Summary Allow target dir to be set to to a different name than table name for hive import Allow target dir to be set to a different name than table name for hive import
        Hide
        Cheolsoo Park added a comment -
        Show
        Cheolsoo Park added a comment - Review board: https://reviews.apache.org/r/5248/
        Cheolsoo Park made changes -
        Attachment SQOOP-483.patch [ 12529824 ]
        Cheolsoo Park made changes -
        Status Open [ 1 ] In Progress [ 3 ]
        Cheolsoo Park made changes -
        Status In Progress [ 3 ] Open [ 1 ]
        Cheolsoo Park made changes -
        Status Open [ 1 ] In Progress [ 3 ]
        Cheolsoo Park made changes -
        Status In Progress [ 3 ] Open [ 1 ]
        Cheolsoo Park made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Cheolsoo Park made changes -
        Attachment SQOOP-483-2.patch [ 12529882 ]
        Hide
        Jarek Jarcec Cecho added a comment -

        Patch committed, thank you Cheolsoo!

        Show
        Jarek Jarcec Cecho added a comment - Patch committed, thank you Cheolsoo!
        Jarek Jarcec Cecho made changes -
        Status Patch Available [ 10002 ] Resolved [ 5 ]
        Fix Version/s 1.4.2-incubating [ 12320141 ]
        Resolution Fixed [ 1 ]
        Hide
        Hudson added a comment -

        Integrated in Sqoop-ant-jdk-1.6 #114 (See https://builds.apache.org/job/Sqoop-ant-jdk-1.6/114/)
        SQOOP-483. Allow target dir to be set to a different name than table name for hive import.

        (Cheolsoo Park via Jarek Jarcec Cecho) (Revision 1342998)

        Result = SUCCESS
        jarcec :
        Files :

        • /sqoop/trunk/src/java/org/apache/sqoop/hive/HiveImport.java
        • /sqoop/trunk/src/java/org/apache/sqoop/hive/TableDefWriter.java
        • /sqoop/trunk/src/test/com/cloudera/sqoop/hive/TestTableDefWriter.java
        Show
        Hudson added a comment - Integrated in Sqoop-ant-jdk-1.6 #114 (See https://builds.apache.org/job/Sqoop-ant-jdk-1.6/114/ ) SQOOP-483 . Allow target dir to be set to a different name than table name for hive import. (Cheolsoo Park via Jarek Jarcec Cecho) (Revision 1342998) Result = SUCCESS jarcec : Files : /sqoop/trunk/src/java/org/apache/sqoop/hive/HiveImport.java /sqoop/trunk/src/java/org/apache/sqoop/hive/TableDefWriter.java /sqoop/trunk/src/test/com/cloudera/sqoop/hive/TestTableDefWriter.java
        Transition Time In Source Status Execution Times Last Executer Last Execution Date
        Open Open In Progress In Progress
        21d 23h 50m 2 Cheolsoo Park 26/May/12 00:38
        In Progress In Progress Open Open
        9s 2 Cheolsoo Park 26/May/12 00:38
        Open Open Patch Available Patch Available
        7s 1 Cheolsoo Park 26/May/12 00:39
        Patch Available Patch Available Resolved Resolved
        1d 6h 39m 1 Jarek Jarcec Cecho 27/May/12 07:18

          People

          • Assignee:
            Cheolsoo Park
            Reporter:
            Cheolsoo Park
          • Votes:
            1 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development