Sqoop
  1. Sqoop
  2. SQOOP-464

Warn when user is importing data into /user/hive/warehouse with flag --hive-import

    Details

    • Type: Improvement Improvement
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 1.4.2
    • Component/s: None
    • Labels:
      None

      Description

      I've recently seen couple of users who were trying to import data into hive (--hive-import) with --target-dir or --warehouse-dir set to /user/hive/warehouse. Sqoop wasn't designed to directly import data into hive warehouse directory. Instead we're importing data into temporary directory first and then by hive call "LOAD DATA" we're instructing hive to do actual load (move to /user/hive/warehouse).

      I would suggest to warn user when he is trying to import data into default hive warehouse directory.

      1. SQOOP-464.patch
        1 kB
        Jarek Jarcec Cecho

        Activity

        Hide
        jiraposter@reviews.apache.org added a comment -

        -----------------------------------------------------------
        This is an automatically generated e-mail. To reply, visit:
        https://reviews.apache.org/r/4364/
        -----------------------------------------------------------

        Review request for Sqoop, Arvind Prabhakar and Bilung Lee.

        Summary
        -------

        I've add parameter validation fragment that is looking for this incorrect usage and warning user.

        This addresses bug SQOOP-464.
        https://issues.apache.org/jira/browse/SQOOP-464

        Diffs


        /src/java/org/apache/sqoop/tool/BaseSqoopTool.java 1300633

        Diff: https://reviews.apache.org/r/4364/diff

        Testing
        -------

        • ant test -Dhadoopversion= {20,23,100}
        • On live CDH based cluster

        Thanks,

        Jarek

        Show
        jiraposter@reviews.apache.org added a comment - ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/4364/ ----------------------------------------------------------- Review request for Sqoop, Arvind Prabhakar and Bilung Lee. Summary ------- I've add parameter validation fragment that is looking for this incorrect usage and warning user. This addresses bug SQOOP-464 . https://issues.apache.org/jira/browse/SQOOP-464 Diffs /src/java/org/apache/sqoop/tool/BaseSqoopTool.java 1300633 Diff: https://reviews.apache.org/r/4364/diff Testing ------- ant test -Dhadoopversion= {20,23,100} On live CDH based cluster Thanks, Jarek
        Hide
        jiraposter@reviews.apache.org added a comment -

        -----------------------------------------------------------
        This is an automatically generated e-mail. To reply, visit:
        https://reviews.apache.org/r/4364/#review6252
        -----------------------------------------------------------

        Ship it!

        +1

        • Arvind

        On 2012-03-15 16:24:38, Jarek Cecho wrote:

        -----------------------------------------------------------

        This is an automatically generated e-mail. To reply, visit:

        https://reviews.apache.org/r/4364/

        -----------------------------------------------------------

        (Updated 2012-03-15 16:24:38)

        Review request for Sqoop, Arvind Prabhakar and Bilung Lee.

        Summary

        -------

        I've add parameter validation fragment that is looking for this incorrect usage and warning user.

        This addresses bug SQOOP-464.

        https://issues.apache.org/jira/browse/SQOOP-464

        Diffs

        -----

        /src/java/org/apache/sqoop/tool/BaseSqoopTool.java 1300633

        Diff: https://reviews.apache.org/r/4364/diff

        Testing

        -------

        * ant test -Dhadoopversion={20,23,100}

        * On live CDH based cluster

        Thanks,

        Jarek

        Show
        jiraposter@reviews.apache.org added a comment - ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/4364/#review6252 ----------------------------------------------------------- Ship it! +1 Arvind On 2012-03-15 16:24:38, Jarek Cecho wrote: ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/4364/ ----------------------------------------------------------- (Updated 2012-03-15 16:24:38) Review request for Sqoop, Arvind Prabhakar and Bilung Lee. Summary ------- I've add parameter validation fragment that is looking for this incorrect usage and warning user. This addresses bug SQOOP-464 . https://issues.apache.org/jira/browse/SQOOP-464 Diffs ----- /src/java/org/apache/sqoop/tool/BaseSqoopTool.java 1300633 Diff: https://reviews.apache.org/r/4364/diff Testing ------- * ant test -Dhadoopversion={20,23,100} * On live CDH based cluster Thanks, Jarek
        Hide
        Arvind Prabhakar added a comment -

        Patch committed. Thanks Jarcec!

        Show
        Arvind Prabhakar added a comment - Patch committed. Thanks Jarcec!
        Hide
        Hudson added a comment -

        Integrated in Sqoop-ant-jdk-1.6 #100 (See https://builds.apache.org/job/Sqoop-ant-jdk-1.6/100/)
        SQOOP-464. Warn when user is importing data into /user/hive/warehouse with flag --hive-import.

        (Jarek Jarcec Cecho via Arvind Prabhakar) (Revision 1304026)

        Result = SUCCESS
        arvind : http://svn.apache.org/viewvc/?view=rev&rev=1304026
        Files :

        • /incubator/sqoop/trunk/src/java/org/apache/sqoop/tool/BaseSqoopTool.java
        Show
        Hudson added a comment - Integrated in Sqoop-ant-jdk-1.6 #100 (See https://builds.apache.org/job/Sqoop-ant-jdk-1.6/100/ ) SQOOP-464 . Warn when user is importing data into /user/hive/warehouse with flag --hive-import. (Jarek Jarcec Cecho via Arvind Prabhakar) (Revision 1304026) Result = SUCCESS arvind : http://svn.apache.org/viewvc/?view=rev&rev=1304026 Files : /incubator/sqoop/trunk/src/java/org/apache/sqoop/tool/BaseSqoopTool.java

          People

          • Assignee:
            Jarek Jarcec Cecho
            Reporter:
            Jarek Jarcec Cecho
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development