Sqoop
  1. Sqoop
  2. SQOOP-327

Mixed update/insert export support for OracleManager

    Details

    • Type: New Feature New Feature
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 1.4.0-incubating
    • Component/s: None
    • Labels:
      None

      Description

      Currently Sqoop export job runs in insert mode (default) or update mode (with --update-key option). When in insert mode, all data are inserted and when in update mode, all data are updated in existing table. This leaves out the use case where some data may need to be updated while the rest needs to be inserted.

      It also causes problems

      • When in insert mode, new data may cause constraint violations if they exist already.
      • When in update mode, it could result in silent dropping of records that do not match the update key.

      The idea is to introduce a new "upsert" mode to update records if they exist in table already or to insert records if they do not exist yet.

      1. SQOOP-327.diff
        23 kB
        Bilung Lee

        Activity

        Hide
        Hudson added a comment -

        Integrated in Sqoop-jdk-1.6 #21 (See https://builds.apache.org/job/Sqoop-jdk-1.6/21/)
        SQOOP-327. Mixed update/insert export for Oracle.

        (Bilung Lee via Arvind Prabhakar)

        arvind : http://svn.apache.org/viewvc/?view=rev&rev=1166930
        Files :

        • /incubator/sqoop/trunk/src/docs/man/sqoop-export.txt
        • /incubator/sqoop/trunk/src/docs/user/export.txt
        • /incubator/sqoop/trunk/src/java/com/cloudera/sqoop/SqoopOptions.java
        • /incubator/sqoop/trunk/src/java/com/cloudera/sqoop/manager/ConnManager.java
        • /incubator/sqoop/trunk/src/java/com/cloudera/sqoop/manager/OracleManager.java
        • /incubator/sqoop/trunk/src/java/com/cloudera/sqoop/mapreduce/JdbcUpsertExportJob.java
        • /incubator/sqoop/trunk/src/java/com/cloudera/sqoop/mapreduce/OracleUpsertOutputFormat.java
        • /incubator/sqoop/trunk/src/java/com/cloudera/sqoop/mapreduce/UpdateOutputFormat.java
        • /incubator/sqoop/trunk/src/java/com/cloudera/sqoop/tool/BaseSqoopTool.java
        • /incubator/sqoop/trunk/src/java/com/cloudera/sqoop/tool/ExportTool.java
        • /incubator/sqoop/trunk/src/test/com/cloudera/sqoop/manager/OracleExportTest.java
        Show
        Hudson added a comment - Integrated in Sqoop-jdk-1.6 #21 (See https://builds.apache.org/job/Sqoop-jdk-1.6/21/ ) SQOOP-327 . Mixed update/insert export for Oracle. (Bilung Lee via Arvind Prabhakar) arvind : http://svn.apache.org/viewvc/?view=rev&rev=1166930 Files : /incubator/sqoop/trunk/src/docs/man/sqoop-export.txt /incubator/sqoop/trunk/src/docs/user/export.txt /incubator/sqoop/trunk/src/java/com/cloudera/sqoop/SqoopOptions.java /incubator/sqoop/trunk/src/java/com/cloudera/sqoop/manager/ConnManager.java /incubator/sqoop/trunk/src/java/com/cloudera/sqoop/manager/OracleManager.java /incubator/sqoop/trunk/src/java/com/cloudera/sqoop/mapreduce/JdbcUpsertExportJob.java /incubator/sqoop/trunk/src/java/com/cloudera/sqoop/mapreduce/OracleUpsertOutputFormat.java /incubator/sqoop/trunk/src/java/com/cloudera/sqoop/mapreduce/UpdateOutputFormat.java /incubator/sqoop/trunk/src/java/com/cloudera/sqoop/tool/BaseSqoopTool.java /incubator/sqoop/trunk/src/java/com/cloudera/sqoop/tool/ExportTool.java /incubator/sqoop/trunk/src/test/com/cloudera/sqoop/manager/OracleExportTest.java
        Hide
        jiraposter@reviews.apache.org added a comment -

        -----------------------------------------------------------
        This is an automatically generated e-mail. To reply, visit:
        https://reviews.apache.org/r/1717/#review1823
        -----------------------------------------------------------

        Ship it!

        +1

        • Arvind

        On 2011-09-06 18:09:15, Bilung Lee wrote:

        -----------------------------------------------------------

        This is an automatically generated e-mail. To reply, visit:

        https://reviews.apache.org/r/1717/

        -----------------------------------------------------------

        (Updated 2011-09-06 18:09:15)

        Review request for Sqoop, Ahmed Radwan, Arvind Prabhakar, and jmhsieh.

        Summary

        -------

        A new option is introduced to allow update records if they exist in table already or to insert records if they do not exist yet.

        This addresses bug SQOOP-327.

        https://issues.apache.org/jira/browse/SQOOP-327

        Diffs

        -----

        src/docs/man/sqoop-export.txt 50052cc

        src/docs/user/export.txt 4401c26

        src/java/com/cloudera/sqoop/SqoopOptions.java d07aecc

        src/java/com/cloudera/sqoop/manager/ConnManager.java f5f5a4b

        src/java/com/cloudera/sqoop/manager/OracleManager.java 1d08c4d

        src/java/com/cloudera/sqoop/mapreduce/JdbcUpsertExportJob.java PRE-CREATION

        src/java/com/cloudera/sqoop/mapreduce/OracleUpsertOutputFormat.java PRE-CREATION

        src/java/com/cloudera/sqoop/mapreduce/UpdateOutputFormat.java d5339d9

        src/java/com/cloudera/sqoop/tool/BaseSqoopTool.java 879c7c8

        src/java/com/cloudera/sqoop/tool/ExportTool.java d156eeb

        src/test/com/cloudera/sqoop/manager/OracleExportTest.java 12858d7

        Diff: https://reviews.apache.org/r/1717/diff

        Testing

        -------

        Thanks,

        Bilung

        Show
        jiraposter@reviews.apache.org added a comment - ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/1717/#review1823 ----------------------------------------------------------- Ship it! +1 Arvind On 2011-09-06 18:09:15, Bilung Lee wrote: ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/1717/ ----------------------------------------------------------- (Updated 2011-09-06 18:09:15) Review request for Sqoop, Ahmed Radwan, Arvind Prabhakar, and jmhsieh. Summary ------- A new option is introduced to allow update records if they exist in table already or to insert records if they do not exist yet. This addresses bug SQOOP-327 . https://issues.apache.org/jira/browse/SQOOP-327 Diffs ----- src/docs/man/sqoop-export.txt 50052cc src/docs/user/export.txt 4401c26 src/java/com/cloudera/sqoop/SqoopOptions.java d07aecc src/java/com/cloudera/sqoop/manager/ConnManager.java f5f5a4b src/java/com/cloudera/sqoop/manager/OracleManager.java 1d08c4d src/java/com/cloudera/sqoop/mapreduce/JdbcUpsertExportJob.java PRE-CREATION src/java/com/cloudera/sqoop/mapreduce/OracleUpsertOutputFormat.java PRE-CREATION src/java/com/cloudera/sqoop/mapreduce/UpdateOutputFormat.java d5339d9 src/java/com/cloudera/sqoop/tool/BaseSqoopTool.java 879c7c8 src/java/com/cloudera/sqoop/tool/ExportTool.java d156eeb src/test/com/cloudera/sqoop/manager/OracleExportTest.java 12858d7 Diff: https://reviews.apache.org/r/1717/diff Testing ------- Thanks, Bilung
        Hide
        jiraposter@reviews.apache.org added a comment -

        On 2011-09-07 04:33:05, Arvind Prabhakar wrote:

        > src/java/com/cloudera/sqoop/mapreduce/OracleUpsertOutputFormat.java, line 37

        > <https://reviews.apache.org/r/1717/diff/5/?file=38115#file38115line37>

        >

        > How about extending from ExportBatchOutputFormat? That would likely work as well unless I am missing something.

        Bilung Lee wrote:

        Yes, both ExportBatchOutputFormat and UpdateOutputFormat are using batch underneath, so extending either one would work fine functionality wise. However, since conceptually upsert is externalized as a "sub-mode" of update, this is why UpdateOutputFormat is the superclass.

        Thanks for the explanation Bilung. That makes sense.

        • Arvind

        -----------------------------------------------------------
        This is an automatically generated e-mail. To reply, visit:
        https://reviews.apache.org/r/1717/#review1779
        -----------------------------------------------------------

        On 2011-09-06 18:09:15, Bilung Lee wrote:

        -----------------------------------------------------------

        This is an automatically generated e-mail. To reply, visit:

        https://reviews.apache.org/r/1717/

        -----------------------------------------------------------

        (Updated 2011-09-06 18:09:15)

        Review request for Sqoop, Ahmed Radwan, Arvind Prabhakar, and jmhsieh.

        Summary

        -------

        A new option is introduced to allow update records if they exist in table already or to insert records if they do not exist yet.

        This addresses bug SQOOP-327.

        https://issues.apache.org/jira/browse/SQOOP-327

        Diffs

        -----

        src/docs/man/sqoop-export.txt 50052cc

        src/docs/user/export.txt 4401c26

        src/java/com/cloudera/sqoop/SqoopOptions.java d07aecc

        src/java/com/cloudera/sqoop/manager/ConnManager.java f5f5a4b

        src/java/com/cloudera/sqoop/manager/OracleManager.java 1d08c4d

        src/java/com/cloudera/sqoop/mapreduce/JdbcUpsertExportJob.java PRE-CREATION

        src/java/com/cloudera/sqoop/mapreduce/OracleUpsertOutputFormat.java PRE-CREATION

        src/java/com/cloudera/sqoop/mapreduce/UpdateOutputFormat.java d5339d9

        src/java/com/cloudera/sqoop/tool/BaseSqoopTool.java 879c7c8

        src/java/com/cloudera/sqoop/tool/ExportTool.java d156eeb

        src/test/com/cloudera/sqoop/manager/OracleExportTest.java 12858d7

        Diff: https://reviews.apache.org/r/1717/diff

        Testing

        -------

        Thanks,

        Bilung

        Show
        jiraposter@reviews.apache.org added a comment - On 2011-09-07 04:33:05, Arvind Prabhakar wrote: > src/java/com/cloudera/sqoop/mapreduce/OracleUpsertOutputFormat.java, line 37 > < https://reviews.apache.org/r/1717/diff/5/?file=38115#file38115line37 > > > How about extending from ExportBatchOutputFormat? That would likely work as well unless I am missing something. Bilung Lee wrote: Yes, both ExportBatchOutputFormat and UpdateOutputFormat are using batch underneath, so extending either one would work fine functionality wise. However, since conceptually upsert is externalized as a "sub-mode" of update, this is why UpdateOutputFormat is the superclass. Thanks for the explanation Bilung. That makes sense. Arvind ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/1717/#review1779 ----------------------------------------------------------- On 2011-09-06 18:09:15, Bilung Lee wrote: ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/1717/ ----------------------------------------------------------- (Updated 2011-09-06 18:09:15) Review request for Sqoop, Ahmed Radwan, Arvind Prabhakar, and jmhsieh. Summary ------- A new option is introduced to allow update records if they exist in table already or to insert records if they do not exist yet. This addresses bug SQOOP-327 . https://issues.apache.org/jira/browse/SQOOP-327 Diffs ----- src/docs/man/sqoop-export.txt 50052cc src/docs/user/export.txt 4401c26 src/java/com/cloudera/sqoop/SqoopOptions.java d07aecc src/java/com/cloudera/sqoop/manager/ConnManager.java f5f5a4b src/java/com/cloudera/sqoop/manager/OracleManager.java 1d08c4d src/java/com/cloudera/sqoop/mapreduce/JdbcUpsertExportJob.java PRE-CREATION src/java/com/cloudera/sqoop/mapreduce/OracleUpsertOutputFormat.java PRE-CREATION src/java/com/cloudera/sqoop/mapreduce/UpdateOutputFormat.java d5339d9 src/java/com/cloudera/sqoop/tool/BaseSqoopTool.java 879c7c8 src/java/com/cloudera/sqoop/tool/ExportTool.java d156eeb src/test/com/cloudera/sqoop/manager/OracleExportTest.java 12858d7 Diff: https://reviews.apache.org/r/1717/diff Testing ------- Thanks, Bilung
        Hide
        jiraposter@reviews.apache.org added a comment -

        On 2011-09-07 04:33:05, Arvind Prabhakar wrote:

        > src/java/com/cloudera/sqoop/mapreduce/OracleUpsertOutputFormat.java, line 37

        > <https://reviews.apache.org/r/1717/diff/5/?file=38115#file38115line37>

        >

        > How about extending from ExportBatchOutputFormat? That would likely work as well unless I am missing something.

        Yes, both ExportBatchOutputFormat and UpdateOutputFormat are using batch underneath, so extending either one would work fine functionality wise. However, since conceptually upsert is externalized as a "sub-mode" of update, this is why UpdateOutputFormat is the superclass.

        • Bilung

        -----------------------------------------------------------
        This is an automatically generated e-mail. To reply, visit:
        https://reviews.apache.org/r/1717/#review1779
        -----------------------------------------------------------

        On 2011-09-06 18:09:15, Bilung Lee wrote:

        -----------------------------------------------------------

        This is an automatically generated e-mail. To reply, visit:

        https://reviews.apache.org/r/1717/

        -----------------------------------------------------------

        (Updated 2011-09-06 18:09:15)

        Review request for Sqoop, Ahmed Radwan, Arvind Prabhakar, and jmhsieh.

        Summary

        -------

        A new option is introduced to allow update records if they exist in table already or to insert records if they do not exist yet.

        This addresses bug SQOOP-327.

        https://issues.apache.org/jira/browse/SQOOP-327

        Diffs

        -----

        src/docs/man/sqoop-export.txt 50052cc

        src/docs/user/export.txt 4401c26

        src/java/com/cloudera/sqoop/SqoopOptions.java d07aecc

        src/java/com/cloudera/sqoop/manager/ConnManager.java f5f5a4b

        src/java/com/cloudera/sqoop/manager/OracleManager.java 1d08c4d

        src/java/com/cloudera/sqoop/mapreduce/JdbcUpsertExportJob.java PRE-CREATION

        src/java/com/cloudera/sqoop/mapreduce/OracleUpsertOutputFormat.java PRE-CREATION

        src/java/com/cloudera/sqoop/mapreduce/UpdateOutputFormat.java d5339d9

        src/java/com/cloudera/sqoop/tool/BaseSqoopTool.java 879c7c8

        src/java/com/cloudera/sqoop/tool/ExportTool.java d156eeb

        src/test/com/cloudera/sqoop/manager/OracleExportTest.java 12858d7

        Diff: https://reviews.apache.org/r/1717/diff

        Testing

        -------

        Thanks,

        Bilung

        Show
        jiraposter@reviews.apache.org added a comment - On 2011-09-07 04:33:05, Arvind Prabhakar wrote: > src/java/com/cloudera/sqoop/mapreduce/OracleUpsertOutputFormat.java, line 37 > < https://reviews.apache.org/r/1717/diff/5/?file=38115#file38115line37 > > > How about extending from ExportBatchOutputFormat? That would likely work as well unless I am missing something. Yes, both ExportBatchOutputFormat and UpdateOutputFormat are using batch underneath, so extending either one would work fine functionality wise. However, since conceptually upsert is externalized as a "sub-mode" of update, this is why UpdateOutputFormat is the superclass. Bilung ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/1717/#review1779 ----------------------------------------------------------- On 2011-09-06 18:09:15, Bilung Lee wrote: ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/1717/ ----------------------------------------------------------- (Updated 2011-09-06 18:09:15) Review request for Sqoop, Ahmed Radwan, Arvind Prabhakar, and jmhsieh. Summary ------- A new option is introduced to allow update records if they exist in table already or to insert records if they do not exist yet. This addresses bug SQOOP-327 . https://issues.apache.org/jira/browse/SQOOP-327 Diffs ----- src/docs/man/sqoop-export.txt 50052cc src/docs/user/export.txt 4401c26 src/java/com/cloudera/sqoop/SqoopOptions.java d07aecc src/java/com/cloudera/sqoop/manager/ConnManager.java f5f5a4b src/java/com/cloudera/sqoop/manager/OracleManager.java 1d08c4d src/java/com/cloudera/sqoop/mapreduce/JdbcUpsertExportJob.java PRE-CREATION src/java/com/cloudera/sqoop/mapreduce/OracleUpsertOutputFormat.java PRE-CREATION src/java/com/cloudera/sqoop/mapreduce/UpdateOutputFormat.java d5339d9 src/java/com/cloudera/sqoop/tool/BaseSqoopTool.java 879c7c8 src/java/com/cloudera/sqoop/tool/ExportTool.java d156eeb src/test/com/cloudera/sqoop/manager/OracleExportTest.java 12858d7 Diff: https://reviews.apache.org/r/1717/diff Testing ------- Thanks, Bilung
        Hide
        jiraposter@reviews.apache.org added a comment -

        -----------------------------------------------------------
        This is an automatically generated e-mail. To reply, visit:
        https://reviews.apache.org/r/1717/#review1779
        -----------------------------------------------------------

        Changes look good. One question/suggestion below.

        src/java/com/cloudera/sqoop/mapreduce/OracleUpsertOutputFormat.java
        <https://reviews.apache.org/r/1717/#comment4061>

        How about extending from ExportBatchOutputFormat? That would likely work as well unless I am missing something.

        • Arvind

        On 2011-09-06 18:09:15, Bilung Lee wrote:

        -----------------------------------------------------------

        This is an automatically generated e-mail. To reply, visit:

        https://reviews.apache.org/r/1717/

        -----------------------------------------------------------

        (Updated 2011-09-06 18:09:15)

        Review request for Sqoop, Ahmed Radwan, Arvind Prabhakar, and jmhsieh.

        Summary

        -------

        A new option is introduced to allow update records if they exist in table already or to insert records if they do not exist yet.

        This addresses bug SQOOP-327.

        https://issues.apache.org/jira/browse/SQOOP-327

        Diffs

        -----

        src/docs/man/sqoop-export.txt 50052cc

        src/docs/user/export.txt 4401c26

        src/java/com/cloudera/sqoop/SqoopOptions.java d07aecc

        src/java/com/cloudera/sqoop/manager/ConnManager.java f5f5a4b

        src/java/com/cloudera/sqoop/manager/OracleManager.java 1d08c4d

        src/java/com/cloudera/sqoop/mapreduce/JdbcUpsertExportJob.java PRE-CREATION

        src/java/com/cloudera/sqoop/mapreduce/OracleUpsertOutputFormat.java PRE-CREATION

        src/java/com/cloudera/sqoop/mapreduce/UpdateOutputFormat.java d5339d9

        src/java/com/cloudera/sqoop/tool/BaseSqoopTool.java 879c7c8

        src/java/com/cloudera/sqoop/tool/ExportTool.java d156eeb

        src/test/com/cloudera/sqoop/manager/OracleExportTest.java 12858d7

        Diff: https://reviews.apache.org/r/1717/diff

        Testing

        -------

        Thanks,

        Bilung

        Show
        jiraposter@reviews.apache.org added a comment - ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/1717/#review1779 ----------------------------------------------------------- Changes look good. One question/suggestion below. src/java/com/cloudera/sqoop/mapreduce/OracleUpsertOutputFormat.java < https://reviews.apache.org/r/1717/#comment4061 > How about extending from ExportBatchOutputFormat? That would likely work as well unless I am missing something. Arvind On 2011-09-06 18:09:15, Bilung Lee wrote: ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/1717/ ----------------------------------------------------------- (Updated 2011-09-06 18:09:15) Review request for Sqoop, Ahmed Radwan, Arvind Prabhakar, and jmhsieh. Summary ------- A new option is introduced to allow update records if they exist in table already or to insert records if they do not exist yet. This addresses bug SQOOP-327 . https://issues.apache.org/jira/browse/SQOOP-327 Diffs ----- src/docs/man/sqoop-export.txt 50052cc src/docs/user/export.txt 4401c26 src/java/com/cloudera/sqoop/SqoopOptions.java d07aecc src/java/com/cloudera/sqoop/manager/ConnManager.java f5f5a4b src/java/com/cloudera/sqoop/manager/OracleManager.java 1d08c4d src/java/com/cloudera/sqoop/mapreduce/JdbcUpsertExportJob.java PRE-CREATION src/java/com/cloudera/sqoop/mapreduce/OracleUpsertOutputFormat.java PRE-CREATION src/java/com/cloudera/sqoop/mapreduce/UpdateOutputFormat.java d5339d9 src/java/com/cloudera/sqoop/tool/BaseSqoopTool.java 879c7c8 src/java/com/cloudera/sqoop/tool/ExportTool.java d156eeb src/test/com/cloudera/sqoop/manager/OracleExportTest.java 12858d7 Diff: https://reviews.apache.org/r/1717/diff Testing ------- Thanks, Bilung
        Hide
        jiraposter@reviews.apache.org added a comment -

        -----------------------------------------------------------
        This is an automatically generated e-mail. To reply, visit:
        https://reviews.apache.org/r/1717/
        -----------------------------------------------------------

        (Updated 2011-09-06 18:09:15.655614)

        Review request for Sqoop, Ahmed Radwan, Arvind Prabhakar, and jmhsieh.

        Summary
        -------

        A new option is introduced to allow update records if they exist in table already or to insert records if they do not exist yet.

        This addresses bug SQOOP-327.
        https://issues.apache.org/jira/browse/SQOOP-327

        Diffs (updated)


        src/docs/man/sqoop-export.txt 50052cc
        src/docs/user/export.txt 4401c26
        src/java/com/cloudera/sqoop/SqoopOptions.java d07aecc
        src/java/com/cloudera/sqoop/manager/ConnManager.java f5f5a4b
        src/java/com/cloudera/sqoop/manager/OracleManager.java 1d08c4d
        src/java/com/cloudera/sqoop/mapreduce/JdbcUpsertExportJob.java PRE-CREATION
        src/java/com/cloudera/sqoop/mapreduce/OracleUpsertOutputFormat.java PRE-CREATION
        src/java/com/cloudera/sqoop/mapreduce/UpdateOutputFormat.java d5339d9
        src/java/com/cloudera/sqoop/tool/BaseSqoopTool.java 879c7c8
        src/java/com/cloudera/sqoop/tool/ExportTool.java d156eeb
        src/test/com/cloudera/sqoop/manager/OracleExportTest.java 12858d7

        Diff: https://reviews.apache.org/r/1717/diff

        Testing
        -------

        Thanks,

        Bilung

        Show
        jiraposter@reviews.apache.org added a comment - ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/1717/ ----------------------------------------------------------- (Updated 2011-09-06 18:09:15.655614) Review request for Sqoop, Ahmed Radwan, Arvind Prabhakar, and jmhsieh. Summary ------- A new option is introduced to allow update records if they exist in table already or to insert records if they do not exist yet. This addresses bug SQOOP-327 . https://issues.apache.org/jira/browse/SQOOP-327 Diffs (updated) src/docs/man/sqoop-export.txt 50052cc src/docs/user/export.txt 4401c26 src/java/com/cloudera/sqoop/SqoopOptions.java d07aecc src/java/com/cloudera/sqoop/manager/ConnManager.java f5f5a4b src/java/com/cloudera/sqoop/manager/OracleManager.java 1d08c4d src/java/com/cloudera/sqoop/mapreduce/JdbcUpsertExportJob.java PRE-CREATION src/java/com/cloudera/sqoop/mapreduce/OracleUpsertOutputFormat.java PRE-CREATION src/java/com/cloudera/sqoop/mapreduce/UpdateOutputFormat.java d5339d9 src/java/com/cloudera/sqoop/tool/BaseSqoopTool.java 879c7c8 src/java/com/cloudera/sqoop/tool/ExportTool.java d156eeb src/test/com/cloudera/sqoop/manager/OracleExportTest.java 12858d7 Diff: https://reviews.apache.org/r/1717/diff Testing ------- Thanks, Bilung
        Hide
        jiraposter@reviews.apache.org added a comment -

        -----------------------------------------------------------
        This is an automatically generated e-mail. To reply, visit:
        https://reviews.apache.org/r/1717/
        -----------------------------------------------------------

        (Updated 2011-09-06 18:05:26.902743)

        Review request for Sqoop, Ahmed Radwan, Arvind Prabhakar, and jmhsieh.

        Summary
        -------

        A new option is introduced to allow update records if they exist in table already or to insert records if they do not exist yet.

        This addresses bug SQOOP-327.
        https://issues.apache.org/jira/browse/SQOOP-327

        Diffs (updated)


        src/test/com/cloudera/sqoop/manager/OracleExportTest.java 12858d7
        src/java/com/cloudera/sqoop/mapreduce/UpdateOutputFormat.java d5339d9
        src/java/com/cloudera/sqoop/tool/BaseSqoopTool.java 879c7c8
        src/java/com/cloudera/sqoop/tool/ExportTool.java d156eeb
        src/java/com/cloudera/sqoop/mapreduce/OracleUpsertOutputFormat.java PRE-CREATION
        src/java/com/cloudera/sqoop/manager/OracleManager.java 1d08c4d
        src/java/com/cloudera/sqoop/mapreduce/JdbcUpsertExportJob.java PRE-CREATION
        src/java/com/cloudera/sqoop/SqoopOptions.java d07aecc
        src/java/com/cloudera/sqoop/manager/ConnManager.java f5f5a4b
        src/docs/man/sqoop-export.txt 50052cc
        src/docs/user/export.txt 4401c26

        Diff: https://reviews.apache.org/r/1717/diff

        Testing
        -------

        Thanks,

        Bilung

        Show
        jiraposter@reviews.apache.org added a comment - ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/1717/ ----------------------------------------------------------- (Updated 2011-09-06 18:05:26.902743) Review request for Sqoop, Ahmed Radwan, Arvind Prabhakar, and jmhsieh. Summary ------- A new option is introduced to allow update records if they exist in table already or to insert records if they do not exist yet. This addresses bug SQOOP-327 . https://issues.apache.org/jira/browse/SQOOP-327 Diffs (updated) src/test/com/cloudera/sqoop/manager/OracleExportTest.java 12858d7 src/java/com/cloudera/sqoop/mapreduce/UpdateOutputFormat.java d5339d9 src/java/com/cloudera/sqoop/tool/BaseSqoopTool.java 879c7c8 src/java/com/cloudera/sqoop/tool/ExportTool.java d156eeb src/java/com/cloudera/sqoop/mapreduce/OracleUpsertOutputFormat.java PRE-CREATION src/java/com/cloudera/sqoop/manager/OracleManager.java 1d08c4d src/java/com/cloudera/sqoop/mapreduce/JdbcUpsertExportJob.java PRE-CREATION src/java/com/cloudera/sqoop/SqoopOptions.java d07aecc src/java/com/cloudera/sqoop/manager/ConnManager.java f5f5a4b src/docs/man/sqoop-export.txt 50052cc src/docs/user/export.txt 4401c26 Diff: https://reviews.apache.org/r/1717/diff Testing ------- Thanks, Bilung
        Hide
        jiraposter@reviews.apache.org added a comment -

        -----------------------------------------------------------
        This is an automatically generated e-mail. To reply, visit:
        https://reviews.apache.org/r/1717/
        -----------------------------------------------------------

        (Updated 2011-09-06 17:42:47.286974)

        Review request for Sqoop, Ahmed Radwan, Arvind Prabhakar, and jmhsieh.

        Summary
        -------

        A new option is introduced to allow update records if they exist in table already or to insert records if they do not exist yet.

        This addresses bug SQOOP-327.
        https://issues.apache.org/jira/browse/SQOOP-327

        Diffs (updated)


        src/docs/man/sqoop-export.txt 50052cc
        src/docs/user/export.txt 4401c26
        src/java/com/cloudera/sqoop/SqoopOptions.java d07aecc
        src/java/com/cloudera/sqoop/manager/ConnManager.java f5f5a4b
        src/java/com/cloudera/sqoop/manager/OracleManager.java 1d08c4d
        src/java/com/cloudera/sqoop/mapreduce/JdbcUpsertExportJob.java PRE-CREATION
        src/java/com/cloudera/sqoop/mapreduce/OracleUpsertOutputFormat.java PRE-CREATION
        src/java/com/cloudera/sqoop/mapreduce/UpdateOutputFormat.java d5339d9
        src/java/com/cloudera/sqoop/tool/BaseSqoopTool.java 879c7c8
        src/java/com/cloudera/sqoop/tool/ExportTool.java d156eeb
        src/test/com/cloudera/sqoop/manager/OracleExportTest.java 12858d7

        Diff: https://reviews.apache.org/r/1717/diff

        Testing
        -------

        Thanks,

        Bilung

        Show
        jiraposter@reviews.apache.org added a comment - ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/1717/ ----------------------------------------------------------- (Updated 2011-09-06 17:42:47.286974) Review request for Sqoop, Ahmed Radwan, Arvind Prabhakar, and jmhsieh. Summary ------- A new option is introduced to allow update records if they exist in table already or to insert records if they do not exist yet. This addresses bug SQOOP-327 . https://issues.apache.org/jira/browse/SQOOP-327 Diffs (updated) src/docs/man/sqoop-export.txt 50052cc src/docs/user/export.txt 4401c26 src/java/com/cloudera/sqoop/SqoopOptions.java d07aecc src/java/com/cloudera/sqoop/manager/ConnManager.java f5f5a4b src/java/com/cloudera/sqoop/manager/OracleManager.java 1d08c4d src/java/com/cloudera/sqoop/mapreduce/JdbcUpsertExportJob.java PRE-CREATION src/java/com/cloudera/sqoop/mapreduce/OracleUpsertOutputFormat.java PRE-CREATION src/java/com/cloudera/sqoop/mapreduce/UpdateOutputFormat.java d5339d9 src/java/com/cloudera/sqoop/tool/BaseSqoopTool.java 879c7c8 src/java/com/cloudera/sqoop/tool/ExportTool.java d156eeb src/test/com/cloudera/sqoop/manager/OracleExportTest.java 12858d7 Diff: https://reviews.apache.org/r/1717/diff Testing ------- Thanks, Bilung
        Hide
        jiraposter@reviews.apache.org added a comment -

        -----------------------------------------------------------
        This is an automatically generated e-mail. To reply, visit:
        https://reviews.apache.org/r/1717/
        -----------------------------------------------------------

        (Updated 2011-09-06 17:37:53.002726)

        Review request for Sqoop, Ahmed Radwan, Arvind Prabhakar, and jmhsieh.

        Summary
        -------

        A new option is introduced to allow update records if they exist in table already or to insert records if they do not exist yet.

        This addresses bug SQOOP-327.
        https://issues.apache.org/jira/browse/SQOOP-327

        Diffs


        src/docs/man/sqoop-export.txt 50052cc
        src/docs/user/export.txt 4401c26
        src/java/com/cloudera/sqoop/SqoopOptions.java d07aecc
        src/java/com/cloudera/sqoop/manager/ConnManager.java f5f5a4b
        src/java/com/cloudera/sqoop/manager/OracleManager.java 1d08c4d
        src/java/com/cloudera/sqoop/mapreduce/JdbcUpsertExportJob.java PRE-CREATION
        src/java/com/cloudera/sqoop/mapreduce/OracleUpsertOutputFormat.java PRE-CREATION
        src/java/com/cloudera/sqoop/mapreduce/UpdateOutputFormat.java d5339d9
        src/java/com/cloudera/sqoop/tool/BaseSqoopTool.java 879c7c8
        src/java/com/cloudera/sqoop/tool/ExportTool.java d156eeb
        src/test/com/cloudera/sqoop/manager/OracleExportTest.java 12858d7

        Diff: https://reviews.apache.org/r/1717/diff

        Testing
        -------

        Thanks,

        Bilung

        Show
        jiraposter@reviews.apache.org added a comment - ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/1717/ ----------------------------------------------------------- (Updated 2011-09-06 17:37:53.002726) Review request for Sqoop, Ahmed Radwan, Arvind Prabhakar, and jmhsieh. Summary ------- A new option is introduced to allow update records if they exist in table already or to insert records if they do not exist yet. This addresses bug SQOOP-327 . https://issues.apache.org/jira/browse/SQOOP-327 Diffs src/docs/man/sqoop-export.txt 50052cc src/docs/user/export.txt 4401c26 src/java/com/cloudera/sqoop/SqoopOptions.java d07aecc src/java/com/cloudera/sqoop/manager/ConnManager.java f5f5a4b src/java/com/cloudera/sqoop/manager/OracleManager.java 1d08c4d src/java/com/cloudera/sqoop/mapreduce/JdbcUpsertExportJob.java PRE-CREATION src/java/com/cloudera/sqoop/mapreduce/OracleUpsertOutputFormat.java PRE-CREATION src/java/com/cloudera/sqoop/mapreduce/UpdateOutputFormat.java d5339d9 src/java/com/cloudera/sqoop/tool/BaseSqoopTool.java 879c7c8 src/java/com/cloudera/sqoop/tool/ExportTool.java d156eeb src/test/com/cloudera/sqoop/manager/OracleExportTest.java 12858d7 Diff: https://reviews.apache.org/r/1717/diff Testing ------- Thanks, Bilung
        Hide
        jiraposter@reviews.apache.org added a comment -

        On 2011-09-06 07:45:14, jmhsieh wrote:

        > src/test/com/cloudera/sqoop/manager/OracleExportTest.java, lines 276-277

        > <https://reviews.apache.org/r/1717/diff/2/?file=37992#file37992line276>

        >

        > Are you upserting the same values? would it make more sense to put new values in and verify them?

        >

        > Also, would it make sense to have some overlapping range to prove that some are updated and some are inserted?

        The values go through the same code path twice. This would also test out both "insert" in first round and "update" in second round.

        On 2011-09-06 07:45:14, jmhsieh wrote:

        > src/java/com/cloudera/sqoop/manager/ConnManager.java, line 284

        > <https://reviews.apache.org/r/1717/diff/2/?file=37985#file37985line284>

        >

        > maybe you should have a '-update' and '-upsert' options instead of introducing modality?

        Boolean "--upsert" option was actually the very first design. The reason to go with a enum now is to allow flexibility for adding new values in the future (such as throwing exception for new rows.) so we won't paint ourselves into a corner.

        On 2011-09-06 07:45:14, jmhsieh wrote:

        > src/docs/user/export.txt, lines 55-59

        > <https://reviews.apache.org/r/1717/diff/2/?file=37983#file37983line55>

        >

        > which mode is default?

        >

        > maybe make it --export-mode with options 'update', 'insert', or 'upsert'? Or maybe just have --update, --insert and --upsert?

        Will clarify the default to be "updateonly". (This is mentioned in "sqoop-export.txt" though.)

        This proposal was also one of several designs considered. It was not chosen because it would introduce unnecessary and meaningless option combination (like "-export-mode insert" and "-update-key col" together) and "upsert" is essentially a "sub-mode" of update mode.

        On 2011-09-06 07:45:14, jmhsieh wrote:

        > src/docs/user/export.txt, lines 177-181

        > <https://reviews.apache.org/r/1717/diff/2/?file=37983#file37983line177>

        >

        > this is confusing. reword to be positive?

        >

        > I think 'mixed' mode means "insert or update" while "updateonly" means update only. maybe change 'mixed' to 'allowinsert'?

        Thanks, "allowinsert" sounds better (will use it in next patch).

        This paragraph is written following from the previous paragraph, so it would make more sense if continuing reading from previous paragraph. Even though, will reword it to make it clear by itself.

        • Bilung

        -----------------------------------------------------------
        This is an automatically generated e-mail. To reply, visit:
        https://reviews.apache.org/r/1717/#review1761
        -----------------------------------------------------------

        On 2011-09-06 00:02:42, Bilung Lee wrote:

        -----------------------------------------------------------

        This is an automatically generated e-mail. To reply, visit:

        https://reviews.apache.org/r/1717/

        -----------------------------------------------------------

        (Updated 2011-09-06 00:02:42)

        Review request for Sqoop, Ahmed Radwan and Arvind Prabhakar.

        Summary

        -------

        A new option is introduced to allow update records if they exist in table already or to insert records if they do not exist yet.

        This addresses bug SQOOP-327.

        https://issues.apache.org/jira/browse/SQOOP-327

        Diffs

        -----

        src/docs/man/sqoop-export.txt 50052cc

        src/docs/user/export.txt 4401c26

        src/java/com/cloudera/sqoop/SqoopOptions.java d07aecc

        src/java/com/cloudera/sqoop/manager/ConnManager.java f5f5a4b

        src/java/com/cloudera/sqoop/manager/OracleManager.java 1d08c4d

        src/java/com/cloudera/sqoop/mapreduce/JdbcUpsertExportJob.java PRE-CREATION

        src/java/com/cloudera/sqoop/mapreduce/OracleUpsertOutputFormat.java PRE-CREATION

        src/java/com/cloudera/sqoop/mapreduce/UpdateOutputFormat.java d5339d9

        src/java/com/cloudera/sqoop/tool/BaseSqoopTool.java 879c7c8

        src/java/com/cloudera/sqoop/tool/ExportTool.java d156eeb

        src/test/com/cloudera/sqoop/manager/OracleExportTest.java 12858d7

        Diff: https://reviews.apache.org/r/1717/diff

        Testing

        -------

        Thanks,

        Bilung

        Show
        jiraposter@reviews.apache.org added a comment - On 2011-09-06 07:45:14, jmhsieh wrote: > src/test/com/cloudera/sqoop/manager/OracleExportTest.java, lines 276-277 > < https://reviews.apache.org/r/1717/diff/2/?file=37992#file37992line276 > > > Are you upserting the same values? would it make more sense to put new values in and verify them? > > Also, would it make sense to have some overlapping range to prove that some are updated and some are inserted? The values go through the same code path twice. This would also test out both "insert" in first round and "update" in second round. On 2011-09-06 07:45:14, jmhsieh wrote: > src/java/com/cloudera/sqoop/manager/ConnManager.java, line 284 > < https://reviews.apache.org/r/1717/diff/2/?file=37985#file37985line284 > > > maybe you should have a '- update' and ' -upsert' options instead of introducing modality? Boolean "--upsert" option was actually the very first design. The reason to go with a enum now is to allow flexibility for adding new values in the future (such as throwing exception for new rows.) so we won't paint ourselves into a corner. On 2011-09-06 07:45:14, jmhsieh wrote: > src/docs/user/export.txt, lines 55-59 > < https://reviews.apache.org/r/1717/diff/2/?file=37983#file37983line55 > > > which mode is default? > > maybe make it --export-mode with options 'update', 'insert', or 'upsert'? Or maybe just have --update, --insert and --upsert? Will clarify the default to be "updateonly". (This is mentioned in "sqoop-export.txt" though.) This proposal was also one of several designs considered. It was not chosen because it would introduce unnecessary and meaningless option combination (like "- export-mode insert" and " -update-key col" together) and "upsert" is essentially a "sub-mode" of update mode. On 2011-09-06 07:45:14, jmhsieh wrote: > src/docs/user/export.txt, lines 177-181 > < https://reviews.apache.org/r/1717/diff/2/?file=37983#file37983line177 > > > this is confusing. reword to be positive? > > I think 'mixed' mode means "insert or update" while "updateonly" means update only. maybe change 'mixed' to 'allowinsert'? Thanks, "allowinsert" sounds better (will use it in next patch). This paragraph is written following from the previous paragraph, so it would make more sense if continuing reading from previous paragraph. Even though, will reword it to make it clear by itself. Bilung ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/1717/#review1761 ----------------------------------------------------------- On 2011-09-06 00:02:42, Bilung Lee wrote: ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/1717/ ----------------------------------------------------------- (Updated 2011-09-06 00:02:42) Review request for Sqoop, Ahmed Radwan and Arvind Prabhakar. Summary ------- A new option is introduced to allow update records if they exist in table already or to insert records if they do not exist yet. This addresses bug SQOOP-327 . https://issues.apache.org/jira/browse/SQOOP-327 Diffs ----- src/docs/man/sqoop-export.txt 50052cc src/docs/user/export.txt 4401c26 src/java/com/cloudera/sqoop/SqoopOptions.java d07aecc src/java/com/cloudera/sqoop/manager/ConnManager.java f5f5a4b src/java/com/cloudera/sqoop/manager/OracleManager.java 1d08c4d src/java/com/cloudera/sqoop/mapreduce/JdbcUpsertExportJob.java PRE-CREATION src/java/com/cloudera/sqoop/mapreduce/OracleUpsertOutputFormat.java PRE-CREATION src/java/com/cloudera/sqoop/mapreduce/UpdateOutputFormat.java d5339d9 src/java/com/cloudera/sqoop/tool/BaseSqoopTool.java 879c7c8 src/java/com/cloudera/sqoop/tool/ExportTool.java d156eeb src/test/com/cloudera/sqoop/manager/OracleExportTest.java 12858d7 Diff: https://reviews.apache.org/r/1717/diff Testing ------- Thanks, Bilung
        Hide
        jiraposter@reviews.apache.org added a comment -

        -----------------------------------------------------------
        This is an automatically generated e-mail. To reply, visit:
        https://reviews.apache.org/r/1717/#review1761
        -----------------------------------------------------------

        I'm -0 about the flags/user interface. Note, I did not carefully review implementation.

        src/docs/user/export.txt
        <https://reviews.apache.org/r/1717/#comment4004>

        which mode is default?

        maybe make it --export-mode with options 'update', 'insert', or 'upsert'? Or maybe just have --update, --insert and --upsert?

        src/docs/user/export.txt
        <https://reviews.apache.org/r/1717/#comment4001>

        this is confusing. reword to be positive?

        I think 'mixed' mode means "insert or update" while "updateonly" means update only. maybe change 'mixed' to 'allowinsert'?

        src/java/com/cloudera/sqoop/manager/ConnManager.java
        <https://reviews.apache.org/r/1717/#comment4002>

        maybe you should have a '-update' and '-upsert' options instead of introducing modality?

        src/test/com/cloudera/sqoop/manager/OracleExportTest.java
        <https://reviews.apache.org/r/1717/#comment4003>

        Are you upserting the same values? would it make more sense to put new values in and verify them?

        Also, would it make sense to have some overlapping range to prove that some are updated and some are inserted?

        • jmhsieh

        On 2011-09-06 00:02:42, Bilung Lee wrote:

        -----------------------------------------------------------

        This is an automatically generated e-mail. To reply, visit:

        https://reviews.apache.org/r/1717/

        -----------------------------------------------------------

        (Updated 2011-09-06 00:02:42)

        Review request for Sqoop, Ahmed Radwan and Arvind Prabhakar.

        Summary

        -------

        A new option is introduced to allow update records if they exist in table already or to insert records if they do not exist yet.

        This addresses bug SQOOP-327.

        https://issues.apache.org/jira/browse/SQOOP-327

        Diffs

        -----

        src/docs/man/sqoop-export.txt 50052cc

        src/docs/user/export.txt 4401c26

        src/java/com/cloudera/sqoop/SqoopOptions.java d07aecc

        src/java/com/cloudera/sqoop/manager/ConnManager.java f5f5a4b

        src/java/com/cloudera/sqoop/manager/OracleManager.java 1d08c4d

        src/java/com/cloudera/sqoop/mapreduce/JdbcUpsertExportJob.java PRE-CREATION

        src/java/com/cloudera/sqoop/mapreduce/OracleUpsertOutputFormat.java PRE-CREATION

        src/java/com/cloudera/sqoop/mapreduce/UpdateOutputFormat.java d5339d9

        src/java/com/cloudera/sqoop/tool/BaseSqoopTool.java 879c7c8

        src/java/com/cloudera/sqoop/tool/ExportTool.java d156eeb

        src/test/com/cloudera/sqoop/manager/OracleExportTest.java 12858d7

        Diff: https://reviews.apache.org/r/1717/diff

        Testing

        -------

        Thanks,

        Bilung

        Show
        jiraposter@reviews.apache.org added a comment - ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/1717/#review1761 ----------------------------------------------------------- I'm -0 about the flags/user interface. Note, I did not carefully review implementation. src/docs/user/export.txt < https://reviews.apache.org/r/1717/#comment4004 > which mode is default? maybe make it --export-mode with options 'update', 'insert', or 'upsert'? Or maybe just have --update, --insert and --upsert? src/docs/user/export.txt < https://reviews.apache.org/r/1717/#comment4001 > this is confusing. reword to be positive? I think 'mixed' mode means "insert or update" while "updateonly" means update only. maybe change 'mixed' to 'allowinsert'? src/java/com/cloudera/sqoop/manager/ConnManager.java < https://reviews.apache.org/r/1717/#comment4002 > maybe you should have a '- update' and ' -upsert' options instead of introducing modality? src/test/com/cloudera/sqoop/manager/OracleExportTest.java < https://reviews.apache.org/r/1717/#comment4003 > Are you upserting the same values? would it make more sense to put new values in and verify them? Also, would it make sense to have some overlapping range to prove that some are updated and some are inserted? jmhsieh On 2011-09-06 00:02:42, Bilung Lee wrote: ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/1717/ ----------------------------------------------------------- (Updated 2011-09-06 00:02:42) Review request for Sqoop, Ahmed Radwan and Arvind Prabhakar. Summary ------- A new option is introduced to allow update records if they exist in table already or to insert records if they do not exist yet. This addresses bug SQOOP-327 . https://issues.apache.org/jira/browse/SQOOP-327 Diffs ----- src/docs/man/sqoop-export.txt 50052cc src/docs/user/export.txt 4401c26 src/java/com/cloudera/sqoop/SqoopOptions.java d07aecc src/java/com/cloudera/sqoop/manager/ConnManager.java f5f5a4b src/java/com/cloudera/sqoop/manager/OracleManager.java 1d08c4d src/java/com/cloudera/sqoop/mapreduce/JdbcUpsertExportJob.java PRE-CREATION src/java/com/cloudera/sqoop/mapreduce/OracleUpsertOutputFormat.java PRE-CREATION src/java/com/cloudera/sqoop/mapreduce/UpdateOutputFormat.java d5339d9 src/java/com/cloudera/sqoop/tool/BaseSqoopTool.java 879c7c8 src/java/com/cloudera/sqoop/tool/ExportTool.java d156eeb src/test/com/cloudera/sqoop/manager/OracleExportTest.java 12858d7 Diff: https://reviews.apache.org/r/1717/diff Testing ------- Thanks, Bilung
        Hide
        jiraposter@reviews.apache.org added a comment -

        -----------------------------------------------------------
        This is an automatically generated e-mail. To reply, visit:
        https://reviews.apache.org/r/1717/
        -----------------------------------------------------------

        Review request for Sqoop, Ahmed Radwan and Arvind Prabhakar.

        Summary
        -------

        A new option is introduced to allow update records if they exist in table already or to insert records if they do not exist yet.

        This addresses bug SQOOP-327.
        https://issues.apache.org/jira/browse/SQOOP-327

        Diffs


        src/docs/man/sqoop-export.txt 50052cc
        src/docs/user/export.txt 4401c26
        src/java/com/cloudera/sqoop/SqoopOptions.java d07aecc
        src/java/com/cloudera/sqoop/manager/ConnManager.java f5f5a4b
        src/java/com/cloudera/sqoop/manager/OracleManager.java 1d08c4d
        src/java/com/cloudera/sqoop/mapreduce/JdbcUpsertExportJob.java PRE-CREATION
        src/java/com/cloudera/sqoop/mapreduce/OracleUpsertOutputFormat.java PRE-CREATION
        src/java/com/cloudera/sqoop/mapreduce/UpdateOutputFormat.java d5339d9
        src/java/com/cloudera/sqoop/tool/BaseSqoopTool.java 879c7c8
        src/java/com/cloudera/sqoop/tool/ExportTool.java d156eeb
        src/test/com/cloudera/sqoop/manager/OracleExportTest.java 12858d7

        Diff: https://reviews.apache.org/r/1717/diff

        Testing
        -------

        Thanks,

        Bilung

        Show
        jiraposter@reviews.apache.org added a comment - ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/1717/ ----------------------------------------------------------- Review request for Sqoop, Ahmed Radwan and Arvind Prabhakar. Summary ------- A new option is introduced to allow update records if they exist in table already or to insert records if they do not exist yet. This addresses bug SQOOP-327 . https://issues.apache.org/jira/browse/SQOOP-327 Diffs src/docs/man/sqoop-export.txt 50052cc src/docs/user/export.txt 4401c26 src/java/com/cloudera/sqoop/SqoopOptions.java d07aecc src/java/com/cloudera/sqoop/manager/ConnManager.java f5f5a4b src/java/com/cloudera/sqoop/manager/OracleManager.java 1d08c4d src/java/com/cloudera/sqoop/mapreduce/JdbcUpsertExportJob.java PRE-CREATION src/java/com/cloudera/sqoop/mapreduce/OracleUpsertOutputFormat.java PRE-CREATION src/java/com/cloudera/sqoop/mapreduce/UpdateOutputFormat.java d5339d9 src/java/com/cloudera/sqoop/tool/BaseSqoopTool.java 879c7c8 src/java/com/cloudera/sqoop/tool/ExportTool.java d156eeb src/test/com/cloudera/sqoop/manager/OracleExportTest.java 12858d7 Diff: https://reviews.apache.org/r/1717/diff Testing ------- Thanks, Bilung

          People

          • Assignee:
            Bilung Lee
            Reporter:
            Bilung Lee
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development