Hive
  1. Hive
  2. HIVE-2617

Insert overwrite table db.tname fails if partition already exists

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.9.0
    • Component/s: Database/Schema, Metastore
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      Insert Overwrite table db.tname fails if partition already exists.
      For example-
      insert overwrite table db.tname PARTITION(part='p') select .. from t2 where part='p'; fails if partition 'p' already exists. Workaround is - use db; and the fire the command.
      From the source code-
      alterPartition(tbl.getTableName(), new Partition(tbl, tpart)); takes String tablename as argument and loses db information. Table table = newTable(tablename) is called to retrieve table from name. But, it relies on currentDatabase value (hence the workaround).

      1. HIVE-2617.patch
        24 kB
        Chinna Rao Lalam
      2. ASF.LICENSE.NOT.GRANTED--HIVE-2617.D843.1.patch
        24 kB
        Phabricator
      3. HIVE-2617.1.patch
        16 kB
        Chinna Rao Lalam

        Activity

        Aniket Mokashi created issue -
        Chinna Rao Lalam made changes -
        Field Original Value New Value
        Attachment HIVE-2617.patch [ 12506277 ]
        Chinna Rao Lalam made changes -
        Assignee Chinna Rao Lalam [ chinnalalam ]
        Hide
        jiraposter@reviews.apache.org added a comment -

        -----------------------------------------------------------
        This is an automatically generated e-mail. To reply, visit:
        https://reviews.apache.org/r/3026/
        -----------------------------------------------------------

        Review request for hive.

        Summary
        -------

        Insert Overwrite table db.tname fails if partition already exists.
        For example-
        insert overwrite table db.tname PARTITION(part='p') select .. from t2 where part='p'; fails if partition 'p' already exists.

        Internal of the code not consider the database name of the table and it is continued with the default database.

        Changed the code like if table contains the database name called the alter partition using the dbname and tablename.

        This addresses bug HIVE-2617.
        https://issues.apache.org/jira/browse/HIVE-2617

        Diffs


        trunk/ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java 1210917
        trunk/ql/src/test/queries/clientpositive/insert2_overwrite_partitions.q PRE-CREATION
        trunk/ql/src/test/results/clientpositive/insert2_overwrite_partitions.q.out PRE-CREATION

        Diff: https://reviews.apache.org/r/3026/diff

        Testing
        -------

        Added new testcase and all unit tests are passed

        Thanks,

        chinna

        Show
        jiraposter@reviews.apache.org added a comment - ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/3026/ ----------------------------------------------------------- Review request for hive. Summary ------- Insert Overwrite table db.tname fails if partition already exists. For example- insert overwrite table db.tname PARTITION(part='p') select .. from t2 where part='p'; fails if partition 'p' already exists. Internal of the code not consider the database name of the table and it is continued with the default database. Changed the code like if table contains the database name called the alter partition using the dbname and tablename. This addresses bug HIVE-2617 . https://issues.apache.org/jira/browse/HIVE-2617 Diffs trunk/ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java 1210917 trunk/ql/src/test/queries/clientpositive/insert2_overwrite_partitions.q PRE-CREATION trunk/ql/src/test/results/clientpositive/insert2_overwrite_partitions.q.out PRE-CREATION Diff: https://reviews.apache.org/r/3026/diff Testing ------- Added new testcase and all unit tests are passed Thanks, chinna
        Hide
        Chinna Rao Lalam added a comment -

        Changed the code like if table contains the database name called the alter partition using the dbname and tablename.

        Show
        Chinna Rao Lalam added a comment - Changed the code like if table contains the database name called the alter partition using the dbname and tablename.
        Chinna Rao Lalam made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Hide
        Namit Jain added a comment -

        The code changes look good - but the test is failing for me.
        Can you reduce the testcase by selecting fewer rows from the table or creating new data
        files with fewer rows ?

        Show
        Namit Jain added a comment - The code changes look good - but the test is failing for me. Can you reduce the testcase by selecting fewer rows from the table or creating new data files with fewer rows ?
        Namit Jain made changes -
        Status Patch Available [ 10002 ] Open [ 1 ]
        Hide
        Phabricator added a comment -

        njain requested code review of "HIVE-2617 [jira] Insert overwrite table db.tname fails if partition already exists".
        Reviewers: JIRA

        HIVE-2617

        Insert Overwrite table db.tname fails if partition already exists.
        For example-
        insert overwrite table db.tname PARTITION(part='p') select .. from t2 where part='p'; fails if partition 'p' already exists. Workaround is - use db; and the fire the command.
        From the source code-
        alterPartition(tbl.getTableName(), new Partition(tbl, tpart)); takes String tablename as argument and loses db information. Table table = newTable(tablename) is called to retrieve table from name. But, it relies on currentDatabase value (hence the workaround).

        TEST PLAN
        EMPTY

        REVISION DETAIL
        https://reviews.facebook.net/D843

        AFFECTED FILES
        ql/src/test/results/clientpositive/insert2_overwrite_partitions.q.out
        ql/src/test/queries/clientpositive/insert2_overwrite_partitions.q
        ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java

        MANAGE HERALD DIFFERENTIAL RULES
        https://reviews.facebook.net/herald/view/differential/

        WHY DID I GET THIS EMAIL?
        https://reviews.facebook.net/herald/transcript/1803/

        Tip: use the X-Herald-Rules header to filter Herald messages in your client.

        Show
        Phabricator added a comment - njain requested code review of " HIVE-2617 [jira] Insert overwrite table db.tname fails if partition already exists". Reviewers: JIRA HIVE-2617 Insert Overwrite table db.tname fails if partition already exists. For example- insert overwrite table db.tname PARTITION(part='p') select .. from t2 where part='p'; fails if partition 'p' already exists. Workaround is - use db; and the fire the command. From the source code- alterPartition(tbl.getTableName(), new Partition(tbl, tpart)); takes String tablename as argument and loses db information. Table table = newTable(tablename) is called to retrieve table from name. But, it relies on currentDatabase value (hence the workaround). TEST PLAN EMPTY REVISION DETAIL https://reviews.facebook.net/D843 AFFECTED FILES ql/src/test/results/clientpositive/insert2_overwrite_partitions.q.out ql/src/test/queries/clientpositive/insert2_overwrite_partitions.q ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java MANAGE HERALD DIFFERENTIAL RULES https://reviews.facebook.net/herald/view/differential/ WHY DID I GET THIS EMAIL? https://reviews.facebook.net/herald/transcript/1803/ Tip: use the X-Herald-Rules header to filter Herald messages in your client.
        Phabricator made changes -
        Attachment HIVE-2617.D843.1.patch [ 12507307 ]
        Chinna Rao Lalam made changes -
        Attachment HIVE-2617.1.patch [ 12507354 ]
        Hide
        Chinna Rao Lalam added a comment -

        Rebased and uploaded the patch

        Show
        Chinna Rao Lalam added a comment - Rebased and uploaded the patch
        Chinna Rao Lalam made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Hide
        Namit Jain added a comment -

        +1

        Show
        Namit Jain added a comment - +1
        Hide
        Namit Jain added a comment -

        Committed. Thanks Chinna

        Show
        Namit Jain added a comment - Committed. Thanks Chinna
        Namit Jain made changes -
        Status Patch Available [ 10002 ] Resolved [ 5 ]
        Hadoop Flags Reviewed [ 10343 ]
        Resolution Fixed [ 1 ]
        Hide
        Hudson added a comment -

        Integrated in Hive-trunk-h0.23.0 #24 (See https://builds.apache.org/job/Hive-trunk-h0.23.0/24/)
        HIVE-2617 Insert overwrite table db.tname fails if partition already exists
        (Chinna Rao Lalam via namit)

        namit : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1214592
        Files :

        • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java
        • /hive/trunk/ql/src/test/queries/clientpositive/insert2_overwrite_partitions.q
        • /hive/trunk/ql/src/test/results/clientpositive/insert2_overwrite_partitions.q.out
        Show
        Hudson added a comment - Integrated in Hive-trunk-h0.23.0 #24 (See https://builds.apache.org/job/Hive-trunk-h0.23.0/24/ ) HIVE-2617 Insert overwrite table db.tname fails if partition already exists (Chinna Rao Lalam via namit) namit : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1214592 Files : /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java /hive/trunk/ql/src/test/queries/clientpositive/insert2_overwrite_partitions.q /hive/trunk/ql/src/test/results/clientpositive/insert2_overwrite_partitions.q.out
        Hide
        Hudson added a comment -

        Integrated in Hive-trunk-h0.21 #1147 (See https://builds.apache.org/job/Hive-trunk-h0.21/1147/)
        HIVE-2617 Insert overwrite table db.tname fails if partition already exists
        (Chinna Rao Lalam via namit)

        namit : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1214592
        Files :

        • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java
        • /hive/trunk/ql/src/test/queries/clientpositive/insert2_overwrite_partitions.q
        • /hive/trunk/ql/src/test/results/clientpositive/insert2_overwrite_partitions.q.out
        Show
        Hudson added a comment - Integrated in Hive-trunk-h0.21 #1147 (See https://builds.apache.org/job/Hive-trunk-h0.21/1147/ ) HIVE-2617 Insert overwrite table db.tname fails if partition already exists (Chinna Rao Lalam via namit) namit : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1214592 Files : /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java /hive/trunk/ql/src/test/queries/clientpositive/insert2_overwrite_partitions.q /hive/trunk/ql/src/test/results/clientpositive/insert2_overwrite_partitions.q.out
        Carl Steinbach made changes -
        Fix Version/s 0.9.0 [ 12317742 ]
        Carl Steinbach made changes -
        Component/s Database/Schema [ 12317906 ]
        Hide
        Ashutosh Chauhan added a comment -

        This issue is closed now. It was released with the fix in 0.9.0. If there is a problem, please open a new jira and link this one with that.

        Show
        Ashutosh Chauhan added a comment - This issue is closed now. It was released with the fix in 0.9.0. If there is a problem, please open a new jira and link this one with that.
        Ashutosh Chauhan made changes -
        Status Resolved [ 5 ] Closed [ 6 ]
        Transition Time In Source Status Execution Times Last Executer Last Execution Date
        Patch Available Patch Available Open Open
        7d 13h 20m 1 Namit Jain 14/Dec/11 06:04
        Open Open Patch Available Patch Available
        6d 6h 49m 2 Chinna Rao Lalam 14/Dec/11 14:13
        Patch Available Patch Available Resolved Resolved
        11h 51m 1 Namit Jain 15/Dec/11 02:05
        Resolved Resolved Closed Closed
        137d 19h 5m 1 Ashutosh Chauhan 30/Apr/12 22:11

          People

          • Assignee:
            Chinna Rao Lalam
            Reporter:
            Aniket Mokashi
          • Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development