Hive
  1. Hive
  2. HIVE-2617

Insert overwrite table db.tname fails if partition already exists

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.9.0
    • Component/s: Database/Schema, Metastore
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      Insert Overwrite table db.tname fails if partition already exists.
      For example-
      insert overwrite table db.tname PARTITION(part='p') select .. from t2 where part='p'; fails if partition 'p' already exists. Workaround is - use db; and the fire the command.
      From the source code-
      alterPartition(tbl.getTableName(), new Partition(tbl, tpart)); takes String tablename as argument and loses db information. Table table = newTable(tablename) is called to retrieve table from name. But, it relies on currentDatabase value (hence the workaround).

      1. HIVE-2617.patch
        24 kB
        Chinna Rao Lalam
      2. ASF.LICENSE.NOT.GRANTED--HIVE-2617.D843.1.patch
        24 kB
        Phabricator
      3. HIVE-2617.1.patch
        16 kB
        Chinna Rao Lalam

        Activity

        Hide
        jiraposter@reviews.apache.org added a comment -

        -----------------------------------------------------------
        This is an automatically generated e-mail. To reply, visit:
        https://reviews.apache.org/r/3026/
        -----------------------------------------------------------

        Review request for hive.

        Summary
        -------

        Insert Overwrite table db.tname fails if partition already exists.
        For example-
        insert overwrite table db.tname PARTITION(part='p') select .. from t2 where part='p'; fails if partition 'p' already exists.

        Internal of the code not consider the database name of the table and it is continued with the default database.

        Changed the code like if table contains the database name called the alter partition using the dbname and tablename.

        This addresses bug HIVE-2617.
        https://issues.apache.org/jira/browse/HIVE-2617

        Diffs


        trunk/ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java 1210917
        trunk/ql/src/test/queries/clientpositive/insert2_overwrite_partitions.q PRE-CREATION
        trunk/ql/src/test/results/clientpositive/insert2_overwrite_partitions.q.out PRE-CREATION

        Diff: https://reviews.apache.org/r/3026/diff

        Testing
        -------

        Added new testcase and all unit tests are passed

        Thanks,

        chinna

        Show
        jiraposter@reviews.apache.org added a comment - ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/3026/ ----------------------------------------------------------- Review request for hive. Summary ------- Insert Overwrite table db.tname fails if partition already exists. For example- insert overwrite table db.tname PARTITION(part='p') select .. from t2 where part='p'; fails if partition 'p' already exists. Internal of the code not consider the database name of the table and it is continued with the default database. Changed the code like if table contains the database name called the alter partition using the dbname and tablename. This addresses bug HIVE-2617 . https://issues.apache.org/jira/browse/HIVE-2617 Diffs trunk/ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java 1210917 trunk/ql/src/test/queries/clientpositive/insert2_overwrite_partitions.q PRE-CREATION trunk/ql/src/test/results/clientpositive/insert2_overwrite_partitions.q.out PRE-CREATION Diff: https://reviews.apache.org/r/3026/diff Testing ------- Added new testcase and all unit tests are passed Thanks, chinna
        Hide
        Chinna Rao Lalam added a comment -

        Changed the code like if table contains the database name called the alter partition using the dbname and tablename.

        Show
        Chinna Rao Lalam added a comment - Changed the code like if table contains the database name called the alter partition using the dbname and tablename.
        Hide
        Namit Jain added a comment -

        The code changes look good - but the test is failing for me.
        Can you reduce the testcase by selecting fewer rows from the table or creating new data
        files with fewer rows ?

        Show
        Namit Jain added a comment - The code changes look good - but the test is failing for me. Can you reduce the testcase by selecting fewer rows from the table or creating new data files with fewer rows ?
        Hide
        Phabricator added a comment -

        njain requested code review of "HIVE-2617 [jira] Insert overwrite table db.tname fails if partition already exists".
        Reviewers: JIRA

        HIVE-2617

        Insert Overwrite table db.tname fails if partition already exists.
        For example-
        insert overwrite table db.tname PARTITION(part='p') select .. from t2 where part='p'; fails if partition 'p' already exists. Workaround is - use db; and the fire the command.
        From the source code-
        alterPartition(tbl.getTableName(), new Partition(tbl, tpart)); takes String tablename as argument and loses db information. Table table = newTable(tablename) is called to retrieve table from name. But, it relies on currentDatabase value (hence the workaround).

        TEST PLAN
        EMPTY

        REVISION DETAIL
        https://reviews.facebook.net/D843

        AFFECTED FILES
        ql/src/test/results/clientpositive/insert2_overwrite_partitions.q.out
        ql/src/test/queries/clientpositive/insert2_overwrite_partitions.q
        ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java

        MANAGE HERALD DIFFERENTIAL RULES
        https://reviews.facebook.net/herald/view/differential/

        WHY DID I GET THIS EMAIL?
        https://reviews.facebook.net/herald/transcript/1803/

        Tip: use the X-Herald-Rules header to filter Herald messages in your client.

        Show
        Phabricator added a comment - njain requested code review of " HIVE-2617 [jira] Insert overwrite table db.tname fails if partition already exists". Reviewers: JIRA HIVE-2617 Insert Overwrite table db.tname fails if partition already exists. For example- insert overwrite table db.tname PARTITION(part='p') select .. from t2 where part='p'; fails if partition 'p' already exists. Workaround is - use db; and the fire the command. From the source code- alterPartition(tbl.getTableName(), new Partition(tbl, tpart)); takes String tablename as argument and loses db information. Table table = newTable(tablename) is called to retrieve table from name. But, it relies on currentDatabase value (hence the workaround). TEST PLAN EMPTY REVISION DETAIL https://reviews.facebook.net/D843 AFFECTED FILES ql/src/test/results/clientpositive/insert2_overwrite_partitions.q.out ql/src/test/queries/clientpositive/insert2_overwrite_partitions.q ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java MANAGE HERALD DIFFERENTIAL RULES https://reviews.facebook.net/herald/view/differential/ WHY DID I GET THIS EMAIL? https://reviews.facebook.net/herald/transcript/1803/ Tip: use the X-Herald-Rules header to filter Herald messages in your client.
        Hide
        Chinna Rao Lalam added a comment -

        Rebased and uploaded the patch

        Show
        Chinna Rao Lalam added a comment - Rebased and uploaded the patch
        Hide
        Namit Jain added a comment -

        +1

        Show
        Namit Jain added a comment - +1
        Hide
        Namit Jain added a comment -

        Committed. Thanks Chinna

        Show
        Namit Jain added a comment - Committed. Thanks Chinna
        Hide
        Hudson added a comment -

        Integrated in Hive-trunk-h0.23.0 #24 (See https://builds.apache.org/job/Hive-trunk-h0.23.0/24/)
        HIVE-2617 Insert overwrite table db.tname fails if partition already exists
        (Chinna Rao Lalam via namit)

        namit : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1214592
        Files :

        • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java
        • /hive/trunk/ql/src/test/queries/clientpositive/insert2_overwrite_partitions.q
        • /hive/trunk/ql/src/test/results/clientpositive/insert2_overwrite_partitions.q.out
        Show
        Hudson added a comment - Integrated in Hive-trunk-h0.23.0 #24 (See https://builds.apache.org/job/Hive-trunk-h0.23.0/24/ ) HIVE-2617 Insert overwrite table db.tname fails if partition already exists (Chinna Rao Lalam via namit) namit : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1214592 Files : /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java /hive/trunk/ql/src/test/queries/clientpositive/insert2_overwrite_partitions.q /hive/trunk/ql/src/test/results/clientpositive/insert2_overwrite_partitions.q.out
        Hide
        Hudson added a comment -

        Integrated in Hive-trunk-h0.21 #1147 (See https://builds.apache.org/job/Hive-trunk-h0.21/1147/)
        HIVE-2617 Insert overwrite table db.tname fails if partition already exists
        (Chinna Rao Lalam via namit)

        namit : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1214592
        Files :

        • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java
        • /hive/trunk/ql/src/test/queries/clientpositive/insert2_overwrite_partitions.q
        • /hive/trunk/ql/src/test/results/clientpositive/insert2_overwrite_partitions.q.out
        Show
        Hudson added a comment - Integrated in Hive-trunk-h0.21 #1147 (See https://builds.apache.org/job/Hive-trunk-h0.21/1147/ ) HIVE-2617 Insert overwrite table db.tname fails if partition already exists (Chinna Rao Lalam via namit) namit : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1214592 Files : /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java /hive/trunk/ql/src/test/queries/clientpositive/insert2_overwrite_partitions.q /hive/trunk/ql/src/test/results/clientpositive/insert2_overwrite_partitions.q.out
        Hide
        Ashutosh Chauhan added a comment -

        This issue is closed now. It was released with the fix in 0.9.0. If there is a problem, please open a new jira and link this one with that.

        Show
        Ashutosh Chauhan added a comment - This issue is closed now. It was released with the fix in 0.9.0. If there is a problem, please open a new jira and link this one with that.

          People

          • Assignee:
            Chinna Rao Lalam
            Reporter:
            Aniket Mokashi
          • Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development