Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-14400

Handle concurrent insert with dynamic partition

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.2.0
    • Fix Version/s: 2.1.1, 2.2.0
    • Component/s: Hive
    • Labels:
      None

      Description

      With multiple users concurrently issuing insert statements on the same partition has a side effect that some queries may not see a partition at the time when they're issued, but will realize the partition is actually there when it is trying to add such partition to the metastore and thus get AlreadyExistsException, because some earlier query just created it (race condition).

      For example, imagine such a table is created:

      create table T (name char(50)) partitioned by (ds string);
      

      and the following two queries are launched at the same time, from different sessions:

      insert into table T partition (ds) values ('Bob', 'today'); -- creates the partition 'today'
      insert into table T partition (ds) values ('Joe', 'today'); -- will fail with AlreadyExistsException
      

        Attachments

        1. HIVE-14400.1.patch
          2 kB
          Wei Zheng
        2. HIVE-14400.2.patch
          3 kB
          Wei Zheng

          Issue Links

            Activity

              People

              • Assignee:
                wzheng Wei Zheng
                Reporter:
                wzheng Wei Zheng
              • Votes:
                0 Vote for this issue
                Watchers:
                4 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: