Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-20890

ACID: Allow whole table ReadLocks to skip all partition locks

Log workAgile BoardRank to TopRank to BottomBulk Copy AttachmentsBulk Move AttachmentsVotersWatch issueWatchersCreate sub-taskConvert to sub-taskMoveLinkCloneLabelsUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    Description

      HIVE-19369 proposes adding a EXCL_WRITE lock which does not wait for any SHARED_READ locks for read operations - in the presence of that lock, the insert overwrite no longer takes an exclusive lock.

      The only exclusive operation will be a schema change or drop table, which should take an exclusive lock on the entire table directly.

      explain locks select * from tpcds_bin_partitioned_orc_1000.store_sales where ss_sold_date_sk=2452626 
      
      +----------------------------------------------------+
      |                      Explain                       |
      +----------------------------------------------------+
      | LOCK INFORMATION:                                  |
      | tpcds_bin_partitioned_orc_1000.store_sales -> SHARED_READ |
      | tpcds_bin_partitioned_orc_1000.store_sales.ss_sold_date_sk=2452626 -> SHARED_READ |
      +----------------------------------------------------+
      

      So the per-partition SHARED_READ locks are no longer necessary, if the lock builder already includes the table-wide SHARED_READ locks.

      The removal of entire partitions is the only part which needs to be taken care of within this semantics as row-removal instead of directory removal (i.e "drop partition" -> "truncate partition" and have the truncation trigger a whole directory cleaner, so that the partition disappears when there are 0 rows left).

      Attachments

        1. HIVE-20890.1.patch
          19 kB
          Denys Kuzmenko
        2. HIVE-20890.2.patch
          23 kB
          Denys Kuzmenko
        3. HIVE-20890.3.patch
          23 kB
          Denys Kuzmenko
        4. HIVE-20890.4.patch
          25 kB
          Denys Kuzmenko
        5. HIVE-20890.5.patch
          25 kB
          Denys Kuzmenko

        Issue Links

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            dkuzmenko Denys Kuzmenko Assign to me
            gopalv Gopal Vijayaraghavan
            Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Time Tracking

              Estimated:
              Original Estimate - Not Specified
              Not Specified
              Remaining:
              Remaining Estimate - 0h
              0h
              Logged:
              Time Spent - 1h
              1h

              Slack

                Issue deployment