Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-16334

Query lock contains the query string, which can cause OOM on ZooKeeper

Log workAgile BoardRank to TopRank to BottomBulk Copy AttachmentsBulk Move AttachmentsVotersWatch issueWatchersCreate sub-taskConvert to sub-taskMoveLinkCloneLabelsUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Closed
    • Major
    • Resolution: Fixed
    • None
    • 3.0.0
    • Locking
    • None

    Description

      When there are big number of partitions in a query this will result in a huge number of locks on ZooKeeper. Since the query object contains the whole query string this might cause serious memory pressure on the ZooKeeper services.

      It would be good to have the possibility to truncate the query strings that are written into the locks

      Attachments

        1. HIVE-16334.2.patch
          10 kB
          Peter Vary
        2. HIVE-16334.3.patch
          10 kB
          Peter Vary
        3. HIVE-16334.4.patch
          10 kB
          Peter Vary
        4. HIVE-16334.patch
          10 kB
          Peter Vary

        Issue Links

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            pvary Peter Vary Assign to me
            pvary Peter Vary
            Votes:
            0 Vote for this issue
            Watchers:
            7 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment