Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-16334

Query lock contains the query string, which can cause OOM on ZooKeeper

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Closed
    • Major
    • Resolution: Fixed
    • None
    • 3.0.0
    • Locking
    • None

    Description

      When there are big number of partitions in a query this will result in a huge number of locks on ZooKeeper. Since the query object contains the whole query string this might cause serious memory pressure on the ZooKeeper services.

      It would be good to have the possibility to truncate the query strings that are written into the locks

      Attachments

        1. HIVE-16334.patch
          10 kB
          Peter Vary
        2. HIVE-16334.2.patch
          10 kB
          Peter Vary
        3. HIVE-16334.3.patch
          10 kB
          Peter Vary
        4. HIVE-16334.4.patch
          10 kB
          Peter Vary

        Issue Links

          relates to

          Improvement - An improvement or enhancement to an existing feature or task. HIVE-17799 Add Ellipsis For Truncated Query In Hive Lock

          • Trivial - Cosmetic problem like misspelt words or misaligned text.
          • Closed
          links to

          Web Link Review link

          Activity

            People

              pvary Peter Vary
              pvary Peter Vary
              Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: