Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-16334

Query lock contains the query string, which can cause OOM on ZooKeeper

    Details

    • Type: Improvement
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 3.0.0
    • Component/s: Locking
    • Labels:
      None

      Description

      When there are big number of partitions in a query this will result in a huge number of locks on ZooKeeper. Since the query object contains the whole query string this might cause serious memory pressure on the ZooKeeper services.

      It would be good to have the possibility to truncate the query strings that are written into the locks

        Attachments

        1. HIVE-16334.patch
          10 kB
          Peter Vary
        2. HIVE-16334.4.patch
          10 kB
          Peter Vary
        3. HIVE-16334.3.patch
          10 kB
          Peter Vary
        4. HIVE-16334.2.patch
          10 kB
          Peter Vary

          Issue Links

            Activity

              People

              • Assignee:
                pvary Peter Vary
                Reporter:
                pvary Peter Vary
              • Votes:
                0 Vote for this issue
                Watchers:
                7 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: