Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-15503

LLAP: Fix use of Runtime.getRuntime.maxMemory in Hive operators

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Critical
    • Resolution: Fixed
    • 2.2.0
    • 2.2.0
    • None
    • None

    Description

      ql/src/java/org/apache/hadoop/hive/ql/exec/GroupByOperator.java:    maxHashTblMemory = (long) (memoryPercentage * Runtime.getRuntime().maxMemory());
      ql/src/java/org/apache/hadoop/hive/ql/exec/TopNHash.java:    // Total Free Memory = maxMemory() - Used Memory;
      ql/src/java/org/apache/hadoop/hive/ql/exec/TopNHash.java:    long totalFreeMemory = Runtime.getRuntime().maxMemory() -
      

      This will not work very well with LLAP because of the memory sharing by executors.

      Attachments

        1. HIVE-15503.1.patch
          57 kB
          Prasanth Jayachandran
        2. HIVE-15503.2.patch
          16 kB
          Prasanth Jayachandran

        Activity

          People

            prasanth_j Prasanth Jayachandran
            prasanth_j Prasanth Jayachandran
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: