Uploaded image for project: 'Apache Drill'
  1. Apache Drill
  2. DRILL-5815

Provide option to set query memory as percent of total

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 1.11.0
    • Fix Version/s: 1.12.0
    • Component/s: None

      Description

      Drill provides a parameter to set the memory per query as a static number which defaults to 2 GB. This number is a wonderful setting for the default Drillbit configuration of 8 GB heap; it allows 2-3 concurrent queries. But, as Drillbit memory increases, the default becomes a bit constraining. While users can change the setting, they seldom do.

      In addition, provide an option that sets memory as a percent of total memory. If the allocation is 10%, say, and total memory is 128 GB, then each query gets ~13GB, which is a big improvement.

      The existing option acts as a floor: the query must receive at least that much memory.

      DOCUMENTATION
      New option should be documented - planner.memory.percent_per_query
      Default - 0.05 (which is equivalent to 5 %)
      To disable feature set to 0.

      More information can be found in pull request description.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                Paul.Rogers Paul Rogers
                Reporter:
                Paul.Rogers Paul Rogers
                Reviewer:
                Boaz Ben-Zvi
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: