Uploaded image for project: 'Apache Drill'
  1. Apache Drill
  2. DRILL-7924

Drill Memory Issues in Kubernetes

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Major
    • Resolution: Not A Problem
    • 1.18.0
    • None
    • None

    Description

      I am specifying a DRILLBIT_MAX_PROC_MEM of 9G, a DRILL_HEAP of 4G, and a DRILL_MAX_DIRECT_MEMORY of 4G and allowing Drill to determine the DRILLBIT_CODE_CACHE_SIZE. When I review the Drill startup logs, I can see that Drill is trying to use those values:

       drill-pod [INFO[] Attempting to start up Drill with the following settings
       drill-pod DRILL_HEAP=4G
       drill-pod DRILL_MAX_DIRECT_MEMORY=4G
       drill-pod DRILLBIT_CODE_CACHE_SIZE=768m
       drill-pod [WARN[] Total Memory Allocation for Drillbit (8GB) exceeds available free memory (3GB)
       drill-pod [WARN[] Drillbit will start up, but can potentially crash due to oversubscribing of system memory.
       drill-pod Starting drillbit, logging to /opt/drill/log/drillbit.out
       drill-pod [INFO[] Attempting to start up Drill with the following settings
       drill-pod DRILL_HEAP=4G
       drill-pod DRILL_MAX_DIRECT_MEMORY=4G
       drill-pod DRILLBIT_CODE_CACHE_SIZE=768m
       drill-pod [WARN[] Total Memory Allocation for Drillbit (8GB) exceeds available free memory (3GB)
       drill-pod [WARN[] Drillbit will start up, but can potentially crash due to oversubscribing of system memory.
       drill-pod drillbit is running.

      But when it is running, the UI doesn't match for the Direct memory and instead indicates that 0GB of memory is available:

      Note that the errors in the log citing memory allocation exceeds available free memory is irrelevant - I get the same issue when I don't get that error.

      What can I do to check if this is a UI bug or if Drill really is allocating 0GB to the Direct Memory despite being told to use 4GB? Any idea how I can fix it if it really is happening?

      Attachments

        Activity

          People

            Unassigned Unassigned
            philstellar Philip Lane
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: