Uploaded image for project: 'Apache Drill'
  1. Apache Drill
  2. DRILL-2878

FragmentExecutor.closeOutResources() is not called if an exception happens in the Foreman before the fragment executor starts running

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • 1.0.0
    • Execution - Flow
    • None

    Description

      When the Foreman sets up the root FragmentExecutor and it needs to wait for data from the remote fragments, the fragment manager is recorded in the work bus and the root fragment executor is not run immediately.
      If an exception happens in the Foreman while setting up the remote fragments, the Foreman cancels all fragments and returns a FAILED message to the client.
      Because the root fragment executor was not run it will never call it's closeOutResources() method and it's fragment context will never be closed.

      You can easily reproduce this by running the following unit test:

      org.apache.drill.exec.server.TestDrillbitResilience#failsWhenSendingFragments
      

      although the test passes successfully because Drill does report the correct failure to the client, the memory leak is not detected and will show up after the test finishes

      Attachments

        1. DRILL-2878.1.patch.txt
          1 kB
          Abdel Hakim Deneche
        2. DRILL-2878.2.patch.txt
          5 kB
          Abdel Hakim Deneche
        3. DRILL-2878.3.patch.txt
          5 kB
          Abdel Hakim Deneche
        4. DRILL-2878.5.patch.txt
          6 kB
          Abdel Hakim Deneche

        Issue Links

          Activity

            People

              jnadeau Jacques Nadeau
              adeneche Abdel Hakim Deneche
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: