[DRILL-2878] FragmentExecutor.closeOutResources() is not called if an exception happens in the Foreman before the fragment executor starts running

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.0.0
• Component/s: Execution - Flow
• Assignee: Jacques Nadeau
• Reporter: Abdel Hakim Deneche
• Created: 25/Apr/15 14:30
• Updated: 10/May/15 16:26
• Resolved: 10/May/15 16:26

Description

When the Foreman sets up the root FragmentExecutor and it needs to wait for data from the remote fragments, the fragment manager is recorded in the work bus and the root fragment executor is not run immediately.
If an exception happens in the Foreman while setting up the remote fragments, the Foreman cancels all fragments and returns a FAILED message to the client.
Because the root fragment executor was not run it will never call it's closeOutResources() method and it's fragment context will never be closed.

You can easily reproduce this by running the following unit test:

org.apache.drill.exec.server.TestDrillbitResilience#failsWhenSendingFragments

although the test passes successfully because Drill does report the correct failure to the client, the memory leak is not detected and will show up after the test finishes

Issue Links

• depends upon: DRILL-2757 Verify operators correctly handle low memory conditions and cancellations
• is related to: DRILL-1542 Early fragment termination causes non running intermediate fragments to error