[DRILL-5104] Foreman sets external sort memory allocation even for a physical plan

• Type: Bug
• Status: Reopened
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 1.8.0
• Assignee: Paul Rogers
• Reporter: Paul Rogers
• Reviewer: Rahul Kumar Challapalli
• Created: 04/Dec/16 07:54
• Updated: 25/Aug/17 17:34

Description

Consider the (disabled) unit test TestSimpleExternalSort.outOfMemoryExternalSort which uses the physical plan xsort/oom_sort_test.json that contains a setting for the amount of memory to allocate:

       {
            ...
            pop:"external-sort",
            ...
            initialAllocation: 1000000,
            maxAllocation: 30000000
        },

When run, the amount of memory is set to 715827882. The reason is that code was added to Foreman to compute the memory to allocate to the external sort:

  private void runPhysicalPlan(final PhysicalPlan plan) throws ExecutionSetupException {
    validatePlan(plan);
    MemoryAllocationUtilities.setupSortMemoryAllocations(plan, queryContext);

The problem is that a physical plan should execute as provided to enable detailed testing.

To solve this problem, move the sort memory setup to the path taken by SQL queries, but not via physical plans.

This change is necessary to re-enable the previously-disabled external sort tests.

Issue Links

• links to: GitHub Pull Request #694
• links to: GitHub Pull Request #703