[IMPALA-8532] Query Plan Page Fails When Profile is Large

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: Impala 3.3.0
• Component/s: Frontend
• Labels: observability
• Epic Color: ghx-label-8
• Assignee: Unassigned
• Reporter: Alan Jackoway
• Created: 10/May/19 14:44
• Updated: 10/May/19 23:58

Description

We discovered that on queries with large profiles, the query plan page with the boxes fails to load.

In Chrome, we saw an increasing number of backend XHR requests, and this JS error:

Uncaught SyntaxError: Unexpected end of JSON input
at JSON.parse (<anonymous>)
at XMLHttpRequest.renderGraph (query_plan?query_id=ec422d433862a401:5b5855a700000000:232)

Our backend requests to the retrieve the plan JSON were returning 1.5MB in between 1.5 and 4 seconds.

I believe the issue is that the code is set up like this:

function renderGraph(ignored_arg) {
  if (req.status != 200) return;
  var json = JSON.parse(req.responseText);
 // Doesn't get here
...
}
...
// Called periodically, fetches the plan JSON from Impala and passes it to renderGraph()
// for display.
function refresh() {
  req = new XMLHttpRequest();
  req.onload = renderGraph;
  req.open("GET", "/query_plan?query_id=ec422d433862a401:5b5855a700000000&json", true);
  req.send();
}

// Force one refresh before starting the timer.
refresh();

setInterval(refresh, 1000);

The issue is that req variable is in global scope. If it takes more than one second to fulfill the request, setInterval has run again and replaced the req variable with a new one that is running. Then in the renderGraph step, either status = 0 (because the variable is new) or the data gets cut off and fails to load. I saw both happening pretty regularly, with no obvious pattern between status=0 and status=200 but data being incomplete.

Some options I see to fix it:

• Change scope of the req variable so that renderGraph always gets the request that is complete rather than the most recent request created
• replace setInterval with an approach that kicks off the new request only when the previous request is complete.

This should be easy to reproduce if you can generate a plan that takes > 1 second to return from your impalad to your browser.