[TEZ-3980] ShuffleRunner: the wake loop needs to check for shutdown

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 0.9.2, 0.10.0
• Assignee: Gopal Vijayaraghavan
• Reporter: Gopal Vijayaraghavan
• Created: 16/Aug/18 04:52
• Updated: 19/Apr/19 19:21
• Resolved: 28/Aug/18 03:38

Description

In the ShuffleRunner threads, there's a loop which does not terminate if the task threads get killed.

          while ((runningFetchers.size() >= numFetchers || pendingHosts.isEmpty())
              && numCompletedInputs.get() < numInputs) {
            inputContext.notifyProgress();
            boolean ret = wakeLoop.await(1000, TimeUnit.MILLISECONDS);
          }

The wakeLoop signal does not exit this out of the loop and is missing a break for shut-down.