[TEZ-4475] VertexStatus is missing in TestLocalMode if DAG finishes too early - causing NPE in unit test

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 0.10.3
• Assignee: László Bodor
• Reporter: László Bodor
• Created: 13/Feb/23 12:57
• Updated: 13/Mar/23 19:53
• Resolved: 14/Feb/23 08:09

Description

this problem is very hard to reproduce, but when I was able to do so, it was like:

2023-02-13 11:32:16,302 INFO  [DAGAppMaster Thread] app.DAGAppMaster (DAGAppMaster.java:startDAG(2545)) - Running DAG: testMultipleClientsWithoutSession2_useDfs
...
2023-02-13 11:32:16,406 INFO  [Thread-675] client.DAGClientImpl (DAGClientImpl.java:getVertexStatusInternal(280)) - getVertexStatusInternal for Sleep, dagCompleted: true, in cache: false

in this case, the latter log message was added here as:

    LOG.info("getVertexStatusInternal for {}, dagCompleted: {}, in cache: {}", vertexName, dagCompleted,
        cachedVertexStatus.containsKey(vertexName));

so, the dag has already completed, but there were no vertex status updates yet (cache was empty), so unit tests failed with an inconvenient NPE

this bug was always there, but got exposed by unit tests added in TEZ-4447
the easiest way to solve this is to simply prevent the dag finish too early

Issue Links

• Is contained by: TEZ-4481 Prepare for Tez 0.10.3 release
• links to: GitHub Pull Request #267