[TEZ-4236] DAGClientServer is not really needed to be started/used in local mode

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 0.10.1
• Assignee: László Bodor
• Reporter: László Bodor
• Created: 01/Oct/20 11:07
• Updated: 09/Sep/22 09:43
• Resolved: 28/Jan/21 21:28

Description

For a customer, we found that they try to run hive on tez local mode tests, and due to strict firewall rules and security policy, the DagClientServer struggles to start or respond to connections (think about corporate laptops here). Instead of investigating the network-related workarounds, let's find another way and decide if DAGAppMaster really needs this server.
LocalClient has an embedded DAGAppMaster, so at the first sight, it doesn't make sense to rely on the network environment in terms of AM usage. Hopefully, AM can become more lightweight in the long term if we try to eliminate pointless network usage.

Issue Links

• causes: TEZ-4282 Possible NPE in LocalClient after TEZ-4236
• relates to: TEZ-1336 Avoid interacting with the DAGAppMaster via RPC
• relates to: TEZ-4447 Collect VertexStatus properly when DAGClientServer is not used (local mode without network)
• relates to: HIVE-26444 Use tez.local.mode.without.network for unit tests
• Testing discovered: TEZ-4277 AsyncDispatcher can hang on serviceStop if the eventhandling thread is in BLOCKED state