[HIVE-20853] Expose ShuffleHandler.registerDag in the llap daemon API

• Type: Improvement
• Status: Closed
• Priority: Critical
• Resolution: Fixed
• Affects Version/s: 3.1.0
• Fix Version/s: 4.0.0-alpha-1
• Component/s: llap
• Assignee: Jaume M
• Reporter: Jaume M
• Created: 01/Nov/18 17:18
• Updated: 17/Nov/22 08:55
• Resolved: 09/Nov/18 20:53

Description

Currently DAGs are only registered when a submitWork is called for that DAG. At this point the crendentials are added to the ShuffleHandler and it can start serving.

However Tez might (and will) schedule tasks to fetch from the ShuffleHandler before anything of this happens and all this tasks will fail which may results in the query failing.

This happens in the scenario in which a LlapDaemon just comes up and tez fetchers try to open a connection before a DAG has been registered.

Adding this API will allow to register the DAG against the Daemon when the AM notices that a new Daemon is up.