[TEZ-1160] Allow users to set non-tez configs on client side

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Duplicate
• Assignee: Unassigned
• Reporter: Ramya Sunil
• Created: 31/May/14 00:15
• Updated: 13/Mar/15 01:10
• Resolved: 13/Mar/15 01:10

Description

We ran into an issue while testing ATS with tez. We wanted to validate if tez AM can write to ATS but we could not do this without updating configs on the entire cluster.

Here is the exact scenario:

1. 10node cluster running with ats=disabled in yarn-site.xml
2. Set ATS=enabled on one host in the cluster and start ATS daemon before test execution
3. Submit a tez job with ats=enabled on cli/tez-site.xml to verify that tezAM writes to ATS.
4. Shutdown ATS.

Step #3 will fail because tezAM reads configs deployed on the cluster i.e RM/NMs and in the configs deployed on the cluster ats=disabled, meaning tezAM does not write to ATS though the client side/tez-site.xml has set ats=enabled.

As a workaround, we need to update the entire cluster with ats=enabled and restart services so that tezAM can write to ATS. It will be nice to have a feature in tez where users can set a non-tez config in tez-site.xml and that config is honored cluster wide for tez AM/tasks.

Issue Links

• duplicates: TEZ-1534 Make client side configs available to AM and tasks
• is duplicated by: TEZ-1534 Make client side configs available to AM and tasks