[PIG-4653] Remove unwanted config set on Tez DAG, vertices and edges

• Type: Improvement
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Fix Version/s: 0.18.0
• Assignee: Rohini Palaniswamy
• Reporter: Rohini Palaniswamy
• Created: 10/Aug/15 16:55
• Updated: 24/May/17 23:03

Description

This is bloating up the data sent in DAG submission and causes some DAGs to fail with java.io.IOException: Requested data length 127200464 is longer than maximum configured RPC length 67108864.

It also overwhelms Tez AM and make it hit OOM while processing getTask requests from 100s of tasks concurrently as the config payload in Input, Output and PigProcessor cause RPC buffers to overflow.