[PIG-4893] Task deserialization time is too long for spark on yarn mode

• Type: Sub-task
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: spark-branch
• Component/s: spark
• Assignee: liyunzhang
• Reporter: liyunzhang
• Created: 12/May/16 08:18
• Updated: 21/Jun/17 09:18
• Resolved: 04/Jul/16 06:53

Description

I found the task deserialization time is a bit long when i run any scripts of pigmix in spark on yarn mode. see the attachment picture. The duration time is 3s but the task deserialization is 13s.

My env is hadoop2.6+spark1.6.

Issue Links

• links to: review board