[TAJO-1651] Too long fetcher default retries

• Type: Improvement
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 0.9.0, 0.10.0, 0.10.1
• Fix Version/s: 0.11.0
• Component/s: Data Shuffle
• Assignee: Jinho Kim
• Reporter: Jinho Kim
• Created: 18/Jun/15 01:48
• Updated: 24/Jul/15 06:59
• Resolved: 24/Jul/15 05:50

Description

The fetcher is getting an intermediate file from PullServer. fetcher will retry, when fails.
Currently, maximum timeout is task-retry(3) * fetch-retry(20) * timeout(120 sec)
I suggest that SHUFFLE_FETCHER_READ_RETRY_MAX_NUM change to 2

SHUFFLE_FETCHER_READ_TIMEOUT("tajo.shuffle.fetcher.read.timeout-sec", 120)
SHUFFLE_FETCHER_READ_RETRY_MAX_NUM("tajo.shuffle.fetcher.read.retry.max-num", 20)