[YARN-10062] Support deploy multiple historyserver in case of sp

• Type: Improvement
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Component/s: yarn
• Assignee: zhoukang
• Reporter: zhoukang
• Created: 27/Dec/19 09:04
• Updated: 20/Jan/20 03:54

Description

In this jira, i want to implement a patch to support history ha.
We can deploy two historyserver and use a load balancer like lvs to support HA.
But there exist error in our production cluster like below:

19/12/13/000000 does not exist.
2019-12-21,13:25:06,822 DEBUG org.apache.hadoop.yarn.webapp.Controller: text/plain; charset=UTF-8: java.io.FileNotFoundException: File /yarn/xxx/staging/history/done/2019/12/13/000000 does not exist.
2019-12-21,13:25:07,530 DEBUG org.apache.hadoop.yarn.webapp.Controller: text/plain; charset=UTF-8: java.io.FileNotFoundException: File /yarn/xxx/staging/history/done/2019/12/13/000000 does not exist.
2019-12-21,13:25:09,910 DEBUG org.apache.hadoop.yarn.webapp.Controller: text/plain; charset=UTF-8: java.io.FileNotFoundException: File /yarn/xxx/staging/history/done/2019/12/13/000000 does not exist.
2019-12-21,13:44:29,044 DEBUG org.apache.hadoop.yarn.webapp.Controller: text/plain; charset=UTF-8: java.io.FileNotFoundException: File /yarn/xxx/staging/history/done/2019/12/13/000000 does not exist.
2019-12-21,13:47:08,154 DEBUG org.apache.hadoop.yarn.webapp.Controller: text/plain; charset=UTF-8: java.io.FileNotFoundException: File /yarn/xxx/staging/history/done/2019/12/13/000000 does not exist.