[YARN-5667] Move HBase backend code in ATS v2 into its separate module

• Type: Task
• Status: Resolved
• Priority: Blocker
• Resolution: Done
• Affects Version/s: 3.0.0-alpha1
• Component/s: yarn
• Target Version/s: 3.0.0-alpha2
• Assignee: Haibo Chen
• Reporter: Haibo Chen
• Created: 26/Sep/16 03:29
• Updated: 20/Jan/17 17:55
• Resolved: 20/Jan/17 17:55

Description

The HBase backend code currently lives along with the core ATS v2 code in hadoop-yarn-server-timelineservice module. Because Resource Manager depends on hadoop-yarn-server-timelineservice, an unnecessary dependency of the RM module on HBase modules is introduced (HBase backend is pluggable, so we do not need to directly pull in HBase jars).

In our internal effort to try ATS v2 with HBase 2.0 which depends on Hadoop 3, we encountered a circular dependency during our builds between HBase2.0 and Hadoop3 artifacts.

hadoop-mapreduce-client-common, hadoop-yarn-client, hadoop-yarn-server-resourcemanager, hadoop-yarn-server-timelineservice, hbase-server, hbase-prefix-tree, hbase-hadoop2-compat, hadoop-mapreduce-client-jobclient, hadoop-mapreduce-client-common]

This jira proposes we move all HBase-backend-related code from hadoop-yarn-server-timelineservice into its own module (possible name is yarn-server-timelineservice-storage) so that core RM modules do not depend on HBase modules any more.

Issue Links

• relates to: YARN-4985 Refactor the coprocessor code & other definition classes into independent packages

Sub-Tasks

• Remove direct references of HBaseTimelineWriter/Reader in core ATS classes: Resolved
• Extract hbase-backend-exclusive utility methods from TimelineStorageUtil: Resolved
• Move ATSv2 HBase backend code into a new module that is only dependent at runtime by yarn servers: Resolved