Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-5667

Move HBase backend code in ATS v2 into its separate module

VotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

    Details

    • Type: Task
    • Status: Resolved
    • Priority: Blocker
    • Resolution: Done
    • Affects Version/s: 3.0.0-alpha1
    • Fix Version/s: None
    • Component/s: yarn
    • Labels:
      None

      Description

      The HBase backend code currently lives along with the core ATS v2 code in hadoop-yarn-server-timelineservice module. Because Resource Manager depends on hadoop-yarn-server-timelineservice, an unnecessary dependency of the RM module on HBase modules is introduced (HBase backend is pluggable, so we do not need to directly pull in HBase jars).

      In our internal effort to try ATS v2 with HBase 2.0 which depends on Hadoop 3, we encountered a circular dependency during our builds between HBase2.0 and Hadoop3 artifacts.

      hadoop-mapreduce-client-common, hadoop-yarn-client, hadoop-yarn-server-resourcemanager, hadoop-yarn-server-timelineservice, hbase-server, hbase-prefix-tree, hbase-hadoop2-compat, hadoop-mapreduce-client-jobclient, hadoop-mapreduce-client-common]
      

      This jira proposes we move all HBase-backend-related code from hadoop-yarn-server-timelineservice into its own module (possible name is yarn-server-timelineservice-storage) so that core RM modules do not depend on HBase modules any more.

        Attachments

        1. part1.yarn5667.prelim.patch
          7 kB
          Haibo Chen
        2. part2.yarn5667.prelim.patch
          51 kB
          Haibo Chen
        3. part3.yarn5667.prelim.patch
          1.04 MB
          Haibo Chen
        4. part4.yarn5667.prelim.patch
          2 kB
          Haibo Chen
        5. part5.yarn5667.prelim.patch
          1 kB
          Haibo Chen
        6. pt3.yarn5667.001.patch
          1.04 MB
          Haibo Chen
        7. pt1.yarn5667.001.patch
          7 kB
          Haibo Chen
        8. pt2.yarn5667.001.patch
          51 kB
          Haibo Chen
        9. pt4.yarn5667.001.patch
          2 kB
          Haibo Chen
        10. pt5.yarn5667.001.patch
          1 kB
          Haibo Chen
        11. pt6.yarn5667.001.patch
          101 kB
          Haibo Chen
        12. New module structure.png
          126 kB
          Haibo Chen
        13. yarn5667-001.tar.gz
          333 kB
          Haibo Chen
        14. pt9.yarn5667.001.patch
          0.5 kB
          Haibo Chen

        Issue Links

          Activity

            People

            • Assignee:
              haibochen Haibo Chen
              Reporter:
              haibochen Haibo Chen

              Dates

              • Created:
                Updated:
                Resolved:

                Issue deployment