Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-913

Umbrella: Add a way to register long-lived services in a YARN cluster

    Details

    • Type: New Feature
    • Status: In Progress
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: 2.5.0, 2.4.1
    • Fix Version/s: None
    • Component/s: api, resourcemanager
    • Labels:
      None

      Description

      In a YARN cluster you can't predict where services will come up -or on what ports. The services need to work those things out as they come up and then publish them somewhere.

      Applications need to be able to find the service instance they are to bond to -and not any others in the cluster.

      Some kind of service registry -in the RM, in ZK, could do this. If the RM held the write access to the ZK nodes, it would be more secure than having apps register with ZK themselves.

        Attachments

        1. RegistrationServiceDetails.txt
          2 kB
          Robert Joseph Evans
        2. 2014-09-03_Proposed_YARN_Service_Registry.pdf
          337 kB
          Steve Loughran
        3. YARN-913-001.patch
          215 kB
          Steve Loughran
        4. 2014-09-08_YARN_Service_Registry.pdf
          327 kB
          Steve Loughran
        5. yarnregistry.pdf
          176 kB
          Steve Loughran
        6. yarnregistry.tla
          14 kB
          Steve Loughran
        7. YARN-913-002.patch
          261 kB
          Steve Loughran
        8. YARN-913-003.patch
          343 kB
          Steve Loughran
        9. YARN-913-003.patch
          374 kB
          Steve Loughran
        10. YARN-913-004.patch
          375 kB
          Steve Loughran
        11. YARN-913-006.patch
          389 kB
          Steve Loughran
        12. YARN-913-007.patch
          394 kB
          Steve Loughran
        13. YARN-913-008.patch
          424 kB
          Steve Loughran
        14. YARN-913-009.patch
          492 kB
          Steve Loughran
        15. YARN-913-010.patch
          497 kB
          Steve Loughran
        16. YARN-913-011.patch
          502 kB
          Steve Loughran
        17. YARN-913-012.patch
          523 kB
          Steve Loughran
        18. YARN-913-013.patch
          526 kB
          Steve Loughran
        19. YARN-913-014.patch
          532 kB
          Steve Loughran
        20. YARN-913-015.patch
          529 kB
          Steve Loughran
        21. YARN-913-016.patch
          544 kB
          Steve Loughran
        22. YARN-913-017.patch
          544 kB
          Steve Loughran
        23. yarnregistry.pdf
          176 kB
          Steve Loughran
        24. YARN-913-018.patch
          545 kB
          Steve Loughran
        25. yarnregistry.pdf
          168 kB
          Steve Loughran
        26. YARN-913-019.patch
          550 kB
          Steve Loughran
        27. YARN-913-020.patch
          550 kB
          Steve Loughran
        28. YARN-913-021.patch
          549 kB
          Steve Loughran

          Issue Links

            Activity

              People

              • Assignee:
                stevel@apache.org Steve Loughran
                Reporter:
                stevel@apache.org Steve Loughran
              • Votes:
                5 Vote for this issue
                Watchers:
                70 Start watching this issue

                Dates

                • Created:
                  Updated:

                  Time Tracking

                  Estimated:
                  Original Estimate - 1.5h Original Estimate - 1.5h
                  1.5h
                  Remaining:
                  Time Spent - 1h Remaining Estimate - 1h
                  1h
                  Logged:
                  Time Spent - 1h Remaining Estimate - 1h
                  1h