XMLWordPrintableJSON

    Details

    • Type: Sub-task
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.10.1
    • Component/s: None
    • Labels:
      None

      Description

      Currently, the YARN AM pretty much dictates how the JobCoordinator works. This creates lot of inflexibility on how we can control failures or even integrate with new system (Mesos).
      For e.g., https://issues.apache.org/jira/browse/SAMZA-465?focusedCommentId=14522043&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14522043

      It would nice to invert the logic to JobCoordinator, and JobCoordinator has a global view of container failures, config changes etc. This simplifies lot of implementation specifics (for e.g., dynamic scaling becomes easier).

      A another nice to have, would be make this logic pluggable.

      e.g.,
      job.clustermanager = org.apache.samza.YarnClusterManager
      job.clustermanager = org.apache.samza.MesosClusterManager
       
      

        Attachments

        1. rb47687 (1).patch
          243 kB
          Jagadish

          Issue Links

            Activity

              People

              • Assignee:
                jagadish1989@gmail.com Jagadish
                Reporter:
                naveenatceg Naveen Somasundaram
              • Votes:
                0 Vote for this issue
                Watchers:
                7 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: