Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.12.0
    • Component/s: None
    • Labels:
      None

      Description

      YARN Node labels were introduced in Hadoop version 2.6, which allows to group nodes with similar characteristics and allows applications to specify where to run. This patch adds support for YARN node labels in Samza.

      In this implementation, node labels are defined directly in yarnConfig in YarnClusterResourceManager. It might be better to have node labels as a part of SamzaResourceRequest and SamzaResource classes, but org.apache.hadoop.yarn.api.records.Container class doesn't contain node label and hence we have nothing to pass to the SamzaResource constructor in onContainersAllocated method of YarnClusterResourceManager class.

      RB Code review: https://reviews.apache.org/r/51633/

        Activity

        Hide
        xinyu Xinyu Liu added a comment -

        Merged and committed. Thanks!

        Show
        xinyu Xinyu Liu added a comment - Merged and committed. Thanks!
        Hide
        jagadish1989@gmail.com Jagadish added a comment - - edited

        Apologies for the delay in getting to this review. Thanks for the patch. Just some nits, and the patch looks shippable! I also had questions/comments (mostly around testing). I've updated the RB

        Thanks,

        Show
        jagadish1989@gmail.com Jagadish added a comment - - edited Apologies for the delay in getting to this review. Thanks for the patch. Just some nits, and the patch looks shippable! I also had questions/comments (mostly around testing). I've updated the RB Thanks,

          People

          • Assignee:
            Unassigned
            Reporter:
            logarithm Maksim Logvinenko
          • Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development