Uploaded image for project: 'Samza'
  1. Samza
  2. SAMZA-816

LocalityManager.register() should not register coordinator stream SystemConsumer in SamzaContainer context

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 0.10.0
    • Fix Version/s: 0.10.0
    • Component/s: None
    • Labels:
      None

      Description

      In current code, LocalityManager.register() is invoked under SamzaContainer context as well, which results in a false positive in the metrics messages_behind_high_watermark: it include the lag the container has on the coordinator stream as well, while there is no logic in the container to consume the coordinator stream. It would be better to make two versions of LocalityManager, one is used by JobCoordinator that reads the locality data, while the other is used by SamzaContainer to write the locality data.

        Attachments

        1. SAMZA-816-1.patch
          19 kB
          Yi Pan
        2. SAMZA-816-0.patch
          19 kB
          Yi Pan

          Activity

            People

            • Assignee:
              nickpan47 Yi Pan
              Reporter:
              nickpan47 Yi Pan
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: