Details

    • Type: Sub-task
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 0.9.0
    • Fix Version/s: 0.9.0
    • Component/s: container, yarn
    • Labels:
      None

      Description

      SAMZA-348 has a detailed design proposal on how we can configure Samza via a stream. Part of this work involves converting the SamzaContainer to retrieve its information via an HTTP/JSON request, rather than from JSON encoded environment variables.

      The three items that we'll need to serve via a "job coordinator" will be the job's config, the job's container:task:SSP mappings, and the job's task:changelog partition mappings.

      We'll also need to assign each container a unique ID, so that it can retrieve its task:SSP mappings (and thus, its task:changelog partition mappings).

      This ticket does not encompass creating a "job coordinator". Instead, we'll just start the HTTP server in three places: the ThreadJob, the ProcessJob, and the YARN AM. In all three cases, a URL will be set via an environment variable, which the SamzaContainer will receive, and use to retrieve its information.

        Attachments

        1. SAMZA-438-3.patch
          91 kB
          Chris Riccomini
        2. SAMZA-438-2.patch
          91 kB
          Chris Riccomini
        3. SAMZA-438-1.patch
          89 kB
          Chris Riccomini
        4. SAMZA-438-0.patch
          88 kB
          Chris Riccomini

          Activity

            People

            • Assignee:
              criccomini Chris Riccomini
              Reporter:
              criccomini Chris Riccomini
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: