Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 0.6.0
    • Fix Version/s: 0.7.0
    • Component/s: yarn
    • Labels:
      None

      Description

      When YARN 2.1.0-beta (non-RC) is released, we should upgrade Samza to it. This includes updating hello-samza, as well. We should also remove the classifier builds in the build.gradle, which allowed for building Samza with multiple versions of YARN. Based on SAMZA-5 discussion, it sounds like 2.1.0-beta+ should remain API-compatible, so we should just use that API (and the latest YARN version) for samza-yarn dependencies.

      1. SAMZA-9.0.patch
        52 kB
        Chris Riccomini

        Issue Links

          Activity

          Hide
          criccomini Chris Riccomini added a comment - - edited

          Attaching a patch that upgrades Samza to YARN 2.2.

          https://reviews.apache.org/r/14767

          1. Upgrade all APIs to YARN 2.2.
          2. Fix a non-daemon thread leak (on failure) in YARN AM.
          3. Removed YARN classifier from builds. Since YARN 2 API is now stable, there should be no need to classify which version of YARN Samza is being built against.
          4. Removed all minCpu and minMem stuff, since YARN eliminated that.

          Testing:

          1. Updated tests with new YARN 2.2. API. All tests pass.
          2. Updated hello-samza locally, and ran with YARN 2.2+this patch.
          3. Killed container. AM re-starts it.
          4. Triggered a failure in the AM, and verified it shuts down correctly.

          Show
          criccomini Chris Riccomini added a comment - - edited Attaching a patch that upgrades Samza to YARN 2.2. https://reviews.apache.org/r/14767 1. Upgrade all APIs to YARN 2.2. 2. Fix a non-daemon thread leak (on failure) in YARN AM. 3. Removed YARN classifier from builds. Since YARN 2 API is now stable, there should be no need to classify which version of YARN Samza is being built against. 4. Removed all minCpu and minMem stuff, since YARN eliminated that. Testing: 1. Updated tests with new YARN 2.2. API. All tests pass. 2. Updated hello-samza locally, and ran with YARN 2.2+this patch. 3. Killed container. AM re-starts it. 4. Triggered a failure in the AM, and verified it shuts down correctly.
          Hide
          criccomini Chris Riccomini added a comment -

          Added a pull request to hello-samza, as well:

          https://github.com/linkedin/hello-samza/pull/6

          Show
          criccomini Chris Riccomini added a comment - Added a pull request to hello-samza, as well: https://github.com/linkedin/hello-samza/pull/6
          Hide
          jghoman Jakob Homan added a comment -

          +1 on JIRA patch. Will take a look at pull request.

          Show
          jghoman Jakob Homan added a comment - +1 on JIRA patch. Will take a look at pull request.
          Hide
          criccomini Chris Riccomini added a comment -

          Cool, holding off on commit until you +1 the pull request as well, since I need to do both commits at the same time (since it's backwards-incompatible).

          Show
          criccomini Chris Riccomini added a comment - Cool, holding off on commit until you +1 the pull request as well, since I need to do both commits at the same time (since it's backwards-incompatible).
          Hide
          jghoman Jakob Homan added a comment -

          +1ed github as well. Good verification that very little had to be changed over there. Nice work.

          Show
          jghoman Jakob Homan added a comment - +1ed github as well. Good verification that very little had to be changed over there. Nice work.
          Hide
          criccomini Chris Riccomini added a comment -

          Merged both GH and SAMZA-9 patch.

          Show
          criccomini Chris Riccomini added a comment - Merged both GH and SAMZA-9 patch.

            People

            • Assignee:
              criccomini Chris Riccomini
              Reporter:
              criccomini Chris Riccomini
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development