Uploaded image for project: 'Samza'
  1. Samza
  2. SAMZA-1340

StreamProcessor does not propagate container failures from StreamTask

        Details

        • Type: Bug
        • Status: Resolved
        • Priority: Major
        • Resolution: Fixed
        • Affects Version/s: 0.13.1
        • Fix Version/s: 0.13.1
        • Component/s: None
        • Labels:
          None

          Description

          Based on Chris Pettitt's observation:

          I'm on Samza's master branch. If my task throws from one of its operators I get back a success state from the LocalApplicationRunner.

          The root cause appears to be in SamzaContainerListener#onContainerFailed in StreamProcessor. There we call a regular stop and have not recorded anywhere that an exception was raised. Consequently onCoordinatorStop is invoked and the processorListener gets its onShutdown callback called instead of onFailure via onCoordinatorFailure.

            Issue Links

            links to

            Web Link GitHub Pull Request #230

              Activity

              Ascending order - Click to sort in descending order
              Hide
              Permalink
              githubbot ASF GitHub Bot added a comment -

              GitHub user navina opened a pull request:

              https://github.com/apache/samza/pull/230

              SAMZA-1340 - StreamProcessor does not propagate container failures from StreamTask

              Storing the exception seen from the container in the `SamzaContainerListener#onFailure(Throwable)` in the StreamProcessor.
              `JobCoordinator#stop` callback inspects this stored exception and invokes the correct callback for StreamProcessorLifecycleListener.
              It is pretty difficult to add all test cases. Suggestion welcome for improving code/testing.

              You can merge this pull request into a Git repository by running:

              $ git pull https://github.com/navina/samza LISAMZA-5272

              Alternatively you can review and apply these changes as the patch at:

              https://github.com/apache/samza/pull/230.patch

              To close this pull request, make a commit to your master/trunk branch
              with (at least) the following in the commit message:

              This closes #230

              commit 839473cf0dea43ba565154c5377b138655db8a5d
              Author: Navina Ramesh <navina@apache.org>
              Date: 2017-06-19T22:49:00Z

              Adding a streamprocessor test to verify containerexception is getting persisted in processor

              commit 5475f6a301a37d9e2ede1d2a8e1e319cdc812fcc
              Author: Navina Ramesh <navina@apache.org>
              Date: 2017-06-20T01:16:27Z

              Removing unused listener in StreamProcessorTestUtils

              commit 213bb2e8afff6f6421cff0697b9b5700ba5c70a1
              Author: Navina Ramesh <navina@apache.org>
              Date: 2017-06-22T02:28:13Z

              Fixing checkstyle

              Show
              githubbot ASF GitHub Bot added a comment - GitHub user navina opened a pull request: https://github.com/apache/samza/pull/230 SAMZA-1340 - StreamProcessor does not propagate container failures from StreamTask Storing the exception seen from the container in the `SamzaContainerListener#onFailure(Throwable)` in the StreamProcessor. `JobCoordinator#stop` callback inspects this stored exception and invokes the correct callback for StreamProcessorLifecycleListener. It is pretty difficult to add all test cases. Suggestion welcome for improving code/testing. You can merge this pull request into a Git repository by running: $ git pull https://github.com/navina/samza LISAMZA-5272 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/samza/pull/230.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #230 commit 839473cf0dea43ba565154c5377b138655db8a5d Author: Navina Ramesh <navina@apache.org> Date: 2017-06-19T22:49:00Z Adding a streamprocessor test to verify containerexception is getting persisted in processor commit 5475f6a301a37d9e2ede1d2a8e1e319cdc812fcc Author: Navina Ramesh <navina@apache.org> Date: 2017-06-20T01:16:27Z Removing unused listener in StreamProcessorTestUtils commit 213bb2e8afff6f6421cff0697b9b5700ba5c70a1 Author: Navina Ramesh <navina@apache.org> Date: 2017-06-22T02:28:13Z Fixing checkstyle
              Hide
              Permalink
              navina Navina Ramesh added a comment -

              Issue resolved by pull request 230
              https://github.com/apache/samza/pull/230

              Show
              navina Navina Ramesh added a comment - Issue resolved by pull request 230 https://github.com/apache/samza/pull/230
              Hide
              Permalink
              githubbot ASF GitHub Bot added a comment -

              Github user asfgit closed the pull request at:

              https://github.com/apache/samza/pull/230

              Show
              githubbot ASF GitHub Bot added a comment - Github user asfgit closed the pull request at: https://github.com/apache/samza/pull/230

                People

                • Assignee:
                  navina Navina Ramesh
                  Reporter:
                  navina Navina Ramesh
                • Votes:
                  0 Vote for this issue
                  Watchers:
                  2 Start watching this issue

                  Dates

                  • Created:
                    Updated:
                    Resolved:

                    Development