Uploaded image for project: 'Samza'
  1. Samza
  2. SAMZA-1346

GroupByContainerCount.balance() should guard against null LocalityManager

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.13.1
    • Component/s: None
    • Labels:
      None

      Description

      While it's less likely after SAMZA-1334, we have seen cases of an NPE in embedded mode.

      org.apache.samza.SamzaException: Failed to run application
      	at org.apache.samza.runtime.LocalApplicationRunner.run(LocalApplicationRunner.java:136)
      	at com.linkedin.beam.runners.samza.runtime.fluent.FluentRuntime$RunnerTask.run(FluentRuntime.java:114)
      	... 1 more
      Caused by: java.lang.NullPointerException
      	at org.apache.samza.container.grouper.task.GroupByContainerCount.balance(GroupByContainerCount.java:92)
      	at org.apache.samza.coordinator.JobModelManager$.readJobModel(JobModelManager.scala:257)
      	at org.apache.samza.coordinator.JobModelManager.readJobModel(JobModelManager.scala)
      	at org.apache.samza.standalone.StandaloneJobCoordinator.<init>(StandaloneJobCoordinator.java:108)
      	at org.apache.samza.standalone.StandaloneJobCoordinatorFactory.getJobCoordinator(StandaloneJobCoordinatorFactory.java:29)
      	at org.apache.samza.processor.StreamProcessor.<init>(StreamProcessor.java:111)
      	at org.apache.samza.processor.StreamProcessor.<init>(StreamProcessor.java:94)
      	at org.apache.samza.runtime.LocalApplicationRunner.createStreamProcessor(LocalApplicationRunner.java:231)
      	at org.apache.samza.runtime.LocalApplicationRunner.lambda$run$0(LocalApplicationRunner.java:125)
      	at org.apache.samza.runtime.LocalApplicationRunner$$Lambda$35/1940982718.accept(Unknown Source)
      	at java.util.ArrayList.forEach(ArrayList.java:1249)
      	at org.apache.samza.runtime.LocalApplicationRunner.run(LocalApplicationRunner.java:121)
      	... 2 more
      

      It should be straight forward to defend against this case and provide better feedback in the logs. E.g. if the locality manager is null, then host affinity is not enabled and we could just defer to group().

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                jmakes Jake Maes
                Reporter:
                jmakes Jake Maes
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: