[NEMO-401] Client stalls on 70th partition under default parallellism

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Labels: build
• Assignee: Unassigned
• Reporter: Gao Zhiyuan
• Created: 02/Aug/19 13:36
• Updated: 02/Aug/19 14:06

Description

1. Description

Nemo has default parallelism of 100. Client stalls at executing 70th partition under streaming policy. Adding `--numEventGenerators=1` in the `usr_args` wouldn't cause this problem.

*Please refer to the attachment for the log.*

1. Steps to reproduce

Execute this script after building the Nemo

start.sh

./bin/run_nexmark.sh \
        -job_id nexmark-stream \
        -executor_json `pwd`/examples/resources/executors/beam_test_executor_resources.json \
        -user_main org.apache.beam.sdk.nexmark.Main \
        -optimization_policy org.apache.nemo.compiler.optimizer.policy.StreamingPolicy \
        -scheduler_impl_class_name org.apache.nemo.runtime.master.scheduler.StreamingScheduler \
        -user_args "--runner=org.apache.nemo.client.beam.NemoRunner --streaming=true --query=$1 --manageResources=false --monitorJobs=true --streamTimeout=30"