Details
-
Bug
-
Status: Closed
-
Major
-
Resolution: Fixed
-
1.7.2
-
None
-
flink1.7.2
linux
java8
kafka1.1
scala 2.1
CDH
Description
I provide a simple example and used resource's picture for this issue
this application is about flink consumer kafka, when submit job such as :
flink-1.7.1/bin/flink run -m yarn-cluster -yn 1 -ynm test -p 5 -cMainConnect3 ./flinkDemo-1.0-SNAPSHOT.jar &
or
flink-1.7.1/bin/flink run -m yarn-cluster -ynm test -cMainConnect4 ./flinkDemo-1.0-SNAPSHOT.jar &
from yarn web ,i see, the total resources are erratic, and in most cases, all the remaining resources of the cluster will be used by the application
i think two ways solve this issue:
1).Add judgment about if -yn * -ys < -p system.exit(1) and Print message: -yn * -ys shoule more than the -p
2).Create taskManager with one solt until -yn * -ys >= -p as default
maybe 1) is more friendly to users