Details
-
Bug
-
Status: Resolved
-
Major
-
Resolution: Fixed
-
3.2.1, 3.4.0
-
None
-
Reviewed
Description
When using the synthetic generator json file example from the doc ( https://hadoop.apache.org/docs/current/hadoop-sls/SchedulerLoadSimulator.html#SYNTH_JSON_input_file_format ), it throws the following exception:
java.lang.IllegalArgumentException: Null user at org.apache.hadoop.security.UserGroupInformation.createRemoteUser(UserGroupInformation.java:1269) at org.apache.hadoop.security.UserGroupInformation.createRemoteUser(UserGroupInformation.java:1256) at org.apache.hadoop.yarn.sls.appmaster.AMSimulator.submitReservationWhenSpecified(AMSimulator.java:191) at org.apache.hadoop.yarn.sls.appmaster.AMSimulator.firstStep(AMSimulator.java:161) at org.apache.hadoop.yarn.sls.scheduler.TaskRunner$Task.run(TaskRunner.java:88) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
So the solution is either:
1) to make user_name a mandatory field, or
2) to set default user in SLS code if the json file does not define it.
IMO, solution 2 might be better, because in most cases (if not all) user_name has no impact on scheduler performance, thus it is reasonable to make it an optional field, which is also consistent with the job.user field in SLS JSON file.