In the current Spark on YARN code, AM always will copy and overwrite its env variables to executors, so we cannot set different values for executors.
To reproduce issue, user could start spark-shell like:
Then check executor env variables by
You will always get am_val instead of executor_val. So we should not let AM to overwrite specifically set executor env variables.