Details
-
Bug
-
Status: Open
-
Major
-
Resolution: Unresolved
-
2.4.4
-
None
-
python2.7
spark2.4
Description
when i submit a spark job , use spark-submit and set option --files file1,
then in python code, i use
//代码占位符 path = str(os.environ["SPARK_YARN_STAGING_DIR"])
return path is None, but this can success in java code
//代码占位符 spark.read().textFile(System.getenv("SPARK_YARN_STAGING_DIR") + "/README.md")
which cause this problem.