[SPARK-9270] Allow --name option in pyspark

• Type: Bug
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 1.4.1, 1.5.0
• Fix Version/s: 1.5.0
• Component/s: PySpark
• Assignee: Cheolsoo Park
• Reporter: Cheolsoo Park
• Created: 23/Jul/15 05:27
• Updated: 24/Jul/15 18:58
• Resolved: 24/Jul/15 18:58

Description

Currently, the app name is hardcoded in pyspark as "PySparkShell", and the app name cannot be changed.

SPARK-8650 fixed this issue for spark-sql.

SPARK-9180 introduced a new option --name for spark-shell.

sparkR is different because SparkContext is not automatically constructed in sparkR, and the app name can be set when initializing SparkContext.

In summary-

shell	able to set app name
pyspark	no
spark-shell	yes via --name
spark-sql	yes via --conf spark.app.name
sparkR	n/a

Issue Links

• is related to: SPARK-8650 Use the user-specified app name priority in SparkSQLCLIDriver or HiveThriftServer2
• relates to: SPARK-9180 Accept --name option in spark-submit
• links to: [Github] Pull Request #7610 (piaozhexiu)