Major Description
java_gateway seems to be missing import for _exception_message:
$ pyspark Python 2.7.13 (default, Apr 4 2017, 08:47:57) Type "copyright", "credits" or "license" for more information. IPython 5.3.0 -- An enhanced Interactive Python. ? -> Introduction and overview of IPython's features. %quickref -> Quick reference. help -> Python's own help system. object? -> Details about 'object', use 'object??' for extra details. 2019-01-06 18:16:56 WARN Utils:66 - Your hostname, TLVMACJ425FVH6 resolves to a loopback address: 127.0.0.1; using 10.196.0.12 instead (on interface en3) 2019-01-06 18:16:56 WARN Utils:66 - Set SPARK_LOCAL_IP if you need to bind to another address 2019-01-06 18:16:56 WARN NativeCodeLoader:62 - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable Setting default log level to "WARN". To adjust logging level use sc.setLogLevel(newLevel). For SparkR, use setLogLevel(newLevel). Welcome to ____ __ / __/__ ___ _____/ /__ _\ \/ _ \/ _ `/ __/ '_/ /__ / .__/\_,_/_/ /_/\_\ version 2.3.2 /_/ Using Python version 2.7.13 (default, Apr 4 2017 08:47:57) SparkSession available as 'spark'. In [1]: from java_gateway import local_connect_and_auth In [2]: local_connect_and_auth(80, "xxx") --------------------------------------------------------------------------- NameError Traceback (most recent call last) <ipython-input-2-bc1a1b76dc77> in <module>() ----> 1 local_connect_and_auth(80, "xxx") /Users/dbirkman/projects/spark-2.3.2-with-pythonrdd-patch/spark/python/pyspark/java_gateway.pyc in local_connect_and_auth(port, auth_secret) 171 return (sockfile, sock) 172 except socket.error as e: --> 173 emsg = _exception_message(e) 174 errors.append("tried to connect to %s, but an error occured: %s" % (sa, emsg)) 175 sock.close() NameError: global name '_exception_message' is not defined