[TOREE-371] $SPARK_HOME environment variable not recognised

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 0.1.0
• Environment: 64 bit Ubuntu 16.04 LTS
• Flags: Important
• Assignee: Unassigned
• Reporter: Peter
• Created: 05/Feb/17 20:49
• Updated: 27/Mar/17 05:37

Description

Hi

toreeapp.py is not recognising $SPARK_HOME, which is exported in .bashrc. toreeapp.py is instead defaulting location of spark.
$SPARK_HOME is read on line 57 in toree.py as follows:
spark_home = Unicode(os.getenv(SPARK_HOME, '/usr/local/spark')

$ echo $SPARK_HOME
/usr/lib/spark
$ sudo jupyter toree install -spark_opts='-master=local[4]'
[ToreeInstall] Installing Apache Toree version 0.1.0.dev8
[ToreeInstall]
Apache Toree is an effort undergoing incubation at the Apache Software
Foundation (ASF), sponsored by the Apache Incubator PMC.

Incubation is required of all newly accepted projects until a further review
indicates that the infrastructure, communications, and decision making process
have stabilized in a manner consistent with other successful ASF projects.

While incubation status is not necessarily a reflection of the completeness
or stability of the code, it does indicate that the project has yet to be
fully endorsed by the ASF.

Additionally, this release is not fully compliant with Apache release policy
and includes a runtime dependency that is licensed as LGPL v3 (plus a static
linking exception). This package is currently under an effort to re-license
(https://github.com/zeromq/jeromq/issues/327).
[ToreeInstall] Creating kernel Scala
[ToreeInstall] Removing existing kernelspec in /usr/local/share/jupyter/kernels/apache_toree_scala
[ToreeInstall] Installed kernelspec apache_toree_scala in /usr/local/share/jupyter/kernels/apache_toree_scala
Traceback (most recent call last):
File "/usr/local/bin/jupyter-toree", line 11, in <module>
sys.exit(main())
File "/usr/local/lib/python3.5/dist-packages/toree/toreeapp.py", line 167, in main
ToreeApp.launch_instance()
File "/usr/local/lib/python3.5/dist-packages/traitlets/config/application.py", line 653, in launch_instance
app.start()
File "/usr/local/lib/python3.5/dist-packages/toree/toreeapp.py", line 164, in start
return self.subapp.start()
File "/usr/local/lib/python3.5/dist-packages/toree/toreeapp.py", line 133, in start
self.create_kernel_json(install_dir, interpreter)
File "/usr/local/lib/python3.5/dist-packages/toree/toreeapp.py", line 90, in create_kernel_json
python_lib_contents = listdir("

{0}

/python/lib".format(self.spark_home))
FileNotFoundError: [Errno 2] No such file or directory: '/usr/local/spark/python/lib'