[ZEPPELIN-734] pyspark importing multiprocessing error

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.5.6
• Fix Version/s: 0.6.0
• Component/s: zeppelin-interpreter
• Assignee: Felix Cheung
• Reporter: Maziyar Boustani
• Created: 10/Mar/16 19:07
• Updated: 18/Mar/16 16:35
• Resolved: 18/Mar/16 16:35

Description

When try to import multiprocessing on Zeppelin using pyspark interpreter it raises an error from /tmp/zeppelin_pyspark.py file.

chrismattmann

%pyspark
from multiprocessing import Manager
manager = Manager()

Traceback (most recent call last):
  File "/tmp/zeppelin_pyspark.py", line 225, in <module>
    eval(compiledCode)
  File "<string>", line 2, in <module>
  File "/data/cluster-local/anaconda/lib/python2.7/site-packages/multiprocessing-2.6.0.2-py2.7-linux-x86_64.egg/multiprocessing/__init__.py", line 104, in Manager
    m.start()
  File "/data/cluster-local/anaconda/lib/python2.7/site-packages/multiprocessing-2.6.0.2-py2.7-linux-x86_64.egg/multiprocessing/managers.py", line 498, in start
    self._process.start()
  File "/data/cluster-local/anaconda/lib/python2.7/site-packages/multiprocessing-2.6.0.2-py2.7-linux-x86_64.egg/multiprocessing/process.py", line 109, in start
    self._popen = Popen(self)
  File "/data/cluster-local/anaconda/lib/python2.7/site-packages/multiprocessing-2.6.0.2-py2.7-linux-x86_64.egg/multiprocessing/forking.py", line 90, in __init__
    sys.stdout.flush()
AttributeError: 'Logger' object has no attribute 'flush'
ERROR   
Took 30 seconds