[TOREE-357] In pyspark_runner.py make sure we always have a code_info available - ASF JIRA

Attach files

Attach Screenshot

Add vote

Voters

Watch issue

Watchers

Create sub-task

Link

Clone

Update Comment Author

Replace String in Comment

Update Comment Visibility

Delete Comments

XML

Word

Printable

JSON

Details

Type: Bug
Status: Open
Priority: Major
Resolution: Unresolved
Affects Version/s: 0.1.0
Fix Version/s: None
Component/s: None
Labels:
- pyspark

Description

In the loop that pull code_info from the queue, make sure we have a new code_info before setting the current one to null. Reason is that in some cases (say Spark Streaming for example), the cell may have finished executing but there is still data coming down