[LIVY-789] Spark data files are not available in interactive session

• Type: Bug
• Status: Open
• Priority: Blocker
• Resolution: Unresolved
• Affects Version/s: 0.7.0
• Fix Version/s: 0.9.0
• Component/s: Interpreter
• Assignee: Unassigned
• Reporter: Dylan Patterson
• Created: 02/Sep/20 15:21
• Updated: 12/Nov/22 22:20

Description

Files passed via the `files` parameter are not available from the main program in interactive mode. This is because livy runs the `fake_shell.py` interpreter in the root directory while yarn stages in a temp dir.  I have not checked whether batch programs suffer the same short coming.

Potential Solutions

1. Set CWD of `fake_shell.py` to yarn staging directory
2. Create temporary working directory and copy necessary files from the `files` parameter