[ZEPPELIN-5811] Shell script for activate conda env

• Type: Improvement
• Status: Open
• Priority: Minor
• Resolution: Unresolved
• Affects Version/s: 0.10.1
• Component/s: r-interpreter, zeppelin-interpreter
• Labels: conda jupyter
• Assignee: Unassigned
• Reporter: Thomas Decaux
• Created: 01/Sep/22 21:36
• Updated: 01/Sep/22 21:36

Description

When using R interpreter, conda to install dependencies, Zeppelin have a very nice feature to activate conda env, but it creates from java code an executable shell script:

 

the following code is executed:

https://github.com/apache/zeppelin/blob/master/zeppelin-jupyter-interpreter/src/main/java/org/apache/zeppelin/jupyter/JupyterKernelInterpreter.java#L199

private String activateCondaEnv(String envName) throws IOException {
...
File scriptFile = Files.createTempFile("zeppelin_jupyter_kernel_", ".sh").toFile();    

try (FileWriter writer = new FileWriter(scriptFile)) {      IOUtils.write(String.format("chmod 777 -R %s \nsource %s/bin/activate \nconda-unpack",              envName, envName),              writer);    }
...
} 

In common case this is very nice and work well.

But in my use case, we have multiple conda envs, I cant add it in the Dockerfile (> 4G). We use "conda pack" to create tar archive, then "wget" from the container to download and "tar -xzf" to install the conda env.

I am wondering, if the shell script created by java could be a simple file inside "/opt/zeppelin/bin":

conda-activate-env.sh:

chmod 777 -R /opt/conda/envs/$1
source /opt/conda/envs/$1/bin/activate 
conda-unpack

So this is possible to override the file and change this behavior easily, in my case:

conda-activate-env.sh:

cd /tmp

wget https://conda-envs/$1.tar.gz
mkdir -p /opt/conda/envs/$1
tar -xzf  $1.tar.gz -C /opt/conda/envs/$1/

chmod 777 -R /opt/conda/envs/$1 
source /opt/conda/envs/$1/bin/activate

I can do a PR very soon