[ZEPPELIN-1967] Passing Z variables to Shell and SQL Interpreters - ASF JIRA

XML

Word

Printable

JSON

It's possible to exchange variables between Scala and Spark
through z.put and z.get.

How to pass a variable to %sh and %sql?

In Jupyter it's possible to do for example as

  ! hadoop fs -put {localfile} {hdfsfile}

where localfile and and hdfsfile are Python variables.

In many notebooks we have to pass small variabels
from Zeppelin notes to external scripts as parameters.

It would be awesome to have something like

%sh
/path/to/script --param8={var1} --param9={var2}

where var1 and var2 would be implied to be fetched as z.get('var1')
and z.get('var2') respectively.

Or similarly for %sql :

%sql
create table dwh.table_{year} stores as parquet
as 
select * from spark_df1 where year = {year}

We miss a lot global variables for %sql and %sh so that a Zeppelin note can be used as a single parametrized
orchestration for a whole workflow.

Escaping is done similarly like in Jupyter.
With double curly braces –

  {{ will be transformed as single left curly brace - {
  }} will be escaped to single right curly brace - }

Parent Feature

ZEPPELIN-2807 Passing Z variables to SQL Interpreter (One part of ZEPPELIN-1967)

ZEPPELIN-2849 Passing Z variables to SHELL Interpreter (One part of ZEPPELIN-1967)

requires

ZEPPELIN-1595 Make ZeppelinContext extensible

links to

GitHub Pull Request #2825

(1 links to)