[ZEPPELIN-555] Add Hadoop mode to the Scalding Interpreter

• Type: New Feature
• Status: Resolved
• Priority: Major
• Resolution: Duplicate
• Fix Version/s: 0.6.0
• Assignee: Sriram Krishnan
• Reporter: Sriram Krishnan
• Created: 03/Jan/16 21:44
• Updated: 14/Apr/17 23:51
• Resolved: 14/Apr/17 23:51

Description

Scalding (https://github.com/twitter/scalding) is a Scala library for writing MapReduce jobs.

ZEPPELIN-526 added a local mode Scalding interpreter to Zeppelin. This issue tracks the addition of the local mode.

AFAICT, this should involve the following steps:

• Provide the hadoop-client jars at runtime, and load the confs via the HADOOP_CONF_DIR (similar to how Spark does it - see https://zeppelin.incubator.apache.org/docs/0.6.0-incubating-SNAPSHOT/interpreter/spark.html)
• Set up the HADOOP_CLASSPATH for Zeppelin
• Make sure that the Scalding REPL starts in Hadoop mode by passing the --hdfs flag (can be done programmatically)

Once the REPL is in Hadoop mode, it can be made to switch to local mode by typing in 'useLocalMode()'.

Issue Links

• is duplicated by: ZEPPELIN-840 Scalding interpreter that works in hdfs mode
• is related to: ZEPPELIN-526 Add a (local mode) Scalding Interpreter to Zeppelin