Details
-
Sub-task
-
Status: Patch Available
-
Blocker
-
Resolution: Unresolved
-
None
-
None
-
None
-
None
Description
MaWo application can be used to run Hadoop UT faster in a Hadoop cluster.
Develop helper scripts to orchestrate end-to-end workflow for running Hadoop UT using MaWo app.
Pre-requisite:
- A Hadoop Cluster with HDFS and YARN installed
- Enable Docker on YARN feature
Helper-scripts
- MaWo_Driver
- create a docker image with latest hadoop source code
- create payload to MaWo app (This is input to mawo app where Each MaWo Task = UT execution of each Hadoop Module)
- Upload payload file to HDFS
- Update MaWo-Launch.json to resolve RM_HOST / Docker Image etc dynamically
- Launch MaWo app in Hadoop cluster