[VXQUERY-179] Using Yarn For Cluster Management

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Assignee: Steven Jacobs
• Reporter: Steven Jacobs
• Created: 22/Sep/15 17:10
• Updated: 01/Oct/15 20:34
• Resolved: 01/Oct/15 20:34

Description

We could use Yarn as our cluster manager. The Apache Hadoop YARN (Yet Another Resource Negotiator) would be a good cluster management tool for VXQuery. If VXQuery can read data from HDFS, then why not also manage the cluster with a tool provided by Hadoop. The solution would replace the current custom python scripts for cluster management.

Issue Links

• duplicates: VXQUERY-130 Add support for running VXQuery using Yarn