[HIVE-8059] MiniHS2 improvements

• Type: Improvement
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 0.13.1
• Component/s: HiveServer2, Testing Infrastructure
• Assignee: Unassigned
• Reporter: Robert Kanter
• Created: 11/Sep/14 20:13
• Updated: 11/Sep/14 20:13

Description

In Oozie, we recently added a Hive 2 action that talks to Hive Server 2. With actions, we typically like to have unit tests that actually run some simple job with that action type. For the Hive 2 action, this required using the MiniHS2, but we ran into a few difficulties:

1. The artifact containing MiniHS2 ("hive-it-unit") isn't published to the maven repo. Luckily, there were only 2 classes that we needed so copying them into Oozie for now wasn't a big deal.
2. Figuring out which system properties, config properties, etc needed to be set for MiniHS2 required looking at some code, the MiniHS2 pom file, and asking people. Better documentation on MiniHS2 would be good.
3. In Oozie, we use our own MiniHDFS and MiniMR clusters for tests. As-is, MiniHS2 can either start and use it's own Mini clusters, or it can use an existing cluster. However, to use an existing cluster, it will fork out to a shell script that we don't have. There was no way to tell MiniHS2 to use an existing cluster without forking. In the end, I had to hack the MiniHS2 constructor to allow this (see here)
   It would be great if these could be fixed so that we can get rid of our hacks and just use MiniHS2 directly.

Special thanks to brocknoland, szehon, and mohitsabharwal for their help through this.

Issue Links

• is related to: OOZIE-1457 Create a Hive Server 2 action