[OOZIE-1613] Update Hadoop to 1.2.1

• Type: Improvement
• Status: Resolved
• Priority: Minor
• Resolution: Duplicate
• Affects Version/s: trunk
• Component/s: core
• Assignee: Unassigned
• Reporter: Robert Kanter
• Created: 15/Nov/13 23:22
• Updated: 29/Aug/14 18:50
• Resolved: 29/Aug/14 18:50

Description

We should update Hadoop 1.1.1 to the latest version, 1.2.1.

I think we'd be able to get rid of some custom stuff, including:

• Hadoop 1.2.1 has hadoop-auth stuff, so we could get rid of the special <hadoop.auth.version> handling in the root pom, which is currently set to 2.0.2-alpha (which is an old alpha version)
• We could stop excluding TestMapReduceActionExecutorUberJar.java from the unit tests
• We can get rid of our copy of AltKerberosAuthenticationHandler.java in the login module (see OOZIE-1104)
• We can use DistCp v2 in each version of Hadoop, which helps with OOZIE-1728

In Hadoop 1.2.x, hadoop-core contains the stuff in hadoop-auth, and we need the hadoop-auth stuff for Oozie client, which would mean that we'd have to put hadoop-core into the Oozie client. And in the server, its likely that hadoop-core 1.2.x isn't 100% compatible with hadoop-auth. So handling this would be quite tricky and risky; its working now with Hadoop 1.1.1 and hadoop-auth 2.0.2-alpha, so I'm reluctant to change it without someone doing extensive testing.

Issue Links

• blocks: OOZIE-1457 Create a Hive Server 2 action
• duplicates: OOZIE-1830 Change hadoop-1 profile to use 1.2.1
• is related to: OOZIE-1728 When an ApplicationMaster restarts, it restarts the launcher job: DistCp followup