[HIVE-6360] Hadoop 2.3 + Tez 0.3

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 0.13.0
• Assignee: Gunther Hagleitner
• Reporter: Gunther Hagleitner
• Created: 03/Feb/14 23:46
• Updated: 12/Aug/14 01:55
• Resolved: 02/Mar/14 02:25

Description

There are some things pending that rely on hadoop 2.3 or tez 0.3. These are not released yet, but will be soon. I'm proposing to collect these in the tez branch and do a merge back once these components have been released at that version.

The things depending on 0.3 or hadoop 2.3 are:

• Zero Copy read for ORC
• Unions in Tez
• Tez on secure clusters
• Changes to DagUtils to reflect tez 0.2 -> 0.3
• Prewarm containers

Issue Links

• depends upon: HIVE-6346 Add Hadoop-2.4.0 shims to hive-tez
• depends upon: HIVE-6255 Change Hive to not pass MRSplitsProto in MRHelpers.createMRInputPayloadWithGrouping
• is blocked by: HIVE-6376 Enable hive to work with tez on secure cluster
• is depended upon by: HIVE-6362 Support union all on tez
• relates to: HIVE-6347 ZeroCopy read path for ORC RecordReader
• relates to: HIVE-6442 load_dyn_part1 is flaky on Tez because it doesn't have the stage re-arranger
• relates to: HIVE-6443 Need to remove unnecessary stats gathering/merging with unions on tez
• relates to: HIVE-6444 Create unique dag id for tez 0.3
• relates to: HIVE-6493 Tez 0.3.0 API change
• relates to: HIVE-6498 Add config vars to be able to set different defaults for tez and mapred
• relates to: HIVE-6501 Change hadoop dependency on tez branch
• relates to: HIVE-6533 Switch to released tez 0.3
• relates to: HIVE-6534 Keep guava on v11 in tez branch
• relates to: HIVE-6524 Update ORC Filedump stripe sizes to match the memory manager changes
• relates to: HIVE-6525 Fix some whitespace issues in GenTezUtils
• relates to: HIVE-6391 Use pre-warm APIs in Tez to improve hive query startup
• links to: rb