[HCATALOG-538] HCatalogStorer fails for 100GB of data with dynamic partitioning (number of partition is 300)

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.4, 0.5
• Fix Version/s: 0.4.1

Environment

Hadoop 0.23.4
HCatalog 0.4

• Assignee: Arup Malakar
• Reporter: Arup Malakar
• Created: 25/Oct/12 21:15
• Updated: 22/Dec/12 02:03
• Resolved: 10/Nov/12 01:15

Description

A hadoop job with 100GB of data and 300 partitions fails. All the maps succeed fine but the commit job fails after that. This looks like a timeout issue as commitJob() takes more than 10 minutes. I am running this on hadoop-0.23.4. I am playing with yarn.nm.liveness-monitor.expiry-interval-ms, yarn.am.liveness-monitor.expiry-interval-ms etc to make it work.

This JIRA is for optimizing the commitJob(), as 10 minutes is too long.
On a side note for storing 100GB of data without partition takes ~12 minutes, same amount of data with 300 partitions fails after 45 minutes. These tests were run on a 10 node cluster.

Issue Links

• relates to: HCATALOG-580 Optimizations in HCAT-538 break e2e tests