[PIG-3754] InputSizeReducerEstimator.getTotalInputFileSize reports incorrect size

• Type: Bug
• Status: Closed
• Priority: Trivial
• Resolution: Fixed
• Affects Version/s: 0.13.0
• Fix Version/s: 0.13.0
• Assignee: Aniket Namadeo Mokashi
• Reporter: Aniket Namadeo Mokashi
• Created: 08/Feb/14 00:16
• Updated: 16/Sep/15 20:56
• Resolved: 07/Mar/14 07:03

Description

If you have more than one input, InputSizeReducerEstimator.getTotalInputFileSize can return incorrect value if one of the loader returns -1 and is not file based (eg- hbase). This causes incorrect reducer estimation and problems in auto.local mode.

If size of input is not found in for any of the inputs, we should bail out with return value of -1.

Issue Links

• breaks: PIG-4679 Performance degradation due to InputSizeReducerEstimator since PIG-3754