[PIG-3552] UriUtil used by reducer estimator should support viewfs

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 0.12.1
• Component/s: impl

Environment

Pig 0.11+

• Patch Info: Patch Available
• Assignee: Akihiro Matsukawa
• Reporter: Akihiro Matsukawa
• Created: 30/Oct/13 22:25
• Updated: 15/Apr/14 20:45
• Resolved: 31/Oct/13 22:02

Description

On Hadoop2 running viewfs throws error

2013-10-30 02:12:46,916 [main] INFO  org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler - Using reducer estimator: org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.InputSizeReducerEstimator
2013-10-30 02:12:46,917 [main] INFO  org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.InputSizeReducerEstimator - BytesPerReducer=1610612736 maxReducers=999 totalInputFileSize=-1
2013-10-30 02:12:46,917 [main] INFO  org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler - Could not estimate number of reducers and no requested or default parallelism set. Defaulting to 1 reducer.

This is caused by the fact that UriUtil.isHDFSFileOrLocalOrS3N only supports files that start with "/", "hdfs:", "file:", "s3n:" but not "viewfs:".

Trivial fix, patch attached.

Testing: ant test-commit passes.