[PIG-4247] S3 properties are not picked up from core-site.xml in local mode

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 0.15.0
• Assignee: Cheolsoo Park
• Reporter: Cheolsoo Park
• Created: 23/Oct/14 19:46
• Updated: 07/Jun/15 03:47
• Resolved: 26/Oct/14 21:12

Description

Even in local mode, fs.s3 properties need to be set if the job accesses s3 files (for eg, registering jars on s3, loading input files on s3, etc). In particular, fs.s3.awsSecretAccessKey and fs.s3.awsAccessKey are usually set in core-site.xml, but since local mode doesn't load core-site.xml, these properties have to be set again in pig.properties. This adds operational overhead because whenever rotating aws keys, pig.properties also needs to be updated. So it would be nice if fs.s3 properties can be set even in local mode.

Issue Links

• breaks: PIG-4556 Local mode is broken in some case by PIG-4247