[PIG-3901] Organize the Pig properties file and document all properties

• Type: Improvement
• Status: Closed
• Priority: Minor
• Resolution: Fixed
• Fix Version/s: 0.13.0
• Labels: conf config documentation properties settings
• Patch Info: Patch Available
• Release Note: Default conf/pig.properties file has been organized, and all properties are documented
• Assignee: Flip Kromer
• Reporter: Flip Kromer
• Created: 17/Apr/14 00:09
• Updated: 07/Jul/14 18:07
• Resolved: 09/May/14 04:15

Description

The current pig.properties file can use some love. Each property should be introduced by a documentation string explaining

• what the feature does,
• what its default and other allowed values are,
• why a user might change it from the default,
• and what might go wrong with each.

The documentation should follow a common format – I propose the following guidelines:

• Each property should supply either a bulleted list of acceptable values, indicating the default; or provide the default value inline with the description
• Don't say 'This setting lets you control whether Pig will decide to use the Hemiconducer feature', say 'Enables the hemiconducer feature, which [...]'
• Don't document the internals of the feature. Describe its impact on job execution or performance.
• Use consistent indentation, title formatting, and block delimiting. (The current patch does not yet do so completely, as I'm figuring it out)
• Place each setting in the appropriate block according to its impact on the user experience.
• Call out Experimental features with `EXPERIMENTAL`, but group them with similar settings.
• If a setting is dangerous, call that out with `WARNING`
• If one value is always appropriate for casual use, or always appropriate for production use, we should call that out. Production use should assume a moderately loaded single rack hadoop cluster according to the major distro's reference configuration – people running massive-scale installations don't need this file's advice.

I've attached a patch that organizes the current properties file and documents everything I felt confident describing. This is a preliminary patch, as I'll need some help documenting many of the currently un-documented ones. Please review what I've written carefully; I have reasonable experience programming Pig but limited familiarity with the experimental features.