[SPARK-13339] Clarify commutative / associative operator requirements for reduce, fold

• Type: Improvement
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 1.6.0
• Fix Version/s: 2.0.0
• Component/s: Documentation
• Target Version/s: 2.0.0
• Assignee: Sean R. Owen
• Reporter: Sean R. Owen
• Created: 16/Feb/16 13:33
• Updated: 06/Sep/16 09:53
• Resolved: 19/Feb/16 10:31

Description

As begun in https://github.com/apache/spark/pull/11091 we should make consistent the documentation for the function supplied to various reduce and fold methods. reduce needs associative and commutative; fold needs associative.

Issue Links

• is duplicated by: SPARK-12844 Spark documentation should be more precise about the algebraic properties of functions in various transformations
• links to: [Github] Pull Request #11217 (srowen)