[SPARK-11503] SQL API audit for Spark 1.6

• Type: Improvement
• Status: Resolved
• Priority: Blocker
• Resolution: Fixed
• Fix Version/s: 1.6.0
• Component/s: SQL
• Target Version/s: 1.6.0
• Assignee: Reynold Xin
• Reporter: Reynold Xin
• Created: 04/Nov/15 17:56
• Updated: 02/Dec/15 00:02
• Resolved: 02/Dec/15 00:02

Description

Umbrella ticket to walk through all newly introduced APIs to make sure they are consistent.

Sub-Tasks

• API audit for distributeBy and localSort: Resolved
• Break aggregate functions into multiple files: Resolved
• Add Python API for repartition and sortWithinPartitions: Resolved
• Remove some SQL aggregation tests: Resolved
• QueryExecutionListener: Resolved
• Break JdbcDialects.scala into multiple files and mark various dialects as private: Resolved
• Dataset Java API: Resolved
• Exchange plan string is too verbose: Resolved
• Dataset should set the resolved encoders internally for maps: Resolved
• Audit function names in FunctionRegistry and corresponding method names shown in functions.scala and functions.py: Resolved
• Move sql.columnar into sql.execution: Resolved
• API audit for GroupedDataset: Resolved
• Add since version for all encoders: Resolved
• API audit for Aggregator: Resolved
• Rename mapGroup -> mapGroups and flatMapGroup -> flatMapGroups: Resolved
• Audit pivot API for 1.6: Resolved
• Mark deprecated methods in SQL with "This will be removed in Spark 2.0.": Resolved
• Use varargs for multiple paths in DataFrameReader: Resolved
• Fix json_tuple and add unit tests for the Python functions added in SPARK-10621: Resolved
• Move implementations of methods back to DataFrame from Queryable: Resolved