[SPARK-9876] Upgrade parquet-mr to 1.8.1

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.5.0
• Fix Version/s: 2.1.0
• Component/s: SQL
• Assignee: Ryan Blue
• Reporter: Cheng Lian
• Shepherd: Cheng Lian
• Created: 12/Aug/15 12:06
• Updated: 01/Dec/16 18:37
• Resolved: 01/Dec/16 18:37

Description

parquet-mr 1.8.1 fixed several issues that affect Spark. For example PARQUET-201 (SPARK-9407).

Issue Links

• is related to: SPARK-6859 Parquet File Binary column statistics error when reuse byte[] among rows
• relates to: SPARK-11153 Turns off Parquet filter push-down for string and binary columns
• links to: [Github] Pull Request #9225 (liancheng)
• links to: [Github] Pull Request #13280 (rdblue)
• links to: [Github] Pull Request #13389 (HyukjinKwon)
• links to: [Github] Pull Request #13445 (yhuai)
• links to: [Github] Pull Request #13450 (yhuai)