[SPARK-18140] Parquet NPE / Update to 1.9

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Duplicate
• Assignee: Unassigned
• Reporter: dori waldman
• Created: 27/Oct/16 11:57
• Updated: 14/Dec/16 18:47
• Resolved: 14/Dec/16 18:47

Description

We are using spark to write parquet files ,
We are getting nullPointerException due to this issue
https://issues.apache.org/jira/browse/PARQUET-544

trying to change to the new fixed parquet version (1.9.0) didnt worked

In submit.sh added following parameters
--jars "../lib/parquet-hadoop-1.9.0.jar"
--conf "spark.driver.userClassPathFirst=true"
--conf "spark.executor.userClassPathFirst=true"

Any advise ? its in our production environment

Thanks

Issue Links

• duplicates: SPARK-13127 Upgrade Parquet to 1.9 (Fixes parquet sorting)