[SPARK-13574] Improve parquet dictionary decoding for strings

• Type: Improvement
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Fix Version/s: 2.0.0
• Component/s: SQL
• Assignee: Nong Li
• Reporter: Nong Li
• Created: 29/Feb/16 20:10
• Updated: 06/Mar/16 21:05
• Resolved: 02/Mar/16 23:03

Description

Currently, the parquet reader will copy the dictionary value for each data value. This is bad for string columns as we explode the dictionary during decode. We should instead, have the data values point to the safe backing memory.

Issue Links

• links to: [Github] Pull Request #11434 (nongli)
• links to: [Github] Pull Request #11454 (nongli)