[SPARK-6052] In JSON schema inference, we should always set containsNull of an ArrayType to true

• Type: Bug
• Status: Resolved
• Priority: Blocker
• Resolution: Fixed
• Fix Version/s: 1.3.0
• Component/s: SQL
• Target Version/s: 1.3.0
• Assignee: Yin Huai
• Reporter: Yin Huai
• Created: 27/Feb/15 05:20
• Updated: 24/Apr/15 00:42
• Resolved: 02/Mar/15 15:19

Description

We should not try to figure out if an array contains null or not because we may miss arrays with null if we do sampling or future data may have nulls in the array.

Issue Links

• links to: [Github] Pull Request #4806 (yhuai)