[SPARK-19375] na.fill() should not change the data type of column

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Duplicate
• Affects Version/s: 1.3.1, 1.4.1, 1.5.2, 1.6.3, 2.0.2, 2.1.0, 2.2.0
• Component/s: SQL
• Assignee: Unassigned
• Reporter: Davies Liu
• Created: 26/Jan/17 20:20
• Updated: 26/Jan/17 20:31
• Resolved: 26/Jan/17 20:31

Description

When these function are called, a column of numeric type will be converted into DoubleType, which will cause unexpected type casting and also precision loss for LongType.

def fill(value: Double)
def fill(value: Double, cols: Array[String])
def fill(value: Double, cols: Seq[String])

Workaround

fill(Map("name" -> v))

Issue Links

• is duplicated by: SPARK-18555 na.fill miss up original values in long integers