Minor Description
import org.apache.spark.sql.functions case class MyRow(value : BigDecimal) val values = List(MyRow(BigDecimal.valueOf(1.23456789))) val dataFrame = spark.createDataFrame(values) dataFrame.show() dataFrame.withColumn("value_rounded", functions.round(dataFrame.col("value"))).show()
This produces output:
+--------------------+ | value| +--------------------+ |1.234567890000000000| +--------------------+ +--------------------+-------------+ | value|value_rounded| +--------------------+-------------+ |1.000000000000000000| 1| +--------------------+-------------+
Same problem occurs when I use round function to filter dataFrame.
