Description
Reduce runtime overhead of a program that creates an primitive array in DataFrame
When a program creates an array in DataFrame, the code generator creates boxing operations. If an array is for primitive type, there are some opportunities for optimizations in generated code to reduce runtime overhead.
Here is a simple example that has generated code with boxing operation
val df = sparkContext.parallelize(Seq(0.0d, 1.0d), 1).toDF
df.selectExpr("Array(value + 1.1d, value + 2.2d)").show
Attachments
Issue Links
- is duplicated by
-
SPARK-16042 Eliminate nullcheck code at projection for an array type
-
- Resolved
-
- links to
