[SPARK-39202] Introduce a `putByteArrays` method to `WritableColumnVector` to support setting multiple duplicate `byte[]`

• Type: Improvement
• Status: Resolved
• Priority: Minor
• Resolution: Won't Fix
• Affects Version/s: 3.4.0
• Component/s: SQL
• Assignee: Unassigned
• Reporter: Yang Jie
• Created: 17/May/22 02:51
• Updated: 19/May/22 04:15
• Resolved: 19/May/22 04:15

Description

Add a  `putByteArrays` method to `WritableColumnVector`

 

int putByteArrays(int rowId, int total, byte[] value) 

 

 

this method used to support setting multiple duplicate `byte[]` to `WritableColumnVector`.

Since `byte[] value` is fixed length, memory can allocated at one time without calling `

reserve(int requiredCapacity)` method many times.

 

This method is applicable to `ColumnVectorUtils.populate` method with `StringType` and partial `DecimalType` scenario, this corresponds to the Vectorized Partition Column filling of Parquet and Orc

 

Issue Links

• links to: [Github] Pull Request #36571 (LuciferYang)
• links to: [Github] Pull Request #36571 (LuciferYang)