Details
-
Sub-task
-
Status: Open
-
Minor
-
Resolution: Unresolved
-
vectorization-branch
-
None
-
None
-
None
Description
In order to prevent a bug, I had to use BytesColumnVector.setVal instead of BytesColumnVector.setRef when creating the output of all string functions. These include TRIM/LTRIM/RTRIM/SUBSTR, which can be made faster if they can use the setRef method instead. That allows them to avoid the cost of copying the data, by just setting a reference to a string (by setting the byte[] pointer, start, and length).
As a future performance enhancement, it would be desirable to be able to use setRef for these functions instead of setVal. But to do that, it is necessary to be able to mark a BytesColumnVector that is being referenced into so it is not reclaimed and re-used. So this would require a design and implementation change to the output column manager used in VectorizationContext.
I'm marking this as "minor" priority because it will result in a small performance enhancement, most likely. It can be deferred for a while.
The following is an example of a query that will exhibit a bug if setRef is used instead of setVal in the implementation of trim and concat functions:
select l_shipmode,
rtrim(concat(l_shipmode,' ')) – incorrect result for this output column
,trim(concat(' ',l_shipmode)) – requires this line for bug to show up
from lineitem_orc
where l_orderkey = 1;