Details
-
Bug
-
Status: Open
-
Major
-
Resolution: Unresolved
-
None
-
None
-
None
-
None
Description
Drill generates code for operators, compile the classes and load them on the fly of a query. However, in large query cases, the generated code will become hundreds KB or could be even bigger. We have seen multiple issues reported when generated code is too big either due to Java's size limit to one method or degrated performance of compiling or executing. Also when I looked at JIT optimization logs, there are many complaining about 'hot method too big'
Some measures can be considered to reduce the code size, such as,
1) For now Drill embed function calls' code directly into generated code, this make one line function call to become 5-10 lines code in generated java classes. If we inject these functions as a private functions of the classes and directly call them in main function body, this could reduce code size while the cost of function call can be erased by JIT inline optimization.
2) Drill generate one variable for each column, if the column number became dozens to one hundred, the code will be redundant, we could consider using an array to store the value vectors and loop on top of it so the code size will be reduced even more.