[IMPALA-2825] Dynamically generate mangled symbols for builtins

• Type: Improvement
• Status: Open
• Priority: Minor
• Resolution: Unresolved
• Affects Version/s: Impala 2.5.0
• Component/s: Backend
• Labels: osx
• Target Version: Product Backlog
• Assignee: Unassigned
• Reporter: Martin Grund
• Created: 08/Jan/16 19:25
• Updated: 09/Aug/16 19:54

Description

Currently Impala uses the information in common/function-registry/impala_functions.py to generate a Java source file containing all the mangled symbols for the builtin catalog. In addition, the names listed in the Python file do not only contain the actual mangled names but some are not and will be mangled on the fly.

A better approach is to list the C++ function signatures in this file (or an exported JSON file) and generate the mangled names dynamically. The reason for this requirement are the different ABI calling conventions on Mac OS and Linux that lead to incompatible mangled names if they are hardcoded in impala_functions.py

On approach to dynamically generate the mangled name is to de-mangle the output of nm with c++filt and build a mapping based on the address in the object. To find all functions, this has to be done as a post build task once everything is build. Since the symbol names may be spread all across Impala it may be required to look at the output of multiple jobs.

I attached a file that I used experimentally to generate a JSON file containing the demangled function signatures for the functions we currently list in impala_functions.py, the code can probably used as an inspiration.

The program can be called with

python mangler.py ./be/build/debug/exprs/libExprs.so

Issue Links

• blocks: IMPALA-2761 Build and Run Impala on OS X