[IMPALA-9371] Run include-what-you-use on all patches

• Type: Improvement
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Component/s: Infrastructure
• Labels: ramp-up
• Target Version: Product Backlog
• Epic Color: ghx-label-2
• Assignee: Unassigned
• Reporter: Tim Armstrong
• Created: 10/Feb/20 19:22
• Updated: 14/May/20 18:52

Description

Impala compile times are longer than necessary because .cc files pull in a lot of headers, especially from the standard C++ library and boost. I looked at the preprocessor output for a couple of files, and it included ~400,000 lines of code, almost entirely from headers.

include-what-you-use is a tool that can identify cases where header includes can be avoided or moved from .h to .cc files to reduce the number of extraneous headers pulled into each compile unit.

Kudu has some tooling to run IWYU in a dev environment and also in their gerrit precommit.

I was able to run iwyu against Impala as follows:

# Clone repo and check out branch to match Impala toolchain's clang version
git clone https://github.com/include-what-you-use/include-what-you-use.git
cd include-what-you-use
git checkout clang_5.0

# Build IWYU against clang from toolchain
cmake -G "Unix Makefiles" -DIWYU_LLVM_ROOT_PATH=/opt/Impala-Toolchain/llvm-5.0.1-p2/ -DCMAKE_CXX_FLAGS=-D_GLIBCXX_USE_CXX11_ABI=0 ..
make -j $(nproc)
IWYU_BUILD_DIR=$(pwd)

# Run IWYU on the impala codebase.
cd $IMPALA_HOME
# Do an ASAN build to force Impala to generate compile_commands.json with compile commands suitable for clang.
./buildall.sh -noclean -notests -asan
# Placeholder for additional arguments to the iwyu tool
IWYU_ARGS=""
PATH=$IWYU_BUILD_DIR:$PATH ~/repos/include-what-you-use/iwyu_tool.py -p . -j $(nproc) -- $IWYU_ARGS > iwyu.txt

Issue Links

• is duplicated by: IMPALA-6833 Use include-what-you-use to reduce build times

Sub-Tasks

• Trial run of IWYU on codebase: Resolved
• Add initial IWYU mappings: Resolved