Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-4701

ccache does not understand that distcc.sh could be clang or gcc

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: Impala 2.8.0
    • Fix Version/s: Impala 2.9.0
    • Component/s: Infrastructure
    • Labels:
      None

      Description

      When distcc is used, the compilation commands start with distcc.sh, not g++ or clang++. ccache then thinks that distcc.sh commands with the same parameters should produce the same results, but that's not true: distcc picks up its compiler from the environment, not from the parameters.

      Perhpas this could be fixed by adding a parameter to distcc for the compiler, rather than relying on the environment, or making distcc-gcc.sh and distcc-clang.sh.

        Activity

        Hide
        tarmstrong Tim Armstrong added a comment -

        IMPALA-4701: make distcc work reliably with clang

        The previous compiler-switching method in distcc had some problems:

        • It duplicated logic from CMakeLists.txt in choosing flags to pass to
          clang
        • In order to switch to ASAN, you needed to remember to change distcc
          settings.
        • The wrapper script approach interacted badly with ccache - GCC and
          Clang-generated artifacts would be treated as equivalent by ccache,
          meaning that if you accidentally build with the wrong compiler setting
          and the artifacts got into the cache you needed to clear ccache.

        Instead of using environment variables to set the compiler, we now
        pass the compiler as an argument to distcc.sh and set things up
        in CMakeLists.txt the same way as ccache. Switching to/from
        clang builds now requires no extra step (aside from cleaning
        out the cmake-generated files with clean.sh).

        Also changes the name of the config file

        Testing:
        Tested switching between debug and asan builds locally.

        Change-Id: I199b245fb14b6c3484b66339a7d4b37d74755af7
        Reviewed-on: http://gerrit.cloudera.org:8080/6493
        Reviewed-by: Tim Armstrong <tarmstrong@cloudera.com>
        Tested-by: Impala Public Jenkins

        Show
        tarmstrong Tim Armstrong added a comment - IMPALA-4701 : make distcc work reliably with clang The previous compiler-switching method in distcc had some problems: It duplicated logic from CMakeLists.txt in choosing flags to pass to clang In order to switch to ASAN, you needed to remember to change distcc settings. The wrapper script approach interacted badly with ccache - GCC and Clang-generated artifacts would be treated as equivalent by ccache, meaning that if you accidentally build with the wrong compiler setting and the artifacts got into the cache you needed to clear ccache. Instead of using environment variables to set the compiler, we now pass the compiler as an argument to distcc.sh and set things up in CMakeLists.txt the same way as ccache. Switching to/from clang builds now requires no extra step (aside from cleaning out the cmake-generated files with clean.sh). Also changes the name of the config file Testing: Tested switching between debug and asan builds locally. Change-Id: I199b245fb14b6c3484b66339a7d4b37d74755af7 Reviewed-on: http://gerrit.cloudera.org:8080/6493 Reviewed-by: Tim Armstrong <tarmstrong@cloudera.com> Tested-by: Impala Public Jenkins

          People

          • Assignee:
            tarmstrong Tim Armstrong
            Reporter:
            jbapple Jim Apple
          • Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development