Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-6363

cscope build step seems racy, breaks compilation

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Critical
    • Resolution: Fixed
    • Affects Version/s: Impala 2.12.0
    • Fix Version/s: Impala 2.12.0
    • Component/s: Infrastructure
    • Labels:

      Description

      In some downstream compilation environments of Impala, the compilation is failing early with little to go on except:

      Generating IR description files
      [  0%] Built target gen_ir_descriptions
      find: ‘./be/src/codegen/CMakeFiles/test-loop.bc.dir/depend.make.tmp’: No such file or directory
      Scanning dependencies of target krb5_realm_override
      Scanning dependencies of target function-registry
      Scanning dependencies of target fb-deps
      Scanning dependencies of target gutil
      Scanning dependencies of target test-loop.bc
      Scanning dependencies of target thrift-cpp
      Scanning dependencies of target impala-parent
      [  0%] Generating files for builtins.
      [  0%] Running FlatBuffers compiler on CatalogObjects.fbs
      [  0%] Running thrift compiler on ErrorCodes.thrift
      ========================================================================
      [  1%] Building CXX object be/src/kudu/security/CMakeFiles/krb5_realm_override.dir/krb5_realm_override.cc.o
      Running mvn  install -DskipTests
      [  1%] Building CXX object be/src/gutil/CMakeFiles/gutil.dir/atomicops-internals-x86.cc.o
      Directory: /data/jenkins/workspace/impala-asf-master-exhaustive-rhel7/repos/Impala/impala-parent
      ========================================================================
      [  1%] Built target test-loop.bc
      [  1%] Building CXX object be/src/gutil/CMakeFiles/gutil.dir/bits.cc.o
      [  1%] Built target function-registry
      [  1%] Built target fb-deps
      [  1%] Running thrift compiler on beeswax.thrift
      [  1%] Running thrift compiler on BackendGflags.thrift
      [  2%] Building CXX object be/src/gutil/CMakeFiles/gutil.dir/callback_internal.cc.o
      [  2%] Running thrift compiler on CatalogInternalService.thrift
      make[2]: *** [CMakeFiles/cscope] Error 1
      make[1]: *** [CMakeFiles/cscope.dir/all] Error 2
      make[1]: *** Waiting for unfinished jobs....
      

      I've seen this twice, and the keys in both are:

      1. No such file or directory on an apparent tempfile, reported by find
      2. Failure related to cscope

      It's possible the cscope target is racy if it is being run with other steps in parallel. From bin/gen-cscope.sh:

      # Generate list of files for Cscope to index
      cd $IMPALA_HOME
      find . -regex '.*\.\(cc\|c\|hh\|h\|java\|thrift\|flex\|y\)$' > cscope.files
      

      If other make steps are being run in parallel, there could be races where items that find is supposed to inspect become unlinked underneath it, and it fails.

      In the environment where I see this, it's happened twice, with successful compilations in between.

        Attachments

          Activity

            People

            • Assignee:
              tarmstrong Tim Armstrong
              Reporter:
              mikesbrown Michael Brown
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: