Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-4681

buildall.sh -notests runs forever and hogs resources

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Critical
    • Resolution: Not A Bug
    • Affects Version/s: Impala 2.8.0
    • Fix Version/s: Impala 2.8.0
    • Component/s: Infrastructure
    • Labels:
      None

      Description

      Running this on my laptop with Ubuntu 12.04 runs forever and basically kills the machine due to excessive resource consumption:

      ./buildall.sh -notests
      

      I've attached the output of

      ./buildall.sh -notests > buildall.out
      

      I aborted after some time. My HEAD was at c2faf4a8a13ba89f2f4f2bc4fbd878cffda53d1f

      I have a feeling this might have something to do with the following change:
      http://gerrit.cloudera.org:8080/4790

      1. buildall.out
        543 kB
        Alexander Behm

        Activity

        Hide
        alex.behm Alexander Behm added a comment -

        I made it work by rebasing Impala-lzo and wiping all CMakeCache.txt

        Show
        alex.behm Alexander Behm added a comment - I made it work by rebasing Impala-lzo and wiping all CMakeCache.txt
        Hide
        tarmstrong Tim Armstrong added a comment -

        Any idea what the cause was? That change shouldn't launch more compile jobs than CPUs.

        I've seen issues like this when the build files were regenerated incorrectly by cmake after using distcc and it ends running the local C++ compiler with distcc parallelism. What were the processes that were consuming all the CPU?

        Show
        tarmstrong Tim Armstrong added a comment - Any idea what the cause was? That change shouldn't launch more compile jobs than CPUs. I've seen issues like this when the build files were regenerated incorrectly by cmake after using distcc and it ends running the local C++ compiler with distcc parallelism. What were the processes that were consuming all the CPU?
        Hide
        jbapple Jim Apple added a comment -

        Note that something similar to this is tested in each public pre-merge test:

        https://github.com/apache/incubator-impala/blob/master/bin/bootstrap_build.sh

        http://jenkins.impala.io:8080/view/Utility/job/ubuntu-14.04-build-only/

        Here are some differences: No Impala-lzo, no distcc, Ubuntu 14.04 (not 12.04), completely stock environment.

        Show
        jbapple Jim Apple added a comment - Note that something similar to this is tested in each public pre-merge test: https://github.com/apache/incubator-impala/blob/master/bin/bootstrap_build.sh http://jenkins.impala.io:8080/view/Utility/job/ubuntu-14.04-build-only/ Here are some differences: No Impala-lzo, no distcc, Ubuntu 14.04 (not 12.04), completely stock environment.

          People

          • Assignee:
            tarmstrong Tim Armstrong
            Reporter:
            alex.behm Alexander Behm
          • Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development