[IMPALA-6198] Error starting cluster for a custom cluster test on a release build

• Type: Bug
• Status: Resolved
• Priority: Blocker
• Resolution: Fixed
• Affects Version/s: Impala 2.11.0
• Fix Version/s: Impala 2.11.0
• Component/s: Infrastructure
• Labels: broken-build
• Epic Color: ghx-label-8
• Assignee: Vuk Ercegovac
• Reporter: Taras Bobrovytsky
• Created: 16/Nov/17 02:59
• Updated: 17/Nov/17 18:00
• Resolved: 17/Nov/17 17:47

Description

We see the following failure on a release exhaustive build:

self = <test_catalog_wait.TestCatalogWait object at 0x54af950>

    @pytest.mark.execute_serially
    def test_delayed_catalog(self):
      """ Tests client interactions with the cluster when one of the daemons,
            impalad[2], is delayed in initializing its local catalog replica."""
    
      # On startup, expect only two executors to be registered.
      self._start_impala_cluster(["--catalog_init_delays=0,0,200000"],
>                                expected_num_executors=2)

custom_cluster/test_catalog_wait.py:48: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
common/custom_cluster_test_suite.py:135: in _start_impala_cluster
    check_call(cmd + options, close_fds=True)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

popenargs = (['/data/jenkins/workspace/impala-umbrella-build-and-test/repos/Impala/bin/start-impala-cluster.py', '--cluster_size=3...umbrella-build-and-test/repos/Impala/logs/custom_cluster_tests', '--log_level=1', '--catalog_init_delays=0,0,200000'],)
kwargs = {'close_fds': True}, retcode = 1
cmd = ['/data/jenkins/workspace/impala-umbrella-build-and-test/repos/Impala/bin/start-impala-cluster.py', '--cluster_size=3'...a-umbrella-build-and-test/repos/Impala/logs/custom_cluster_tests', '--log_level=1', '--catalog_init_delays=0,0,200000']

    def check_call(*popenargs, **kwargs):
        """Run command with arguments.  Wait for command to complete.  If
        the exit code was zero then return, otherwise raise
        CalledProcessError.  The CalledProcessError object will have the
        return code in the returncode attribute.
    
        The arguments are the same as for the Popen constructor.  Example:
    
        check_call(["ls", "-l"])
        """
        retcode = call(*popenargs, **kwargs)
        cmd = kwargs.get("args")
        if cmd is None:
            cmd = popenargs[0]
        if retcode:
>           raise CalledProcessError(retcode, cmd)
E           CalledProcessError: Command '['/data/jenkins/workspace/impala-umbrella-build-and-test/repos/Impala/bin/start-impala-cluster.py', '--cluster_size=3', '--num_coordinators=3', '--log_dir=/data/jenkins/workspace/impala-umbrella-build-and-test/repos/Impala/logs/custom_cluster_tests', '--log_level=1', '--catalog_init_delays=0,0,200000']' returned non-zero exit status 1