[SAMZA-970] Problems with integration tests

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.10.0
• Fix Version/s: 0.10.1, 0.11.0
• Component/s: test

Environment

OS X 10.11.5

• Assignee: Xinyu Liu
• Reporter: Robert Crim
• Created: 22/Jun/16 01:33
• Updated: 29/Jul/16 20:50
• Resolved: 29/Jul/16 20:50

Description

I've had a couple of problems running the integration tests.

First, scipy wouldn't install. Bumping the virtualenv version changed this behaviour (failing in different ways) until 15.0.1, which seems OK.

Then I get this failure after zopkio tries to run the first test.

2016-06-21 18:14:34,691 naarad [INFO] Available graphing modules:  
2016-06-21 18:14:34,697 zopkio.test_runner [INFO] Setting up configuration: single execution
2016-06-21 18:14:34,697 deployment [INFO] Current working directory: /private/tmp/samza-test
2016-06-21 18:14:34,697 deployment [INFO] Downloading: https://archive.apache.org/dist/hadoop/common/hadoop-2.6.1/hadoop-2.6.1.tar.gz
2016-06-21 18:25:10,895 deployment [INFO] Downloading: http://www.us.apache.org/dist/kafka/0.8.2.0/kafka_2.9.2-0.8.2.0.tgz
2016-06-21 18:25:12,992 deployment [INFO] Downloading: http://archive.apache.org/dist/zookeeper/zookeeper-3.4.3/zookeeper-3.4.3.tar.gz
2016-06-21 18:25:48,654 deployment [INFO] Deploying zookeeper_instance_0 on host: localhost
2016-06-21 18:25:51,251 deployment [INFO] Deploying yarn_rm_instance_0 on host: localhost
2016-06-21 18:25:59,116 deployment [INFO] Deploying yarn_nm_instance_0 on host: localhost
2016-06-21 18:26:07,995 deployment [INFO] Deploying kafka_instance_0 on host: localhost
2016-06-21 18:26:09,732 zopkio.test_runner [ERROR] Aborting single execution due to setup_suite failure:
Traceback (most recent call last):
  File "/tmp/samza-test/samza-integration-tests/lib/python2.7/site-packages/zopkio/test_runner.py", line 189, in run
    self.deployment_module.setup_suite()
  File "/tmp/samza-test/scripts/deployment.py", line 85, in setup_suite
    'install_path': samza_install_path,
TypeError: Can't instantiate abstract class SamzaJobYarnDeployer with abstract methods kill_all_process

2016-06-21 18:26:09,732 zopkio.test_runner [INFO] Execution of configuration: single execution complete
2016-06-21 18:26:09,733 zopkio.test_runner [INFO] test_samza_job----skipped
2016-06-21 18:26:09,733 zopkio.test_runner [INFO] ['NoneType\n']
2016-06-21 18:26:09,733 zopkio.test_runner [INFO] test_container_performance----skipped
2016-06-21 18:26:09,733 zopkio.test_runner [INFO] ['NoneType\n']
2016-06-21 18:26:09,733 zopkio.test_runner [INFO] test_kafka_read_write_performance----skipped
2016-06-21 18:26:09,733 zopkio.test_runner [INFO] ['NoneType\n']
2016-06-21 18:26:10,038 zopkio [INFO] Exiting zopkio

I'm happy to post more verbose output in both failure modes. I'm curious if anyone has this problem, or I'm missing something simple.

I also had problems on my linux vm, but suspect that be more due to state of python in my Arch install.