[MESOS-5888] SlaveAuthorizerTest/ViewFlags is flaky

• Type: Bug
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Fix Version/s: 1.1.0
• Component/s: test
• Labels: flaky-test mesosphere
• Epic Link: flaky
• Assignee: Neil Conway
• Reporter: Neil Conway
• Shepherd: Vinod Kone
• Created: 22/Jul/16 12:15
• Updated: 26/Apr/17 17:00
• Resolved: 27/Jul/16 23:59

Description

Observed on internal CI:

[09:52:45] :	 [Step 10/10] [ RUN      ] SlaveAuthorizerTest/1.ViewFlags
[09:52:45]W:	 [Step 10/10] I0722 09:52:45.797574 22980 containerizer.cpp:196] Using isolation: posix/cpu,posix/mem,filesystem/posix,network/cni
[09:52:45]W:	 [Step 10/10] I0722 09:52:45.800644 22980 linux_launcher.cpp:101] Using /sys/fs/cgroup/freezer as the freezer hierarchy for the Linux launcher
[09:52:45]W:	 [Step 10/10] I0722 09:52:45.801910 22996 slave.cpp:198] Agent started on 338)@172.30.2.80:35421
[09:52:45]W:	 [Step 10/10] I0722 09:52:45.801923 22996 slave.cpp:199] Flags at startup: --acls="" --appc_simple_discovery_uri_prefix="http://" --appc_store_dir="/tmp/mesos/store/appc" --authenticate_http_readonly="true" --authenticate_http_readwrite="true" --authenticatee="crammd5" --authentication_backoff_factor="1secs" --authorizer="local" --cgroups_cpu_enable_pids_and_tids_count="false" --cgroups_enable_cfs="false" --cgroups_hierarchy="/sys/fs/cgroup" --cgroups_limit_swap="false" --cgroups_root="mesos" --container_disk_watch_interval="15secs" --containerizers="mesos" --credential="/mnt/teamcity/temp/buildTmp/SlaveAuthorizerTest_1_ViewFlags_ISUu44/credential" --default_role="*" --disk_watch_interval="1mins" --docker="docker" --docker_kill_orphans="true" --docker_registry="https://registry-1.docker.io" --docker_remove_delay="6hrs" --docker_socket="/var/run/docker.sock" --docker_stop_timeout="0ns" --docker_store_dir="/tmp/mesos/store/docker" --docker_volume_checkpoint_dir="/var/run/mesos/isolators/docker/volume" --enforce_container_disk_quota="false" --executor_registration_timeout="1mins" --executor_shutdown_grace_period="5secs" --fetcher_cache_dir="/mnt/teamcity/temp/buildTmp/SlaveAuthorizerTest_1_ViewFlags_ISUu44/fetch" --fetcher_cache_size="2GB" --frameworks_home="" --gc_delay="1weeks" --gc_disk_headroom="0.1" --hadoop_home="" --help="true" --hostname_lookup="true" --http_authenticators="basic" --http_command_executor="false" --http_credentials="/mnt/teamcity/temp/buildTmp/SlaveAuthorizerTest_1_ViewFlags_ISUu44/http_credentials" --image_provisioner_backend="copy" --initialize_driver_logging="true" --isolation="posix/cpu,posix/mem" --launcher_dir="/mnt/teamcity/work/4240ba9ddd0997c3/build/src" --logbufsecs="0" --logging_level="INFO" --oversubscribed_resources_interval="15secs" --perf_duration="10secs" --perf_interval="1mins" --qos_correction_interval_min="0ns" --quiet="false" --recover="reconnect" --recovery_timeout="15mins" --registration_backoff_factor="10ms" --resources="cpus:2;gpus:0;mem:1024;disk:1024;ports:[31000-32000]" --revocable_cpu_low_priority="true" --sandbox_directory="/mnt/mesos/sandbox" --strict="true" --switch_user="true" --systemd_enable_support="true" --systemd_runtime_directory="/run/systemd/system" --version="false" --work_dir="/mnt/teamcity/temp/buildTmp/SlaveAuthorizerTest_1_ViewFlags_ISUu44"
[09:52:45]W:	 [Step 10/10] I0722 09:52:45.802104 22996 credentials.hpp:86] Loading credential for authentication from '/mnt/teamcity/temp/buildTmp/SlaveAuthorizerTest_1_ViewFlags_ISUu44/credential'
[09:52:45]W:	 [Step 10/10] I0722 09:52:45.802160 22996 slave.cpp:336] Agent using credential for: test-principal
[09:52:45]W:	 [Step 10/10] I0722 09:52:45.802170 22996 credentials.hpp:37] Loading credentials for authentication from '/mnt/teamcity/temp/buildTmp/SlaveAuthorizerTest_1_ViewFlags_ISUu44/http_credentials'
[09:52:45]W:	 [Step 10/10] I0722 09:52:45.802223 22996 http.cpp:883] Using default 'basic' HTTP authenticator for realm 'mesos-agent-readonly'
[09:52:45]W:	 [Step 10/10] I0722 09:52:45.802264 22996 http.cpp:883] Using default 'basic' HTTP authenticator for realm 'mesos-agent-readwrite'
[09:52:45]W:	 [Step 10/10] I0722 09:52:45.802472 22996 slave.cpp:519] Agent resources: cpus(*):2; mem(*):1024; disk(*):1024; ports(*):[31000-32000]
[09:52:45]W:	 [Step 10/10] I0722 09:52:45.802495 22996 slave.cpp:527] Agent attributes: [  ]
[09:52:45]W:	 [Step 10/10] I0722 09:52:45.802500 22996 slave.cpp:532] Agent hostname: ip-172-30-2-80.mesosphere.io
[09:52:45]W:	 [Step 10/10] I0722 09:52:45.802726 22996 process.cpp:3341] Handling HTTP event for process 'slave(338)' with path: '/slave(338)/flags'
[09:52:45]W:	 [Step 10/10] I0722 09:52:45.802738 22999 state.cpp:57] Recovering state from '/mnt/teamcity/temp/buildTmp/SlaveAuthorizerTest_1_ViewFlags_ISUu44/meta'
[09:52:45]W:	 [Step 10/10] I0722 09:52:45.802819 23000 status_update_manager.cpp:200] Recovering status update manager
[09:52:45]W:	 [Step 10/10] I0722 09:52:45.802901 22994 containerizer.cpp:522] Recovering containerizer
[09:52:45]W:	 [Step 10/10] I0722 09:52:45.802971 22996 http.cpp:270] HTTP GET for /slave(338)/flags from 172.30.2.80:46288
[09:52:45]W:	 [Step 10/10] I0722 09:52:45.804105 22995 process.cpp:3341] Handling HTTP event for process 'slave(338)' with path: '/slave(338)/state'
[09:52:45]W:	 [Step 10/10] I0722 09:52:45.804378 23001 http.cpp:270] HTTP GET for /slave(338)/state from 172.30.2.80:46290
[09:52:45] :	 [Step 10/10] ../../src/tests/slave_authorization_tests.cpp:316: Failure
[09:52:45] :	 [Step 10/10] Value of: (response).get().status
[09:52:45] :	 [Step 10/10]   Actual: "503 Service Unavailable"
[09:52:45] :	 [Step 10/10] Expected: OK().status
[09:52:45] :	 [Step 10/10] Which is: "200 OK"
[09:52:45] :	 [Step 10/10] Agent has not finished recovery
[09:52:45] :	 [Step 10/10] ../../src/tests/slave_authorization_tests.cpp:320: Failure
[09:52:45]W:	 [Step 10/10] I0722 09:52:45.805032 23001 slave.cpp:767] Agent terminating
[09:52:45] :	 [Step 10/10] parse: syntax error at line 1 near: Agent has not finished recovery
[09:52:45] :	 [Step 10/10] [  FAILED  ] SlaveAuthorizerTest/1.ViewFlags, where TypeParam = mesos::internal::tests::Module (10 ms)

Issue Links

• is duplicated by: MESOS-5802 SlaveAuthorizerTest/0.ViewFlags is flaky.