Details
Description
Also tested on Linux, where this test succeeds. GLOG_v=1 output of this test on macOS:
I0304 09:33:08.532002 155725824 master.cpp:414] Master 8be09e79-ff3b-49bf-86e9-cde00fbdcdaa (172.18.8.49) started on 172.18.8.49:56584 I0304 09:33:08.532045 155725824 master.cpp:417] Flags at startup: --acls="" --agent_ping_timeout="15secs" --agent_reregister_timeout="10mins" --allocation_interval="1secs" --allocator="hierarchical" --authenticate_agents="true" --authenticate_frameworks="true" --authenticate_http_frameworks="true" --authenticate_http_readonly="true" --authenticate_http_readwrite="true" --authentication_v0_timeout="15secs" --authenticators="crammd5" --authorizers="local" --credentials="/private/var/folders/0b/srgwj7vd2037pygpz1fpyqgm0000gn/T/uCWwLH/credentials" --filter_gpu_resources="true" --framework_sorter="drf" --help="false" --hostname_lookup="true" --http_authenticators="basic" --http_framework_authenticators="basic" --initialize_driver_logging="true" --log_auto_initialize="true" --logbufsecs="0" --logging_level="INFO" --max_agent_ping_timeouts="5" --max_completed_frameworks="50" --max_completed_tasks_per_framework="1000" --max_operator_event_stream_subscribers="1000" --max_unreachable_tasks_per_framework="1000" --memory_profiling="false" --min_allocatable_resources="cpus:0.01|mem:32" --port="5050" --publish_per_framework_metrics="true" --quiet="false" --recovery_agent_removal_limit="100%" --registry="in_memory" --registry_fetch_timeout="1mins" --registry_gc_interval="15mins" --registry_max_agent_age="2weeks" --registry_max_agent_count="102400" --registry_store_timeout="100secs" --registry_strict="false" --require_agent_domain="false" --role_sorter="drf" --root_submissions="true" --version="false" --webui_dir="/usr/local/share/mesos/webui" --work_dir="/private/var/folders/0b/srgwj7vd2037pygpz1fpyqgm0000gn/T/uCWwLH/master" --zk_session_timeout="10secs" I0304 09:33:08.532878 155725824 master.cpp:466] Master only allowing authenticated frameworks to register I0304 09:33:08.532889 155725824 master.cpp:472] Master only allowing authenticated agents to register I0304 09:33:08.532896 155725824 master.cpp:478] Master only allowing authenticated HTTP frameworks to register I0304 09:33:08.532903 155725824 credentials.hpp:37] Loading credentials for authentication from '/private/var/folders/0b/srgwj7vd2037pygpz1fpyqgm0000gn/T/uCWwLH/credentials' I0304 09:33:08.533071 155725824 master.cpp:522] Using default 'crammd5' authenticator I0304 09:33:08.533094 155725824 authenticator.cpp:520] Initializing server SASL I0304 09:33:08.551656 155725824 auxprop.cpp:73] Initialized in-memory auxiliary property plugin I0304 09:33:08.551702 155725824 http.cpp:965] Creating default 'basic' HTTP authenticator for realm 'mesos-master-readonly' I0304 09:33:08.551745 155725824 http.cpp:965] Creating default 'basic' HTTP authenticator for realm 'mesos-master-readwrite' I0304 09:33:08.551766 155725824 http.cpp:965] Creating default 'basic' HTTP authenticator for realm 'mesos-master-scheduler' I0304 09:33:08.551785 155725824 master.cpp:603] Authorization enabled I0304 09:33:08.551923 154116096 whitelist_watcher.cpp:77] No whitelist given I0304 09:33:08.551964 151969792 hierarchical.cpp:208] Initialized hierarchical allocator process I0304 09:33:08.553930 151969792 master.cpp:2103] Elected as the leading master! I0304 09:33:08.553966 151969792 master.cpp:1638] Recovering from registrar I0304 09:33:08.554018 153579520 registrar.cpp:339] Recovering registrar I0304 09:33:08.556378 155725824 registrar.cpp:383] Successfully fetched the registry (0B) in 2.342912ms I0304 09:33:08.556512 155725824 registrar.cpp:487] Applied 1 operations in 38854ns; attempting to update the registry I0304 09:33:08.558737 153579520 registrar.cpp:544] Successfully updated the registry in 2.206976ms I0304 09:33:08.558776 153579520 registrar.cpp:416] Successfully recovered registrar I0304 09:33:08.558888 153042944 master.cpp:1752] Recovered 0 agents from the registry (136B); allowing 10mins for agents to reregister I0304 09:33:08.558929 155725824 hierarchical.cpp:248] Skipping recovery of hierarchical allocator: nothing to recover I0304 09:33:08.561846 162198976 sched.cpp:232] Version: 1.8.0 I0304 09:33:08.562060 155189248 sched.cpp:336] New master detected at master@172.18.8.49:56584 I0304 09:33:08.562099 155189248 sched.cpp:401] Authenticating with master master@172.18.8.49:56584 I0304 09:33:08.562110 155189248 sched.cpp:408] Using default CRAM-MD5 authenticatee I0304 09:33:08.562196 154116096 authenticatee.cpp:97] Initializing client SASL I0304 09:33:08.564996 154116096 authenticatee.cpp:121] Creating new client SASL connection I0304 09:33:08.565150 154652672 master.cpp:10017] Authenticating scheduler-3af4a67f-d839-4c35-9c97-20c584107ab1@172.18.8.49:56584 I0304 09:33:08.565213 152506368 authenticator.cpp:414] Starting authentication session for crammd5-authenticatee(1)@172.18.8.49:56584 I0304 09:33:08.565263 151969792 authenticator.cpp:98] Creating new server SASL connection I0304 09:33:08.565356 153579520 authenticatee.cpp:213] Received SASL authentication mechanisms: CRAM-MD5 I0304 09:33:08.565383 153579520 authenticatee.cpp:239] Attempting to authenticate with mechanism 'CRAM-MD5' I0304 09:33:08.565415 153042944 authenticator.cpp:204] Received SASL authentication start I0304 09:33:08.565488 153042944 authenticator.cpp:326] Authentication requires more steps I0304 09:33:08.565538 155189248 authenticatee.cpp:259] Received SASL authentication step I0304 09:33:08.565593 155725824 authenticator.cpp:232] Received SASL authentication step I0304 09:33:08.565610 155725824 auxprop.cpp:109] Request to lookup properties for user: 'test-principal' realm: 'nfnt.local' server FQDN: 'nfnt.local' SASL_AUXPROP_VERIFY_AGAINST_HASH: false SASL_AUXPROP_OVERRIDE: false SASL_AUXPROP_AUTHZID: false I0304 09:33:08.565620 155725824 auxprop.cpp:181] Looking up auxiliary property '*userPassword' I0304 09:33:08.565639 155725824 auxprop.cpp:181] Looking up auxiliary property '*cmusaslsecretCRAM-MD5' I0304 09:33:08.565644 155725824 auxprop.cpp:109] Request to lookup properties for user: 'test-principal' realm: 'nfnt.local' server FQDN: 'nfnt.local' SASL_AUXPROP_VERIFY_AGAINST_HASH: false SASL_AUXPROP_OVERRIDE: false SASL_AUXPROP_AUTHZID: true I0304 09:33:08.565650 155725824 auxprop.cpp:131] Skipping auxiliary property '*userPassword' since SASL_AUXPROP_AUTHZID == true I0304 09:33:08.565654 155725824 auxprop.cpp:131] Skipping auxiliary property '*cmusaslsecretCRAM-MD5' since SASL_AUXPROP_AUTHZID == true I0304 09:33:08.565673 155725824 authenticator.cpp:318] Authentication success I0304 09:33:08.565719 154116096 authenticatee.cpp:299] Authentication success I0304 09:33:08.565749 154652672 master.cpp:10049] Successfully authenticated principal 'test-principal' at scheduler-3af4a67f-d839-4c35-9c97-20c584107ab1@172.18.8.49:56584 I0304 09:33:08.565798 152506368 authenticator.cpp:432] Authentication session cleanup for crammd5-authenticatee(1)@172.18.8.49:56584 I0304 09:33:08.565877 151969792 sched.cpp:513] Successfully authenticated with master master@172.18.8.49:56584 I0304 09:33:08.565889 151969792 sched.cpp:817] Sending SUBSCRIBE call to master@172.18.8.49:56584 I0304 09:33:08.565968 151969792 sched.cpp:850] Will retry registration in 788.765853ms if necessary I0304 09:33:08.566045 153579520 master.cpp:2879] Received SUBSCRIBE call for framework 'default' at scheduler-3af4a67f-d839-4c35-9c97-20c584107ab1@172.18.8.49:56584 I0304 09:33:08.566072 153579520 master.cpp:2175] Authorizing framework principal 'test-principal' to receive offers for roles '{ * }' I0304 09:33:08.566187 155189248 master.cpp:2960] Subscribing framework default with checkpointing disabled and capabilities [ MULTI_ROLE, RESERVATION_REFINEMENT ] I0304 09:33:08.566610 155189248 master.cpp:10247] Adding framework 8be09e79-ff3b-49bf-86e9-cde00fbdcdaa-0000 (default) at scheduler-3af4a67f-d839-4c35-9c97-20c584107ab1@172.18.8.49:56584 with roles { } suppressed I0304 09:33:08.566761 153042944 hierarchical.cpp:337] Added framework 8be09e79-ff3b-49bf-86e9-cde00fbdcdaa-0000 I0304 09:33:08.566838 153042944 hierarchical.cpp:1592] Performed allocation for 0 agents in 35704ns I0304 09:33:08.566835 154116096 sched.cpp:744] Framework registered with 8be09e79-ff3b-49bf-86e9-cde00fbdcdaa-0000 I0304 09:33:08.566920 154116096 sched.cpp:758] Scheduler::registered took 29603ns W0304 09:33:08.568495 162198976 process.cpp:2829] Attempted to spawn already running process files@172.18.8.49:56584 I0304 09:33:08.568799 162198976 containerizer.cpp:314] Using isolation { environment_secret, filesystem/posix, posix/mem, posix/cpu } I0304 09:33:08.568997 162198976 provisioner.cpp:298] Using default backend 'copy' I0304 09:33:08.569422 162198976 cluster.cpp:510] Creating default 'local' authorizer I0304 09:33:08.569931 154652672 slave.cpp:265] Mesos agent started on (1)@172.18.8.49:56584 I0304 09:33:08.569950 154652672 slave.cpp:266] Flags at startup: --acls="" --appc_simple_discovery_uri_prefix="http://" --appc_store_dir="/private/var/folders/0b/srgwj7vd2037pygpz1fpyqgm0000gn/T/uCWwLH/9aURxR/store/appc" --authenticate_http_executors="true" --authenticate_http_readonly="true" --authenticate_http_readwrite="true" --authenticatee="crammd5" --authentication_backoff_factor="1secs" --authentication_timeout_max="1mins" --authentication_timeout_min="5secs" --authorizer="local" --container_disk_watch_interval="15secs" --containerizers="mesos" --credential="/private/var/folders/0b/srgwj7vd2037pygpz1fpyqgm0000gn/T/uCWwLH/9aURxR/credential" --default_role="*" --disk_watch_interval="1mins" --docker="docker" --docker_kill_orphans="true" --docker_registry="https://registry-1.docker.io" --docker_remove_delay="6hrs" --docker_socket="/var/run/docker.sock" --docker_stop_timeout="0ns" --docker_store_dir="/private/var/folders/0b/srgwj7vd2037pygpz1fpyqgm0000gn/T/uCWwLH/9aURxR/store/docker" --docker_volume_checkpoint_dir="/var/run/mesos/isolators/docker/volume" --enforce_container_disk_quota="false" --executor_registration_timeout="1mins" --executor_reregistration_timeout="2secs" --executor_shutdown_grace_period="5secs" --fetcher_cache_dir="/private/var/folders/0b/srgwj7vd2037pygpz1fpyqgm0000gn/T/uCWwLH/9aURxR/fetch" --fetcher_cache_size="2GB" --fetcher_stall_timeout="1mins" --frameworks_home="/private/var/folders/0b/srgwj7vd2037pygpz1fpyqgm0000gn/T/uCWwLH/9aURxR/frameworks" --gc_delay="1weeks" --gc_disk_headroom="0.1" --gc_non_executor_container_sandboxes="false" --help="false" --hostname_lookup="true" --http_command_executor="false" --http_credentials="/private/var/folders/0b/srgwj7vd2037pygpz1fpyqgm0000gn/T/uCWwLH/9aURxR/http_credentials" --http_heartbeat_interval="30secs" --initialize_driver_logging="true" --isolation="posix/cpu,posix/mem" --jwt_secret_key="/private/var/folders/0b/srgwj7vd2037pygpz1fpyqgm0000gn/T/uCWwLH/9aURxR/jwt_secret_key" --launcher="posix" --launcher_dir="/Users/jan/Documents/mesos/build/src" --logbufsecs="0" --logging_level="INFO" --max_completed_executors_per_framework="150" --memory_profiling="false" --network_cni_metrics="true" --network_cni_root_dir_persist="false" --oversubscribed_resources_interval="15secs" --port="5051" --qos_correction_interval_min="0ns" --quiet="false" --reconfiguration_policy="equal" --recover="reconnect" --recovery_timeout="15mins" --registration_backoff_factor="10ms" --resources="cpus:2;gpus:0;mem:1024;disk:1024;ports:[31000-32000]" --runtime_dir="/var/folders/0b/srgwj7vd2037pygpz1fpyqgm0000gn/T/MasterLoadTest_SimultaneousBatchedRequests_pGAmOZ" --sandbox_directory="/mnt/mesos/sandbox" --strict="true" --switch_user="true" --version="false" --work_dir="/var/folders/0b/srgwj7vd2037pygpz1fpyqgm0000gn/T/MasterLoadTest_SimultaneousBatchedRequests_HPSwD1" --zk_session_timeout="10secs" I0304 09:33:08.570118 154652672 credentials.hpp:86] Loading credential for authentication from '/private/var/folders/0b/srgwj7vd2037pygpz1fpyqgm0000gn/T/uCWwLH/9aURxR/credential' I0304 09:33:08.570180 154652672 slave.cpp:298] Agent using credential for: test-principal I0304 09:33:08.570189 154652672 credentials.hpp:37] Loading credentials for authentication from '/private/var/folders/0b/srgwj7vd2037pygpz1fpyqgm0000gn/T/uCWwLH/9aURxR/http_credentials' I0304 09:33:08.570266 154652672 http.cpp:965] Creating default 'basic' HTTP authenticator for realm 'mesos-agent-executor' I0304 09:33:08.570284 154652672 http.cpp:986] Creating default 'jwt' HTTP authenticator for realm 'mesos-agent-executor' I0304 09:33:08.570320 154652672 http.cpp:965] Creating default 'basic' HTTP authenticator for realm 'mesos-agent-readonly' I0304 09:33:08.570339 154652672 http.cpp:986] Creating default 'jwt' HTTP authenticator for realm 'mesos-agent-readonly' I0304 09:33:08.570369 154652672 http.cpp:965] Creating default 'basic' HTTP authenticator for realm 'mesos-agent-readwrite' I0304 09:33:08.570381 154652672 http.cpp:986] Creating default 'jwt' HTTP authenticator for realm 'mesos-agent-readwrite' I0304 09:33:08.570448 154652672 disk_profile_adaptor.cpp:80] Creating default disk profile adaptor module I0304 09:33:08.571094 154652672 slave.cpp:613] Agent resources: [{"name":"cpus","scalar":{"value":2.0},"type":"SCALAR"},{"name":"mem","scalar":{"value":1024.0},"type":"SCALAR"},{"name":"disk","scalar":{"value":1024.0},"type":"SCALAR"},{"name":"ports","ranges":{"range":[{"begin":31000,"end":32000}]},"type":"RANGES"}] I0304 09:33:08.571168 154652672 slave.cpp:621] Agent attributes: [ ] I0304 09:33:08.571175 154652672 slave.cpp:630] Agent hostname: 172.18.8.49 I0304 09:33:08.571245 155189248 status_update_manager_process.hpp:379] Pausing operation status update manager I0304 09:33:08.571254 151969792 task_status_update_manager.cpp:181] Pausing sending task status updates I0304 09:33:08.571534 153042944 state.cpp:67] Recovering state from '/var/folders/0b/srgwj7vd2037pygpz1fpyqgm0000gn/T/MasterLoadTest_SimultaneousBatchedRequests_HPSwD1/meta' I0304 09:33:08.571595 152506368 slave.cpp:7150] Finished recovering checkpointed state from '/var/folders/0b/srgwj7vd2037pygpz1fpyqgm0000gn/T/MasterLoadTest_SimultaneousBatchedRequests_HPSwD1/meta', beginning agent recovery I0304 09:33:08.571663 151969792 task_status_update_manager.cpp:207] Recovering task status update manager I0304 09:33:08.571744 154116096 containerizer.cpp:797] Recovering Mesos containers I0304 09:33:08.571820 154116096 containerizer.cpp:1123] Recovering isolators I0304 09:33:08.571966 154652672 containerizer.cpp:1162] Recovering provisioner I0304 09:33:08.572166 154116096 provisioner.cpp:494] Provisioner recovery complete I0304 09:33:08.572340 153579520 composing.cpp:339] Finished recovering all containerizers I0304 09:33:08.572407 151969792 slave.cpp:7607] Recovering executors I0304 09:33:08.572433 151969792 slave.cpp:7760] Finished recovery I0304 09:33:08.572990 153042944 task_status_update_manager.cpp:181] Pausing sending task status updates I0304 09:33:08.572988 151969792 slave.cpp:1258] New master detected at master@172.18.8.49:56584 I0304 09:33:08.572998 154116096 status_update_manager_process.hpp:379] Pausing operation status update manager I0304 09:33:08.573030 151969792 slave.cpp:1323] Detecting new master I0304 09:33:08.582975 155725824 slave.cpp:1350] Authenticating with master master@172.18.8.49:56584 I0304 09:33:08.583046 155725824 slave.cpp:1359] Using default CRAM-MD5 authenticatee I0304 09:33:08.583168 155189248 authenticatee.cpp:121] Creating new client SASL connection I0304 09:33:08.583304 154652672 master.cpp:10017] Authenticating slave(1)@172.18.8.49:56584 I0304 09:33:08.583359 153042944 authenticator.cpp:414] Starting authentication session for crammd5-authenticatee(2)@172.18.8.49:56584 I0304 09:33:08.583444 154116096 authenticator.cpp:98] Creating new server SASL connection I0304 09:33:08.583549 151969792 authenticatee.cpp:213] Received SASL authentication mechanisms: CRAM-MD5 I0304 09:33:08.583575 151969792 authenticatee.cpp:239] Attempting to authenticate with mechanism 'CRAM-MD5' I0304 09:33:08.583644 153579520 authenticator.cpp:204] Received SASL authentication start I0304 09:33:08.583765 153579520 authenticator.cpp:326] Authentication requires more steps I0304 09:33:08.583822 155725824 authenticatee.cpp:259] Received SASL authentication step I0304 09:33:08.583878 152506368 authenticator.cpp:232] Received SASL authentication step I0304 09:33:08.583897 152506368 auxprop.cpp:109] Request to lookup properties for user: 'test-principal' realm: 'nfnt.local' server FQDN: 'nfnt.local' SASL_AUXPROP_VERIFY_AGAINST_HASH: false SASL_AUXPROP_OVERRIDE: false SASL_AUXPROP_AUTHZID: false I0304 09:33:08.583911 152506368 auxprop.cpp:181] Looking up auxiliary property '*userPassword' I0304 09:33:08.583926 152506368 auxprop.cpp:181] Looking up auxiliary property '*cmusaslsecretCRAM-MD5' I0304 09:33:08.583940 152506368 auxprop.cpp:109] Request to lookup properties for user: 'test-principal' realm: 'nfnt.local' server FQDN: 'nfnt.local' SASL_AUXPROP_VERIFY_AGAINST_HASH: false SASL_AUXPROP_OVERRIDE: false SASL_AUXPROP_AUTHZID: true I0304 09:33:08.583951 152506368 auxprop.cpp:131] Skipping auxiliary property '*userPassword' since SASL_AUXPROP_AUTHZID == true I0304 09:33:08.583962 152506368 auxprop.cpp:131] Skipping auxiliary property '*cmusaslsecretCRAM-MD5' since SASL_AUXPROP_AUTHZID == true I0304 09:33:08.583977 152506368 authenticator.cpp:318] Authentication success I0304 09:33:08.584046 155189248 authenticatee.cpp:299] Authentication success I0304 09:33:08.584084 154652672 master.cpp:10049] Successfully authenticated principal 'test-principal' at slave(1)@172.18.8.49:56584 I0304 09:33:08.584144 153042944 authenticator.cpp:432] Authentication session cleanup for crammd5-authenticatee(2)@172.18.8.49:56584 I0304 09:33:08.584230 154116096 slave.cpp:1450] Successfully authenticated with master master@172.18.8.49:56584 I0304 09:33:08.584408 154116096 slave.cpp:1897] Will retry registration in 18.232947ms if necessary I0304 09:33:08.584504 154652672 master.cpp:6744] Received register agent message from slave(1)@172.18.8.49:56584 (172.18.8.49) I0304 09:33:08.584586 154652672 master.cpp:3976] Authorizing agent providing resources 'cpus:2; mem:1024; disk:1024; ports:[31000-32000]' with principal 'test-principal' I0304 09:33:08.584825 151969792 master.cpp:6811] Authorized registration of agent at slave(1)@172.18.8.49:56584 (172.18.8.49) I0304 09:33:08.584914 151969792 master.cpp:6926] Registering agent at slave(1)@172.18.8.49:56584 (172.18.8.49) with id 8be09e79-ff3b-49bf-86e9-cde00fbdcdaa-S0 I0304 09:33:08.585204 153042944 registrar.cpp:487] Applied 1 operations in 172209ns; attempting to update the registry I0304 09:33:08.588699 154652672 registrar.cpp:544] Successfully updated the registry in 3.442176ms I0304 09:33:08.588752 155725824 master.cpp:6974] Admitted agent 8be09e79-ff3b-49bf-86e9-cde00fbdcdaa-S0 at slave(1)@172.18.8.49:56584 (172.18.8.49) I0304 09:33:08.588891 155725824 master.cpp:7019] Registered agent 8be09e79-ff3b-49bf-86e9-cde00fbdcdaa-S0 at slave(1)@172.18.8.49:56584 (172.18.8.49) with cpus:2; mem:1024; disk:1024; ports:[31000-32000] I0304 09:33:08.588960 153042944 hierarchical.cpp:638] Added agent 8be09e79-ff3b-49bf-86e9-cde00fbdcdaa-S0 (172.18.8.49) with cpus:2; mem:1024; disk:1024; ports:[31000-32000] (allocated: {}) I0304 09:33:08.589026 154116096 slave.cpp:1483] Registered with master master@172.18.8.49:56584; given agent ID 8be09e79-ff3b-49bf-86e9-cde00fbdcdaa-S0 I0304 09:33:08.589082 152506368 task_status_update_manager.cpp:188] Resuming sending task status updates I0304 09:33:08.589284 153042944 hierarchical.cpp:1592] Performed allocation for 1 agents in 233200ns I0304 09:33:08.589432 154116096 slave.cpp:1518] Checkpointing SlaveInfo to '/var/folders/0b/srgwj7vd2037pygpz1fpyqgm0000gn/T/MasterLoadTest_SimultaneousBatchedRequests_HPSwD1/meta/slaves/8be09e79-ff3b-49bf-86e9-cde00fbdcdaa-S0/slave.info' I0304 09:33:08.589438 155725824 master.cpp:9832] Sending offers [ 8be09e79-ff3b-49bf-86e9-cde00fbdcdaa-O0 ] to framework 8be09e79-ff3b-49bf-86e9-cde00fbdcdaa-0000 (default) at scheduler-3af4a67f-d839-4c35-9c97-20c584107ab1@172.18.8.49:56584 I0304 09:33:08.589450 153579520 status_update_manager_process.hpp:385] Resuming operation status update manager GMOCK WARNING: Uninteresting mock function call - returning directly. Function call: resourceOffers(0x7fa835f20060, @0x7000093ffc90 { 160-byte object <D8-3F 16-0A 01-00 00-00 00-00 00-00 00-00 00-00 5F-00 00-00 00-00 00-00 00-00 00-00 00-00 00-00 04-00 00-00 04-00 00-00 90-11 80-36 A8-7F 00-00 00-00 00-00 00-00 00-00 00-00 00-00 00-00 00-00 ... 40-0C 80-36 A8-7F 00-00 60-0C 80-36 A8-7F 00-00 D0-0C 80-36 A8-7F 00-00 40-0D 80-36 A8-7F 00-00 00-12 80-36 A8-7F 00-00 00-00 00-00 00-00 00-00 00-13 80-36 A8-7F 00-00 00-00 00-00 00-00 00-00> }) NOTE: You can safely ignore the above warning unless this call should not happen. Do not suppress it by blindly adding an EXPECT_CALL() if you don't mean to enforce the call. See https://github.com/google/googletest/blob/master/googlemock/docs/CookBook.md#knowing-when-to-expect for details. I0304 09:33:08.589700 155189248 sched.cpp:914] Scheduler::resourceOffers took 120402ns I0304 09:33:08.590303 154652672 process.cpp:3588] Handling HTTP event for process 'master' with path: '/master//state' I0304 09:33:08.590291 154116096 slave.cpp:1567] Forwarding agent update {"operations":{},"resource_providers":{},"resource_version_uuid":{"value":"6UdB5nN7Q5mO2mldTj04qw=="},"slave_id":{"value":"8be09e79-ff3b-49bf-86e9-cde00fbdcdaa-S0"},"update_oversubscribed_resources":false} I0304 09:33:08.590579 154652672 master.cpp:8097] Ignoring update on agent 8be09e79-ff3b-49bf-86e9-cde00fbdcdaa-S0 at slave(1)@172.18.8.49:56584 (172.18.8.49) as it reports no changes I0304 09:33:08.590616 154652672 process.cpp:3588] Handling HTTP event for process 'master' with path: '/master//state' I0304 09:33:08.590795 154652672 process.cpp:3588] Handling HTTP event for process 'master' with path: '/master//state' I0304 09:33:08.590939 154652672 process.cpp:3588] Handling HTTP event for process 'master' with path: '/master//state' I0304 09:33:08.591362 154652672 http.cpp:1105] HTTP GET for /master//state from 172.18.8.49:56587 I0304 09:33:08.591537 154652672 http.cpp:1105] HTTP GET for /master//state from 172.18.8.49:56585 I0304 09:33:08.591660 154652672 http.cpp:1105] HTTP GET for /master//state from 172.18.8.49:56586 I0304 09:33:08.591835 154652672 process.cpp:3588] Handling HTTP event for process 'master' with path: '/master//state' I0304 09:33:08.591910 154652672 http.cpp:1105] HTTP GET for /master//state from 172.18.8.49:56588 I0304 09:33:08.592375 155725824 process.cpp:3588] Handling HTTP event for process 'master' with path: '/master//state' I0304 09:33:08.592515 155725824 http.cpp:1105] HTTP GET for /master//state from 172.18.8.49:56589 I0304 09:33:08.592854 154652672 process.cpp:3588] Handling HTTP event for process 'master' with path: '/master//state' I0304 09:33:08.593078 151969792 http.cpp:1105] HTTP GET for /master//state from 172.18.8.49:56590 I0304 09:33:08.593305 154116096 http.cpp:1105] HTTP GET for /master//state from 172.18.8.49:56591 I0304 09:33:08.593653 154652672 process.cpp:3588] Handling HTTP event for process 'master' with path: '/master//state' I0304 09:33:08.593994 151969792 http.cpp:1105] HTTP GET for /master//state from 172.18.8.49:56592 I0304 09:33:08.594394 154652672 process.cpp:3588] Handling HTTP event for process 'master' with path: '/master//state' I0304 09:33:08.594696 155725824 http.cpp:1105] HTTP GET for /master//state from 172.18.8.49:56593 I0304 09:33:08.594790 155725824 process.cpp:3588] Handling HTTP event for process 'master' with path: '/master//state' I0304 09:33:08.595019 154116096 process.cpp:3588] Handling HTTP event for process 'master' with path: '/master//state' I0304 09:33:08.595108 154116096 http.cpp:1105] HTTP GET for /master//state from 172.18.8.49:56594 I0304 09:33:08.595299 154116096 process.cpp:3588] Handling HTTP event for process 'master' with path: '/master//state' I0304 09:33:08.595571 154116096 process.cpp:3588] Handling HTTP event for process 'master' with path: '/master//state' I0304 09:33:08.595687 154116096 http.cpp:1105] HTTP GET for /master//state from 172.18.8.49:56595 I0304 09:33:08.595847 154116096 process.cpp:3588] Handling HTTP event for process 'master' with path: '/master//state' I0304 09:33:08.596002 154116096 http.cpp:1105] HTTP GET for /master//state from 172.18.8.49:56596 I0304 09:33:08.596168 154116096 process.cpp:3588] Handling HTTP event for process 'master' with path: '/master//state' I0304 09:33:08.596256 154116096 http.cpp:1105] HTTP GET for /master//state from 172.18.8.49:56597 I0304 09:33:08.596415 154116096 process.cpp:3588] Handling HTTP event for process 'master' with path: '/master//state' I0304 09:33:08.596702 154116096 process.cpp:3588] Handling HTTP event for process 'master' with path: '/master//state' I0304 09:33:08.596807 154116096 http.cpp:1105] HTTP GET for /master//state from 172.18.8.49:56598 I0304 09:33:08.597060 154116096 process.cpp:3588] Handling HTTP event for process 'master' with path: '/master//state' I0304 09:33:08.597148 154116096 http.cpp:1105] HTTP GET for /master//state from 172.18.8.49:56599 I0304 09:33:08.597292 154116096 http.cpp:1105] HTTP GET for /master//state from 172.18.8.49:56600 I0304 09:33:08.597504 154116096 process.cpp:3588] Handling HTTP event for process 'master' with path: '/master//state' I0304 09:33:08.597703 154116096 process.cpp:3588] Handling HTTP event for process 'master' with path: '/master//state' I0304 09:33:08.597767 154116096 http.cpp:1105] HTTP GET for /master//state from 172.18.8.49:56601 I0304 09:33:08.597879 154116096 http.cpp:1105] HTTP GET for /master//state from 172.18.8.49:56602 I0304 09:33:08.598171 154116096 process.cpp:3588] Handling HTTP event for process 'master' with path: '/master//state' I0304 09:33:08.598326 154116096 http.cpp:1105] HTTP GET for /master//state from 172.18.8.49:56603 I0304 09:33:08.598453 154116096 http.cpp:1105] HTTP GET for /master//state from 172.18.8.49:56604 I0304 09:33:08.598582 154116096 process.cpp:3588] Handling HTTP event for process 'master' with path: '/master//state' I0304 09:33:08.598826 154116096 http.cpp:1105] HTTP GET for /master//state from 172.18.8.49:56605 I0304 09:33:08.598996 154116096 process.cpp:3588] Handling HTTP event for process 'master' with path: '/master//state' I0304 09:33:08.599050 154116096 http.cpp:1105] HTTP GET for /master//state from 172.18.8.49:56606 I0304 09:33:08.599421 152506368 http.cpp:1105] HTTP GET for /master//state from 172.18.8.49:56607 I0304 09:33:08.599522 152506368 process.cpp:3588] Handling HTTP event for process 'master' with path: '/master//state' I0304 09:33:08.599740 153042944 http.cpp:1105] HTTP GET for /master//state from 172.18.8.49:56608 I0304 09:33:08.600147 153042944 process.cpp:3588] Handling HTTP event for process 'master' with path: '/master//state' I0304 09:33:08.600383 154652672 http.cpp:1105] HTTP GET for /master//state from 172.18.8.49:56609 I0304 09:33:08.600750 153579520 process.cpp:3588] Handling HTTP event for process 'master' with path: '/master//state' I0304 09:33:08.601006 153042944 http.cpp:1105] HTTP GET for /master//state from 172.18.8.49:56610 I0304 09:33:08.601270 155725824 process.cpp:3588] Handling HTTP event for process 'master' with path: '/master//state' I0304 09:33:08.601516 151969792 http.cpp:1105] HTTP GET for /master//state from 172.18.8.49:56611 I0304 09:33:08.601781 151969792 process.cpp:3588] Handling HTTP event for process 'master' with path: '/master//state' I0304 09:33:08.601981 153579520 http.cpp:1105] HTTP GET for /master//state from 172.18.8.49:56612 I0304 09:33:08.602298 155725824 process.cpp:3588] Handling HTTP event for process 'master' with path: '/master//state' I0304 09:33:08.602507 154652672 http.cpp:1105] HTTP GET for /master//state from 172.18.8.49:56613 I0304 09:33:08.602847 153042944 process.cpp:3588] Handling HTTP event for process 'master' with path: '/master//state' I0304 09:33:08.603041 155725824 http.cpp:1105] HTTP GET for /master//state from 172.18.8.49:56614 I0304 09:33:08.603472 155189248 process.cpp:3588] Handling HTTP event for process 'master' with path: '/master//state' I0304 09:33:08.603693 155725824 http.cpp:1105] HTTP GET for /master//state from 172.18.8.49:56615 I0304 09:33:08.603796 155725824 process.cpp:3588] Handling HTTP event for process 'master' with path: '/master//state' I0304 09:33:08.603912 155725824 process.cpp:3588] Handling HTTP event for process 'master' with path: '/master//state' I0304 09:33:08.604063 154116096 process.cpp:3588] Handling HTTP event for process 'master' with path: '/master//state' I0304 09:33:08.604144 154116096 http.cpp:1105] HTTP GET for /master//state from 172.18.8.49:56616 I0304 09:33:08.604316 154116096 http.cpp:1105] HTTP GET for /master//state from 172.18.8.49:56617 I0304 09:33:08.604408 154116096 process.cpp:3588] Handling HTTP event for process 'master' with path: '/master//state' I0304 09:33:08.604558 154116096 process.cpp:3588] Handling HTTP event for process 'master' with path: '/master//state' I0304 09:33:08.604635 154116096 http.cpp:1105] HTTP GET for /master//state from 172.18.8.49:56618 I0304 09:33:08.604871 154116096 process.cpp:3588] Handling HTTP event for process 'master' with path: '/master//state' I0304 09:33:08.604929 154116096 http.cpp:1105] HTTP GET for /master//state from 172.18.8.49:56619 I0304 09:33:08.605044 154116096 process.cpp:3588] Handling HTTP event for process 'master' with path: '/master//state' I0304 09:33:08.605130 154116096 process.cpp:3588] Handling HTTP event for process 'master' with path: '/master//state' I0304 09:33:08.605288 154116096 http.cpp:1105] HTTP GET for /master//state from 172.18.8.49:56620 I0304 09:33:08.605425 154116096 process.cpp:3588] Handling HTTP event for process 'master' with path: '/master//state' I0304 09:33:08.605536 154116096 http.cpp:1105] HTTP GET for /master//state from 172.18.8.49:56621 I0304 09:33:08.605674 154116096 process.cpp:3588] Handling HTTP event for process 'master' with path: '/master//state-summary' I0304 09:33:08.605806 154116096 http.cpp:1105] HTTP GET for /master//state from 172.18.8.49:56622 I0304 09:33:08.605931 154116096 process.cpp:3588] Handling HTTP event for process 'master' with path: '/master//state-summary' W0304 09:33:08.606092 156262400 process.cpp:903] Failed to accept socket: Failed to accept: Too many open files I0304 09:33:08.606139 154116096 http.cpp:1105] HTTP GET for /master//state from 172.18.8.49:56623 W0304 09:33:08.606158 156262400 process.cpp:903] Failed to accept socket: Failed to accept: Too many open files I0304 09:33:08.606242 154116096 process.cpp:3588] Handling HTTP event for process 'master' with path: '/master//state-summary' W0304 09:33:08.606266 156262400 process.cpp:903] Failed to accept socket: Failed to accept: Too many open files I0304 09:33:08.606422 154116096 process.cpp:3588] Handling HTTP event for process 'master' with path: '/master//state-summary' I0304 09:33:08.606519 154116096 http.cpp:1105] HTTP GET for /master//state from 172.18.8.49:56624 I0304 09:33:08.606779 154116096 process.cpp:3588] Handling HTTP event for process 'master' with path: '/master//state-summary' W0304 09:33:08.606808 156262400 process.cpp:903] Failed to accept socket: Failed to accept: Too many open files W0304 09:33:08.606873 156262400 process.cpp:903] Failed to accept socket: Failed to accept: Too many open files W0304 09:33:08.606973 156262400 process.cpp:903] Failed to accept socket: Failed to accept: Too many open files I0304 09:33:08.607213 154116096 http.cpp:1105] HTTP GET for /master//state-summary from 172.18.8.49:56625 W0304 09:33:08.607342 156262400 process.cpp:903] Failed to accept socket: Failed to accept: Too many open files I0304 09:33:08.607362 154116096 process.cpp:3588] Handling HTTP event for process 'master' with path: '/master//state-summary' W0304 09:33:08.607401 156262400 process.cpp:903] Failed to accept socket: Failed to accept: Too many open files W0304 09:33:08.607494 156262400 process.cpp:903] Failed to accept socket: Failed to accept: Too many open files I0304 09:33:08.607522 154116096 process.cpp:3588] Handling HTTP event for process 'master' with path: '/master//state-summary' W0304 09:33:08.607733 156262400 process.cpp:903] Failed to accept socket: Failed to accept: Too many open files W0304 09:33:08.607800 156262400 process.cpp:903] Failed to accept socket: Failed to accept: Too many open files W0304 09:33:08.607904 156262400 process.cpp:903] Failed to accept socket: Failed to accept: Too many open files W0304 09:33:08.608388 156262400 process.cpp:903] Failed to accept socket: Failed to accept: Too many open files W0304 09:33:08.608449 156262400 process.cpp:903] Failed to accept socket: Failed to accept: Too many open files W0304 09:33:08.608860 156262400 process.cpp:903] Failed to accept socket: Failed to accept: Too many open files W0304 09:33:08.608920 156262400 process.cpp:903] Failed to accept socket: Failed to accept: Too many open files W0304 09:33:08.608996 156262400 process.cpp:903] Failed to accept socket: Failed to accept: Too many open files W0304 09:33:08.609063 156262400 process.cpp:903] Failed to accept socket: Failed to accept: Too many open files W0304 09:33:08.609180 156262400 process.cpp:903] Failed to accept socket: Failed to accept: Too many open files W0304 09:33:08.609259 156262400 process.cpp:903] Failed to accept socket: Failed to accept: Too many open files W0304 09:33:08.609463 156262400 process.cpp:903] Failed to accept socket: Failed to accept: Too many open files W0304 09:33:08.609509 156262400 process.cpp:903] Failed to accept socket: Failed to accept: Too many open files W0304 09:33:08.609791 156262400 process.cpp:903] Failed to accept socket: Failed to accept: Too many open files ../src/tests/master_load_tests.cpp:369: Failure (response).failure(): Connection reset by peer I0304 09:33:08.610236 154116096 http.cpp:1105] HTTP GET for /master//state-summary from 172.18.8.49:56626 I0304 09:33:08.610378 154116096 process.cpp:3588] Handling HTTP event for process 'master' with path: '/master//state-summary' I0304 09:33:08.610442 154116096 process.cpp:3588] Handling HTTP event for process 'master' with path: '/master//state-summary' I0304 09:33:08.610693 154116096 process.cpp:3588] Handling HTTP event for process 'master' with path: '/master//state-summary' I0304 09:33:08.610785 154116096 http.cpp:1105] HTTP GET for /master//state-summary from 172.18.8.49:56627 I0304 09:33:08.610836 154116096 process.cpp:3588] Handling HTTP event for process 'master' with path: '/master//state-summary' *** Aborted at 1551688388 (unix time) try "date -d @1551688388" if you are using GNU date *** I0304 09:33:08.610894 154116096 process.cpp:3588] Handling HTTP event for process 'master' with path: '/master//state-summary' I0304 09:33:08.610960 154116096 process.cpp:3588] Handling HTTP event for process 'master' with path: '/master//state-summary' I0304 09:33:08.611002 154116096 process.cpp:3588] Handling HTTP event for process 'master' with path: '/master//state-summary' W0304 09:33:08.611004 156262400 process.cpp:903] Failed to accept socket: Failed to accept: Too many open files I0304 09:33:08.611052 154116096 process.cpp:3588] Handling HTTP event for process 'master' with path: '/master//state-summary' W0304 09:33:08.611069 156262400 process.cpp:903] Failed to accept socket: Failed to accept: Too many open files W0304 09:33:08.611168 156262400 process.cpp:903] Failed to accept socket: Failed to accept: Too many open files W0304 09:33:08.611263 156262400 process.cpp:903] Failed to accept socket: Failed to accept: Too many open files I0304 09:33:08.611289 154116096 process.cpp:3588] Handling HTTP event for process 'master' with path: '/master//state-summary' I0304 09:33:08.611392 154116096 process.cpp:3588] Handling HTTP event for process 'master' with path: '/master//state-summary' I0304 09:33:08.611455 154116096 process.cpp:3588] Handling HTTP event for process 'master' with path: '/master//state-summary' I0304 09:33:08.611503 154116096 process.cpp:3588] Handling HTTP event for process 'master' with path: '/master//frameworks' I0304 09:33:08.611557 154652672 slave.cpp:912] Agent terminating I0304 09:33:08.611654 154116096 process.cpp:3588] Handling HTTP event for process 'master' with path: '/master//frameworks' I0304 09:33:08.611786 154116096 process.cpp:3588] Handling HTTP event for process 'master' with path: '/master//frameworks' I0304 09:33:08.611863 154116096 http.cpp:1105] HTTP GET for /master//state-summary from 172.18.8.49:56628 I0304 09:33:08.612150 151969792 http.cpp:1122] HTTP GET for /master//state-summary from 172.18.8.49:56625: '200 OK' after 6.70592ms I0304 09:33:08.612270 155189248 http.cpp:1122] HTTP GET for /master//state from 172.18.8.49:56587: '200 OK' after 21.9651ms I0304 09:33:08.612383 155189248 http.cpp:1122] HTTP GET for /master//state from 172.18.8.49:56588: '200 OK' after 21.3161ms W0304 09:33:08.612423 156262400 process.cpp:903] Failed to accept socket: Failed to accept: Too many open files I0304 09:33:08.612493 155189248 http.cpp:1122] HTTP GET for /master//state from 172.18.8.49:56589: '200 OK' after 20.6269ms W0304 09:33:08.612502 156262400 process.cpp:903] Failed to accept socket: Failed to accept: Too many open files I0304 09:33:08.612586 155189248 http.cpp:1122] HTTP GET for /master//state from 172.18.8.49:56586: '200 OK' after 21.653ms I0304 09:33:08.612707 155189248 http.cpp:1122] HTTP GET for /master//state from 172.18.8.49:56585: '200 OK' after 21.782ms I0304 09:33:08.612846 155189248 http.cpp:1122] HTTP GET for /master//state from 172.18.8.49:56590: '200 OK' after 19.9299ms I0304 09:33:08.612959 155189248 http.cpp:1122] HTTP GET for /master//state from 172.18.8.49:56591: '200 OK' after 19.3091ms I0304 09:33:08.613097 155189248 http.cpp:1122] HTTP GET for /master//state from 172.18.8.49:56594: '200 OK' after 17.4001ms I0304 09:33:08.613246 155189248 http.cpp:1122] HTTP GET for /master//state from 172.18.8.49:56595: '200 OK' after 17.1779ms I0304 09:33:08.613346 155189248 http.cpp:1122] HTTP GET for /master//state from 172.18.8.49:56593: '200 OK' after 17.7869ms PC: @ 0x101786aaa mesos::internal::tests::BlockingAuthorizerProcess::getObjectApprover() *** SIGSEGV (@0x0) received by PID 9500 (TID 0x700009171000) stack trace: *** @ 0x7fff72501b3d _sigtramp @ 0x1 (unknown) I0304 09:33:08.613466 155189248 http.cpp:1122] HTTP GET for /master//state from 172.18.8.49:56592: '200 OK' after 18.5211ms I0304 09:33:08.613574 155189248 http.cpp:1122] HTTP GET for /master//state from 172.18.8.49:56596: '200 OK' after 16.9779ms I0304 09:33:08.613687 155189248 http.cpp:1122] HTTP GET for /master//state from 172.18.8.49:56599: '200 OK' after 16.212ms I0304 09:33:08.613782 155189248 http.cpp:1122] HTTP GET for /master//state from 172.18.8.49:56600: '200 OK' after 15.9401ms I0304 09:33:08.613922 155189248 http.cpp:1122] HTTP GET for /master//state from 172.18.8.49:56597: '200 OK' after 16.6989ms @ 0x10178728b _ZN6lambda8internal7PartialIZN7process8dispatchINS2_5OwnedIN5mesos14ObjectApproverEEENS5_8internal5tests25BlockingAuthorizerProcessERK6OptionINS5_13authorization7SubjectEERKNSC_6ActionESG_SJ_EENS2_6FutureIT_EERKNS2_3PIDIT0_EEMSO_FSM_T1_T2_EOT3_OT4_EUlNSt3__110unique_ptrINS2_7PromiseIS7_EENS10_14default_deleteIS13_EEEEOSE_OSH_PNS2_11ProcessBaseEE_JS16_SE_SH_NS10_12placeholders4__phILi1EEEEE13invoke_expandIS1B_NS10_5tupleIJS16_SE_SH_S1E_EEENS1H_IJOS1A_EEEJLm0ELm1ELm2ELm3EEEEDTclsr5cpp17E6invokeclsr3stdE7forwardISL_Efp_Espcl6expandclsr3stdE3getIXT2_EEclsr3stdE7forwardISO_Efp0_EEclsr3stdE7forwardISS_Efp2_EEEEOSL_OSO_N5cpp1416integer_sequenceImJXspT2_EEEEOSS_ I0304 09:33:08.614030 155189248 http.cpp:1122] HTTP GET for /master//state from 172.18.8.49:56601: '200 OK' after 15.5369ms I0304 09:33:08.614161 155189248 http.cpp:1122] HTTP GET for /master//state from 172.18.8.49:56598: '200 OK' after 16.4539ms I0304 09:33:08.614300 155189248 http.cpp:1122] HTTP GET for /master//state from 172.18.8.49:56602: '200 OK' after 15.3411ms @ 0x1017871f1 _ZNO6lambda12CallableOnceIFvPN7process11ProcessBaseEEE10CallableFnINS_8internal7PartialIZNS1_8dispatchINS1_5OwnedIN5mesos14ObjectApproverEEENSB_8internal5tests25BlockingAuthorizerProcessERK6OptionINSB_13authorization7SubjectEERKNSI_6ActionESM_SP_EENS1_6FutureIT_EERKNS1_3PIDIT0_EEMSU_FSS_T1_T2_EOT3_OT4_EUlNSt3__110unique_ptrINS1_7PromiseISD_EENS16_14default_deleteIS19_EEEEOSK_OSN_S3_E_JS1C_SK_SN_NS16_12placeholders4__phILi1EEEEEEEclEOS3_ I0304 09:33:08.614394 155189248 http.cpp:1122] HTTP GET for /master//state from 172.18.8.49:56605: '200 OK' after 14.0411ms I0304 09:33:08.614491 155189248 http.cpp:1122] HTTP GET for /master//state from 172.18.8.49:56604: '200 OK' after 14.5521ms I0304 09:33:08.614651 155189248 http.cpp:1122] HTTP GET for /master//state from 172.18.8.49:56603: '200 OK' after 15.0139ms I0304 09:33:08.614799 155189248 http.cpp:1122] HTTP GET for /master//state from 172.18.8.49:56606: '200 OK' after 13.6389ms @ 0x108e00fc1 process::ProcessBase::consume() @ 0x108dfd669 process::ProcessManager::resume() I0304 09:33:08.614954 155189248 http.cpp:1122] HTTP GET for /master//state from 172.18.8.49:56609: '200 OK' after 11.999ms @ 0x108e4468a _ZNSt3__114__thread_proxyINS_5tupleIJNS_10unique_ptrINS_15__thread_structENS_14default_deleteIS3_EEEEZN7process14ProcessManager12init_threadsEvE3$_8EEEEEPvSB_ @ 0x7fff7250a305 _pthread_body @ 0x7fff7250d26f _pthread_start @ 0x7fff72509415 thread_start I0304 09:33:08.615038 155189248 http.cpp:1122] HTTP GET for /master//state from 172.18.8.49:56607: '200 OK' after 13.2539ms