Uploaded image for project: 'Mesos'
  1. Mesos
  2. MESOS-7604

SlaveTest.ExecutorReregistrationTimeoutFlag aborts on Windows

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 1.4.0
    • 1.6.0
    • test

    • Windows

    • Mesosphere Sprint 74
    • 1

    Description

      [ RUN      ] SlaveTest.ExecutorReregistrationTimeoutFlag
rk ae9679b1-67c9-4db6-8187-0641b0e929d2-0000
I0601 23:53:23.488337  2748 master.cpp:1156] Master terminating
I0601 23:53:23.492337  2728 hierarchical.cpp:579] Removed agent ae9679b1-67c9-4db6-8187-0641b0e929d2-S0
I0601 23:53:23.530340  1512 cluster.cpp:162] Creating default 'local' authorizer
I0601 23:53:23.544342  2728 master.cpp:436] Master f07f4fdd-cd91-4d62-bf33-169b20d02020 (ip-172-20-128-1.ec2.internal) started on 172.20.128.1:51241
I0601 23:53:23.545341  2728 master.cpp:438] Flags at startup: --acls="" --agent_ping_timeout="15secs" --agent_reregister_timeout="10mins" --allocation_interval="1secs" --allocator="HierarchicalDRF" --authenticate_agents="false" --authenticate_frameworks="false" --authenticate_http_frameworks="true" --authenticate_http_readonly="true" --authenticate_http_readwrite="true" --authenticators="crammd5" --authorizers="local" --credentials="C:\temp\FWZORI\credentials" --filter_gpu_resources="true" --framework_sorter="drf" --help="false" --hostname_lookup="true" --http_authenticators="basic" --http_framework_authenticators="basic" --initialize_driver_logging="true" --log_auto_initialize="true" --logbufsecs="0" --logging_level="INFO" --max_agent_ping_timeouts="5" --max_completed_frameworks="50" --max_completed_tasks_per_framework="1000" --max_unreachable_tasks_per_framework="1000" --port="5050" --quiet="false" --recovery_agent_removal_limit="100%" --registry="in_memory" --registry_fetch_timeout="1mins" --registry_gc_interval="15mins" --registry_max_agent_age="2weeks" --registry_max_agent_count="102400" --registry_store_timeout="100secs" --registry_strict="false" --root_submissions="true" --user_sorter="drf" --version="false" --webui_dir="/webui" --work_dir="C:\temp\FWZORI\master" --zk_session_timeout="10secs"
I0601 23:53:23.550338  2728 master.cpp:515] Master only allowing authenticated HTTP frameworks to register
I0601 23:53:23.550338  2728 credentials.hpp:37] Loading credentials for authentication from 'C:\temp\FWZORI\credentials'
I0601 23:53:23.552338  2728 http.cpp:975] Creating default 'basic' HTTP authenticator for realm 'mesos-master-readonly'
I0601 23:53:23.553339  2728 http.cpp:975] Creating default 'basic' HTTP authenticator for realm 'mesos-master-readwrite'
I0601 23:53:23.554340  2728 http.cpp:975] Creating default 'basic' HTTP authenticator for realm 'mesos-master-scheduler'
I0601 23:53:23.555341  2728 master.cpp:640] Authorization enabled
I0601 23:53:23.570340  2124 master.cpp:2159] Elected as the leading master!
I0601 23:53:23.570340  2124 master.cpp:1698] Recovering from registrar
I0601 23:53:23.573341  1920 registrar.cpp:389] Successfully fetched the registry (0B) in 0ns
I0601 23:53:23.573341  1920 registrar.cpp:493] Applied 1 operations in 0ns; attempting to update the registry
I0601 23:53:23.575342  1920 registrar.cpp:550] Successfully updated the registry in 0ns
I0601 23:53:23.576344  1920 registrar.cpp:422] Successfully recovered registrar
I0601 23:53:23.577342  2728 master.cpp:1797] Recovered 0 agents from the registry (167B); allowing 10mins for agents to re-register
I0601 23:53:23.595341  1512 containerizer.cpp:230] Using isolation: windows/cpu,filesystem/windows,environment_secret
I0601 23:53:23.596343  1512 provisioner.cpp:255] Using default backend 'copy'
I0601 23:53:23.626343  3976 slave.cpp:248] Mesos agent started on (133)@172.20.128.1:51241
I0601 23:53:23.627342  3976 slave.cpp:249] Flags at startup: --appc_simple_discovery_uri_prefix="http://" --appc_store_dir="C:\temp\kglZbS\store\appc" --authenticate_http_readonly="true" --authenticate_http_readwrite="true" --authenticatee="crammd5" --authentication_backoff_factor="1secs" --authorizer="local" --container_disk_watch_interval="15secs" --containerizers="mesos" --default_role="*" --disk_watch_interval="1mins" --docker="docker" --docker_kill_orphans="true" --docker_registry="https://registry-1.docker.io" --docker_remove_delay="6hrs" --docker_socket="//./pipe/docker_engine" --docker_stop_timeout="0ns" --docker_store_dir="C:\temp\kglZbS\store\docker" --docker_volume_checkpoint_dir="/var/run/mesos/isolators/docker/volume" --enforce_container_disk_quota="false" --executor_registration_timeout="1mins" --executor_reregistration_timeout="15secs" --executor_shutdown_grace_period="5secs" --fetcher_cache_dir="C:\temp\kglZbS\fetch" --fetcher_cache_size="2GB" --frameworks_home="" --gc_delay="1weeks" --gc_disk_headroom="0.1" --hadoop_home="" --help="false" --hostname_lookup="true" --http_command_executor="false" --http_credentials="C:\temp\kglZbS\http_credentials" --http_heartbeat_interval="30secs" --initialize_driver_logging="true" --isolation="windows/cpu" --launcher="windows" --launcher_dir="C:\Users\Administrator\workspace\mesos\Mesos_CI-build\FLAG\Plain\label\mesos-ec2-windows\mesos\build\src" --logbufsecs="0" --logging_level="INFO" --max_completed_executors_per_framework="150" --oversubscribed_resources_interval="15secs" --port="5051" --qos_correction_interval_min="0ns" --quiet="false" --recover="reconnect" --recovery_timeout="15mins" --registration_backoff_factor="10ms" --resources="cpus:2;gpus:0;mem:1024;disk:1024;ports:[31000-32000]" --runtime_dir="C:\temp\kglZbS" --sandbox_directory="C:\mesos\sandbox" --strict="true" --version="false" --work_dir="C:\temp\b1wVnd"
I0601 23:53:23.632310  3976 credentials.hpp:37] Loading credentials for authentication from 'C:\temp\kglZbS\http_credentials'
I0601 23:53:23.634342  3976 http.cpp:975] Creating default 'basic' HTTP authenticator for realm 'mesos-agent-readonly'
I0601 23:53:23.635347  3976 http.cpp:975] Creating default 'basic' HTTP authenticator for realm 'mesos-agent-readwrite'
I0601 23:53:23.640344  3976 slave.cpp:552] Agent resources: cpus(*):2; mem(*):1024; disk(*):1024; ports(*):[31000-32000]
I0601 23:53:23.641345  3976 slave.cpp:560] Agent attributes: [  ]
I0601 23:53:23.641345  3976 slave.cpp:565] Agent hostname: ip-172-20-128-1.ec2.internal
I0601 23:53:23.641345  2124 status_update_manager.cpp:177] Pausing sending status updates
I0601 23:53:23.643345  1512 sched.cpp:232] Version: 1.4.0
I0601 23:53:23.645345  1920 sched.cpp:336] New master detected at master@172.20.128.1:51241
I0601 23:53:23.646344  1920 sched.cpp:365] Authentication is not available on this platform. Attempting to register without authentication
I0601 23:53:23.647344  4168 master.cpp:2811] Received SUBSCRIBE call for framework 'default' at scheduler-895fb702-afb9-42fe-8802-83c47f72432e@172.20.128.1:51241
I0601 23:53:23.647344  4168 master.cpp:2195] Authorizing framework principal 'test-principal' to receive offers for roles '{ * }'
I0601 23:53:23.648345  2728 state.cpp:62] Recovering state from 'C:\temp\b1wVnd\meta'
I0601 23:53:23.649345  4168 master.cpp:2811] Received SUBSCRIBE call for framework 'default' at scheduler-895fb702-afb9-42fe-8802-83c47f72432e@172.20.128.1:51241
I0601 23:53:23.649345  4168 master.cpp:2195] Authorizing framework principal 'test-principal' to receive offers for roles '{ * }'
I0601 23:53:23.649345  3976 status_update_manager.cpp:203] Recovering status update manager
I0601 23:53:23.650308  4168 master.cpp:2888] Subscribing framework default with checkpointing enabled and capabilities [  ]
I0601 23:53:23.650308  4496 containerizer.cpp:582] Recovering containerizer
I0601 23:53:23.652308  4168 master.cpp:2888] Subscribing framework default with checkpointing enabled and capabilities [  ]
I0601 23:53:23.652308  1920 sched.cpp:759] Framework registered with f07f4fdd-cd91-4d62-bf33-169b20d02020-0000
I0601 23:53:23.652308  4168 master.cpp:2898] Framework f07f4fdd-cd91-4d62-bf33-169b20d02020-0000 (default) at scheduler-895fb702-afb9-42fe-8802-83c47f72432e@172.20.128.1:51241 already subscribed, resending acknowledgement
I0601 23:53:23.653309  3976 hierarchical.cpp:294] Added framework f07f4fdd-cd91-4d62-bf33-169b20d02020-0000
I0601 23:53:23.658309  1920 provisioner.cpp:416] Provisioner recovery complete
I0601 23:53:23.659309  2748 slave.cpp:6119] Finished recovery
I0601 23:53:23.662308  4496 status_update_manager.cpp:177] Pausing sending status updates
I0601 23:53:23.662308  2748 slave.cpp:945] New master detected at master@172.20.128.1:51241
I0601 23:53:23.663310  2748 slave.cpp:969] No credentials provided. Attempting to register without authentication
I0601 23:53:23.663310  2748 slave.cpp:980] Detecting new master
I0601 23:53:23.664309  3976 master.cpp:5425] Received register agent message from slave(133)@172.20.128.1:51241 (ip-172-20-128-1.ec2.internal)
I0601 23:53:23.665309  3976 master.cpp:3657] Authorizing agent without a principal
I0601 23:53:23.666309  2124 master.cpp:5564] Registering agent at slave(133)@172.20.128.1:51241 (ip-172-20-128-1.ec2.internal) with id f07f4fdd-cd91-4d62-bf33-169b20d02020-S0
I0601 23:53:23.668309  3976 registrar.cpp:493] Applied 1 operations in 0ns; attempting to update the registry
I0601 23:53:23.670310  3976 registrar.cpp:550] Successfully updated the registry in 0ns
I0601 23:53:23.674311  1920 slave.cpp:1148] Registered with master master@172.20.128.1:51241; given agent ID f07f4fdd-cd91-4d62-bf33-169b20d02020-S0
I0601 23:53:23.674311  4168 master.cpp:5642] Registered agent f07f4fdd-cd91-4d62-bf33-169b20d02020-S0 at slave(133)@172.20.128.1:51241 (ip-172-20-128-1.ec2.internal) with cpus(*):2; mem(*):1024; disk(*):1024; ports(*):[31000-32000]
I0601 23:53:23.675309  3976 status_update_manager.cpp:184] Resuming sending status updates
I0601 23:53:23.676309  2728 hierarchical.cpp:546] Added agent f07f4fdd-cd91-4d62-bf33-169b20d02020-S0 (ip-172-20-128-1.ec2.internal) with cpus(*):2; mem(*):1024; disk(*):1024; ports(*):[31000-32000] (allocated: {})
I0601 23:53:23.681309  1920 slave.cpp:1206] Forwarding total oversubscribed resources {}
I0601 23:53:23.682309  2124 master.cpp:6295] Received update of agent f07f4fdd-cd91-4d62-bf33-169b20d02020-S0 at slave(133)@172.20.128.1:51241 (ip-172-20-128-1.ec2.internal) with total oversubscribed resources {}
I0601 23:53:23.686309  2124 master.cpp:7252] Sending 1 offers to framework f07f4fdd-cd91-4d62-bf33-169b20d02020-0000 (default) at scheduler-895fb702-afb9-42fe-8802-83c47f72432e@172.20.128.1:51241
I0601 23:53:23.694308  2728 master.cpp:3872] Processing ACCEPT call for offers: [ f07f4fdd-cd91-4d62-bf33-169b20d02020-O0 ] on agent f07f4fdd-cd91-4d62-bf33-169b20d02020-S0 at slave(133)@172.20.128.1:51241 (ip-172-20-128-1.ec2.internal) for framework f07f4fdd-cd91-4d62-bf33-169b20d02020-0000 (default) at scheduler-895fb702-afb9-42fe-8802-83c47f72432e@172.20.128.1:51241
I0601 23:53:23.694308  2728 master.cpp:3424] Authorizing framework principal 'test-principal' to launch task 792f1a13-d0ee-4e98-a4c8-82b9849adfbe
I0601 23:53:23.705309  2728 master.cpp:9265] Adding task 792f1a13-d0ee-4e98-a4c8-82b9849adfbe with resources cpus(*)(allocated: *):2; mem(*)(allocated: *):1024; disk(*)(allocated: *):1024; ports(*)(allocated: *):[31000-32000] on agent f07f4fdd-cd91-4d62-bf33-169b20d02020-S0 at slave(133)@172.20.128.1:51241 (ip-172-20-128-1.ec2.internal)
I0601 23:53:23.707309  2728 master.cpp:4527] Launching task 792f1a13-d0ee-4e98-a4c8-82b9849adfbe of framework f07f4fdd-cd91-4d62-bf33-169b20d02020-0000 (default) at scheduler-895fb702-afb9-42fe-8802-83c47f72432e@172.20.128.1:51241 with resources cpus(*)(allocated: *):2; mem(*)(allocated: *):1024; disk(*)(allocated: *):1024; ports(*)(allocated: *):[31000-32000] on agent f07f4fdd-cd91-4d62-bf33-169b20d02020-S0 at slave(133)@172.20.128.1:51241 (ip-172-20-128-1.ec2.internal)
I0601 23:53:23.710311  4496 slave.cpp:1632] Got assigned task '792f1a13-d0ee-4e98-a4c8-82b9849adfbe' for framework f07f4fdd-cd91-4d62-bf33-169b20d02020-0000
I0601 23:53:23.721310  3976 hierarchical.cpp:871] Updated allocation of framework f07f4fdd-cd91-4d62-bf33-169b20d02020-0000 on agent f07f4fdd-cd91-4d62-bf33-169b20d02020-S0 from cpus(*)(allocated: *):2; mem(*)(allocated: *):1024; disk(*)(allocated: *):1024; ports(*)(allocated: *):[31000-32000] to cpus(*)(allocated: *):2; mem(*)(allocated: *):1024; disk(*)(allocated: *):1024; ports(*)(allocated: *):[31000-32000]
I0601 23:53:23.723309  4496 slave.cpp:1913] Authorizing task '792f1a13-d0ee-4e98-a4c8-82b9849adfbe' for framework f07f4fdd-cd91-4d62-bf33-169b20d02020-0000
I0601 23:53:23.727311  4496 slave.cpp:2100] Launching task '792f1a13-d0ee-4e98-a4c8-82b9849adfbe' for framework f07f4fdd-cd91-4d62-bf33-169b20d02020-0000
I0601 23:53:23.738310  4496 slave.cpp:7078] Launching executor '792f1a13-d0ee-4e98-a4c8-82b9849adfbe' of framework f07f4fdd-cd91-4d62-bf33-169b20d02020-0000 with resources cpus(*)(allocated: *):0.1; mem(*)(allocated: *):32 in work directory 'C:\temp\b1wVnd\slaves\f07f4fdd-cd91-4d62-bf33-169b20d02020-S0\frameworks\f07f4fdd-cd91-4d62-bf33-169b20d02020-0000\executors\792f1a13-d0ee-4e98-a4c8-82b9849adfbe\runs\1cbce8a6-ae59-484f-b898-e2ea6396d2a9'
I0601 23:53:23.741310  4496 slave.cpp:2795] Launching container 1cbce8a6-ae59-484f-b898-e2ea6396d2a9 for executor '792f1a13-d0ee-4e98-a4c8-82b9849adfbe' of framework f07f4fdd-cd91-4d62-bf33-169b20d02020-0000
I0601 23:53:23.743311  3976 containerizer.cpp:1056] Starting container 1cbce8a6-ae59-484f-b898-e2ea6396d2a9
I0601 23:53:23.750272  4496 slave.cpp:2329] Queued task '792f1a13-d0ee-4e98-a4c8-82b9849adfbe' for executor '792f1a13-d0ee-4e98-a4c8-82b9849adfbe' of framework f07f4fdd-cd91-4d62-bf33-169b20d02020-0000
I0601 23:53:23.797314  1856 launcher.cppReceived SUBSCRIBED event
Subscribed executor on ip-172-20-128-1.ec2.internal
Received LAUNCH event
Starting task 792f1a13-d0ee-4e98-a4c8-82b9849adfbe
Running 'C:\Users\Administrator\workspace\mesos\Mesos_CI-build\FLAG\Plain\label\mesos-ec2-windows\mesos\build\src\mesos-containerizer.exe launch <POSSIBLY-SENSITIVE-DATA>'
Forked command at 1180
:140] Forked child with pid '4980' for container '1cbce8a6-ae59-484f-b898-e2ea6396d2a9'
I0601 23:53:23.798315  1856 containerizer.cpp:1722] Checkpointing container's forked pid 4980 to 'C:\temp\b1wVnd\meta\slaves\f07f4fdd-cd91-4d62-bf33-169b20d02020-S0\frameworks\f07f4fdd-cd91-4d62-bf33-169b20d02020-0000\executors\792f1a13-d0ee-4e98-a4c8-82b9849adfbe\runs\1cbce8a6-ae59-484f-b898-e2ea6396d2a9\pids\forked.pid'
I0601 23:53:24.029322  1856 slave.cpp:3825] Got registration for executor '792f1a13-d0ee-4e98-a4c8-82b9849adfbe' of framework f07f4fdd-cd91-4d62-bf33-169b20d02020-0000 from executor(1)@172.20.128.1:51637
I0601 23:53:24.043324  2124 slave.cpp:2542] Sending queued task '792f1a13-d0ee-4e98-a4c8-82b9849adfbe' to executor '792f1a13-d0ee-4e98-a4c8-82b9849adfbe' of framework f07f4fdd-cd91-4d62-bf33-169b20d02020-0000 at executor(1)@172.20.128.1:51637
I0601 23:53:24.185328  2728 slave.cpp:4295] Handling status update TASK_RUNNING (UUID: d814fd3c-25f8-4307-a8a7-3235177322b9) for task 792f1a13-d0ee-4e98-a4c8-82b9849adfbe of framework f07f4fdd-cd91-4d62-bf33-169b20d02020-0000 from executor(1)@172.20.128.1:51637
I0601 23:53:24.192329  2124 status_update_manager.cpp:323] Received status update TASK_RUNNING (UUID: d814fd3c-25f8-4307-a8a7-3235177322b9) for task 792f1a13-d0ee-4e98-a4c8-82b9849adfbe of framework f07f4fdd-cd91-4d62-bf33-169b20d02020-0000
I0601 23:53:24.195328  2124 status_update_manager.cpp:834] Checkpointing UPDATE for status update TASK_RUNNING (UUID: d814fd3c-25f8-4307-a8a7-3235177322b9) for task 792f1a13-d0ee-4e98-a4c8-82b9849adfbe of framework f07f4fdd-cd91-4d62-bf33-169b20d02020-0000
I0601 23:53:24.197327  2728 slave.cpp:4735] Forwarding the update TASK_RUNNING (UUID: d814fd3c-25f8-4307-a8a7-3235177322b9) for task 792f1a13-d0ee-4e98-a4c8-82b9849adfbe of framework f07f4fdd-cd91-4d62-bf33-169b20d02020-0000 to master@172.20.128.1:51241
I0601 23:53:24.199362  4496 master.cpp:6440] Status update TASK_RUNNING (UUID: d814fd3c-25f8-4307-a8a7-3235177322b9) for task 792f1a13-d0ee-4e98-a4c8-82b9849adfbe of framework f07f4fdd-cd91-4d62-bf33-169b20d02020-0000 from agent f07f4fdd-cd91-4d62-bf33-169b20d02020-S0 at slave(133)@172.20.128.1:51241 (ip-172-20-128-1.ec2.internal)
I0601 23:53:24.198328  2728 slave.cpp:4645] Sending acknowledgement for status update TASK_RUNNING (UUID: d814fd3c-25f8-4307-a8a7-3235177322b9) for task 792f1a13-d0ee-4e98-a4c8-82b9849adfbe of framework f07f4fdd-cd91-4d62-bf33-169b20d02020-0000 to executor(1)@172.20.128.1:51637
I0601 23:53:24.199362  4496 master.cpp:6502] Forwarding status update TASK_RUNNING (UUID: d814fd3c-25f8-4307-a8a7-3235177322b9) for task 792f1a13-d0ee-4e98-a4c8-82b9849adfbe of framework f07f4fdd-cd91-4d62-bf33-169b20d02020-0000
I0601 23:53:24.200330  4496 master.cpp:8507] Updating the state of task 792f1a13-d0ee-4e98-a4c8-82b9849adfbe of framework f07f4fdd-cd91-4d62-bf33-169b20d02020-0000 (latest state: TASK_RUNNING, status update state: TASK_RUNNING)
I0601 23:53:24.207330  4496 master.cpp:5190] Processing ACKNOWLEDGE call d814fd3c-25f8-4307-a8a7-3235177322b9 for task 792f1a13-d0ee-4e98-a4c8-82b9849adfbe of framework f07f4fdd-cd91-4d62-bf33-169b20d02020-0000 (default) at scheduler-895fb702-afb9-42fe-8802-83c47f72432e@172.20.128.1:51241 on agent f07f4fdd-cd91-4d62-bf33-169b20d02020-S0
I0601 23:53:24.208281  4168 status_update_manager.cpp:395] Received status update acknowledgement (UUID: d814fd3c-25f8-4307-a8a7-3235177322b9) for task 792f1a13-d0ee-4e98-a4c8-82b9849adfbe of framework f07f4fdd-cd91-4d62-bf33-169b20d02020-0000
I0601 23:53:24.209329  4168 status_update_manager.cpp:834] Checkpointing ACK for status update TASK_RUNNING (UUID: d814fd3c-25f8-4307-a8a7-3235177322b9) for task 792f1a13-d0ee-4e98-a4c8-82b9849adfbe of framework f07f4fdd-cd91-4d62-bf33-169b20d02020-0000
I0601 23:53:24.211320  4496 slave.cpp:817] Agent terminating
I0601 23:53:24.212329  4168 master.cpp:1314] Agent f07f4fdd-cd91-4d62-bf33-169b20d02020-S0 at slave(133)@172.20.128.1:51241 (ip-172-20-128-1.ec2.internal) disconnected
I0601 23:53:24.212329  4168 master.cpp:3195] Disconnecting agent f07f4fdd-cd91-4d62-bf33-169b20d02020-S0 at slave(133)@172.20.128.1:51241 (ip-172-20-128-1.ec2.internal)
I0601 23:53:24.212329  4168 master.cpp:3214] Deactivating agent f07f4fdd-cd91-4d62-bf33-169b20d02020-S0 at slave(133)@172.20.128.1:51241 (ip-172-20-128-1.ec2.internal)
I0601 23:53:24.213330  1920 hierarchical.cpp:674] Agent f07f4fdd-cd91-4d62-bf33-169b20d02020-S0 deactivated
I0601 23:53:24.214330  1512 containerizer.cpp:230] Using isolation: windows/cpu,filesystem/windows,environment_secret
I0601 23:53:24.215328  1512 provisioner.cpp:255] Using default backend 'copy'
I0601 23:53:24.254333  2728 slave.cpp:248] Mesos agent started on (134)@172.20.128.1:51241
I0601 23:53:24.254333  2728 slave.cpp:249] Flags at startup: --appc_simple_discovery_uri_prefix="http://" --appc_store_dir="C:\temp\kglZbS\store\appc" --authenticate_http_readonly="true" --authenticate_http_readwrite="true" --authenticatee="crammd5" --authentication_backoff_factor="1secs" --authorizer="local" --container_disk_watch_interval="15secs" --containerizers="mesos" --default_role="*" --disk_watch_interval="1mins" --docker="docker" --docker_kill_orphans="true" --docker_registry="https://registry-1.docker.io" --docker_remove_delay="6hrs" --docker_socket="//./pipe/docker_engine" --docker_stop_timeout="0ns" --docker_store_dir="C:\temp\kglZbS\store\docker" --docker_volume_checkpoint_dir="/var/run/mesos/isolators/docker/volume" --enforce_container_disk_quota="false" --executor_registration_timeout="1mins" --executor_reregistration_timeout="15secs" --executor_shutdown_grace_period="5secs" --fetcher_cache_dir="C:\temp\kglZbS\fetch" --fetcher_cache_size="2GB" --frameworks_home="" --gc_delay="1weeks" --gc_disk_headroom="0.1" --hadoop_home="" --help="false" --hostname_lookup="true" --http_command_executor="false" --http_credentials="C:\temp\kglZbS\http_credentials" --http_heartbeat_interval="30secs" --initialize_driver_logging="true" --isolation="windows/cpu" --launcher="windows" --launcher_dir="C:\Users\Administrator\workspace\mesos\Mesos_CI-build\FLAG\Plain\label\mesos-ec2-windows\mesos\build\src" --logbufsecs="0" --logging_level="INFO" --max_completed_executors_per_framework="150" --oversubscribed_resources_interval="15secs" --port="5051" --qos_correction_interval_min="0ns" --quiet="false" --recover="reconnect" --recovery_timeout="15mins" --registration_backoff_factor="10ms" --resources="cpus:2;gpus:0;mem:1024;disk:1024;ports:[31000-32000]" --runtime_dir="C:\temp\kglZbS" --sandbox_directory="C:\mesos\sandbox" --strict="true" --version="false" --work_dir="C:\temp\b1wVnd"
I0601 23:53:24.258334  2728 credentials.hpp:37] Loading credentials for authentication from 'C:\temp\kglZbS\http_credentials'
I0601 23:53:24.261334  2728 http.cpp:975] Creating default 'basic' HTTP authenticator for realm 'mesos-agent-readonly'
I0601 23:53:24.262333  2728 http.cpp:975] Creating default 'basic' HTTP authenticator for realm 'mesos-agent-readwrite'
I0601 23:53:24.269332  2728 slave.cpp:552] Agent resources: cpus(*):2; mem(*):1024; disk(*):1024; ports(*):[31000-32000]
I0601 23:53:24.269332  2728 slave.cpp:560] Agent attributes: [  ]
I0601 23:53:24.269332  2728 slave.cpp:565] Agent hostname: ip-172-20-128-1.ec2.internal
I0601 23:53:24.270332  2748 status_update_manager.cpp:177] Pausing sending status updates
I0601 23:53:24.279335  4496 state.cpp:62] Recovering state from 'C:\temp\b1wVnd\meta'
I0601 23:53:24.280333  4496 state.cpp:710] No committed checkpointed resources found at 'C:\temp\b1wVnd\meta\resources\resources.info'
E0601 23:53:24.308333  3976 slave.cpp:6110] EXIT with status 1: Failed to perform recovery: Incompatible agent info detected.
------------------------------------------------------------
Old agent info:
hostname: "ip-172-20-128-1.ec2.internal"
resources {
  name: "cpus"
  type: SCALAR
  scalar {
    value: 2
  }
  role: "*"
}
resources {
  name: "mem"
  type: SCALAR
  scalar {
    value: 1024
  }
  role: "*"
}
resources {
  name: "disk"
  type: SCALAR
  scalar {
    value: 1024
  }
  role: "*"
}
resources {
  name: "ports"
  type: RANGES
  ranges {
    range {
      begin: 31000
      end: 32000
    }
  }
  role: "*"
}
id {
  value: "f07f4fdd-cd91-4d62-bf33-169b20d02020-S0"
}
checkpoint: true
port: 51241

------------------------------------------------------------
New agent info:
hostname: "ip-172-20-128-1.ec2.internal"
resources {
  name: "cpus"
  type: SCALAR
  scalar {
    value: 2
  }
  role: "*"
}
resources {
  name: "mem"
  type: SCALAR
  scalar {
    value: 1024
  }
  role: "*"
}
resources {
  name: "disk"
  type: SCALAR
  scalar {
    value: 1024
  }
  role: "*"
}
resources {
  name: "ports"
  type: RANGES
  ranges {
    range {
      begin: 31000
      end: 32000
    }
  }
  role: "*"
}
id {
  value: "latest"
}
checkpoint: true
port: 51241

------------------------------------------------------------
To remedy this do as follows:
Step 1: rm -f C:\temp\b1wVnd\meta\slaves\latest
        This ensures agent doesn't recover old live executors.
Step 2: Restart the agent.

      Attachments

        Issue Links

          is related to

          Improvement - An improvement or enhancement to an existing feature or task. MESOS-7540 Add an agent flag for executor re-registration timeout.

          • Major - Major loss of function.
          • Resolved
          relates to

          Bug - A problem which impairs or prevents the functions of the product. MESOS-5881 Semantics of `os::symlink` differ across POSIX and Windows

          • Major - Major loss of function.
          • Resolved

          Bug - A problem which impairs or prevents the functions of the product. MESOS-5939 `os::size` doesn't follow symlinks correctly on Windows.

          • Major - Major loss of function.
          • Resolved

          Bug - A problem which impairs or prevents the functions of the product. MESOS-6735 `os::realpath` semantics differ between Windows and POSIX

          • Major - Major loss of function.
          • Resolved

          Activity

            People

              andschwa Andrew Schwartzmeyer
              kaysoky Joseph Wu
              Joseph Wu Joseph Wu
              Votes:
              1 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: