Details
Critical Description
I’m trying to launch docker container with universal containerizer, mesos 1.2.0. But getting error “Failed to parse the image manifest: Docker v2 image manifest validation failed: ‘signatures’ field size must be at least one”. And if I switch to docker containerizer, app is starting normally.
We are working with private docker registry v2 backed by nexus repository manager 3.1.0
cat /etc/mesos-slave/docker_registry https://docker.company.ru cat /etc/mesos-slave/docker_config { "auths": { "docker.company.ru": { "auth": "........" } } }
Here agent's log:
I0405 22:00:49.860234 44856 slave.cpp:4346] Received ping from slave-observer(7)@10.34.1.31:5050 I0405 22:00:50.327030 44865 slave.cpp:1625] Got assigned task 'md_kafka_broker.2f58917d-1a32-11e7-ad66-02424dd04a14' for framework 5ad97c04-d982-49d3-ac4f-53c468993190-0000 I0405 22:00:50.327785 44865 slave.cpp:1785] Launching task 'md_kafka_broker.2f58917d-1a32-11e7-ad66-02424dd04a14' for framework 5ad97c04-d982-49d3-ac4f-53c468993190-0000 I0405 22:00:50.329324 44865 paths.cpp:547] Trying to chown '/export/intssd/mesos-slave/workdir/slaves/5ad97c04-d982-49d3-ac4f-53c468993190-S1/frameworks/5ad97c04-d982-49d3-ac4f-53c468993190-0000/executors/md_kafka_broker.2f58917d-1a32-11e7-ad66-02424dd04a14/runs/f82f5f69-87a3-4586-b4cc-b91d285dcaff' to user 'dockdata' I0405 22:00:50.329607 44865 slave.cpp:6896] Checkpointing ExecutorInfo to '/export/intssd/mesos-slave/workdir/meta/slaves/5ad97c04-d982-49d3-ac4f-53c468993190-S1/frameworks/5ad97c04-d982-49d3-ac4f-53c468993190-0000/executors/md_kafka_broker.2f58917d-1a32-11e7-ad66-02424dd04a14/executor.info' I0405 22:00:50.330531 44865 slave.cpp:6472] Launching executor 'md_kafka_broker.2f58917d-1a32-11e7-ad66-02424dd04a14' of framework 5ad97c04-d982-49d3-ac4f-53c468993190-0000 with resources cpus(*)(allocated: general_marathon_service_role):0.1; mem(*)(allocated: general_marathon_service_role):32 in work directory '/export/intssd/mesos-slave/workdir/slaves/5ad97c04-d982-49d3-ac4f-53c468993190-S1/frameworks/5ad97c04-d982-49d3-ac4f-53c468993190-0000/executors/md_kafka_broker.2f58917d-1a32-11e7-ad66-02424dd04a14/runs/f82f5f69-87a3-4586-b4cc-b91d285dcaff' I0405 22:00:50.331244 44865 slave.cpp:6919] Checkpointing TaskInfo to '/export/intssd/mesos-slave/workdir/meta/slaves/5ad97c04-d982-49d3-ac4f-53c468993190-S1/frameworks/5ad97c04-d982-49d3-ac4f-53c468993190-0000/executors/md_kafka_broker.2f58917d-1a32-11e7-ad66-02424dd04a14/runs/f82f5f69-87a3-4586-b4cc-b91d285dcaff/tasks/md_kafka_broker.2f58917d-1a32-11e7-ad66-02424dd04a14/task.info' I0405 22:00:50.331568 44862 docker.cpp:1106] Skipping non-docker container I0405 22:00:50.331822 44865 slave.cpp:2118] Queued task 'md_kafka_broker.2f58917d-1a32-11e7-ad66-02424dd04a14' for executor 'md_kafka_broker.2f58917d-1a32-11e7-ad66-02424dd04a14' of framework 5ad97c04-d982-49d3-ac4f-53c468993190-0000 I0405 22:00:50.331966 44865 slave.cpp:884] Successfully attached file '/export/intssd/mesos-slave/workdir/slaves/5ad97c04-d982-49d3-ac4f-53c468993190-S1/frameworks/5ad97c04-d982-49d3-ac4f-53c468993190-0000/executors/md_kafka_broker.2f58917d-1a32-11e7-ad66-02424dd04a14/runs/f82f5f69-87a3-4586-b4cc-b91d285dcaff' I0405 22:00:50.332582 44861 containerizer.cpp:993] Starting container f82f5f69-87a3-4586-b4cc-b91d285dcaff for executor 'md_kafka_broker.2f58917d-1a32-11e7-ad66-02424dd04a14' of framework 5ad97c04-d982-49d3-ac4f-53c468993190-0000 I0405 22:00:50.333286 44862 metadata_manager.cpp:168] Looking for image 'docker.company.ru/company-infra/kafka:0.10.2.0-16' I0405 22:00:50.333627 44879 registry_puller.cpp:247] Pulling image 'docker.company.ru/company-infra/kafka:0.10.2.0-16' from 'docker-manifest://docker.company.rucompany-infra/kafka?0.10.2.0-16#https' to '/export/intssd/mesos-slave/docker-store/staging/aV2yko' E0405 22:00:50.834630 44872 slave.cpp:4642] Container 'f82f5f69-87a3-4586-b4cc-b91d285dcaff' for executor 'md_kafka_broker.2f58917d-1a32-11e7-ad66-02424dd04a14' of framework 5ad97c04-d982-49d3-ac4f-53c468993190-0000 failed to start: Failed to parse the image manifest: Docker v2 image manifest validation failed: 'signatures' field size must be at least one I0405 22:00:50.835008 44853 containerizer.cpp:2069] Destroying container f82f5f69-87a3-4586-b4cc-b91d285dcaff in PROVISIONING state I0405 22:00:50.835127 44853 containerizer.cpp:2124] Waiting for the provisioner to complete provisioning before destroying container f82f5f69-87a3-4586-b4cc-b91d285dcaff I0405 22:00:50.835273 44844 provisioner.cpp:484] Ignoring destroy request for unknown container f82f5f69-87a3-4586-b4cc-b91d285dcaff I0405 22:00:50.836199 44837 slave.cpp:4754] Executor 'md_kafka_broker.2f58917d-1a32-11e7-ad66-02424dd04a14' of framework 5ad97c04-d982-49d3-ac4f-53c468993190-0000 has terminated with unknown status I0405 22:00:50.837193 44837 slave.cpp:3816] Handling status update TASK_FAILED (UUID: efd419db-5350-48bf-b612-8e5b5685b9a0) for task md_kafka_broker.2f58917d-1a32-11e7-ad66-02424dd04a14 of framework 5ad97c04-d982-49d3-ac4f-53c468993190-0000 from @0.0.0.0:0 E0405 22:00:50.837766 44846 slave.cpp:4097] Failed to update resources for container f82f5f69-87a3-4586-b4cc-b91d285dcaff of executor 'md_kafka_broker.2f58917d-1a32-11e7-ad66-02424dd04a14' running task md_kafka_broker.2f58917d-1a32-11e7-ad66-02424dd04a14 on status update for terminal task, destroying container: Container not found W0405 22:00:50.837962 44865 composing.cpp:630] Attempted to destroy unknown container f82f5f69-87a3-4586-b4cc-b91d285dcaff I0405 22:00:50.838018 44877 status_update_manager.cpp:323] Received status update TASK_FAILED (UUID: efd419db-5350-48bf-b612-8e5b5685b9a0) for task md_kafka_broker.2f58917d-1a32-11e7-ad66-02424dd04a14 of framework 5ad97c04-d982-49d3-ac4f-53c468993190-0000 I0405 22:00:50.838081 44877 status_update_manager.cpp:500] Creating StatusUpdate stream for task md_kafka_broker.2f58917d-1a32-11e7-ad66-02424dd04a14 of framework 5ad97c04-d982-49d3-ac4f-53c468993190-0000 I0405 22:00:50.838560 44877 status_update_manager.cpp:832] Checkpointing UPDATE for status update TASK_FAILED (UUID: efd419db-5350-48bf-b612-8e5b5685b9a0) for task md_kafka_broker.2f58917d-1a32-11e7-ad66-02424dd04a14 of framework 5ad97c04-d982-49d3-ac4f-53c468993190-0000 I0405 22:00:50.838708 44877 status_update_manager.cpp:377] Forwarding update TASK_FAILED (UUID: efd419db-5350-48bf-b612-8e5b5685b9a0) for task md_kafka_broker.2f58917d-1a32-11e7-ad66-02424dd04a14 of framework 5ad97c04-d982-49d3-ac4f-53c468993190-0000 to the agent I0405 22:00:50.838860 44878 slave.cpp:4256] Forwarding the update TASK_FAILED (UUID: efd419db-5350-48bf-b612-8e5b5685b9a0) for task md_kafka_broker.2f58917d-1a32-11e7-ad66-02424dd04a14 of framework 5ad97c04-d982-49d3-ac4f-53c468993190-0000 to master@10.34.1.31:5050 I0405 22:00:50.839059 44878 slave.cpp:4150] Status update manager successfully handled status update TASK_FAILED (UUID: efd419db-5350-48bf-b612-8e5b5685b9a0) for task md_kafka_broker.2f58917d-1a32-11e7-ad66-02424dd04a14 of framework 5ad97c04-d982-49d3-ac4f-53c468993190-0000 I0405 22:00:50.848595 44866 status_update_manager.cpp:395] Received status update acknowledgement (UUID: efd419db-5350-48bf-b612-8e5b5685b9a0) for task md_kafka_broker.2f58917d-1a32-11e7-ad66-02424dd04a14 of framework 5ad97c04-d982-49d3-ac4f-53c468993190-0000 I0405 22:00:50.848696 44866 status_update_manager.cpp:832] Checkpointing ACK for status update TASK_FAILED (UUID: efd419db-5350-48bf-b612-8e5b5685b9a0) for task md_kafka_broker.2f58917d-1a32-11e7-ad66-02424dd04a14 of framework 5ad97c04-d982-49d3-ac4f-53c468993190-0000 I0405 22:00:50.848801 44866 status_update_manager.cpp:531] Cleaning up status update stream for task md_kafka_broker.2f58917d-1a32-11e7-ad66-02424dd04a14 of framework 5ad97c04-d982-49d3-ac4f-53c468993190-0000 I0405 22:00:50.849365 44850 slave.cpp:3105] Status update manager successfully handled status update acknowledgement (UUID: efd419db-5350-48bf-b612-8e5b5685b9a0) for task md_kafka_broker.2f58917d-1a32-11e7-ad66-02424dd04a14 of framework 5ad97c04-d982-49d3-ac4f-53c468993190-0000 I0405 22:00:50.849431 44850 slave.cpp:6875] Completing task md_kafka_broker.2f58917d-1a32-11e7-ad66-02424dd04a14