[MESOS-10196] The task program runs successfully but the task status is failed

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Not A Bug
• Affects Version/s: 1.9.0, 1.10.0
• Component/s: executor

Environment

Ubuntu 16.04

mesos master 1.10.0

mesos slave 1.9.0

python 3.7.3

• Assignee: Unassigned
• Reporter: clancyhuang
• Created: 04/Nov/20 07:05
• Updated: 29/May/21 10:27
• Resolved: 29/May/21 10:27

Description

When testing mesos to execute the task by default executor, I found that the task status is failed but in fact the task was executed successfully.I tested two shell scripts, one is very simple

python -V > /root/test.txt

,The other is a script about image processing.
I am sure they are all working properly, but I get an error:REASON_EXECUTOR_TERMINATED.
The stderr of the task has no output, and the stdout is correct,the mesos agent has such log output

I1104 11:34:35.337236 35682 slave.cpp:3657] Launching container 65c98d6f-fcf8-4be4-89a8-7fe53b5c30ef for executor 'default' of framework d915071b-c275-4321-afd5-134b86ebadf3-0002
I1104 11:34:35.337371 35685 containerizer.cpp:1396] Starting container 65c98d6f-fcf8-4be4-89a8-7fe53b5c30ef
I1104 11:34:35.337563 35685 containerizer.cpp:3323] Transitioning the state of container 65c98d6f-fcf8-4be4-89a8-7fe53b5c30ef from STARTING to PROVISIONING after 76800ns
I1104 11:34:35.338893 35685 containerizer.cpp:3323] Transitioning the state of container 65c98d6f-fcf8-4be4-89a8-7fe53b5c30ef from PROVISIONING to PREPARING after 1.321216ms
I1104 11:34:35.340224 35703 switchboard.cpp:316] Container logger module finished preparing container 65c98d6f-fcf8-4be4-89a8-7fe53b5c30ef; IOSwitchboard server is not required
I1104 11:34:35.341944 35707 linux_launcher.cpp:492] Launching container 65c98d6f-fcf8-4be4-89a8-7fe53b5c30ef and cloning with namespaces
I1104 11:34:35.346983 35704 containerizer.cpp:3323] Transitioning the state of container 65c98d6f-fcf8-4be4-89a8-7fe53b5c30ef from PREPARING to ISOLATING after 8.082944ms
I1104 11:34:35.347719 35704 containerizer.cpp:3323] Transitioning the state of container 65c98d6f-fcf8-4be4-89a8-7fe53b5c30ef from ISOLATING to FETCHING after 730880ns
I1104 11:34:35.348254 35737 containerizer.cpp:3323] Transitioning the state of container 65c98d6f-fcf8-4be4-89a8-7fe53b5c30ef from FETCHING to RUNNING after 539136ns
I1104 11:34:58.060906 35680 slave.cpp:7406] Current disk usage 73.86%. Max allowed age: 1.130070981247558days
I1104 11:35:58.062266 35708 slave.cpp:7406] Current disk usage 73.86%. Max allowed age: 1.129549109651991days
I1104 11:36:58.062948 35741 slave.cpp:7406] Current disk usage 73.87%. Max allowed age: 1.129005310066273days
I1104 11:37:58.063513 35703 slave.cpp:7406] Current disk usage 73.88%. Max allowed age: 1.128437717518472days
I1104 11:38:30.242969 35740 containerizer.cpp:3161] Container 65c98d6f-fcf8-4be4-89a8-7fe53b5c30ef has exited
I1104 11:38:30.243052 35740 containerizer.cpp:2620] Destroying container 65c98d6f-fcf8-4be4-89a8-7fe53b5c30ef in RUNNING state
I1104 11:38:30.243072 35740 containerizer.cpp:3323] Transitioning the state of container 65c98d6f-fcf8-4be4-89a8-7fe53b5c30ef from RUNNING to DESTROYING after 3.91491408213333mins
I1104 11:38:30.243252 35672 linux_launcher.cpp:576] Asked to destroy container 65c98d6f-fcf8-4be4-89a8-7fe53b5c30ef
I1104 11:38:30.243350 35672 linux_launcher.cpp:618] Destroying cgroup '/sys/fs/cgroup/freezer/mesos/65c98d6f-fcf8-4be4-89a8-7fe53b5c30ef'
I1104 11:38:30.243768 35679 cgroups.cpp:2854] Freezing cgroup /sys/fs/cgroup/freezer/mesos/65c98d6f-fcf8-4be4-89a8-7fe53b5c30ef
I1104 11:38:30.243961 35671 cgroups.cpp:1242] Successfully froze cgroup /sys/fs/cgroup/freezer/mesos/65c98d6f-fcf8-4be4-89a8-7fe53b5c30ef after 110848ns
I1104 11:38:30.244160 35683 cgroups.cpp:2872] Thawing cgroup /sys/fs/cgroup/freezer/mesos/65c98d6f-fcf8-4be4-89a8-7fe53b5c30ef
I1104 11:38:30.244272 35683 cgroups.cpp:1271] Successfully thawed cgroup /sys/fs/cgroup/freezer/mesos/65c98d6f-fcf8-4be4-89a8-7fe53b5c30ef after 67840ns
I1104 11:38:30.244668 35690 linux_launcher.cpp:650] Destroying cgroup '/sys/fs/cgroup/systemd/mesos/65c98d6f-fcf8-4be4-89a8-7fe53b5c30ef'
I1104 11:38:30.245975 35726 slave.cpp:6856] Executor 'default' of framework d915071b-c275-4321-afd5-134b86ebadf3-0002 exited with status 0
I1104 11:38:30.246995 35726 slave.cpp:5737] Handling status update TASK_FAILED (Status UUID: 03cbd06b-cad8-4624-bf65-36a1a83ea39e) for task onextest1 of framework d915071b-c275-4321-            afd5-134b86ebadf3-0002 from @0.0.0.0:0
W1104 11:38:30.247347 35727 containerizer.cpp:2419] Ignoring update for unknown container 65c98d6f-fcf8-4be4-89a8-7fe53b5c30ef
I1104 11:38:30.247728 35730 task_status_update_manager.cpp:328] Received task status update TASK_FAILED (Status UUID: 03cbd06b-cad8-4624-bf65-36a1a83ea39e) for task onextest1 of fram            ework d915071b-c275-4321-afd5-134b86ebadf3-0002
I1104 11:38:30.248106 35731 slave.cpp:6276] Forwarding the update TASK_FAILED (Status UUID: 03cbd06b-cad8-4624-bf65-36a1a83ea39e) for task onextest1 of framework d915071b-c275-4321-a            fd5-134b86ebadf3-0002 to master@master:5050
I1104 11:38:30.355559 35735 task_status_update_manager.cpp:401] Received task status update acknowledgement (UUID: 03cbd06b-cad8-4624-bf65-36a1a83ea39e) for task onextest1 of framewo            rk d915071b-c275-4321-afd5-134b86ebadf3-0002
I1104 11:38:30.355821 35732 slave.cpp:6967] Cleaning up executor 'default' of framework d915071b-c275-4321-afd5-134b86ebadf3-0002