[MESOS-9383] Registry puller needs to implement retry upon failing image pulls

• Type: Improvement
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Component/s: containerization
• Assignee: Unassigned
• Reporter: Jason Lai
• Shepherd: Gilbert Song
• Created: 11/Nov/18 20:57
• Updated: 11/Nov/18 20:58

Description

UCR's RegistryPuller doesn't support retry upon failures on fetching container images and below is an example of the issue:

mesos-slave.{hostname}.root.log.INFO.20181016-193329.90745.6.gz:E1029 21:28:16.114250 90838 slave.cpp:6191] Container '28a416f4-9b3f-41f5-b343-17281c796c81' for executor '747f92c4-a0d7-4b8a-927a-596feac33af1-1-4' of framework 3dcc744f-016c-6579-9b82-6325424502d2-9999 failed to start: Collect failed: Failed to perform 'curl': curl: (18) transfer closed with 9732 bytes remaining to read
mesos-slave.{hostname}.root.log.INFO.20181016-193329.90745.6.gz:E1029 21:28:16.118737 90829 slave.cpp:6191] Container 'e80c5a81-c849-4aa3-bae3-a1d55fa1b9da' for executor '747f92c4-a0d7-4b8a-927a-596feac33af1-2-4' of framework 3dcc744f-016c-6579-9b82-6325424502d2-9999 failed to start: Collect failed: Failed to perform 'curl': curl: (18) transfer closed with 9732 bytes remaining to read

On the other hand, such logic was implemented long time ago in Docker and quite necessary for UCR at scale.