[MESOS-6203] Mesos fetcher fails to extract tar archive

• Type: Bug
• Status: Resolved
• Priority: Critical
• Resolution: Duplicate
• Component/s: fetcher
• Flags: Important
• Assignee: Unassigned
• Reporter: Mark Hampson
• Created: 18/Sep/16 18:23
• Updated: 18/Sep/16 18:31
• Resolved: 18/Sep/16 18:31

Description

I'm trying to download and extract URL with mesos-fetcher. I have explicitly specified to extract archive in CommandInfo.URL. The url to download and extract is http://localhost:8000/tarball/cc4c8adb-7fb4-44ff-b5e6-78448e54db25.tar

Fetcher writes to stderr the following logs:

I0918 20:06:44.951944 5922 fetcher.cpp:409] Fetcher Info: {"cache_directory":"\/tmp\/mesos\/fetch\/slaves\/785e0582-fcfb-43ab-affc-3f65f8d0c36a-S2","items":[{"action":"BYPASS_CACHE","uri":{"executable":false,"extract":true,"value":"http:\/\/localhost:8000\/tarball\/cc4c8adb-7fb4-44ff-b5e6-78448e54db25.tar"}}],"sandbox_directory":"\/tmp\/mesos\/slaves\/785e0582-fcfb-43ab-affc-3f65f8d0c36a-S2\/frameworks\/785e0582-fcfb-43ab-affc-3f65f8d0c36a-0007\/executors\/d29609d7-5d23-48c3-8639-8ed1ad1052fc\/runs\/d1426b22-3181-4684-a6eb-a6728e2edb20"}
I0918 20:06:44.952720 5922 fetcher.cpp:364] Fetching URI 'http://localhost:8000/tarball/cc4c8adb-7fb4-44ff-b5e6-78448e54db25.tar'
I0918 20:06:44.952733 5922 fetcher.cpp:238] Fetching directly into the sandbox directory
I0918 20:06:44.952739 5922 fetcher.cpp:176] Fetching URI 'http://localhost:8000/tarball/cc4c8adb-7fb4-44ff-b5e6-78448e54db25.tar'
I0918 20:06:44.952744 5922 fetcher.cpp:126] Downloading resource from 'http://localhost:8000/tarball/cc4c8adb-7fb4-44ff-b5e6-78448e54db25.tar' to '/tmp/mesos/slaves/785e0582-fcfb-43ab-affc-3f65f8d0c36a-S2/frameworks/785e0582-fcfb-43ab-affc-3f65f8d0c36a-0007/executors/d29609d7-5d23-48c3-8639-8ed1ad1052fc/runs/d1426b22-3181-4684-a6eb-a6728e2edb20/cc4c8adb-7fb4-44ff-b5e6-78448e54db25.tar'
W0918 20:06:44.963593 5922 fetcher.cpp:260] Copying instead of extracting resource from URI with 'extract' flag, because it does not seem to be an archive: http://localhost:8000/tarball/cc4c8adb-7fb4-44ff-b5e6-78448e54db25.tar
I0918 20:06:44.963609 5922 fetcher.cpp:441] Fetched 'http://localhost:8000/tarball/cc4c8adb-7fb4-44ff-b5e6-78448e54db25.tar' to '/tmp/mesos/slaves/785e0582-fcfb-43ab-affc-3f65f8d0c36a-S2/frameworks/785e0582-fcfb-43ab-affc-3f65f8d0c36a-0007/executors/d29609d7-5d23-48c3-8639-8ed1ad1052fc/runs/d1426b22-3181-4684-a6eb-a6728e2edb20/cc4c8adb-7fb4-44ff-b5e6-78448e54db25.tar'
I0918 20:06:45.183472 5937 exec.cpp:132] Version: 0.23.0
I0918 20:06:45.185164 5943 exec.cpp:206] Executor registered on slave 785e0582-fcfb-43ab-affc-3f65f8d0c36a-S2

Logs indicate

Copying instead of extracting resource from URI with 'extract' flag, because it does not seem to be an archive: http://localhost:8000/tarball/cc4c8adb-7fb4-44ff-b5e6-78448e54db25.tar

This log corresponds to line 289 in fetcher.cpp which happens due to failure of tar extraction:
https://github.com/apache/mesos/blob/master/src/launcher/fetcher.cpp#L289

The logs show that the source path ends with ".tar", but line 64 returns false and the extraction never happens.
https://github.com/apache/mesos/blob/master/src/launcher/fetcher.cpp#L64

What am I missing?
Thanks in advance.

Issue Links

• is a clone of: MESOS-2935 Fetcher doesn't extract from .tar files