[MESOS-7133] mesos-fetcher fails with openssl-related output.

• Type: Bug
• Status: Resolved
• Priority: Blocker
• Resolution: Fixed
• Affects Version/s: 1.1.1, 1.2.0
• Fix Version/s: 1.1.1, 1.2.0, 1.2.1, 1.3.0
• Target Version/s: 1.1.0, 1.2.0, 1.3.0
• Sprint: Mesosphere Sprint 51, Mesosphere Sprint 52
• Assignee: Till Toenshoff
• Reporter: Till Toenshoff
• Shepherd: Adam B
• Created: 16/Feb/17 01:17
• Updated: 23/Feb/17 02:47
• Resolved: 23/Feb/17 02:47

Description

Running a task as non root user while having a fetcherinfo setup for downloading some .zip or .tar.gz may cause the fetcher to break.

I0215 03:52:55.702874  4800 fetcher.cpp:531] Fetcher Info: {"cache_directory":"\/tmp\/mesos\/fetch\/slaves\/5c12449d-a933-44aa-ad03-5a9a2ff0161e-S4\/core","items":[{"action":"BYPASS_CACHE","uri":{"extract":true,"value":"https:\/\/downloads.mesosphere.com\/elastic\/assets\/1.0.4-5.1.2\/executor.zip"}},{"action":"BYPASS_CACHE","uri":{"extract":true,"value":"https:\/\/downloads.mesosphere.com\/libmesos-bundle\/libmesos-bundle-1.9-argus-1.1.x-2.tar.gz"}},{"action":"BYPASS_CACHE","uri":{"extract":true,"value":"https:\/\/artifacts.elastic.co\/downloads\/elasticsearch\/elasticsearch-5.1.2.tar.gz"}},{"action":"BYPASS_CACHE","uri":{"extract":true,"value":"https:\/\/downloads.mesosphere.com\/java\/jre-8u112-linux-x64.tar.gz"}},{"action":"BYPASS_CACHE","uri":{"extract":true,"value":"https:\/\/downloads.mesosphere.com\/elastic\/assets\/1.0.4-5.1.2\/bootstrap.zip"}},{"action":"BYPASS_CACHE","uri":{"extract":true,"value":"https:\/\/downloads.mesosphere.com\/elastic\/assets\/1.0.4-5.1.2\/elastic-scheduler.zip"}},{"action":"BYPASS_CACHE","uri":{"extract":true,"value":"https:\/\/github.com\/elastic\/elasticsearch-support-diagnostics\/releases\/download\/5.1\/support-diagnostics-5.1-dist.zip"}},{"action":"BYPASS_CACHE","uri":{"extract":false,"output_file":"config-templates\/elasticsearch","value":"http:\/\/api.elastic.marathon.l4lb.thisdcos.directory\/v1\/artifacts\/template\/96a656ca-8a10-469c-9f4f-22a6f4d7264d\/ingest\/server\/elasticsearch"}},{"action":"BYPASS_CACHE","uri":{"extract":true,"value":"https:\/\/artifacts.elastic.co\/downloads\/packs\/x-pack\/x-pack-5.1.2.zip"}}],"sandbox_directory":"\/var\/lib\/mesos\/slave\/slaves\/5c12449d-a933-44aa-ad03-5a9a2ff0161e-S4\/frameworks\/5c12449d-a933-44aa-ad03-5a9a2ff0161e-0002\/executors\/ingest__491a21e0-b984-49df-a015-b4df0b43f83a\/runs\/1d282b1a-5403-461e-ae55-b675daf6fcb5","user":"core"}
I0215 03:52:55.705590  4800 fetcher.cpp:442] Fetching URI 'https://downloads.mesosphere.com/elastic/assets/1.0.4-5.1.2/executor.zip'
I0215 03:52:55.705608  4800 fetcher.cpp:283] Fetching directly into the sandbox directory
I0215 03:52:55.705631  4800 fetcher.cpp:220] Fetching URI 'https://downloads.mesosphere.com/elastic/assets/1.0.4-5.1.2/executor.zip'
I0215 03:52:55.705653  4800 fetcher.cpp:163] Downloading resource from 'https://downloads.mesosphere.com/elastic/assets/1.0.4-5.1.2/executor.zip' to '/var/lib/mesos/slave/slaves/5c12449d-a933-44aa-ad03-5a9a2ff0161e-S4/frameworks/5c12449d-a933-44aa-ad03-5a9a2ff0161e-0002/executors/ingest__491a21e0-b984-49df-a015-b4df0b43f83a/runs/1d282b1a-5403-461e-ae55-b675daf6fcb5/executor.zip'
W0215 03:52:55.947074  4800 openssl.cpp:402] Failed SSL connections will be downgraded to a non-SSL socket
I0215 03:52:55.947113  4800 openssl.cpp:424] CA directory path unspecified! NOTE: Set CA directory path with LIBPROCESS_SSL_CA_DIR=<dirpath>
I0215 03:52:55.947124  4800 openssl.cpp:429] Will not verify peer certificate!
NOTE: Set LIBPROCESS_SSL_VERIFY_CERT=1 to enable peer certificate verification
I0215 03:52:55.947131  4800 openssl.cpp:435] Will only verify peer certificate if presented!
NOTE: Set LIBPROCESS_SSL_REQUIRE_CERT=1 to require peer certificate verification
Could not load key file '/run/dcos/pki/tls/private/mesos-slave.key' (OpenSSL error #33558541): error:0200100D:system library:fopen:Permission denied

The variable LIBPROCESS_SSL_KEY_FILE obviously was handed from the agent to the mesos-fetcher. The fetcher does a SUID shortly after spawning but before initializing libprocess. libprocess gets initialized via subprocess call in static Try<bool> extract(). The file linked by that environment variable is root-only readable and hence that failure.

Issue Links

• is related to: MESOS-6751 Mesos should allow for selective environment inheritance.