[MESOS-8250] Fetcher cache does not ignore query parameters in URIs

• Type: Bug
• Status: Accepted
• Priority: Minor
• Resolution: Unresolved
• Affects Version/s: 1.3.0
• Component/s: fetcher

Environment

Ubuntu 14.04

• Assignee: Unassigned
• Reporter: Timmy Zhu
• Created: 21/Nov/17 01:03
• Updated: 12/Jul/19 20:05

Description

I'm starting tasks on Mesos with S3 HTTP presigned URLs, so they all have long query parameters, which change each time the link is generated, even though the file content is the same. The Mesos fetcher cache does not ignore these query parameters, so the uri is never cached.

It would be nice to remove query params from the uri before using it as the cache key. Judging from other issues (MESOS-1686, MESOS-3367), the fetcher not removing query parameters also contributes to unexpected behavior from the viewpoint of the user.

Issue Links

• relates to: MESOS-1686 URI query strings are preserved in local filename
• relates to: MESOS-3367 Mesos fetcher does not extract archives for URI with parameters