[MESOS-3919] Design URI fetcher interfaces. - ASF JIRA

Attach files

Attach Screenshot

Voters

Watch issue

Watchers

Link

Clone

Update Comment Author

Replace String in Comment

Update Comment Visibility

Delete Comments

XML

Word

Printable

JSON

Details

Type: Task
Status: Resolved
Priority: Major
Resolution: Fixed
Affects Version/s: None
Fix Version/s: None
Component/s: None
Labels:
- twitter

Epic Link:
URI Fetcher

Description

The design doc can be found here:
https://docs.google.com/document/d/1p8tmQrGtxG6keZVo19gvHPr9WHxeny6PHTVofcLWqco/edit?usp=sharing

The main design goal here is to decouple artifacts fetching from artifacts cache management. This proposal describes the interfaces for fetching artifacts, which are designed to be extensible so that we can add custom fetching support. Artifacts cache management is orthogonal to artifacts fetching, and is not the goal of this doc.

The existing Fetcher contains both artifacts fetching logics and cache management logics. The former can leverage the new artifacts fetching interfaces described in this doc (i.e., uri::Fetcher) to fetch artifacts, or stay the same (using mesos-fetcher binary). Eventually, the existing Fetcher will use uri::Fetcher to download artifacts and we’ll be retiring the ‘mesos-fetcher’ binary.