[YARN-7301] Create stable Scheduler API

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Component/s: yarn
• Assignee: Unassigned
• Reporter: Robert Kanter
• Created: 09/Oct/17 18:17
• Updated: 09/Oct/17 18:17

Description

Currently, it's not practical for a user to create their own scheduler. Besides it being a large undertaking, the API is a mess. A few of the problems:

1. We make incompatible changes to YarnScheduler sometimes (see YARN-7138).
2. Many methods in YarnScheduler are marked as @Public @Stable, but the class itself has no annotations, which defaults to @Private.
3. We often cast a YarnScheduler to an AbstractYarnScheduler, which means that custom schedulers must also subclass AbstractYarnScheduler or they'll get a ClassCastException. However, AbstractYarnScheduler is @Private @Unstable.

It could be useful to provide a proper usable API for custom schedulers.

Issue Links

• relates to: YARN-7138 Fix incompatible API change for YarnScheduler involved by YARN-5221