[YARN-8320] [Umbrella] Support CPU isolation for latency-sensitive (LS) service

• Type: New Feature
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Component/s: nodemanager
• Assignee: Unassigned
• Reporter: Jiandan Yang
• Created: 18/May/18 09:17
• Updated: 18/Jun/18 20:03

Description

Currently NodeManager uses “cpu.cfs_period_us”, “cpu.cfs_quota_us” and “cpu.shares” to isolate cpu resource. However,

• Linux Completely Fair Scheduling (CFS) is a throughput-oriented scheduler; no support for differentiated latency
• Request latency of services running on container may be frequent shake when all containers share cpus, and latency-sensitive services can not afford in our production environment.

So we need more fine-grained cpu isolation.

Here we propose a solution using cgroup cpuset to binds containers to different processors, this is inspired by the isolation technique in Borg system.