[YARN-5585] [Atsv2] Reader side changes for entity prefix and support for pagination via additional filters

• Type: Sub-task
• Status: Resolved
• Priority: Critical
• Resolution: Fixed
• Fix Version/s: 2.9.0, YARN-5355, 3.0.0-beta1, YARN-5355-branch-2
• Component/s: timelinereader
• Labels: yarn-5355-merge-blocker
• Hadoop Flags: Reviewed
• Assignee: Rohith Sharma K S
• Reporter: Rohith Sharma K S
• Created: 30/Aug/16 13:00
• Updated: 21/Oct/17 06:05
• Resolved: 07/Jan/17 16:41

Description

TimelineReader REST API's provides lot of filters to retrieve the applications. Along with those, it would be good to add new filter i.e fromId so that entities can be retrieved after the fromId.

Current Behavior : Default limit is set to 100. If there are 1000 entities then REST call gives first/last 100 entities. How to retrieve next set of 100 entities i.e 101 to 200 OR 900 to 801?
Example : If applications are stored database, app-1 app-2 ... app-10.
getApps?limit=5 gives app-1 to app-5. But to retrieve next 5 apps, there is no way to achieve this.

So proposal is to have fromId in the filter like getApps?limit=5&&fromId=app-5 which gives list of apps from app-6 to app-10.

Since ATS is targeting large number of entities storage, it is very common use case to get next set of entities using fromId rather than querying all the entites. This is very useful for pagination in web UI.

Issue Links

• is blocked by: YARN-5715 introduce entity prefix for return and sort order
• is depended upon by: YARN-6027 Support fromid(offset) filter for /flows API
• is required by: YARN-6064 Support fromId for flowRuns and flow/flowRun apps REST API's
• relates to: YARN-6027 Support fromid(offset) filter for /flows API
• relates to: YARN-6047 Documentation updates for TimelineService v2