[YARN-6455] Enhance the timelinewriter.flush() race condition fix

• Type: Sub-task
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.0.0-alpha2
• Fix Version/s: 2.9.0, YARN-5355, YARN-5355-branch-2, 3.0.0-alpha4
• Component/s: yarn
• Target Version/s: 3.0.0-alpha4
• Hadoop Flags: Reviewed
• Assignee: Haibo Chen
• Reporter: Haibo Chen
• Created: 07/Apr/17 04:00
• Updated: 21/Oct/17 06:05
• Resolved: 27/Apr/17 09:49

Description

YARN-6376 fixes timelinewriter.flush() race condition among concurrent putEntities() calls and periodical flush by TimelineCollectorManager by synchronizing on the writer object.

Synchronizing on the writer is still a little brittle there, because there is a getWriter method which lets callers access the writer without synchronizing on it. AppLevelTimelineCollector#AppLevelAggregator#agregate() does this in line 152: getWriter().write(...) In this case it doesn't flush, but if that were to be added, that would re-introduce the race fixed in YARN-6376.

Instead of exposing the writer, perhaps it would be better to have the sub-classes call #putEntities instead. It defers to the private writeTimelineEntities which does the same work to get the context:
TimelineCollectorContext context = getTimelineEntityContext();

Issue Links

• relates to: YARN-6376 Exceptions caused by synchronous putEntities requests can be swallowed