Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-4233 YARN Timeline Service plugin: ATS v1.5
  3. YARN-4716

TimelineClient to implement Flushable; propagate to writer

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Resolved
    • Major
    • Resolution: Duplicate
    • 2.8.0
    • None
    • timelineserver
    • None

    Description

      I need a flush() operation in the timeline client. Knowing the lifecycle of my app, I do explicitly want to flush up events at certain points (app start event, app end, without relying on an async flush client)

      Right now, in tests, the time it takes for an event to propagate is: flush delay + scan delay, you are looking at 2 seconds minimum, per test case, with less deterministic outcomes.

      In production, those big app lifecycle events are so important my client code currently explicitly flushes my own event queue, and expects them to reach the destination

      With the filesystem writer, I've lost those durability guarantees.

      Implementing Flushable.flush() would let my app tell the timeline client to write; it'd be a no-op on the http client, but for the filesystem writer (or any other async writer), it'd be expected to force a write to the durable medium.

      Attachments

        Issue Links

          Activity

            People

              Unassigned Unassigned
              stevel@apache.org Steve Loughran
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: