Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-2928 YARN Timeline Service v.2: alpha 1
  3. YARN-3949

ensure timely flush of timeline writes

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • YARN-2928
    • 2.9.0, 3.0.0-alpha1
    • timelineserver
    • None
    • Reviewed

    Description

      Currently flushing of timeline writes is not really handled. For example, HBaseTimelineWriterImpl relies on HBase's BufferedMutator to batch and write puts asynchronously. However, BufferedMutator may not flush them to HBase unless the internal buffer fills up.

      We do need a flush functionality first to ensure that data are written in a reasonably timely manner, and to be able to ensure some critical writes are done synchronously (e.g. key lifecycle events).

      Attachments

        1. YARN-3949-YARN-2928.001.patch
          11 kB
          Sangjin Lee
        2. YARN-3949-YARN-2928.002.patch
          11 kB
          Sangjin Lee
        3. YARN-3949-YARN-2928.002.patch
          11 kB
          Sangjin Lee
        4. YARN-3949-YARN-2928.003.patch
          12 kB
          Sangjin Lee
        5. YARN-3949-YARN-2928.004.patch
          12 kB
          Sangjin Lee
        6. YARN-3949-YARN-2928.004.patch
          13 kB
          Sangjin Lee

        Issue Links

          relates to

          Sub-task - The sub-task of the issue YARN-3908 Bugs in HBaseTimelineWriterImpl

          • Major - Major loss of function.
          • Resolved

          Activity

            People

              sjlee0 Sangjin Lee
              sjlee0 Sangjin Lee
              Votes:
              0 Vote for this issue
              Watchers:
              10 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: