Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-5355 YARN Timeline Service v.2: alpha 2
  3. YARN-6850

Ensure that supplemented timestamp is stored only for flow run metrics

VotersStop watchingWatchersLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Reviewed

    Description

      In timeline service v2, ColumnHelper#getPutTimestamp supplements the timestamp and is called by ColumnHelper#store. This is not conditional and called for every put.

      We need to ensure that the cell timestamps for metrics in entity and application (and sub application) tables are "correct" timestamps since we will be enabling TTLs for these cells.

      The supplemented timestamp is to be used only in the flow run table by the coprocessor which intercepts all reads & writes to cells in this table. It looks at the supplemented timestamp to figure out which app id this particular cell belongs to. This is done in order to ensure no collision occurs when two apps belonging to same flow run write the same metric at the same timestamp.

      Discovered in the discussion in YARN-4455

      Attachments

        1. YARN-6850-YARN-5355.01.patch
          10 kB
          Varun Saxena

        Issue Links

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            varun_saxena Varun Saxena
            vrushalic Vrushali C
            Votes:
            0 Vote for this issue
            Watchers:
            7 Stop watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment