Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-9802 YARN Timeline Service v2 (post GA features)
  3. YARN-9403

GET /apps/{appid}/entities/YARN_APPLICATION accesses application table instead of entity table

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 3.2.0
    • 3.4.0
    • ATSv2

    Description

      "GET /apps/{appid}/entities/YARN_APPLICATION"

      accesses application table instead of entity table. As per the doc, With this API, you can query generic entities identified by cluster ID, application ID and per-framework entity type. But it also provides all the apps when entityType is set to YARN_APPLICATION. It should only access Entity Table through GenericEntityReader.

      Wrong Output: With YARN_APPLICATION entityType, all applications listed from application tables.

      [hbase@yarn-ats-3 centos]$ curl -s "http://yarn-ats-3:8198/ws/v2/timeline/apps/application_1553258815132_0002/entities/YARN_APPLICATION?user.name=hbase&userid=hbase&flowname=word%20count" | jq .
      [
        {
          "metrics": [],
          "events": [],
          "createdtime": 1553258922721,
          "idprefix": 0,
          "isrelatedto": {},
          "relatesto": {},
          "info": {
            "UID": "ats!application_1553258815132_0002",
            "FROM_ID": "ats!hbase!word count!1553258922721!application_1553258815132_0002"
          },
          "configs": {},
          "type": "YARN_APPLICATION",
          "id": "application_1553258815132_0002"
        },
        {
          "metrics": [],
          "events": [],
          "createdtime": 1553258825918,
          "idprefix": 0,
          "isrelatedto": {},
          "relatesto": {},
          "info": {
            "UID": "ats!application_1553258815132_0001",
            "FROM_ID": "ats!hbase!word count!1553258825918!application_1553258815132_0001"
          },
          "configs": {},
          "type": "YARN_APPLICATION",
          "id": "application_1553258815132_0001"
        }
      ]
      

      Right Output: With correct entity type (MAPREDUCE_JOB) it accesses entity table for given applicationId and entityType.

      [hbase@yarn-ats-3 centos]$ curl -s "http://yarn-ats-3:8198/ws/v2/timeline/apps/application_1553258815132_0002/entities/MAPREDUCE_JOB?user.name=hbase&userid=hbase&flowname=word%20count" | jq .
      [
        {
          "metrics": [],
          "events": [],
          "createdtime": 1553258926667,
          "idprefix": 0,
          "isrelatedto": {},
          "relatesto": {},
          "info": {
            "UID": "ats!application_1553258815132_0002!MAPREDUCE_JOB!0!job_1553258815132_0002",
            "FROM_ID": "ats!hbase!word count!1553258922721!application_1553258815132_0002!MAPREDUCE_JOB!0!job_1553258815132_0002"
          },
          "configs": {},
          "type": "MAPREDUCE_JOB",
          "id": "job_1553258815132_0002"
        }
      ]
      

      Flow Activity and Flow Run tables can also be accessed using similar way.

      GET /apps/{appid}/entities/YARN_FLOW_ACTIVITY
      GET /apps/{appid}/entities/YARN_FLOW_RUN
      

      Attachments

        1. YARN-9403-004.patch
          19 kB
          Prabhu Joseph
        2. YARN-9403-003.patch
          19 kB
          Prabhu Joseph
        3. YARN-9403-002.patch
          82 kB
          Prabhu Joseph
        4. YARN-9403-001.patch
          29 kB
          Prabhu Joseph

        Issue Links

          Activity

            People

              groot Ashutosh Gupta
              prabhujoseph Prabhu Joseph
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 40m
                  40m